Difference in tweet count received and billed for


#1

Hi,

We are currently testing datasift with our product, before using for production. We are having a problem with missed tweets on which I had a query.

We are using the Java client library and using the streaming API to listen to the tweets. our typical code…
user.createDefinition()
consumer = querydef.getConsumer(StreamConsumer.TYPE_HTTP, new Siftstream())
consumer.consume()

In our onInteraction class I have the following
onInteraction(StreamConsumer c, Interaction i) throws EInvalidData {
try {
datacnt++

This way I am tracking the datacnt which is the number of tweets received by us.

At the end of the run I find that the number of tweets we receive is lesser by a significant number to what is shown on the dashboard.
We have built a query with sampling rate such that we receive around 300 tweets a minute.
At this rate what could be the reason we are missing the tweets?

Thanks and Regards,
Shiva


#2

Hi,

We are in a very similar position. We too are using the Java streaming API and count our interactions as they come through.

There are, for certain times / days large discrepancies in what the dashboard is saying, and the data that we are receiving - are there any known issues around this?

Thanks,
Rob.


#3

We are currently investigating a number of issues which we believe may be resulting in a small number of interactions being lost before they are delivered. We have a number of stability releases planned for release over the coming weeks which should resolve any cases where interactions may have been lost.