Twitter mentions stream - missing tweets?



we’re currently looking into using DataSift as the source of data for our apps and have run a couple of test during the day.

One of the most important ones was collecting mentions for a relatively high-profile account like @techcrunch. I created a very simple stream with only one filter, namely:

twitter.mentions in “techcrunch”

…and I left it to run for approx. 1.5 hrs. It only caught 28 interactions which struck me as odd, so I went to compare Twitter’s search results with DataSift’s export. Plenty of tweets which were showing up in Twitter’s search weren’t caught by the stream. I can provide the exported JSON and the screenshot of the Twitter search results page for the relevant period if needed - I don’t see an attachment option here on the Discussion forum, but I can send it by email.

Have I missed something (i.e. is there a subtle CSDL quirk I’m not aware of, which could cause the issue), or is the stream simply incomplete?



For what it’s worth, I just tried the same with the CSDL of

twitter.mentions == “techcrunch”

…and got similar holes in the data.



One thing to bear in mind is that twitter.mentions only looks at Tweets - not retweets. If you would like to include the @mentions in retweets, you also need to include twitter.retweet.mentions in your CSDL.

If you would like to provide more details such as the stream hash you have been using, the method by which you were consuming this data, and your results, please feel free to drop us a line at the DataSift Support Helpdesk.