Every time: "Waiting for data" message


#1

Hello.
I have a problem and I hope you can help me. I’ve compiled a hash for imo very weakly selective condition:
twitter.text CONTAINS “the”

But I always get “Waiting for data” response message from Streaming API. What is going on? For about one hour I didn’t obtain any tweet.


#2

I’ve had the same symptoms. the stream is generating data when previewed via the DataSift UI, but all I’m getting in the stream are “tick” messages. I’ve only tried one stream (known high volume).


#3

The same situation for rule:
interaction.type == “twitter” AND interaction.content CONTAINS “the”

Is somebody here from the DataSift crew, who might give a clue to solve this problem?


#4

Oh, my bad!
There is no way to obtain data without signed license for theirs source :wink: I apologise for the confusion!


#5

You will not be able to receive any data without first signing the data source licence (if applicable).

Please note:

Running a stream like "twitter.text contains "the"" may be a banned stream. It would have been banned because it will return close to 1000 interactions per second, very quickly using all your credit, and sending you over your Twitter rate limit. Our client libraries use the example:

interaction.content contains "football"

This tends to return approximately 1 interaction per second.


#6

hi my problem is slightly different and I’m pretty sure it doesn’t related to license or data rate. the stream works fine via the UI - I get plenty of results coming through if I run in the UI either by recording or previewing.

my problem is that when I connect via the API I only get the {“tick”… messages. same in the browser, although I can put 2 browser windows side-by-side (the GUI on preview and the streaming URL) and see data coming through on one and not the other

the code used to work, but I didn’t use it extensively. I’ve read as much as I can find on the DataSift website and I can’t figure out whether streaming is a 2 step process or 1-step.

I’ve tried compiling first or just using the hash that I get from the DataSift UI (it actually gives the same hash which is not surprising).

so my question is: if I perform step 1 as I “GET” the following URL:

https://stream.datasift.com/?username=<user_id>&api_key=<my_api_key>&hash=9a0f422329cbf9bd26f742bc01d082bb

is this enough to begin receiving data (I’m pretty sure this is what happened when my code worked a few months ago) OR do I have to perform a second step to start the data flowing?


#7

silly me for using GT and LT symbols in an html form. the url is

https://stream.datasift.com/?username=(my_user_name)&api_key=(my_api_key)&hash=9a0f422329cbf9bd26f742bc01d082bb


#8

Last Question: does the message {“tick”:1367154961,“status”:“connected”,“message”:“Waiting for data”} mean DataSift is waiting for data that satisfies my filters so that it can deliver to me.

or does it mean DataSift is waiting for me to do something? (like make another call to start the stream running)?


#9

ok - I think I’ve found my problem. I’m using the wrong structure of the streaming endpoint url

http://stream.datasift.com/[hash]?username=[username]&api_key=[api_key]

works in a browser window. testing code now


#10

yep - works fine. I guess what threw me is that the first url structure I was using was returning 200 response codes and providing the “tick” messages. Not sure why that would be. ok, now on to doing something useful with that streamed data.


#11

Good stuff! http://stream.datasift.com/[hash]?username=[username]&api_key=[api_key] is the corrct format to connect to the live streaming API. 

Regarding ticks, you will receive a tick with the status 'initialised' as soon as you successfully connect to the streaming API (you do not need to have connected with a stream hash to receive this). You will receive ticks with status 'connected' if you are connected to the streaming API, and you do not receive any interactions for over 30 seconds. These are used as a Keep-Alive mechanism. You can read more about ticks in the Streaming API documentation.