Pull connector issue

#1

hi,

I configured a pull connector for the push delivery of historic query. I am using cygwin to perform curl queries. The pull connector get created fine. The logs show the following:

ID: 69e7e355fd3fcc…
Name: connectorpull
Status: active
Created at: Tue May 06 10:49:30 IST 2014
Last request: null
Last success: null
Output Type: pull
Output Params:
format = json_new_line

But, when I try to pull data using the following commands, nothing is returned either on the command line, nor in the folder not even 200/204/400

command:

curl -X POST ‘https://api.datasift.com/v1/pull’
-d ‘id=69e7e355fd3fcc…’
-H ‘Authorization: 0’

curl -X POST ‘https://api.datasift.com/v1/pull’
-d ‘id=69e7e355fd3fcc…’
-H ‘Authorization: 0’

Please, suggest a resolution for this.

Thanks,
Sharmistha

#2

When you hit /push/get for that Push subscription, what is the value of  'remaining_bytes'? If the query you are running has not yet returned any data, hitting /pull should not return any data.

Hitting /pull directly via cURL will simply print the interactions to your terminal - they will not appear in any other folder

#3

Calling push/get using the id of the connector gave the following output:

{“subscriptions”:[],“count”:0}
What does this output mean ? This is a pull connector for a historics query ranging over 1 month. I have manually checked twitter for existence of data for the query in the give time period. Therefore, the correct result cannot be zero.

The steps I have followed for the historic query creations are as follows:

  1. Create a stream online. Record the hash value.
  2. Create a new historic query with the hash and other values. Record the historic ID.
  3. Create a pull connector with historic ID. Record the ID,
  4. Start the historic ID.

The historic query processed completely. I checked that using historics/view.

I am not sure which step is going wrong.

#4

The Push subscription may not be showing up in the response as it has finished. Try adding the following parameter to your API call:

include_finished=1

As this query has finished, remaining_bytes will likely be set to null, as any remaining data will have been dropped. Are you able to provide your DataSift username for me to take a closer look?