After successful push, status is retrying but push/log returns nothing


#1

I successfully push a very simple JSCDL with an sftp server (facebook.message contains_any “Personal”)

My code to create the push definition is
PushDefinition pushDef = user.createPushDefinition();
pushDef.setOutputType(“sftp”);
pushDef.setOutputParam(“host”, HOSTNAME);
pushDef.setOutputParam(“port”, “22”)
pushDef.setOutputParam(“auth.username”, SFTP_USERNAME);
pushDef.setOutputParam(“auth.password”, SFTP_PASSWORD);
pushDef.setOutputParam(“directory”, “/files/test”);
pushDef.setOutputParam(“delivery_frequency”, “60”);
pushDef.setOutputParam(“max_size”, “10485760”);

(Internally datasift must confirm connectivity because when I was setting this up I initially had the wrong permissions on the SFTP directory and the push/create command returned an error. I have also confirmed from a separate server that I can log in and create files)

When I list my subscriptions, it comes back with the 1 above, but the status is set to “retrying” (and I indeed get no records pushed into my SFTP directory)

The push/logs returns no errors however:

api.datasift.com/push/log?page=1&per_page=20&order_by=request_time&order_dir=desc&api_key=APIKEY&&username=USERNAME

{
“success”: true,
“count”: 0,
“log_entries”: []
}

Any thoughts on how I can debug this issue further?


#2

OK actually the permissions on the target directory were still wrong… the push/create errors if the directory cannot be accessed, but it doesn’t check whether its writable - and the push/log doesn’t log anything in that case. So my fault, though you might want to tidy up the error handling?


#3

Thanks Alex. I will ensure our Engineering Team is aware of this.


#4

We have now updated our Push to (S)FTP connectors, to ensure we correctly log errors around target directory permissions.