How to tell if Push to FTP upload is complete or still underway


#1

We have several FTP subscriptions set up and are thinking of increasing the frequency of delivery. One question is how to avoid our side downloading files from the FTP until they are completely uploaded. I know different systems handle this problem in different ways. For example, the sending system (DataSift) could send the file with a different, temporary filename and only rename it to the expected format after the upload is complete. Or, the sending system could make a checksum available so the receiving system can verify. There are other possibilities too but I am pretty sure DataSift doesn’t do them (e.g. sending a second “done” file indicating the primary file is uploaded completely).

Can anyone shed light on this? Thank you!


#2

Take a look at the Push Status Codes page. When a Push Subscription has finished delivering everything and the Historic has completed or the live stream has been stopped, the status of the subscription will change to 'finished'. You can use the /push/get REST API endpoint to periodically check on the status of your Push Subscriptions.


#3

Thanks Jason…but I think my active FTP Push subscriptions always have a status of “active” right? While DataSift is uploading a particular file to my FTP I just am trying to figure out if there is a way to know it is done uploading that file or not.


#4

Ah - yes an active Push Subscription will have the status of 'active'. Unfortunately we provide no way to tell if we have completed uploading an individual file to your FTP server. I believe this is something that is handled by the FTP protocol itself.