Honouring Twitter delete messages post-export


#1

If I have a recorded stream I understand Datasift will process delete messages on that.

However, if I have already downloaded a copy before you process a delete, is there is an implied responsibility for me as a customer to continually re-download the recorded stream and diff with my local copy in order to identify these deletes?
Not sure what the retention policy is you would advise on here. What would Twitter require as a minimum?

The re-download approach I outlined sounds horrid and so would be better facilitated if Datasift maintain a “deletes-since-last-download” file to alert me to any delete messages that I would have to apply to my local copy.


#2

DataSift will process any deletes received on a recording, and will not deliver any deleted Tweets. If Tweets are subsequently deleted, you are not required to delete them from your store. You are only responsible for deleting a Tweet if you receive a delete notification from DataSift.

The best way to ensure you always receive every delete notification would be to run live streams rather than recordings and exports.


#3

OK, thanks.

Deletes are messy - especially if you take it to the logical conclusion that all derived aggregate stats must also be re-evaluated after delete of a tweet in case there remains a potential for information leakage in these summaries. This seems like a very grey area in terms of contractual responsibilities.

So for some apps there may be an advantage to taking recorded streams rather than live ones if, as you suggest, it frees them from the contractual responsibility of assuring removal of all traces of a tweet.