BREAKING CHANGE: ODP for Twitter - Mapping Update


#1

If you are using our Open Data Processing product to process Twitter data from Gnip please be aware that we will be updating the mapping configuration soon.

This change will alter the structure of the data you receive from DataSift, which potentially could be a breaking change for your application.

Why are we making this change?

We recently announced that as of September 1 we would cease to augment Twitter data with Klout data provided by DataSift in our Open Data Processing (ODP) product. Instead, we would pass through Klout data provided by Gnip such that it would be mapped and filterable in CSDL using the existing klout.score target.

However, we subsequently discovered that the way we changed the data mapping to facilitate this update caused an issue for some customers. We therefore rolled back this change so we could improve the solution.

In this new change we will again pass through Klout data from Gnip, instead of using the DataSift augmentation, but we have updated the data mapping to make it more suitable for our customers.

When will the change be applied?

6th October 2016.

How will the format of delivered data to my application change?

The changes are fully detailed with examples here:
https://gist.github.com/andimiller/7c6a5e00964cc660e25c9a7bd95d634f

You will see that we’ve added in a klout object at the top level, with a score inside it, this reflects the normal DataSift output for klout scores. We’ve also moved the whole gnip namespace into a top level object called gnip, away from it’s previous location of unmapped.gnip.

Note that you can filter on klout.score, however the gnip.* fields are not filterable in CSDL.

If you have any questions or concerns about this change, please contact DataSift Support.