Delay and completeness of the Facebook feed


#1
  1. What is typical delay for Facebook data?

  2. Is it reasonable to expect each and every FB message matching the keyword to show up in the DataSift stream?

Here is some background. I have started a keyword based stream:

interaction.content contains "nosuchkeyword"
Test facebook messages I have posted with this specific keyword do not show in DataSift stream, even after 30 min or so. Twitter messages show up in under few minutes.

#2

After more investigation the issue seem to be Facebook related and not related to DataSift. I can not find my status updates using facebook native search api.


#3

We use the Facebook Open Graph API to access public Facebook posts. Unlike Twitter, which has a robust Firehose delivery model, the data returned via Facebook’s Open Graph API isn't guaranteed by Facebook to return all public posts. Running an analysis, we observed that it is not unusual for the volume of posts returned to drop sharply during the day.


#4

what about shared links? should they appear in the datasift stream?


#5

Yes. If the Facebook post is collected by DataSift, we will pick up any entities (including links) that are contained in the post.


#6

Jason,
If I write a post in a wall that contains a link, it is indeed collected by Datasift and included as a link entity. What I meant is when you share some content in a website using social share plugins. It will create a post in you wall with some info regarding the content you have shared and a link to that. However, I have not managed to receive in the Datasift stream any content generated in such a way.


#7

I am not sure whether posts generated on your profile by Social Plugins are available via the Graph API. I will have to look into this further.


#8

Thanks. I would appreciate it.


#9

Hi Jason, was there any follow-up to this? I’m trying to understand how to characterise the scope/accuracy of DataSift results (from sources other than twitter)

I similarly ran a simple filter and posted some test messages on both FB (2 status posts with the keyword, each status post had 4 comments posted by others which included the keyword) and Twitter (3 tweets).

Results: All the tweets showed up pretty quickly (as you’d expect via the firehose); but very mixed results on the FB

Summary:

  • Only one of 2 of my own FB posts to my own timeline showed up in the DataSift filter results
  • for each of the 2 primary posts that I made, there were at least 4 comments posted by my FB friends that used the keyword. None of these comments appeared in the stream.
  • Interestingly, of the 2 primary test posts, the one that showed up in Datasift results did not appear in the FB search results for that keyword; and vice-versa, the post that appeared in DataSift, did not appear in the FB search results

any thoughts?


#10

The same is still true - We use the Facebook Open Graph API to access public Facebook posts. Unfortunately the data returned via Facebook’s Open Graph API isn't guaranteed by Facebook to return all public posts, so sometimes we simily will not receive certain Facebook posts.

We are currently working on improving our Facebook data source in order to return more consistent results.


#11

thanks - I understand what you are working with.

I’m getting some odd results though, where posts that are returned through the API aren’t turning up in the DataSift feed, and vice versa.

may be different facets of the same problem, but would be interesting to get an idea on your thoughts on some specific instnaces

I sent a bunch of files to one of your colleagues - maybe I should send them to you?

regards,
a.


#12

These instances where you receive posts through DataSift, but not through the Facebook API are perfect examples of why you may sometimes miss posts in DataSift - Facebook does not guarantee delivery of it's messages through the Open Graph API to any users. While this is the case, the Facebook data source will be provided at no cost, as it is a 'best effort' service, carrying no delivery guarantees. 


#13

thanks Jason. makes sense.


#14

Hi, I have a question about Facebook shares. What I need to do is to find all shares made from my site.

The site does use the http://www.facebook.com/sharer.php to share links, example — http://oi40.tinypic.com/34ycpxd.jpg

You can see that it shares a link that starts from http://sharinghub.wegohealth.com (the post content does not contain that link though). So what I was trying to do is to use the next query: facebook.link substr “sharinghub.wegohealth.com

But that does not seem to be working. Is the query wrong or does it related to the problems described above?

BTW, I also tried to add fb:app_id meta tag to the page and filter by “source” which didn’t work either.

Best,
Alex


#15

Also tried to search by message content (event though this variant is not suitable since we have lots of messages, but just to check data delivery) and it did not work also. The next query:
facebook.message substr “Find useful resources and information” OR facebook.link substr “sharinghub.wegohealth.com


#16

Any help, please?


#17

Unfortunately it is not currently possible to receive shares of Facebook posts via DataSift.