Regularly updating a CSDL


#1

Just wondering what the best way to go about this is…

My situation is this:

I am searching twitter for a specific hashTag based on a user on our system. Every time a new user is created, I want to add a new hashTag to the CSDL.

I’m using the PHP client library and I have a record() method similar to this:

$user = new DataSift_User(
$options[‘datasift’][‘username’],
$options[‘datasift’][‘apikey’]
);
$definition = new DataSift_Definition($user, $this->_csdl);
$consumer = $definition->getConsumer(
DataSift_StreamConsumer::TYPE_HTTP,
array($this, ‘save’),
array($this, ‘stopped’)
);
$consumer->consume();

My save function just adds the twitter entry into my database.

The bit I’m unclear on, is how to change my CSDL. I’m unsure what happens when I re run this script. Will it create a separate consumer, running alongside the existing one? Or will it simply update the current one? And then, if I delete a user and I want to remove a hashTag from my stream, how will that work? I guess if it is updating the current stream, then no problem, but if it is adding additional consumers everytime I call this script, then how can I interact with a specific one?

I may be missing some basic knowledge here, but this is really confusing me, so any help would be much appreciated. I’ve trawled the docs and can’t seem to find the answer anywhere.

Thanks in advance,

Jon


#2

 

Each time you "change" your CSDL, you technically compile a new rule (see Version Control for full details). Whether you want to add or remove hashtags from your stream, you will still have to hit the /compile endpoint, to compile the "new" rule. Any old rules you were running will still exist with their old stream hashes. You can use the /usage endpoint to query any streams you processed in the last day.

I hope this helps.

Jason

DataSift Support