Cannot stop consumer


#1

I am using datasift-java library to create a stream and to receive data from it via TYPE_HTTP consumer.
However sometimes after I stop consuming via consumer.stop() Billing UI still shows that credit is decreasing. It proceeds decreasing even several hours after it and even after I KILL my java application until my credit reaches zero. So looks like an issue on Datasift side.

To troubleshoot: please check user jmors2 - its credit is decreasing right now when I am writing this post (Jan,11 - Jan,12). I faced this situation several times already.

Here is my java code to start consuming and to stop it:



User user = new User(credentials.getUsername(), credentials.getPassword());
Definition def = user.createDefinition(experiment.getCsdlFilter());
StreamConsumer underlyingConsumer = def.getConsumer(StreamConsumer.TYPE_HTTP,
this);

        underlyingConsumer.consume();
        ...
        underlyingConsumer.stop();

Could you please look into this problem and advise how to fix it?

Thanks,
Artsiom


#2

From the code you have given me, it all looks correct. Could you please zip up your Java code, and send it to support@datasift.com so I can try to reproduce the problem.

Thanks


#3

Jason, thanks for looking into it.

Unfortunately I did not manage to reproduce this problem on a smaller piece of code than my whole client-server application.

However while troubleshooting I’ve noticed one strange thing in datasift-java library. It seems when HttpThread stops it does not notify HttpClient in any way that connections and low-level system resources should be released as proposed by HttpComponents guide. I guess sometimes it could lead to connections being opened infinitely.

So I have modified HttpThread in datasift-java locally and re-compiled it. Added following finally block into HttpThread class:

finally {
try {
if (get != null) {
get.abort();
}

                    if (client != null) {
                        client.getConnectionManager().shutdown();
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }

I am basically doing HttpMethod.abort() and shutting down connection manager.

After this change the issue I faced before disappeared. Is my understanding right and datasift-java code should be fixed like this or I am missing something about Datasift communication specifics?

Thanks,
Artsiom


#4

Thanks for raising this. We will do some investigation into this issue, and I will post the results here.


#5

Hi Jason,

Could you please update on the status of this issue please? I have temporary done code changes and recompiled datasift-java locally however I would prefer getting official fixed version if possible.

Thanks,
Artsiom


#6

The fix went live this morning. You'll need to download the latest version of the Java client library from our GitHub page.

Let us know if you experience any further problems. We're here to help.


#7

Thanks for your efforts. I tried the fix however it seems there is a problem in the fix in datasift-java library. reader.close() invocation that was added to HttpThread class never finishes, it hangs as I see. I guess this is because underlying stream is being continuously populated by new coming tweets. As mentioned in http://hc.apache.org/httpcomponents-client-ga/tutorial/html/fundamentals.html#d4e143 if you don’t need to consume the whole entity you should do method.abort() - this works for me. Could you please take a look at this? Also JFYI in HttpComponents guide they recommend doing client.getConnectionManager().shutdown(); in finally block. Hope this will help.

Many Thanks,
Artsiom


#8

 

Thank you! Please download the latest version of the Java client library again.

The engineers assure me that it's working now. It appears that there was a compatibility issue between different runtimes.


#9

I too seem to be experiencing something similar. I have taken the sample “LiveStream.java” program provided with the Java library, updated the “Definition” to a different query, and while my Java program stopped running a couple of hours ago, my credits are slowly being eaten away even though I have no open connections. It’s literally the sample Java program but with

        Definition def = user.createDefinition("interaction.type == \"twitter\" and (interaction.content contains \"iphone\" and (interaction.geo exists))");

instead of the original Definition which just included "interaction.type == “twitter” and (interaction.content contains “music”);


#10

 

We believe the issue is a back-end problem, and has been raised as a high priority issue. We are pushing some changes to our testing environment today regarding this issue.
 
We hope to have this fixed as soon as possible, and I will keep you all updated with any progress.

#11

Hi Jason,

Could you please update the community with the status of this issue? I’m not spending any more money on DataSift until I trust that you are not ripping me or some other customer off.

The fact that you haven’t responded in 17 days says A LOT and I’m not going to get into the implications of what that means. I’m sure you are very aware, or I hope so, that your customers can put the pieces together to understand that something is very, very wrong in your back-end systems.

Sorry, the failure of your billing system also indicates failures of your data collection and filtering systems, the entire your product platform is built on.

Sorry for being so harsh, but I’ve “been there and done that” with big companies and from what I’m seeing from DataSift right now is just kind of amateurishness and not indicative of very large scale company like I thought you were.

-Marcus


#12

Apologies for the lack of updates on the status of this issue. We pushed a small updated this week, which has fixed this problem.