At 17:15 yesterday, my streaming application started receiving the following message:
“message”: “This streaming API service node is currently unavailable, please reconnect immediately.”
The DataSift client returned (Datasift\DatasiftStream\DatasiftStream.cs line 223): “Unkown exception:
Unable to read data from the transport connection: The connection was closed.”. It then stopped the stream. As the streamer needs to run 24/7, it needs to retry (using the reconnect interval rules) whatever error occurs and not terminate.
In my application, if the stream stops, it will be restarted. This should never occur as the DataSift client should deal with this (it was only put in place in the event that ‘MaxRetries’ was exceeded). Because the stream stopped, it was restarted, but the same error occurred immediately (the DataSift client is not using the reconnect interval rules in this scenario).
The restart / unknown error cycle continued until an OutOfMemoryException was thrown. The server does have plenty of ram (24GB) so this suggests that the DataSift client is not releasing resources.
Please can you look at implementing retries for all stream outcomes and check resources are being released.