Null Values in 'id' Fields - MongoDB w/ C#


#1

Hi there,

So I’ve been working on building our in house system for handling consumed streams and I’m using C# to pull data out of MongoDB, but I’m finding something strange… All fields in all documents labelled ‘id’ have a null value when queried from C#, however when using ssh to interface with mongo directly, they do not. ‘id’ is the only field in which this oddity occurs, which leads me to believe that the name of the field is the problem. Now obviously this is usually the point where google is my friend and I find that someone somewhere has found an answer to my problem and posted it somewhere on the internet, alas this time this is not the case. Anybody had the same/similar issues? Any advise on how to get around this?

Thanks in advance.

Jonathan


#2

How are you getting the data into MongoDB in the first place? Are you writing it in with your C# code, or using the DataSift MongoDB Push Connector to write straight from DataSift, into a MongoDB Collection?

I ran a quick test, pushing some data into Mongo using the Mongo Push Connector, and the value of the "_id" field for each MongoDB document is somelthing like the following - ObjectId("529f093e591f90ec628b4568"). Could you be having some problems reading that value? 


#3

I’m using the push connector to populate the collection, then using C# to model and query the collection. I’m not having any problems accessing the ObjectId, ‘_id’ contains the ObjectId of the document as I would expect, it is only fields named ‘id’ deeper within the document structure that are coming up null. For example ‘interaction.interaction.id’ or ‘interaction.twitter.id’. It’s really strange, everything else is coming through perfectly.

Just to give you an idea here’s a snippet of a couple of my modelling classes:

[BsonIgnoreExtraElements]
public class Entity
{
[BsonId]
public ObjectId _id { get; set; }
[BsonElementAttribute(“interaction”)]
public Interaction interaction { get; set; }

    [BsonIgnoreExtraElements]
    public class Interaction
    {
        [BsonElementAttribute("interaction")]
        public Interaction_ interaction_ { get; set; }
    }

    [BsonIgnoreExtraElements]
    public class Interaction_
    {
        public String id { get; set; }
    }</code>

Obviously there are a whole load of other methods in there accessing other fields, this is simplified somewhat for clarity.

Thanks again!
Jonathan


#4

It seems very strange that you would be able to grab every other field without a problem, and only have issues with the ID fields. Can you share the DataSift username of the account you are using to push this data into Mongo? (or raise a ticket with support.datasift.com if you would prefer to keep that private)


#5

Of course, the username is Salience. I shouldn’t imagine that it would be a problem with the streams, because as I mentioned in my first post, I can get the values from the id fields when I interface with mongo directly, it just seems to be when I attempt to retrieve those values using C#. Very strange indeed.


#6

Do you still have these Push subscriptions to MongoDB running? There does not seem to be an active Push subscription on your account. It looks like you are consuming your streams via your client library, and writing the results to MongoDB from there.

Either way, if you can see the *.id values when you are using the MongoDB CLI, but can't see them when you attempt to pull the data out using the MongoDB C# Driver, it suggests that there may be an issue with either the driver, or the code on your side.

I'll admit I'm not familiar with C#, though comparing the code you shared with the C# Driver documentation, it seems that you are doing the right thing:

Adding the [BsonIgnoreExtraElements] attribute to your interaction class will ensure that we ignore any attributes you have not specified within the class, and any attributes you have specified (such as the id), will be included in your interaction object. I assume you specify every other attribute you include in your interaction object in the same place as the ID? I'm wondering if this could have anything to do with inheritance?


#7

If I remember correctly we stopped the steam, I am using the data that we consumed from the stream before we stopped it for testing and building the system. but yeah you’re correct, the interaction class looks like this:

[BsonIgnoreExtraElements] public class Interaction { [BsonElementAttribute("interaction")] public Interaction_ interaction_ { get; set; } public Demographic demographic { get; set; } public Klout klout { get; set; } public Language language { get; set; } public Links links { get; set; } public Salience salience { get; set; } public Trends trends { get; set; } public Twitter twitter { get; set; } }

It is my suspicion that it could be something to do with the C# driver, but I can’t figure out how to get around it. If I’m perfectly honest I’m not entirely sure if it’s even a field that we’re actually going to need for our purposes, but it certainly is strange and slightly concerning that I can’t get at it.


#8

My recommendation for now would be to raise a ticket against the MongoDB C# Driver to see if the team behind that have any ideas. Sorry I can't be of any more use for now!


#9

No problem, I’ll be sure to do that.

Thanks anyway
Jonathan


#10

Just a follow up, I fixed this problem. It turns out that if you name any attribute within an entity class ‘id’ or any variant thereof, you’re always going to return a null value regardless of whether there is data in the field that you are trying to retrieve from the collection. This is because ‘id’ is forcemapped to ‘_id’ for use with the ObjectId. The way around it is to rename the attribute to something other than a variant on ‘id’ and use the [BsonAttributeElement] tag to cast it back to ‘id’. For example:

        <code>[BsonElementAttribute("id")]</code>
        <code>public Int64 idint { get; set; }</code>