New Posts New Posts RSS Feed: EntityState is detached when querying synchronously...
  FAQ FAQ  Forum Search   Calendar   Register Register  Login Login

EntityState is detached when querying synchronously...
 Post Reply Post Reply
Author
  Share Topic Share Topic  Topic Search Topic Search  Topic Options Topic Options
HunterMayer View Drop Down
Newbie
Newbie
Avatar

Joined: 10-Jun-2010
Location: Beaverton OR
Posts: 11 		Post Options Post Options   Quote HunterMayer Quote  Post ReplyReply Direct Link To This Post Topic: EntityState is detached when querying synchronously...
    Posted: 31-Jan-2011 at 3:56pm
Yes, those clear up my misconceptions. And ultimately #1 led to us finding the problem point in our code base. We found where the error point was occurring. It was not the threading setting. We failed either way, AuthorizedThreadID set to null or not.

Regarding thread checks. The concurrency checks in place were all good (except in one place, and that has a fix in the works). Thank you for encouraging me to try that out and get our threading model in line with the changes made in the v5.0.5.0 release. I knew setting that to null so it would work was a long shot that would need repair. Adding the setting the AuthorizedThreadID in the concurrency code already present removed it as an issue and our pre-v5.0.5.0 induced threading errors went away in total. I did that not fully understanding what wrath i might be incurring... well I knew there could be hell to pay, but did it anyway.

So here is how item #1 led us to the solution. What I did find was the following. You tell me if I should not code this way, but it seemed an OK way to go (at the time). When I cast the IEntityQuery to the IEntityQuery<> version to make use the Any() extension method, I got reports exceptions on the fail machines. I never actually called Any. I had left the IEntityQuery as is and was still calling FirstOrNullEntity() on cast I got exceptions. (and I was a bad dev and did not have the operator on the failing machine capture the details, so I owe you that). I am just excited that after working on this issue for a while a working solution is in place. Which is good, we ship later this week.

Again this is some machines and not all. But as long as I don't cast IEntityQuery.FirstOrNullEntity() works. It is absolutely bizarre. I have not experienced this but in a case where the cast object actually involved a different implementation than expected. For giggles we took the generic interface aspect out and use the EntityQuery class. In all cases the same was true.

To be honest... The layout of this framework I am working in probably lead me to (coupled with personality flaws in regards to my tendency to over design). It would be easier to explain he class layout in uml. Showing the strategy pattern using generics allowing for use of covariance... its complicated. I wont get into it. But if you ever see those symptoms again maybe this is a clue of some sort. You can drop the "maybe your design is too complicated line" but also say "hey are you cast IEntityQuery and getting funny results afterwards?" Maybe a simple project might illustrate it, if I ever get the time to make one, I will share it.

Short of the long.
We are working again on all machines that exhibited the failure simply by not casting the IEntityQuery to the IEntityQuery<> in the generic base class set up to allow covariance I used in the strategy pattern. If you want more info ask, but I suggest we go offline and talk UML, etc. for specifics. I've muddied up the forums enough this week with kludge talk to last a month... or longer...


Edited by HunterMayer - 31-Jan-2011 at 9:00pm
Back to Top
DenisK View Drop Down
IdeaBlade
IdeaBlade


Joined: 25-Aug-2010
Posts: 715 		Post Options Post Options   Quote DenisK Quote  Post ReplyReply Direct Link To This Post Posted: 27-Jan-2011 at 5:23pm
Hi HunterMayer;

1. Generally, the "best" way to check for data is to use .Count() or .AsScalarAsync.Count() or .Any() (sync or async).

2. The EM is not designed to be thread safe. We do no testing of this and can't predict what problems someone may run into.

3. It is recommended to create an EM per thread.

4. Entity caches are not shared across EMs. This lack of sharing has nothing to do with multi-threading. What we do share is metadata about entities in the application. This metadata is thread-safe and accessible on any thread.

I hope this answers your questions.
Back to Top
HunterMayer View Drop Down
Newbie
Newbie
Avatar

Joined: 10-Jun-2010
Location: Beaverton OR
Posts: 11 		Post Options Post Options   Quote HunterMayer Quote  Post ReplyReply Direct Link To This Post Posted: 27-Jan-2011 at 9:57am
1. As suspected, sounds like I need a better is approach to determining if a query results in no data. Regardless of the issue at hand (which this really has no bearing on other than this approach brought the issue up), how would you check for empty. My first attempt was count... I chose to check the top item for existence. What is generally the quickest? (hoping to obtain a solution that uses the least amount of server resources and is thus fast).

2. I can't control the threads requests come in on. I am leveraging DevForce and adapting it to our domain specific DAL service. Some calls are main UI, some are background processes doing periodic updating. I am very familiar with concurrency, its issues, pitfalls, the pains, and concur that avoiding it usually preferred to embracing it (my personal motto anyway)  I do know that the second request is all on the same thread (the main thread), but not the thread the EM was created with. In this case that thread has served its purpose, and met its end never to run again (so I am told by the initiators). Is reusing an entity manager across threads prone to issues beyond just concurrency access? We jumped from version 6.0.4.0 to 6.0.6.0. I read the warnings in total, understood the devil would have to be paid at some point. To get it running prior to release, it was set to null. Nothing said that it should not be done, but that the feint of heart should not manage there own threads... which is generally true. In my case the entity manager is always created on a background thread initially, used a couple times on that thread, stored, then allowed to be reused by whatever request (regardless of thread needs it), however I do ensure only one thread is using it at a time. In this scenarios there is only one thread accessing/using an EM at a time and this issue it is being done on the main thread.

3. Given what was stated in 2, this appears to match your assessment. Another thread creates EM, main thread uses it and some machines get an entity back that is Detached when caching is turned on. Do I need to create a new EM for each thread? That would be possible to do in our case since its just an additional key for any given source I suppose to make sure that the the EM and the activity.

I let you know what happens when I turn back on caching and use an EM per thread... by the way does each EM share a cache? I suspect I will have the same issue, but it will be the third try that fails. That is my hypothesis anyway...

Here is a list of my questions extracted from the wall of text:
1. What is generally the quickest way to check for a query that would return data?
2. Is reusing an EM across threads prone to issues beyond just concurrency access?
3. Do I need to create a new EM for each TID (thread ID)?
4. Will an EM created in one thread share cache data with an, essentially the same, EM created on another thread?

Somewhere I swear I read the answer for #4 but for the life of me I can not find the answer...
Back to Top
DenisK View Drop Down
IdeaBlade
IdeaBlade


Joined: 25-Aug-2010
Posts: 715 		Post Options Post Options   Quote DenisK Quote  Post ReplyReply Direct Link To This Post Posted: 26-Jan-2011 at 5:55pm
Hi HunterMayer;

Ok, this is what came out of the discussion.

1. You said that you've turned off AsyncNavigation (UseAsyncNavigation = false) but in your IsEmpty method, you're calling IsPendingEntity, which is only meaningful for async navigation. An entity queried directly, as with a FirstOrNullEntity query, will never return a pending entity.

2. You've turned off the cross threading checks from the EM (AuthorizedThreadId = null). Please remember that EM is not thread safe and AuthorizedThreadId was added as a warning to developers to stay away from multi-threading unless they have extensive familiarity with it. If you have not set the AuthorizedThreadId to null, we would have thrown an exception if the current thread id does not match with the "authorized" thread id.

3. In conclusion, from what we can gather from your posts, the EM might be created and queried against a background thread and the problem with the detached entities seems to arise when the EM is used on the main thread. So our suggestion at this time is to not use an EM across threads.

Hope this helps.
Back to Top
DenisK View Drop Down
IdeaBlade
IdeaBlade


Joined: 25-Aug-2010
Posts: 715 		Post Options Post Options   Quote DenisK Quote  Post ReplyReply Direct Link To This Post Posted: 26-Jan-2011 at 11:17am
Hi HunterMayer;

Please stand by. This sounds like a threading issue but let me discuss and confirm first with a senior engineer.
Back to Top
HunterMayer View Drop Down
Newbie
Newbie
Avatar

Joined: 10-Jun-2010
Location: Beaverton OR
Posts: 11 		Post Options Post Options   Quote HunterMayer Quote  Post ReplyReply Direct Link To This Post Posted: 25-Jan-2011 at 2:03pm
This is a rather obtuse way to answer DennisK's question about my use of predicate builder.

T is the entity of your choice. I can not share code or any form there of, so I really don't mean to be obtuse, however, you could call me lazy. This should convey the conceptual idea... I hope...

//The start of the query
IEntityQuery<T> baseEntityQuery = entityManager.GetQuery<T>();

//The next step, which happens elsewhere in the query building system
//If we had constraints we add them here* instead of this line (using PredicateDescription)
contraintedEntityQuery = baseEntityQuery.Where<T>(PredicateBuilder.True<T>()); //getting everything

//And the specific entity Type in particular has the following happening too
finalEntityQuery.Include(entity => entity.ZeroToManyNavigationProperty).OrderBy(entity => entity.RecordId).Skip(page * pageSize).Take(pageSize);


The names have been obfuscated to show general intent, but I hope it conveys the point. I really don't have time to work up a sample that fails against adventure works right now. But I see the value to make sure we are solved and can pass around a sharable project to make sure it is fixed, etc. etc. But I would be happy to send one to our failing machines for testing purposes (eventually I will have a machine that does).

*I can talk more about how constraints are built up (using dynamic linq and PredicateDescription), if you think that it might matter. Additional property constraints are are built up using a PredicateDescription method since we do not know the parameter names until the user picks something at run-time. Sound sufficiently complicated? Its not fun to troubleshoot, but it has been working like a charm (for reasonably small sets of constraints (<1000 values to constrain)) I hope this aspect is ultimately just not a factor at all.


Edited by HunterMayer - 25-Jan-2011 at 2:04pm
Back to Top
HunterMayer View Drop Down
Newbie
Newbie
Avatar

Joined: 10-Jun-2010
Location: Beaverton OR
Posts: 11 		Post Options Post Options   Quote HunterMayer Quote  Post ReplyReply Direct Link To This Post Posted: 25-Jan-2011 at 1:37pm
Here is the more accurate scenario (I will attempt to answer the additional questions posed by DennisK in the new description)

entityManager.ExecuteQuery(...) is NOT where this problem exhibits... I got a mixed message, my bad. I have since got clarification from the witnesses.

The following code fails on the assert and EntityState is found to be Detached.

bool IsEmpty(IEntityQuery queryToTest)
{
    var first = queryToTest.FirstOrNullEntity() as Entity;
    var ea = first.EntityAspect;
    Contract.Assert(ea.EntityState != EntityState.Detached);
    return !ea.IsPendingEntity; // for the sake of argument pending means it is not empty
}

BTW: Feel free to kibitz the implementation, I find it highly suspect that is not optimal, and I am in search of the most optimal "get in/get out" check for emptiness, but I digress.

When IsEmpty is called for the first time, IsEmpty would be false (as expected). There is a lot of data matching the query. Run the same query again, using the same entity manager, not necessarily the same thread, against the cached results, and then you see the entity is detached, null, and not even pending... This particular thread is was running on the main thread on the second query. Running execute query DOES produce data by the way. I got mixed reports of the fail point which produced my original post (as being unable to reproduce the problem locally might be the cause of that snafu)

The only setting we set on the EntityManager is as follows (see below for our workaround. it may be a clue):

entityManager.AuthorizedThreadId = null;
entityManager.UseAsyncNavigation = false;

This would happen on different data sources (same schema, but it didn't matter what data source we tried). None of them had less than 3 million records if that matters (I never thought to try this on a database with only a few rows in it, we simply don't have a running scenario where that is not true, except for the moment the data collection effort starts)... Unless you mean data source as in SQL Server 2005, SQL Server 2008... in that case SQL Server 2005 in all cases.

The query does contain an inner join. I can share the query details to a certain degree. Hopefully enough to be helpful. I will reply to this post, so as to not obfuscate this to much.

Not every development machine could/can recreate this error. It is consistent on a machine that fails*. I suspect, if I was to make a broad statement about the machines that see this issue, they are more top end development machines (relative to our environment).

What we know now:

If you turn off caching this issue goes away entirely with no other code change. It is noticeably slower (as to be expected).
I.E.,
entityManager.DefaultQueryStrategy = QueryStrategy.DataSourceOnly;

Thoughts?


Edited by HunterMayer - 25-Jan-2011 at 1:51pm
Back to Top
DenisK View Drop Down
IdeaBlade
IdeaBlade


Joined: 25-Aug-2010
Posts: 715 		Post Options Post Options   Quote DenisK Quote  Post ReplyReply Direct Link To This Post Posted: 24-Jan-2011 at 11:31am
Hi HunterMayer;

A few things.

1. "3 of the 5 fail the assert checks in the snippet on the SECOND query". Could you clarify what you meant by the "SECOND query" ? Did you mean the second assert?

2. Could you send me a code snippet on the entityQuery that you constructed using PredicateBuilder? I'd like to do a repro as well.

3. Are the data sources the same on all 5 development machines?

4. We have made a number of improvements and bug fixes in 6.0.7 relating to navigation. If you have time, you might want to upgrade and see if the issue still occurs. Meanwhile, I'll see if I can dig deeper once I get your entityQuery code snippet.
Back to Top
HunterMayer View Drop Down
Newbie
Newbie
Avatar

Joined: 10-Jun-2010
Location: Beaverton OR
Posts: 11 		Post Options Post Options   Quote HunterMayer Quote  Post ReplyReply Direct Link To This Post Posted: 21-Jan-2011 at 1:49pm
[See post below for actual fail scenario, ExecuteQuery is not where this failed, but the results were the same... a detached entity] 

I got a weird one... well from my perspective its weird... and it took use a while to root out, since I am only DevForce aware person on the team... I might guess what is happening under the hood, but I will not jump to any conclusions... but I digress. 

Here is the weird... Can you explain why (more importantly how to fix, as I really don't care about plumbing issues frankly ;) the following:

2 out of 5 dev machines on my site run this following oversimplified code snippet (insert model of your choosing) without error. 3 of the 5 fail the assert checks in the snippet on the SECOND query. I do not have a machine here (I work offsite) that fails. They are keen on sending me one since this is a show stopper for us. I would like to get them running in the meantime.

entityManager.UseAsyncNavigation = false;

var results = entityManager.ExecuteQuery(entityQuery);
foreach (var item in results)
{
    if (item is IbEm.Entity)
    {
        var e = item as IbEm.Entity;
        var ea = e.EntityAspect;
        Contract.Assert(ea.EntityState != EntityState.Detached);
        Contract.Assert(!ea.IsNullEntity);
    }
}


I am using DevForce universal version 6.0.6.
VS2010 in all cases
The entityQuery parameter used in the execute is an untyped IEntityQuery partly made using PredicateBuilder
The entityManager is not accessing data via BOS (I.E., it is making a direct connect)
The two machines that consistently never have detached entities upon enumerating the results are an XP, and Vista. The three that always fail on the second query (the first one works!) are a Vista and two are Windows 7 development environments.

What are the settings I should care about. We want synchronous behavior in the enumeration. This feels like a race condition with the state (albeit non-error) under the hood somewhere.

Do you have any hints as I proceed? I'd like to not have to re-enumerate the list (or listen for pending completion events if that even applies here). Waiting (blocked) until an entity is ready is most desirable. Do I need to wait for an event somewhere...

What are your thoughts?

What do I need to have set (or do) to keep this state from being encountered when enumerating.

Thanks!!!


Edited by HunterMayer - 25-Jan-2011 at 2:06pm
Back to Top
 Post Reply Post Reply

Forum Jump Forum Permissions View Drop Down