DevForce Classic: Query Add Span

Query Add Span

Printed From: IdeaBlade
Category: DevForce
Forum Name: DevForce Classic
Forum Discription: For .NET 2.0
URL: http://www.ideablade.com/forum/forum_posts.asp?TID=568
Printed Date: 23-Apr-2025 at 6:38pm

Topic: Query Add Span

Posted By: Markh
Subject: Query Add Span
Date Posted: 29-Nov-2007 at 4:14pm

When adding a span entity relation to a query and retrieving data based on DataSourceOnly. I would expect the Persistence Manager cache to update child entities that do not exist in the database.

Execute the following query

Console.WriteLine(GetCustomers()[0].ContactName);

Console.WriteLine(GetCustomers()[0].OrderSummaries.Count);

Console.ReadLine();

remove an OrderSummary record from the database

Now execute the same query the child count in both queries is the same. (ie Displays records that have been deleted from the database)

Console.WriteLine(GetCustomers()[0].ContactName);

Console.WriteLine(GetCustomers()[0].OrderSummaries.Count);

Console.ReadLine();

static EntityList<Model.Customer> GetCustomers()

{

EntityQuery query = new EntityQuery(typeof(Model.Customer));

query.AddSpan(Model.EntityRelations.Customer_OrderSummary);

return PersistenceManager.DefaultManager.GetEntities<Model.Customer>(query, QueryStrategy.DataSourceOnly);

}

Can you confirm that this result is expected?

What would be the best approach to update child entities in the Default Persistence Manager?

Replies:

Posted By: pkarsh
Date Posted: 30-Nov-2007 at 3:24pm

Assuming that within the same program you run the query, get the count of OrderSummaries, then, while the program is waiting for a response to the Console.Readline() call, you delete an OrderSummary that is associated with that Customer from the database, and then run the query again, the behavior you report is what is expected. If you had made a change to the OrderSummary record in the database instead of deleting it, the DataSourceOnly query strategy would cause that change to be put in the cache. In other words, to answer the last question in your post, the DataSourceOnly query strategy would in fact update child entities in the PersistenceManager. The DataSourceOnly query strategy does not, however, remove the Entity from the cache even though it no longer exists in the database. I grant you that the documentation may not be totally explicit on that point.

If what you really want is for entities not in the database to not be in the cache, you can do a Clear() method call on the PersistenceManager prior to calling GetEntities again. If you are concerned about the situation where you try to save an entity which has in fact been deleted from the database table, consult the tutorial on "Handling Concurrency Conflicts" in the intermediate section of the Tutorials, which addresses this situation among other things.

Posted By: jozza
Date Posted: 02-Dec-2007 at 9:03pm

Hi Paul

Thanks for your reply. Our issue is not so much during the saving of entities, because as you suggested, we receive a concurrency violation if we try to update a record that is no longer in the database.

Our problem only really occurs when we want to view data and its relationships eg. In a tree view. One would expect that if one was to use the DataSourceOnly query strategy that all the relevant data (relationships included) should accurately represent the data in the database. Alas, this is not the case. The data which has been deleted from the database (by other means) is still in the cache after the DataSourceOnly query has been executed. And this, for us, is a bad thing.

We don't want to have to clear everything out of the persistence manager cache everytime we want to refresh a node on the tree.

The fact that the DataSourceOnly query adds/updates New and Modified records from the database, but doesn't remove records that are no longer in the database lacks a certain amount of symmetry which one would expect from such a query.

We've managed to work around the problem by overriding the GetManagagedChildren method in our base entity, so that it forcibly removes and then re-adds the related children. This minimises the amount of data that needs to be cleared from the cache, and thereby minimises the amount of data that needs to be refetched. Note, we take care not to remove any entities in the cache that have the "Added" row state (because even though they're not in the database, they have been added by us in the devforce code and are yet to be persisted. It would be a shame to lose them).

I guess we would like to know why Devforce doesn't remove items that have been deleted from the database from the cache after a DataSourceOnly query.

Cheers

Jacob

Posted By: pkarsh
Date Posted: 04-Dec-2007 at 2:36pm

Hi Jacob

Here is a response to your question that was written by Wasrd Bell, the Product Manager for DevForce and the author of "Ward's Blog" on our Web site.

I agree with you that "The fact that the DataSourceOnly query adds/updates New and Modified records from the database, but doesn't remove records that are no longer in the database lacks a certain amount of symmetry "

Permit me to restate the use case.
If I issue a DataSourceOnly query of an Order graph that is already in cache AND
supplement the query with a span that asks for the related OrderDetails AND
there exists an OrderDetail of that Order in cache (call it "Detail-X") BUT
Detail-X has been deleted in the database THEN
Detail-X will still be present in the cache.
In other words, DevForce will not have detected that Detail-X was NOT among the details returned in the query. Therefore, Detail-X continues to appear to be a detail of the Order when, in fact, it no longer exists as of the moment of refresh.

This is manifestly the wrong result.

[Aside: this is a problem only for the related entities. If our query had been for Orders alone and there was an unmodified Order-X that had been deleted, DevForce would have silently removed it from the cache as expected. The problem is that unmodified related objects are not removed.]

There are a number of ways we could have done this properly. We could have silently queried the cache first, before going to the database, and then confirmed that all of the same entities came back (whether the root entities and their related entities). We would then discard any unmodified entities (root or related) that were not among those fetch results.

Now I understand why we don't do so for the DataSourceOnly query. By specifying DataSourceOnly, you told DevForce to take a performance short cut - to get the data directly from the data source and fix the cache only as necessary.

Unfortunately, we have since discovered that CacheThenDataSource (the default) and DataSourceThenCache don't cleanup deleted related items. I believe they should be cleaned up and have added the issue to our defect tracking system.

I do not know when we will implement a repair. Frankly, we have not run into this in five years and, in my judgement, we have more urgent priorities. So, while I am embarassed about the problem, I will have to endure that embarassment for awhile.

You, on the other hand, need a more immediate solution.

Clearing the entire cache is a terrible idea and completely unnecessary.

I don't think that I would override the GetManagedChildren method. That seems a bit extreme for what should be a relatively rare case.

I think that I would tie into the fetched event (fired by DevForce whenever it returns from the data source after a query) so that I cleaned things up when they happened. For one, this ensures that you only clean up after the fetch succeeds.

Before you start, make this change to your model's business object base class (you DO have one, yes?)
* Define an IQueryCleanup interface
* IQueryCleanup has a single read/write property called CleanUpGuid backed by a (nonpersistent) mGuid field.
* the base class explicitly implements your custom interface, IQueryCleanup so it can get/set mGuid.

Now write a FetchedHandler:

Were going to do a mark-and-sweep here. The handler does something like the following:
* sets a local variable called qSpans to the spans on the query (call query.GetSpans()); your handler has to learn about the query but that's a detail for you.
* If no spans in the query, exit the hander (no problem)
* set a local variable, aCleanUpGuid to Guid.New();
* the handler receives a list of savedEntities from DevForce.
* Foreach entity in savedEntities, cast as IQueryCleanUp and set CleanUpGuid = aCleanUpGuid
* Make a list of just the rootEntities in savedEntities

-- the following is the easy way ... not necessarily the most efficient possible but I'd try this first

* set a local variable, entityGraphs, to the relevant parts of the graphs of the root entities with the expression:

pm.GetEntityGraph(rootEntities, qSpans, DataRowState.Unmodified | DataRowState.Deleted);

this returns all related entities that are either unmodified or scheduled for deletion anyway (might as well remove them too).

* Foreach entity in entityGraphs, if it implements IQueryCleanUp and if CleanUpGuid <> aCleanUpGuid, then remove the entity (or, safer, call anEntity.Refresh()).
[NB: this is pseudo code that I haven't tried. It ought to work ;-} ]

------------

Clearly there is extra processing here which you won't need with every query in your application. You might want to encapsulate it in a special query method of an object that wraps your PersistenceManager.

That method would hook up the Fetched handler before performing the spanned query and then remove it when the query returned. You wouldn't even have to hook it up if the query lacked spans. But then you wouldn't call this method unless there were spans, would you.

Remember to use a Try/Catch to ensure that the wiring/dewiring survives exceptions.

Hope this helps. Sorry for the problem.

Ward

Posted By: travisvlowe
Date Posted: 30-Jan-2009 at 12:36pm

Has this issue been fixed in DevForce EF?

Posted By: GregD
Date Posted: 03-Feb-2009 at 5:36pm

It doesn't appear that it has (I tested it), and I couldn't find it in the issues database. So I've re-entered it today.

Posted By: travisvlowe
Date Posted: 04-Feb-2009 at 1:45pm

The solution proposed below would certainly work for queries with spans, but as Ward points out, it would not work for queries without spans including lazy-load queries. Is there any plan for handling that scenario?

Posted By: CarlosIsaza
Date Posted: 22-Feb-2010 at 9:43pm

It has been over a year since the last posting to this topic but I beleive it remains to be an important issue for users of DevForce. We started using DevForce in one of our projects a couple of years ago, and this problem has deterred us from moving further with the product and rather consider using LINQ to SQL to build the entity models for new compnents of the application. Nonetheless, I regularly come back to IdeaBlade's web site to chek for new developments and the possibility of using it for the data model of our application. We considered many alternatives to solve the issue even including adding a refetch method for each of our data entities to remove deleted elements from the cache but we could not find an acceptable broad, generic solution. Clearing the cache is a non-viable option in our case because we load many "supporting" lists to the controls that we cannot clear every time a record needs to be reloaded from the database. However, if we could find a way of removing a selected list of entitties and their children from the cache, that would be a great help as we would then be able to get only the required entities from the database again. Is there a way to accomplish this?

Posted By: GregD
Date Posted: 24-Feb-2010 at 3:39pm

Originally posted by CarlosIsaza

...if we could find a way of removing a selected list of entitties and their children from the cache, that would be a great help as we would then be able to get only the required entities from the database again. Is there a way to accomplish this?

< ="-" ="text/; =utf-8">< name="ProgId" ="Word.">< name="Generator" ="Microsoft Word 12">< name="Originator" ="Microsoft Word 12">Sure. EntityManager has a RemoveEntities() method to which you can pass an IEnumerable<T>. Thus you can make a list of entities, populate it with any entities you like, and pass that list to the method, whereupon the entities will be removed from the cache.

Posted By: CarlosIsaza
Date Posted: 24-Feb-2010 at 5:28pm

Thanks for your reply Greg. I have tried this approach and it works fine to remove the entity (or list of entities) from the cache but it does not seem to remove the related children entities. Is there a recursive way to remove the entity and its related children entities or if not, how can I find the children entities of an entity to apply the RemoveEntities method recursively?