New Posts New Posts RSS Feed: Clearing the cache
  FAQ FAQ  Forum Search   Calendar   Register Register  Login Login

Clearing the cache
 Post Reply Post Reply
Author
  Share Topic Share Topic  Topic Search Topic Search  Topic Options Topic Options
Customer View Drop Down
Senior Member
Senior Member
Avatar
User Submitted Questions to Support

Joined: 30-May-2007
Location: United States
Posts: 260 		Post Options Post Options   Quote Customer Quote  Post ReplyReply Direct Link To This Post Topic: Clearing the cache
    Posted: 19-Jul-2007 at 10:46am

We have been doing some tests on the amount of memory consumed when DevForce loads back entities and it’s quite significant. For example, a 25MB table in SQL server consumes 100MB when using a plain .Net DataSet and about 200MB when using a DevForce entity cache (note: the numbers are approximate). Our application will not typically load back all of this data at once, but in a user session that lasts for a number of hours it’s quite possible that they’ll gradually trigger the loading of a large amount of data back to the client, and therefore consume a lot of memory.

 

We therefore want to use a strategy of clearing out child entity lists that are no longer needed. I fully understand that we lose any subsequent performance advantage that cached data would provide, but a client session that incrementally consumes memory without releasing it as a user steps through data will be a significant problem for us.

 

The typical scenario would be something like this (and I’ll revert to the old customer-order scenario to explain it).

 

A form has two grids. The top grid has a list of customers. The bottom grid has a list of related orders for the selected customer.

 

As the user steps through each customer, the related orders are loaded back. As more and more orders are retrieved, more and more memory is consumed but not released.

 

Is it possible to:

 

(1) Clear the orders cache so that it only ever holds the list of orders that relate to the selected customer, or

(2) Clear the cache of all orders when the form is closed, or

(3) Clear the cache of all customers and orders when the form is closed?

 
Back to Top
IdeaBlade View Drop Down
Moderator Group
Moderator Group
Avatar

Joined: 30-May-2007
Location: United States
Posts: 353 		Post Options Post Options   Quote IdeaBlade Quote  Post ReplyReply Direct Link To This Post Posted: 19-Jul-2007 at 10:52am

The PersistenceManager’s internal datastore is a .NET dataset.  We should have similar memory footprint for the data as .NET DataSet.

 Here are the answers to your 3 questions:

  1. clear the orders cache so that it only ever holds the list of orders that relate to the selected customer, or

Yes. You can use Entity.RemoveFromManager or PersistenceManager.RemoveEntity overload for individual order removal from the PM’s cache.

For multiple orders, you can query (order.CustomerId not equal to CurrentCustomer.Id etc) for a list of orders using a FetchStrategy of CacheOnly to obtain the list of orders to be removed.  Then call PersistenceManager.RemoveEntities with the returned list.  

  1. clear the cache of all orders when the form is closed, or

Same as item 1. 

  1. clear the cache of all customers and orders when the form is closed?

You can do similar removal as in item 1. 

Alternatively, you may consider using a different PM for the target form.  Create a new PM on Form_Load.  Simply delete the PM on Form_Close.  This way you main application’s PM is completely isolated from the Form’s PM and its cache data.

Note: this technique only works well if none of the data changed inside the form is needed back in the main PM.

 

Yes. You can remove business objects from the PM’s cache easily.  See below…
Back to Top
IdeaBlade View Drop Down
Moderator Group
Moderator Group
Avatar

Joined: 30-May-2007
Location: United States
Posts: 353 		Post Options Post Options   Quote IdeaBlade Quote  Post ReplyReply Direct Link To This Post Posted: 19-Jul-2007 at 10:56am
A few additional observations.
 
1) If you decide to remove from manager, you should know that this clears the entire Query Cache (not just the queries associated with the fetched type). That means that the PM no longer remembers that it queried anything. Subsequent queries go to database until the Q-cache is built back up.
 
This is essential practice because removing from manager invalidates at least one query and it is near impossible to guess which other queries are affected. You removed a Company; that invalidates all Company queries but also Order queries because you might have queried for orders of a Company. And so on.
 
There is an overload of the remove in which you tell the PM not to clear the Q-cache [Pm.RemoveEntities(IEnumerable, bool)].  You can manipulate the Q-cache directly too. These techniques are for special cases wherein you can be certain that no query is adversely affected by such removal or Q-cache manipulation. You cannot be certain in the scenario presented so I strong recommend that you do not use these techniques.
 
2) I'm surprised you found such a large footprint. I'd like to see the case when you have time.
 
3) Many customers anticipate problems with cache memory ... that never in fact arrive. As always with performance tuning, it helps to measure, measure, measure.
 
4) The separate PM approach is the best and easiest for large collections of read-only data. For example, you might have a UI in which the user roots around in a huge Product Catalog. The user is profoundly unclear about what he wants and so retrieves an enormous number of products before settling on the three items he really wants.
 
The catalog search UI relies upon a "Catalog PM" within the search UI. The floundering user's queries fill out this pm, not the Main Pm. You will ultimately discard the Catalog PM (or clear it).
 
This technique works fine for selecting data or even modifying that data while it resides in the Catalog PM. 
 
Just remember to copy the selected/modified items from the Catalog PM to the Main PM before purging the Catalog PM; this copy is a simple, one-line statement.
 
BTW, the search UI can assess the size of the Catalog PM and flush it only when it hits some max (after the user leaves the search module of course).
 
Observe that the Main PM's Q-cache is unaffected by these operations.
 
5) I'm not sure it was clear: it is easy to clear the entity cache of an entire type; you don't have to do anything fancy with any queries; see PM.RemoveEntities(Type, DataRowState).
 
Back to Top
 Post Reply Post Reply

Forum Jump Forum Permissions View Drop Down