DevForce Community Forum : Get only changed entities from the database http://www.ideablade.com/forum/ This is an XML content feed of; DevForce Community Forum : DevForce Classic : Get only changed entities from the database Tue, 14 Apr 2026 23:20:15 -700 Sun, 09 Dec 2007 19:31:42 -700 http://blogs.law.harvard.edu/tech/rss Web Wiz Forums 9.69 360 www.ideablade.com/forum/RSS_post_feed.asp?TID=598 DevForce Community Forum http://www.ideablade.com/forum/forum_images/IdeaBlade_logo_tm.png http://www.ideablade.com/forum/ Get only changed entities from the database : Thankd David... I will take this... http://www.ideablade.com/forum/forum_posts.asp?TID=598&PID=2149#2149 Author: sieepvvalk
Subject: 598
Posted: 09-Dec-2007 at 7:31pm

Thankd David... I will take this into consideration.]]> Sun, 09 Dec 2007 19:31:42 -700 http://www.ideablade.com/forum/forum_posts.asp?TID=598&PID=2149#2149 Get only changed entities from the database : Option 2 is better than Option... http://www.ideablade.com/forum/forum_posts.asp?TID=598&PID=2148#2148 Author: davidklitzke
Subject: 598
Posted: 09-Dec-2007 at 5:38pm

 Option 2 is better than Option 3 because:
 
(1) Option 2 takes less time for the database to execute (one time comparison versus one time comparison plus one NOT IN comparison.
 
(2) Option 2 takes less processing time on the client to formulate.
 
As the number of items in the entity cache gets larger and larger, Option 2 becomes faster and faster relative to Option 3.
]]> Sun, 09 Dec 2007 17:38:09 -700 http://www.ideablade.com/forum/forum_posts.asp?TID=598&PID=2148#2148 Get only changed entities from the database : Thank you for the suggestion.... http://www.ideablade.com/forum/forum_posts.asp?TID=598&PID=2147#2147 Author: sieepvvalk
Subject: 598
Posted: 09-Dec-2007 at 12:16pm

Thank you for the suggestion.  I want to clarify my understanding of your solution and see if there is any better way to do this.  I will present 3 options and hopefully you can help me to figure out which of these is the best way to do things with DevForce.
 
OPTION 1: This is the most basic way we can query for new orders:
 
(1) I run a query that returns 30 orders meeting a certain criteria
(2) 10 new orders are added to the system over the next 10 minutes
(3) I run the same query again, and this time i should get 40 orders.  If I use DevForce, I will have to use a DataSourceOnly or DataSourceThenCache QueryStrategy because I need to check the database to get the new orders.  This strategy will send all 40 orders back over the network, clogging network resources.
 
OPTION 2: Your solution is to follow these steps:
 
(1) I run a query that returns 30 orders meeting a certain criteria
(2) I store the date that the query was last run as our (LastRunDate)
(3) 10 new orders are added to the system over the next 10 minutes.
(4) I run a query to get all orders which match my criteria that were added since our (LastRunDate).
(5) I merge these new orders with my old orders, thus minimizing network traffic.
 
OPTION 3: Is options 2 a more optimized solution than the following?
 
(1) I run a query that returns 30 orders meeting a certain criteria
(3) 10 new orders are added to the system over the next 10 minutes.
(4) I run a query to get all orders which match my criteria and I add a "NOT" clause which excludes order which match the primary keys of the orders already in the list.
(5) I merge these new orders with my old orders, thus minimizing network traffic.
 
I am wondering what the most optimized way to do this would be.  I suppose that since our primary key is automatically indexed, the best processing speed (on the server side) would be to eliminate certain records based on their primary key.  This however poses the problem that we are adding a good amount of additional outgoing network traffic to send the list of primary keys to the server.  A big concern with our application is speed, so I would like to know which way would be more likely to minimize the overall query time for new orders.
 
I appreciate your help with this issue.
]]> Sun, 09 Dec 2007 12:16:05 -700 http://www.ideablade.com/forum/forum_posts.asp?TID=598&PID=2147#2147 Get only changed entities from the database : I think you are... http://www.ideablade.com/forum/forum_posts.asp?TID=598&PID=2146#2146 Author: davidklitzke
Subject: 598
Posted: 09-Dec-2007 at 10:31am

I think  you are asking for all the entities that have been added or changed using DevForce (or perhaps all entities that have been added or changed recently).
 
My recommendation (if this is possible) is to add CreationTimeStamp (CrtnTs) and ModificationTimeStamp (ModTs) columns (as in Employee table in IdeaBlade Tutorial database).  Then you just have to query for objects which have creation or modification time stamps newer than a particular time.
 
With respect to "CacheThenDataSource", I agree that earlier definitions were definitely misleading.  Fortunately, we recently cleaned up this documentation.  Here is the new description.
 
CacheThenDataSource
 
Check the query cache to see if the code has previously submitted the current query or an identifiable superset of it.[1]  If so, satisfy the query from the entity cache, and skip the trip to the datasource.

If the query cache contains no query matching or encompassing the current query, DevForce will go to the datasource for the requested data.  Having retrieved data from the datasource into the entity cache, it will then obtain from the entity cache the final set of entity references it returns to the caller.  By so doing, it will return references to new entities that exist only in the cache, as well as those that duplicate rows from the datasource.

Some overloads of GetEntity() and GetEntities() permit search for entities by primary key. These queries work differently from those that specify arbitrary criteria. For primary key queries, there is no preliminary search of the query cache for a matching query: rather, DevForce searches the entity cache directly for the provided key value or values. If matching entities are not found in the entity cache, the datasource is also searched.


[1] For example, if you’re now asking for all Customers from the Eastern region, and you previously retrieved all Customers (without qualification), the all-Customers-without-qualification query will be recognized as a superset of the all-Customers-from-the-Eastern-region query, and the new query will be satisfied from the entity cache.



Edited by davidklitzke - 09-Dec-2007 at 10:32am]]> Sun, 09 Dec 2007 10:31:48 -700 http://www.ideablade.com/forum/forum_posts.asp?TID=598&PID=2146#2146 Get only changed entities from the database : Is there any way to get ONLYentities... http://www.ideablade.com/forum/forum_posts.asp?TID=598&PID=2145#2145 Author: sieepvvalk
Subject: 598
Posted: 08-Dec-2007 at 1:44pm

Is there any way to get ONLY entities which have been added or changed using DevForce?
 
An application we created gets a list of entities at startup.  Every 60 seconds, it re-downloads the list from the server to make sure the user is always up to date with the latest changes.  In order to do this, we are using a QueryStrategy of "DataSourceThanCache", but this causes ALL the data to be refreshed from the server.  This causes excess network traffic and that is something we would like to avoid when only a few records change every 60 seconds.
 
It seems that the "CacheThanDataSource" FetchStrategy is a bit misleading, because it doesn't check the cache THEN get the additional records from the server... it actually should be "CacheORDataSource".  Is there something that really does perform a "CacheThanDataSource" query to minimize network traffic?
]]> Sat, 08 Dec 2007 13:44:08 -700 http://www.ideablade.com/forum/forum_posts.asp?TID=598&PID=2145#2145