My best guess is that it's choking on the data during serialization.  Do you have the query which resulted in the failure, and the expected resuts?  If you can send me your EDMX and the query I can take a look, although I might need your DB too if nothing stands out.  Send to kimj@ ...

 

The default message size used by DevForce is actually 2G, the maximum allowed, unless you've modified the bindings in either the web.config or added a ServiceReferences.ClientConfig.  There's also the

   <httpRuntime maxRequestLength="sizeinKB"/>

element you can set in the web.config if you are sending large amounts of data. 

 

When using the AsyncParallelTask DevForce will fire off a new thread, and a new service request, for each action in the task.  So with 10 actions you potentially have 10 threads, and 10 requests outstanding to the service at one time.  DevForce also opens a new database connection to serve each request, so you also may have 10 open database connections too.  My new guess is that something - WCF, the firewall, the database - is complaining about a possible DoS attack.

 

A few things to look at in your code -

  - Make sure that you've called LoginAsync (or ConnectAsync and LoginAsync) prior to executing the AsyncParallelTask.  If you haven't, every action in the task will first try to connect/login before running the query, so there may be more service activity than expected. 

 - If you don't already have one, add an exception handler to the AsyncParallelTask to try to trap errors.

 - Maybe add an EntityServerError handler on the client.  This event should always fire for any errors and may help if the error is otherwise being handled somewhere else.

 

The grid, just by calling the getter on the DealCost property, was causing the async navigation logic to kick in for this related entity, resulting in another async query to the server for each row.  We likely have some bug in not correctly identifying when a related entity has resolved to the NullEntity vs. when it still needs retrieval, so I will look into that.  (In answer to question C.)

 

As for why these would cause timeouts, I have no idea. :)  The async navigation logic works a bit differently than the usual async queries and async tasks so I'll need to look into this too.  It's possible that one side (and I'm really not sure which) was just too busy processing all those individual queries of DealCost for each deal ID.   Also, when a timeout exception is received the client will retry once before throwing the exception to the caller.  So if your handler never saw the timeout exception then the retry succeeded (question A).

 

Question B) - it's not clear that the server was hung up, at least based on the log.  The server does start responding to queries again after those timeout messages.  If it was hung up, then maybe it did run into a problem of too many outstanding requests ... and what that magic number is I don't know.