Any experiences with scaling out with BOS
Printed From: IdeaBlade
Category: DevForce
Forum Name: DevForce Classic
Forum Discription: For .NET 2.0
URL: http://www.ideablade.com/forum/forum_posts.asp?TID=4016
Printed Date: 16-Apr-2024 at 5:34am
Topic: Any experiences with scaling out with BOS
Posted By: ctoth
Subject: Any experiences with scaling out with BOS
Date Posted: 04-Mar-2013 at 9:49am
We will probably need to scale out for some big customers in the near future. We use DevForce 2010 currently with a desktop application solution. We will perform some experiments but I want to go ahead and query the experience of other folks here regarding this matter.
Our plan is to deploy BOS to multiple servers, those would serve as a load balanced tier above the DB tier. I haven't found too many topics about this, a couple of them suggests that using Windows Server's load balancing feature should just work. I also found a Word document white paper titled "DevForce Business Object Server Performance". That says:
"Scalability
Since the Business Object Server is effectively stateless,
the server scales very well and there are a number of good options if
additional performance is required:
- Use a
high-end processor - Since the server is not memory-bound, the throughput
scales well with increasing clock speed.
- Use
more processing units – Moving to a quad core processor or a multi-socket
configuration will increase throughput by about 80% for every doubling of
processing units.
Add more servers – This effectively adds more
processing units but scales almost linearly because the memory subsystem is not
shared among cores."
Please confirm that the caching is done on the client side and the BOS is absolutely stateless. That would really mean that it can scale out easily. At the same time I'm wondering where is the catch. Because there's always one. Right now I can see two catches:
1. The freshness of the client-side cache. If one client modifies a bunch of business object, when will another client through another BOS (or the same BOS) see that?
2. What if we have also some stored procedures which goes down to the DB tier and manipulates the data in bulk (for performance reasons).
Extra question from the quoted section: how multi-threaded is the BOS? The doc states to "use more processing units". BOS will take automatically advantage of extra cores?
Thanks,
Csaba
|
Replies:
Posted By: sbelini
Date Posted: 06-Mar-2013 at 10:02am
Csaba,
- Entity caching is not done on the server, only on the clients.
The server does users and metadata only. In case a client hit a different server, (i.e. in load balancing) the user will be re-logged in the second server.
- As for "catches" 1 & 2, the client will need to refetch the entities in order to updated. A way to "know" entities were changed would be subscribing to a push service.
- DevForce does not do anything in special in regards to multithreading - this task is left for the OS to manage.
sbelini.
|
Posted By: ctoth
Date Posted: 06-Mar-2013 at 10:35am
Thanks for the heads up sbelini.
I realized that I forgot to mention a very important fact: we use the Classic version of DevForce 2010, not the version for EntityFramework. I think your statements (except notification) are true for the classic also. I need to look at how refetch goes with Classic.
I'm new to IdeaBlade BTW.
|
Posted By: sbelini
Date Posted: 06-Mar-2013 at 11:05am
There is not "Classic version of DevForce 2010".
What is the exact DevForce version number you are using? (You can see it in any DevForce assembly)
|
Posted By: ctoth
Date Posted: 06-Mar-2013 at 11:16am
You are right. I wanted to say DevForce Classic. I guess I opened this topic in the wrong forum.
|
Posted By: sbelini
Date Posted: 06-Mar-2013 at 11:26am
|
No problem. I'm moving it.
|
Posted By: ctoth
Date Posted: 06-Mar-2013 at 11:35am
|
The exact version is "3.7.5.0".
|
Posted By: sbelini
Date Posted: 06-Mar-2013 at 12:05pm
Thanks.
You are indeed using DevForce Classic.
The answer still applies though.
sbelini.
|
Posted By: ctoth
Date Posted: 12-Mar-2013 at 4:30pm
I was experimenting a little with BOSes. We use remoting/RPC instead of
WCF to connect to the BOS. In order to rule out the effect of database
blocking to the results I just introduced dummy work to be done on the
server (Thread.Sleep). The following finding is true for a single BOS
setup: I went up to 10 connected clients. Their work tasks executed
parallelly on the server, but not more than 2 at a time from a given
client. So if all the 8 clients issued 10 work tasks request of 20
seconds of work at the same time, than a pair of work items executed at
once only at a time. So 20 work items were done after 20 seconds, the
next 20 (a pair of tasks from each client) the next 20 seconds and so
on.
This is even true if I connect with a single client to the BOS and issue 15 tasks. Only a pair is executed at once.
Question:
what kind of parameter or setting causes that? Is there a configuration
or properties switch which tunes thread pool size or connection pool
size for the BOS? ASP.NET has a 25 thread/CPU default limit, but maybe
that's only for the web server core listener which spawns the actual
worker threads later? Is there some configuration for the BOS itself?
Thanks,
Csaba
|
Posted By: ctoth
Date Posted: 12-Mar-2013 at 4:35pm
BTW, multiple BOS setup is just like that. The BOS really seems to be stateless and that allows using more of them. I'll try Network Load Balancing setup soon.
|
Posted By: kimj
Date Posted: 18-Mar-2013 at 12:00pm
|
Regarding the issue with only 2 connections from a client at one time - this is the default setting for a client remoting channel. To change it you can set the clientConnectionLimit in a client-side app.config file. I've attached a sample. uploads/11/App.zip - uploads/11/App.zip
|
Posted By: ctoth
Date Posted: 18-Mar-2013 at 1:28pm
|
Thank you so much! I couldn't find that!
|
Posted By: ctoth
Date Posted: 18-Mar-2013 at 1:33pm
|
To anybody who reads this thread: this (clientConnectionLimit) is about legacy RPC communication and not for WCF.(The stateless behavior of BOS is true for every setup though).
|
Posted By: ctoth
Date Posted: 19-Mar-2013 at 3:10pm
Hi kimj,
Could you give more instructions to the app.config. The thing is that we configure the RPC in the IdeaBlade.ibconfig, there's a section in it:
<remoting>
<remotePersistenceEnabled>true</remotePersistenceEnabled>
<communicationsTechnology>Remoting</communicationsTechnology>
<remoteBaseURL>http://10.10.131.118</remoteBaseURL>
<serverPort>80</serverPort>
<serviceName>BOS/PersistenceServer.rem</serviceName>
<serverDetectTimeoutMilliseconds>0</serverDetectTimeoutMilliseconds>
<proxyPort>0</proxyPort>
</remoting>
This corresponds to the IdeaBlade.Util.IdeaBladeConfig.RemotingInfo
class. If I add the section what you sent to our app.config, I get exceptions. I tried to remove portions of it (for example the client/welknown) to trim it down to the barely needed from.
The exceptions are below:
http://www.ideablade.com/forum/uploads/1737/clientconnlimit_exception1.txt - uploads/1737/clientconnlimit_exception1.txt
uploads/1737/clientconnlimit_exception2.txt - uploads/1737/clientconnlimit_exception2.txt
I think the exception comes from the client side. The Ideablade.Persistence.dll should be accessible to the binary, but that FileNotFoundException maybe just some aftermath, and the original one is the System.Runtime.Remoting.RemotingConfigHandler.ConfigureRemoting? The second exception comes when I remove the client section. I'm scratching my head.
|
Posted By: kimj
Date Posted: 19-Mar-2013 at 4:15pm
The .ibconfig information is for DevForce only, and is not used by the Remoting system. Generally, if you don't include a .config file with the system.runtime.remoting section, DevForce will configure remoting using information in the .ibconfig, along with both DevForce and Remoting defaults.
Here, since you need to modify a configuration setting which is not supported within the .ibconfig file, you'll need to specify the system.runtime.remoting information entirely so that the Remoting system can auto-configure from the .config file.
I'm not sure what other settings you might have in your web.config file (eg, for authentication), but based on the .ibconfig information you show and the .config file I sent previously, the only change needed to the info in the .config is to set the correct URL:
url="http://10.10.131.118/BOS/PersistenceServer.rem"
You'll still need your .ibconfig file too, as DevForce will still use that. So, on the client side, you'll have both the .ibconfig and .config files, or you can consolidate into a .config file containing both DevForce and Remoting information.
Also make sure that the references to the DevForce assemblies are set as "copy local" so that the assemblies will be copied to your application bin folder.
|
Posted By: ctoth
Date Posted: 03-Apr-2013 at 10:56am
Finally I figured it out. In our case I made the following changes to our app.config (as opposed to the one you linked, which defines channels and providers and whatnot):
<system.net>
<connectionManagement>
<clear/>
<add address="*" maxconnection="10"/>
</connectionManagement>
</system.net>
That seem to increase the "clientConnectionLimit" limit also. In my test setup I managed to get more dummy RPC work processed in parallel this way.
|
Posted By: ctoth
Date Posted: 03-Apr-2013 at 11:16am
Some links on the matter (increasing connection limit of 2 on the client side):
Last comment by http://social.msdn.microsoft.com/profile/twisterrob/?ws=usercard-mini - TWiStErRob :
http://social.msdn.microsoft.com/forums/en-US/netfxnetcom/thread/1f863f20-09f9-49a5-8eee-17a89b591007 - http://social.msdn.microsoft.com/forums/en-US/netfxnetcom/thread/1f863f20-09f9-49a5-8eee-17a89b591007
Topic opening post:
http://stackoverflow.com/questions/866350/how-can-i-programmatically-remove-the-2-connection-limit-in-webclient - http://stackoverflow.com/questions/866350/how-can-i-programmatically-remove-the-2-connection-limit-in-webclient
Some people are complaining about the fact that 2 is some kind of RFC standard (for HTTP 1.1?), but honestly I don't care.
|
Posted By: kimj
Date Posted: 03-Apr-2013 at 7:00pm
|
Thanks for the follow up and links!
|
|