New Posts New Posts RSS Feed: Notification Service WebHost Error
  FAQ FAQ  Forum Search   Calendar   Register Register  Login Login

Notification Service WebHost Error
 Post Reply Post Reply
Author
  Share Topic Share Topic  Topic Search Topic Search  Topic Options Topic Options
JamesM View Drop Down
Newbie
Newbie


Joined: 28-Sep-2011
Posts: 11 		Post Options Post Options   Quote JamesM Quote  Post ReplyReply Direct Link To This Post Topic: Notification Service WebHost Error
    Posted: 03-Feb-2012 at 8:40am
This is a continuation of the this post: "http://www.ideablade.com/forum/forum_posts.asp?TID=3237&title=notification-service-subscription-timeout-exception"

While we had thought we had resolved the issues with our subscription to the notification service, we continued to see VERY sporadic instances of the error.  We had been able to reproduce the error in development once, we were able to capture error is three locations. Windows event viewer, BOS debug log, and in the client exception handling.

Windows Event Viewer

WebHost failed to process a request.
 Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/12547953
 Exception: System.ServiceModel.ServiceActivationException: The service '/[Redacted]/EntityServer.svc' cannot be activated due to an exception during compilation.  The exception message is: Object reference not set to an instance of an object.. ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at IdeaBlade.EntityModel.Server.EntityServerHostFactory.CreateServiceHost(String constructorString, Uri[] baseAddresses)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CreateService(String normalizedVirtualPath)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.ActivateService(String normalizedVirtualPath)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath)
   --- End of inner exception stack trace ---
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath)
   at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath)
 Process Name: w3wp

BOS Debug Log

IdeaBlade.EntityModel.Server -->
NotificationServiceManager::AsyncNotify -->
Removing subscriber after error The message could not be transferred within the allotted timeout of 00:01:00. There was no space available in the reliable channel's transfer window. The time allotted to this operation may have been a portion of a longer timeout. 

Client Exception Capture

IdeaBlade.EntityModel.EntityServerException: Unable to subscribe to service.  Make sure the EntityServer is listening for INotificationService operations. ---> IdeaBlade.Core.IdeaBladeException: Unable to subscribe to service.  Make sure the EntityServer is listening for INotificationService operations. ---> System.ServiceModel.CommunicationException: The server did not provide a meaningful reply; this might be caused by a contract mismatch, a premature session shutdown or an internal server error.

Server stack trace: 
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at IdeaBlade.EntityModel.INotificationService.Subscribe(SessionBundle sessionBundle, SubscribeWorkState workstate)
   at IdeaBlade.EntityModel.NotificationCallbackManagerBase.Subscribe(SessionBundle sessionBundle, SubscribeWorkState workstate)
   --- End of inner exception stack trace ---
   at IdeaBlade.EntityModel.RemoteEntityServerProxyBase.CheckConnection(Exception pException)
   at IdeaBlade.EntityModel.EntityServerProxy.RegisterCallback(SessionBundle sessionBundle, SubscribeWorkState workstate)
   at IdeaBlade.EntityModel.EntityManager.<RegisterCallbackAsyncCore>b__68(RegisterCallbackOperation op)
   --- End of inner exception stack trace ---
   at IdeaBlade.EntityModel.AsyncProcessor`1.<>c__DisplayClass2.<.ctor>b__0(TArgs args)
   at IdeaBlade.EntityModel.AsyncProcessor`1.Signal()
   at IdeaBlade.EntityModel.AsyncProcessor`1.<Execute>b__5(Object x)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   at MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(Object source, Delegate method, Object args, Int32 numArgs, Delegate catchHandler)

-----------------------
Here is our config for server

<?xml version="1.0"?>
<configuration>
  <configSections>
    <section name="ideablade.configuration" type="IdeaBlade.Core.Configuration.IdeaBladeSection, IdeaBlade.Core"/>
  </configSections> 
  <connectionStrings>
	[Redacted]
 </connectionStrings>  
  <ideablade.configuration version="6.00" xmlns="http://schemas.ideablade.com/2010/IdeaBladeConfig" > 
    <logging logFile="log\DebugLog.xml"
              archiveLogs="true"/>
    <notificationService enabled="true"/>    
  </ideablade.configuration> 
  <system.serviceModel>
    <serviceHostingEnvironment aspNetCompatibilityEnabled="false"
                               multipleSiteBindingsEnabled="true" />
    <services>
      <service name="EntityService">
        <endpoint address=""
                  binding="customBinding" bindingConfiguration="compressedBinaryBinding"
                  contract="IdeaBlade.EntityModel.IEntityServiceContract" />
      </service> 
      <service name="IdeaBlade.EntityModel.Server.EntityServer">
        <endpoint address=""
                  binding="customBinding" bindingConfiguration="compressedBinaryBinding"
                  contract="IdeaBlade.EntityModel.IEntityServerContract" />
        <endpoint address="NotificationService"
                  binding="wsDualHttpBinding" bindingConfiguration="dualBinding"
                  contract="IdeaBlade.EntityModel.INotificationService" /> 
      </service>
    </services>
    <bindings> 
      <customBinding>
        <binding name="compressedBinaryBinding" sendTimeout="00:05:00" receiveTimeout="00:05:00">
          <gzipMessageEncoding>
            <readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647" />
          </gzipMessageEncoding>
          <httpTransport maxReceivedMessageSize="2147483647" />
        </binding>
 
      </customBinding>
      <wsDualHttpBinding>
        <binding name="dualBinding"
                 messageEncoding="Mtom"
                 receiveTimeout="00:10:00"
                 maxReceivedMessageSize="2147483647"
                 maxBufferPoolSize="2147483647">
          <reliableSession inactivityTimeout="01:00:00" />
          <security mode="None"  />
        </binding>
      </wsDualHttpBinding>
    </bindings> 
    <extensions>
      <bindingElementExtensions>
        <add name="gzipMessageEncoding" type="IdeaBlade.Core.Wcf.Extensions.GZipMessageEncodingElement, IdeaBlade.Core"/>
      </bindingElementExtensions> 
    </extensions>
  </system.serviceModel> 
  <system.web>
    <compilation debug="true" targetFramework="4.0" />
  </system.web> 
  <appSettings>
    <add key="NotificationPollTimeSpan" value="00:00:05"/>
  </appSettings> 
</configuration>
And our Client Config

<ideablade.configuration version="6.00" xmlns="http://schemas.ideablade.com/2010/IdeaBladeConfig"> 
    <logging logFile="DebugLog.xml"/>
    <objectServer
      remoteBaseURL="[Redacted]"
      serverPort="80"
      serviceName="[Redacted]\EntityService.svc"
      >
      <clientSettings isDistributed="true" />
    </objectServer> 
    <notificationService clientPort="39199"/>
  </ideablade.configuration>

The full stack trace of the exception thrown server side appears to be hidden somewhere in "NotificationServiceManager::AsyncNotify" perhaps there is a way of capturing more of what is happening there, we are unaware of how that is being captured and logged.

Our client is a WPF and we have explicitly declared the server side configuration for notification service and provided explicit files for  EntityServer.svc and EntityService.svc in our BOS project.

Is there there explicit configuration that needs to be handled to allow for a load balanced environment?
Back to Top
kimj View Drop Down
IdeaBlade
IdeaBlade
Avatar

Joined: 09-May-2007
Posts: 1391 		Post Options Post Options   Quote kimj Quote  Post ReplyReply Direct Link To This Post Posted: 03-Feb-2012 at 3:28pm
Are you seeing this error only in a load balanced environment?
 
The wsDualHttpBinding which the notification service uses does require "sticky" sessions in your load balancer.
 
You also need to ensure that you assign the same SessionEncryptionKey in the serverSettings element in the ideablade.configuration section so that DevForce can authenticate user credentials correctly on all servers.
 
The Event Viewer error about the compilation error with the EntityServer.svc is the most telling. Is it possibly occurring either when the application pool has recycled or the request is being processed by another server in the cluster?
Back to Top
JamesM View Drop Down
Newbie
Newbie


Joined: 28-Sep-2011
Posts: 11 		Post Options Post Options   Quote JamesM Quote  Post ReplyReply Direct Link To This Post Posted: 06-Feb-2012 at 8:59am
We can only reliably reproduce the error in a load balanced environment, however we have gotten the error a handful of times in development where we don't have a load balancer.

Can you point us in the direction of documentation about setting up "Sticky" sessions on a load balancer? This concept is new to us.

Can you provide an example for the SessionEncryptionKey?

As per our other thread "http://www.ideablade.com/forum/forum_posts.asp?TID=3237&title=notification-service-subscription-timeout-exception" we tried to explicitly configure the wsDualHttpBinding clientBaseAddress, but because we are using ClickOnce for our WPF client and we can't edit the config files without resigning the manifest.

We assume we need to set up the binding in code. Again we are assuming that when we use the flag for turning the notification service on, Ideablade sets this binding up in code for us and we would like to be able to explicitly configure it ourselves. Do you have an example of building this endpoint in code, grabbing the machinename.

We are trying multiple avenues of attack to try to solve this issue.
Back to Top
kimj View Drop Down
IdeaBlade
IdeaBlade
Avatar

Joined: 09-May-2007
Posts: 1391 		Post Options Post Options   Quote kimj Quote  Post ReplyReply Direct Link To This Post Posted: 06-Feb-2012 at 10:39am
How to set up sticky sessions of client affinity is dependent on your configuration, but here's some very brief information for Windows Server 2008 - http://ddkonline.blogspot.com/2010/10/how-to-check-that-windows-server-2008.html.
 
 
DevForce will use default configuration settings if nothing (or insufficient information) is specified in either the .config or via code.  To explicitly configure the client settings in code you can use the ServiceProxyEvents class.  I don't think we have anything showing how to configure the notification binding, but here's some information to get you started - http://drc.ideablade.com/xwiki/bin/view/Documentation/code-sample-custom-client-configuration.  I can work up a sample for the notification binding if necessary.
 
Back to Top
 Post Reply Post Reply

Forum Jump Forum Permissions View Drop Down