I have Installed Sitecore 9 Update 1
together with Sitecore Experience Commerce 9 Update 1
on my dev VM.
I was not using it for a while and wanted to check something in Experience Analytics.
I was pretty astonished when I saw these exceptions instead of expected charts:
It was working nicely last time when I left the virtual machine. Hmmm.
Problem
Exceptions seen:
The ListControl list cannot be displayed due to a server error. Contact you system administrator.
The LineChart graph cannot be displayed due to a server error. Contact you system administrator.
When I dug deeper I saw these exceptions in Firefox coming as responses to requests towards xconnect to receive data.
Exceptions in Firefox:
{message: “An error has occurred.”,
exceptionMessage: “An error occurred while sending the request.”,…}
exceptionMessage : “An error occurred while sending the request.”
exceptionType : “System.Net.Http.HttpRequestException”
innerException : {message: “An error has occurred.”,…}
exceptionMessage : “The underlying connection was closed:
An unexpected error occurred on a send.”
exceptionType : “System.Net.WebException”
innerException : {message: “An error has occurred.”,…}
exceptionMessage :
“Unable to read data from the transport connection:
An existing connection was forcibly closed by the remote host.”
exceptionType : “System.IO.IOException”
innerException :
{message: “An error has occurred.”,…}
exceptionMessage : “An existing connection was forcibly closed by the remote host”
exceptionType : “System.Net.Sockets.SocketException”
message : “An error has occurred.”
stackTrace : ”
at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult) ↵
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)”
message : “An error has occurred.”
stackTrace : ” at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) ↵
at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)”
message : “An error has occurred.”
stackTrace : ”
at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) ↵
at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)”
message : “An error has occurred.”
stackTrace :
” at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func1 taskFactory)
In Sitecore log I saw these exceptions:
Click to collapse/expand the full exception
5340 09:58:15 INFO Cache created: ‘ExperienceAnalytics.DimensionItems’ (max size: 2MB, running total: 7120MB) 5340 09:58:15 ERROR [Sitecore Services]: HTTP GET URL http://sxc9.sc/sitecore/api/ao/aggregates/all/46511EC0EC7A492BA90F96BAB33B7456/all?&dateGrouping=collapsed&&dateFrom=04-04-2018&dateTo=02-07-2018&keyGrouping=by-key
Exception System.Net.Http.HttpRequestException: An error occurred while sending the request. —> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult) at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult) — End of inner exception stack trace — at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar) — End of inner exception stack trace — at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar) — End of inner exception stack trace — at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func
1 taskFactory)
at Sitecore.ExperienceAnalytics.Core.Repositories.ReferenceData.ReferenceDataSegmentReader.Get(IEnumerable
1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Aggregation.Repositories.AggregationSegmentReader.Get(IEnumerable`1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetReportSegments() at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetRequestType() at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.GetModelFromBindingContext(HttpActionContext actionContext, ModelBindingContext bindingContext) at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext) at System.Web.Http.ModelBinding.ModelBinderParameterBinding.ExecuteBindingAsync(ModelMetadataProvider metadataProvider, HttpActionContext actionContext, CancellationToken cancellationToken) at System.Web.Http.Controllers.HttpActionBinding.d__0.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ExceptionFilterResult.d__0.MoveNext() 5340 09:58:15 ERROR [Sitecore Services]: HTTP GET URL http://sxc9.sc/sitecore/api/ao/aggregates/all/46511EC0EC7A492BA90F96BAB33B7456/all?&dateGrouping=collapsed&&dateFrom=04-04-2018&dateTo=02-07-2018&keyGrouping=by-keyException System.Net.Http.HttpRequestException: An error occurred while sending the request. —> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult) at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult) — End of inner exception stack trace — at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar) — End of inner exception stack trace — at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar) — End of inner exception stack trace — at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func
1 taskFactory)
at Sitecore.ExperienceAnalytics.Core.Repositories.ReferenceData.ReferenceDataSegmentReader.Get(IEnumerable
1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Aggregation.Repositories.AggregationSegmentReader.Get(IEnumerable`1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetReportSegments() at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetRequestType() at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.GetModelFromBindingContext(HttpActionContext actionContext, ModelBindingContext bindingContext) at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext) at System.Web.Http.ModelBinding.ModelBinderParameterBinding.ExecuteBindingAsync(ModelMetadataProvider metadataProvider, HttpActionContext actionContext, CancellationToken cancellationToken) at System.Web.Http.Controllers.HttpActionBinding.d__0.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ExceptionFilterResult.d__0.MoveNext() 2376 09:58:15 ERROR [Sitecore Services]: HTTP GET URL http://sxc9.sc/sitecore/api/ao/aggregates/all/46511EC0EC7A492BA90F96BAB33B7456/all?&dateGrouping=by-auto&&keyTop=8&keyOrderBy=visits-Desc&dateFrom=04-04-2018&dateTo=02-07-2018&keyGrouping=by-keyException System.Net.Http.HttpRequestException: An error occurred while sending the request. —> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.Socket.BeginReceive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags, AsyncCallback callback, Object state) at System.Net.Sockets.NetworkStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state) — End of inner exception stack trace — at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar) — End of inner exception stack trace — at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar) — End of inner exception stack trace — at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func
1 taskFactory)
at Sitecore.ExperienceAnalytics.Core.Repositories.ReferenceData.ReferenceDataSegmentReader.Get(IEnumerable
1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Aggregation.Repositories.AggregationSegmentReader.Get(IEnumerable`1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetReportSegments() at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetRequestType() at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.GetModelFromBindingContext(HttpActionContext actionContext, ModelBindingContext bindingContext) at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext) at System.Web.Http.ModelBinding.ModelBinderParameterBinding.ExecuteBindingAsync(ModelMetadataProvider metadataProvider, HttpActionContext actionContext, CancellationToken cancellationToken) at System.Web.Http.Controllers.HttpActionBinding.d__0.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ExceptionFilterResult.d__0.MoveNext() 2376 09:58:15 ERROR [Sitecore Services]: HTTP GET URL http://sxc9.sc/sitecore/api/ao/aggregates/all/46511EC0EC7A492BA90F96BAB33B7456/all?&dateGrouping=by-auto&&keyTop=8&keyOrderBy=visits-Desc&dateFrom=04-04-2018&dateTo=02-07-2018&keyGrouping=by-keyException System.Net.Http.HttpRequestException: An error occurred while sending the request. —> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.Socket.BeginReceive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags, AsyncCallback callback, Object state) at System.Net.Sockets.NetworkStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state) — End of inner exception stack trace — at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar) — End of inner exception stack trace — at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar) — End of inner exception stack trace — at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func
1 taskFactory)
at Sitecore.ExperienceAnalytics.Core.Repositories.ReferenceData.ReferenceDataSegmentReader.Get(IEnumerable
1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Aggregation.Repositories.AggregationSegmentReader.Get(IEnumerable1 keys, NameValueCollection readingPreferences)
at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetReportSegments()
at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetRequestType()
at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.GetModelFromBindingContext(HttpActionContext actionContext, ModelBindingContext bindingContext)
at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext)
at System.Web.Http.ModelBinding.ModelBinderParameterBinding.ExecuteBindingAsync(ModelMetadataProvider metadataProvider, HttpActionContext actionContext, CancellationToken cancellationToken)
at System.Web.Http.Controllers.HttpActionBinding.d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ExceptionFilterResult.d__0.MoveNext()
4228 09:58:15 ERROR Exception when executing agent aggregation/pathAnalyzerHistoryAgent
Exception: Sitecore.XConnect.XdbCollectionUnavailableException
Message: An error occurred while sending the request.
Source: Sitecore.Xdb.Common.Web
at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func
1 taskFactory) at Sitecore.XConnect.Client.XConnectSynchronousExtensions.SuspendContextLock(Func`1 taskFactory) at Sitecore.XConnect.Client.Configuration.SitecoreXConnectClientConfiguration.Initialize(XmlNode configNode) at Sitecore.Configuration.DefaultFactory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert, IFactoryHelper helper) at Sitecore.Configuration.DefaultFactory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert) at Sitecore.Configuration.DefaultFactory.CreateObject(String configPath, String[] parameters, Boolean assert) at Sitecore.XConnect.Client.Configuration.SitecoreXConnectClientConfiguration.GetClient(String clientConfigPath) at Sitecore.PathAnalyzer.Processing.Agents.TreeAggregatorAgent.Execute() at Sitecore.Analytics.Core.BackgroundService.Run()Nested Exception
Exception: System.Net.Http.HttpRequestException Message: An error occurred while sending the request. Source: mscorlib at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Common.Web.CommonWebApiClient
1.d__37.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.Xdb.Common.Web.CommonWebApiClient
1.d__32.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.XConnect.Client.WebApi.ConfigurationWebApiClient.d__4.MoveNext()Nested Exception
Exception: System.Net.WebException Message: The underlying connection was closed: An unexpected error occurred on a send. Source: System at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
Nested Exception
Exception: System.IO.IOException Message: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. Source: System at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)
Nested Exception
Exception: System.Net.Sockets.SocketException Message: An existing connection was forcibly closed by the remote host Source: System at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult) at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
Unfortunately, this didn’t help me much.
Solution
I had a bit of a chat with Vlad Iobagiu on Sitecore Slack and he pointed out that the key could be this line (from the exceptions) and that I should check certificates:
Exception System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote
I headed to MMC console to check certificates and their validity.
I have also checked permissions of certificates but everything seems to be configured and set properly.
I have then opened IIS Manager and checked what certificate is set for xconnect site.
I have found out that no certificate is selected:
but also Bingo! 🙂
I just had to set it to proper certificate with name “site_name.xconnect”. Just replace “site_name” prefix with your IIS site name. In my case the certificate name was “sxc9.xconnect”.
and all good!
No more exceptions:
Also xconnect site gives proper response:
Victory is mine 🙂
Hopefully this will help somebody with same exception and save you some hours of digging.
However, one mystery still remains.
How and why certificate was not set for xconnect site and why it was “reset”??
Cheers!