I have Installed Sitecore 9 Update 1 together with Sitecore Experience Commerce 9 Update 1 on my dev VM.

I was not using it for a while and wanted to check something in Experience Analytics.

I was pretty astonished when I saw these exceptions instead of expected charts:

analyticsexception

shocked.jpg

It was working nicely last time when I left the virtual machine. Hmmm.

Problem

Exceptions seen:

The ListControl list cannot be displayed due to a server error. Contact you system administrator.

The LineChart graph cannot be displayed due to a server error. Contact you system administrator.

When I dug deeper I saw these exceptions in Firefox coming as responses to requests towards xconnect to receive data.

Exceptions in Firefox:

{message: “An error has occurred.”,
exceptionMessage: “An error occurred while sending the request.”,…}
exceptionMessage : “An error occurred while sending the request.”
exceptionType : “System.Net.Http.HttpRequestException”
innerException : {message: “An error has occurred.”,…}
exceptionMessage : “The underlying connection was closed:
An unexpected error occurred on a send.”
exceptionType : “System.Net.WebException”
innerException : {message: “An error has occurred.”,…}
exceptionMessage :
“Unable to read data from the transport connection:
An existing connection was forcibly closed by the remote host.”
exceptionType : “System.IO.IOException”
innerException :
{message: “An error has occurred.”,…}
exceptionMessage : “An existing connection was forcibly closed by the remote host”
exceptionType : “System.Net.Sockets.SocketException”
message : “An error has occurred.”
stackTrace : ”
at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult) ↵
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)”
message : “An error has occurred.”
stackTrace : ” at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) ↵
at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)”
message : “An error has occurred.”
stackTrace : ”
at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) ↵
at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)”
message : “An error has occurred.”
stackTrace :
” at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func1 taskFactory)

In Sitecore log I saw these exceptions:

 

Click to collapse/expand the full exception

5340 09:58:15 INFO Cache created: ‘ExperienceAnalytics.DimensionItems’ (max size: 2MB, running total: 7120MB) 5340 09:58:15 ERROR [Sitecore Services]: HTTP GET URL http://sxc9.sc/sitecore/api/ao/aggregates/all/46511EC0EC7A492BA90F96BAB33B7456/all?&dateGrouping=collapsed&&dateFrom=04-04-2018&dateTo=02-07-2018&keyGrouping=by-key

Exception System.Net.Http.HttpRequestException: An error occurred while sending the request. —> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult) at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult) — End of inner exception stack trace — at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar) — End of inner exception stack trace — at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar) — End of inner exception stack trace — at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func1 taskFactory)
at Sitecore.ExperienceAnalytics.Core.Repositories.ReferenceData.ReferenceDataSegmentReader.Get(IEnumerable1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Aggregation.Repositories.AggregationSegmentReader.Get(IEnumerable`1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetReportSegments() at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetRequestType() at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.GetModelFromBindingContext(HttpActionContext actionContext, ModelBindingContext bindingContext) at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext) at System.Web.Http.ModelBinding.ModelBinderParameterBinding.ExecuteBindingAsync(ModelMetadataProvider metadataProvider, HttpActionContext actionContext, CancellationToken cancellationToken) at System.Web.Http.Controllers.HttpActionBinding.d__0.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ExceptionFilterResult.d__0.MoveNext() 5340 09:58:15 ERROR [Sitecore Services]: HTTP GET URL http://sxc9.sc/sitecore/api/ao/aggregates/all/46511EC0EC7A492BA90F96BAB33B7456/all?&dateGrouping=collapsed&&dateFrom=04-04-2018&dateTo=02-07-2018&keyGrouping=by-key

Exception System.Net.Http.HttpRequestException: An error occurred while sending the request. —> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult) at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult) — End of inner exception stack trace — at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar) — End of inner exception stack trace — at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar) — End of inner exception stack trace — at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func1 taskFactory)
at Sitecore.ExperienceAnalytics.Core.Repositories.ReferenceData.ReferenceDataSegmentReader.Get(IEnumerable1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Aggregation.Repositories.AggregationSegmentReader.Get(IEnumerable`1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetReportSegments() at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetRequestType() at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.GetModelFromBindingContext(HttpActionContext actionContext, ModelBindingContext bindingContext) at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext) at System.Web.Http.ModelBinding.ModelBinderParameterBinding.ExecuteBindingAsync(ModelMetadataProvider metadataProvider, HttpActionContext actionContext, CancellationToken cancellationToken) at System.Web.Http.Controllers.HttpActionBinding.d__0.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ExceptionFilterResult.d__0.MoveNext() 2376 09:58:15 ERROR [Sitecore Services]: HTTP GET URL http://sxc9.sc/sitecore/api/ao/aggregates/all/46511EC0EC7A492BA90F96BAB33B7456/all?&dateGrouping=by-auto&&keyTop=8&keyOrderBy=visits-Desc&dateFrom=04-04-2018&dateTo=02-07-2018&keyGrouping=by-key

Exception System.Net.Http.HttpRequestException: An error occurred while sending the request. —> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.Socket.BeginReceive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags, AsyncCallback callback, Object state) at System.Net.Sockets.NetworkStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state) — End of inner exception stack trace — at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar) — End of inner exception stack trace — at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar) — End of inner exception stack trace — at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func1 taskFactory)
at Sitecore.ExperienceAnalytics.Core.Repositories.ReferenceData.ReferenceDataSegmentReader.Get(IEnumerable1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Aggregation.Repositories.AggregationSegmentReader.Get(IEnumerable`1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetReportSegments() at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetRequestType() at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.GetModelFromBindingContext(HttpActionContext actionContext, ModelBindingContext bindingContext) at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext) at System.Web.Http.ModelBinding.ModelBinderParameterBinding.ExecuteBindingAsync(ModelMetadataProvider metadataProvider, HttpActionContext actionContext, CancellationToken cancellationToken) at System.Web.Http.Controllers.HttpActionBinding.d__0.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Web.Http.Controllers.ExceptionFilterResult.d__0.MoveNext() 2376 09:58:15 ERROR [Sitecore Services]: HTTP GET URL http://sxc9.sc/sitecore/api/ao/aggregates/all/46511EC0EC7A492BA90F96BAB33B7456/all?&dateGrouping=by-auto&&keyTop=8&keyOrderBy=visits-Desc&dateFrom=04-04-2018&dateTo=02-07-2018&keyGrouping=by-key

Exception System.Net.Http.HttpRequestException: An error occurred while sending the request. —> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.Socket.BeginReceive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags, AsyncCallback callback, Object state) at System.Net.Sockets.NetworkStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state) — End of inner exception stack trace — at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar) — End of inner exception stack trace — at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar) — End of inner exception stack trace — at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func1 taskFactory)
at Sitecore.ExperienceAnalytics.Core.Repositories.ReferenceData.ReferenceDataSegmentReader.Get(IEnumerable1 keys, NameValueCollection readingPreferences) at Sitecore.ExperienceAnalytics.Aggregation.Repositories.AggregationSegmentReader.Get(IEnumerable1 keys, NameValueCollection readingPreferences)
at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetReportSegments()
at Sitecore.ExperienceAnalytics.Api.RequestTypeResolver.GetRequestType()
at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.GetModelFromBindingContext(HttpActionContext actionContext, ModelBindingContext bindingContext)
at Sitecore.ExperienceAnalytics.Api.Http.ModelBinding.ReportQueryModelBinder.BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext)
at System.Web.Http.ModelBinding.ModelBinderParameterBinding.ExecuteBindingAsync(ModelMetadataProvider metadataProvider, HttpActionContext actionContext, CancellationToken cancellationToken)
at System.Web.Http.Controllers.HttpActionBinding.d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ExceptionFilterResult.d__0.MoveNext()
4228 09:58:15 ERROR Exception when executing agent aggregation/pathAnalyzerHistoryAgent
Exception: Sitecore.XConnect.XdbCollectionUnavailableException
Message: An error occurred while sending the request.
Source: Sitecore.Xdb.Common.Web
at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func1 taskFactory) at Sitecore.XConnect.Client.XConnectSynchronousExtensions.SuspendContextLock(Func`1 taskFactory) at Sitecore.XConnect.Client.Configuration.SitecoreXConnectClientConfiguration.Initialize(XmlNode configNode) at Sitecore.Configuration.DefaultFactory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert, IFactoryHelper helper) at Sitecore.Configuration.DefaultFactory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert) at Sitecore.Configuration.DefaultFactory.CreateObject(String configPath, String[] parameters, Boolean assert) at Sitecore.XConnect.Client.Configuration.SitecoreXConnectClientConfiguration.GetClient(String clientConfigPath) at Sitecore.PathAnalyzer.Processing.Agents.TreeAggregatorAgent.Execute() at Sitecore.Analytics.Core.BackgroundService.Run()

Nested Exception

Exception: System.Net.Http.HttpRequestException Message: An error occurred while sending the request. Source: mscorlib at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Common.Web.CommonWebApiClient1.d__37.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Sitecore.Xdb.Common.Web.CommonWebApiClient1.d__32.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.XConnect.Client.WebApi.ConfigurationWebApiClient.d__4.MoveNext()

Nested Exception

Exception: System.Net.WebException Message: The underlying connection was closed: An unexpected error occurred on a send. Source: System at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)

Nested Exception

Exception: System.IO.IOException Message: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. Source: System at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult) at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)

Nested Exception

Exception: System.Net.Sockets.SocketException Message: An existing connection was forcibly closed by the remote host Source: System at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult) at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)

 

Unfortunately, this didn’t help me much.

Solution

I had a bit of a chat with Vlad Iobagiu on Sitecore Slack and he pointed out that the key could be this line (from the exceptions) and that I should check certificates:

Exception System.Net.Http.HttpRequestException: 
An error occurred while sending the request. ---> 
System.Net.WebException: The underlying connection was closed:
 An unexpected error occurred on a send. --->
System.IO.IOException: Unable to read data from the transport connection:
An existing connection was forcibly closed by the remote host. --->
System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote

I headed to MMC console to check certificates and their validity.

I have also checked permissions of certificates but everything seems to be configured and set properly.

I have then opened IIS Manager and checked what certificate is set for xconnect site.

I have found out that no certificate is selected:

xConnectIISManagerWrong.PNG

what

but also Bingo! 🙂

I just had to set it to proper certificate with name “site_name.xconnect”. Just replace “site_name” prefix with your IIS site name. In my case the certificate name was “sxc9.xconnect”.

xConnectIISManager.PNG

and all good!

No more exceptions:

analyticsworking

Also xconnect site gives proper response:

xconnectworking

Yes

Victory is mine 🙂

Hopefully this will help somebody with same exception and save you some hours of digging.

However, one mystery still remains.

How and why certificate was not set for xconnect site and why it was “reset”??

mystery

Cheers!