During the last Sitecore Hackathon, I’ve been through a very strange and annoying issue with my Contacts, while developing our Face Login module.
This happens in Sitecore 9.1, but previous versions should also be affected.
When the issue starts, your Experience Profile gets “frozen” with the contacts that you already have. Any new contact will not appear.
Your Indexer log also will start to show this error:
2019-03-07 11:44:31.739 -03:00 [Error] The attempt to recover from previous failure has not been successful. There will be another attempt. Attempts count: 35 Sitecore.Xdb.Collection.Failures.DataProviderException: *** [xdb_collection.GetContactsChanges], Line 27. Errno 50000: Sync token is no longer valid for [Contacts] table. ---> System.Data.SqlClient.SqlException: *** [xdb_collection.GetContactsChanges], Line 27. Errno 50000: Sync token is no longer valid for [Contacts] table. at System.Data.SqlClient.SqlCommand.<>c.<ExecuteDbDataReaderAsync>b__180_0(Task`1 result) at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Sql.Common.Extensions.DbCommandExtensions.<>c__DisplayClass1_0.<<ExecuteReaderWithRetryAsync>b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Sql.Common.Extensions.DbCommandExtensions.<ExecuteReaderWithRetryAsync>d__1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Sql.Common.Extensions.SqlCommandExtensions.<ExecuteReaderWithRetryAsync>d__1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Sql.Common.Extensions.SqlCommandExtensions.<ExecuteReaderWithRetryAsync>d__0.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Collection.Data.SqlServer.Managers.SqlDataRecordsManager`2.<>c__DisplayClass75_0.<<ReadChanges>b__0>d.MoveNext() --- End of inner exception stack trace --- at Sitecore.Xdb.Collection.Data.SqlServer.Managers.SqlDataRecordsManager`2.<>c__DisplayClass75_0.<<ReadChanges>b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Collection.Data.SqlServer.Managers.SqlDataRecordsManager`2.<ReadChanges>d__75.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Collection.Data.SqlServer.Managers.SqlDataRecordsManager`2.<GetChanges>d__51.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Collection.Data.SqlServer.SqlDataProvider.<GetChanges>d__16.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Collection.DataProviderCountersDecorator.<GetChanges>d__14.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Collection.DataProviderCountersDecorator.<GetChanges>d__14.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Collection.Indexing.Indexer.<>c__DisplayClass9_0.<<GetChanges>b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Collection.Indexing.IndexerExtensions.<IndexNextChangesSimple>d__0.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Collection.Indexing.SingleThreadedIndexer.<IndexNextChangesWithTiming>d__8.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.Collection.Indexing.SingleThreadedIndexer.<RunInThread>d__6.MoveNext()
What’s going on?
Good question… Wanted to take a look at my Shard databases, more specifically at the ContactFacets database:
SELECT * FROM [xdb_collection].[ContactFacets] WHERE FacetKey='Avatar'
This will list the Avatar facet of my contacts:
Look closer to the FacetData field – this is where the serialized Avatar Facet is stored. Due to the size of the image I uploaded, this will be a very long string.
In my case, this string was truncated:
{"@odata.type":"#Bookshelf.Repository.XConnect.Facets.BookshelfPicture","Picture":"/9j/4AAQSkZJRgABAQEASABIAAD (... ... ... ) 8QP7K1LxJHrUJm
Resulting in a value that cannot be de-serialized, and thus causing all issues previously described.
How to fix it?
In order to fix this issue, you will need to get rid of your broken Avatars from Shard0 and Shard1 databases.
- Run this SQL (make sure to point to your database names)
DELETE FROM [Collection.Shard0].[xdb_collection].[ContactFacets] where FacetKey='Avatar' DELETE FROM [Collection.Shard1].[xdb_collection].[ContactFacets] where FacetKey='Avatar'
This script is deleting all Avatars from all Contacts. You can add more filters to delete only the corrupted ones
- Restart your Indexer Service
- Request an XConnect rebuild:
XConnectSearchIndexer -rr
After that, your Experience Profile will get back to normal.
How to avoid this issue to re-appear?
The Avatar Facet was not designed to store huge files – an Avatar usually has a size of 200×200. In order to avoid this issue in the future, this is what I’ve done:
- Created a custom Facet to store the Base64 information of the big picture;
- Limited the upload of files to 1.5Mb maximum;
- When a picture is uploaded, two versions are stored by XConnect:
- Picture Facet (Custom) – Stores the original size of the image
- Avatar Facet (OOTB) – Stores a smaller version of the image, resized to the width of 200px
By doing this, my new contacts are appearing correctly on Experience Profile.
Leave a Reply