Sitecore Xperiences - The things I've seen as a Sitecore Developer

PhantomJS Screenshots not being generated with Windows Authentication turned on

It has been observed with several versions of Sitecore 8 and above (empty versions) – including 8.2 Update 2 – that the screenshot generation is broken when the site has Windows Authentication turned ON.

How Windows Authentication is turned on on IIS

Symptoms

When attempting to start up a test in any level, the Test Start dialog will show screenshots of your website to demonstrate how your tested content will display. This can be experienced, for instance, if you trigger the Page Test command inside Experience Editor:

Create Page Test

When the Page Test dialog appears you will noticed screenshots are all broken:

Page Test - Screenshot errors 1

Page Test - Screenshot errors 2

Sitecore log shows the following messages during the generation of screenshots:

ManagedPoolThread #16 19:09:27 INFO Job started: Generate screenshots
18756 19:09:28 INFO [Content Testing]: PhantomJS console: 
Resource error
id: 0
url: http://sc82rev161221/?sc_mode=edit&sc_itemid={110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9}&sc_disable_edit=yes&sc_lang=en&sc_version=1&sc_vercomp=0
errorCode: 5
errorString: Operation canceled
!--error--!
ManagedPoolThread #16 19:09:28 INFO Job ended: Generate screenshots (units processed: 1)
ManagedPoolThread #5 19:09:30 INFO Job started: Generate screenshots
28116 19:09:30 INFO [Content Testing]: PhantomJS console: 
Resource error
id: 0
url: http://sc82rev161221/?sc_mode=edit&sc_itemid={110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9}&sc_disable_edit=yes&sc_lang=en&sc_version=1&sc_vercomp=0
errorCode: 5
errorString: Operation canceled
!--error--!
ManagedPoolThread #5 19:09:30 INFO Job ended: Generate screenshots (units processed: 1)

Root Cause

In the end the root cause is not exactly Windows Authentication, but the fact that PhantomJs, when confronted with the Windows Authentication prompt, don’t know how to proceed.

WIndows Auth Prompt

Unable to type Login and Password at this prompt, PhantomJS will get back a “403 – Forbidden” error and fail creating screenshots.

Short Story: the root cause is that PhantomJS doesn’t have proper permissions to access the filesystem and write down the screenshot images. This issue thus, may also happen in scenarios where Windows Authentication is not enabled.

How Windows Authentication is being used

We are using Windows Authentication to achieve a specific goal at our environment: During Sitecore development, we have multiple Sitecore instances with different purposes (DEV, QA, UAT), being constantly updated by TeamCity, our Continuous Integration server. One of these instances (generally UAT) is exposed to public so our clients can access from outside. However, we don’t want unrelated people accessing this address, so we lock it down with Windows Authentication and provide our clients one or multiple internal accounts to login and access the website. Unrelated people would be stopped at the Windows Authentication prompt.

Ideal solution would be for us to know exactly what folder(s) to enable Anonymous authentication, so the website itself is protected and still PhantonJS can properly generate screenshots.

Solution

With the help of Sitecore Helpdesk we were able to determine how to tweak our website, enabling PhantomJS to properly generate screenshots of our website, and still sheltering it from being accessed by anyone but the people we want.

The solution consists of A) enabling Anonymous authentication for the site root node and B) disabling Anonymous authentication for every sub-folder and files

Powershell Script

Simple to understand, hard to implement.

Can we have this automatized? Yes of course! With the assistance of my college Eduardo Pereira  (thank you very much, my friend!) I was able to end up with the following PowerShell script:

Import-Module WebAdministration
$Str_IISSiteName = "Your-Website-Name-On-IIS"
$Str_PSPath = "IIS:\Sites\" + $Str_IISSiteName + "\"
$Obj_Site = Get-Item $Str_PSPath
$Obj_Items = Get-ChildItem $Obj_Site.physicalPath
write-host "Changing authentication method on " $Str_IISSiteName
Set-WebConfigurationProperty -Filter "/system.webServer/security/authentication/AnonymousAuthentication" -Name Enabled -Value True -PSPath $Str_PSPath
Set-WebConfigurationProperty -Filter "/system.webServer/security/authentication/WindowsAuthentication" -Name Enabled -Value True -PSPath $Str_PSPath
Foreach ($Obj_Item in $Obj_Items)
{
    write-host "Changing authentication method on" $Str_PSPath$Obj_Item
    Set-WebConfigurationProperty -Filter "/system.webServer/security/authentication/AnonymousAuthentication" -Name Enabled -Value False -PSPath $Str_PSPath -location $Obj_Item
    Set-WebConfigurationProperty -Filter "/system.webServer/security/authentication/WindowsAuthentication" -Name Enabled -Value True -PSPath $Str_PSPath -location $Obj_Item
}

Important: you may need to give proper access to enable PowerShell to properly execute the script above. According to this StackOverflow article, that very well describes the issue and solution, you may have to firstly execute the steps below:

  1. Open IIS Manager
  2. Click the server name in the tree on the left
  3. Right hand pane, Management section, double click Configuration Editor
  4. At the top, choose the section system.webServer/security/authentication/anonymousAuthentication
    Right hand pane, click Unlock Section
  5. At the top, choose the section system.webServer/security/authentication/windowsAuthentication
    Right hand pane, click Unlock Section
Posted in Phantom JS, Support Ticket

Richtext Editor - Table Properties adds cells to the table

When you edit the properties of a table in the content editor of most page types, extra cells are added. Check the following animation to see the issue:

Table Issue

 

This issue has been reproduced in empty (clean) instances of Sitecore 8.1 rev 160302, 160519 and Sitecore 8.2 rev 161221, and has been registered by Sitecore as a bug at their Bug Tracking System with the reference number 101901.

Solution

To resolve the issue, please update the Telerik RTE assemblies to the newer version:

  1. Extract the Telerik.Web.UI.dll and Telerik.Web.UI.Skins.dll assemblies into your bin folder (replacing the existing ones);
  2. Replace the Telerik.Web.UI.XML file in your bin folder with the attached one;
  3. Add the Telerik.Web.UI assembly binding to your web.config:
    <runtime>
     <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
     ...
     <dependentAssembly>
     <assemblyIdentity name="Telerik.Web.UI" publicKeyToken="121fae78165ba3d4" />
     <bindingRedirect oldVersion="2015.1.401.45" newVersion="2015.3.930.45" />
     </dependentAssembly>
     </assemblyBinding>
    </runtime>
Posted in Uncategorized

Language specific MediaProvider breaking icons at Media Library

UPDATE: Kamruz Jaman has contributed with a comment for a simpler, less intrusive and more generalist solution:  Inside the GetMediaUrl method of your custom MediaProvider, check for the “Thumbnail” property of your MediaUrlOptions parameter. If it is true, make your code fall back to the base, such as:

if (mediaOptions.Thumbnail)
return base.GetMediaUrl(mediaItem, mediaOptions);

During an upgrade from Sitecore 6.5 to Sitecore 8.1 one issue were noticed at the Media Library: when a Media Item is uploaded, the icon appears broken.

Broken Icon

A custom MediaProvider is used at this build to modify the way Sitecore generates Media Item urls. All it does is to add the language at beginning of the media URL so we can have language-specific media items. This is important to allow the same Media Item (etc: an image or PDF) to be served in multiple languages.

Root cause

When a media item is uploaded to the Media Library, Sitecore will fill the “Icon” field with the URL of the media item itself. Because of the custom MediaProvider, language is being added to the beginning of the generated URL. Apparently that was not a problem in Sitecore 6.5, but things seems to have changed at Sitecore 8.

Technical explanation by Sitecore

A Support Ticket #476256 were created to allow further investigation from Sitecore on this issue. According to the technical team:

“The behavior you reported is expected as tree node icons are rendered by Sitecore.Shell.Applications.ContentManager.Sidebars.Tree.RenderIcon(Item item) method which does not use language definition, embedded to URL, to resolve item language. So when the Icon field contains URL with language definition embedded, it breaks the method functionality and leads to incorrect resolution of the item’s icon.

When you have versioned media item, the icon is rendered for specific language automatically even when language is not specified deliberately. For instance, if you have media item with two language versions and different images for each language, and use -/media/ItemGUID.ashx?h=16&thn=1&w=16 string in the Icon field, language specific icon will be rendered when you switch the language in the Content Editor.”

This explains the causes but doesn’t fix the issue: how can we skip or bypass the custom MediaProvider, avoiding the language to be added to the URL when the “Icon” field is filled?

Solution

In order to fix that we will need to extend and override the Sitecore.Resources.Media.Media class, responsible for the upload and update of metadata when a Media Item is uploaded to the Media Library, then modify all Media Types to use your class instead of the original.

Step 1 – Extend the Media class

This is going to be responsible for removing the language from the URL when a Media Item is uploaded. In bold you see the code responsible for that, rest is basically native code to ensure compliance with default Sitecore behavior.

using Sitecore;
using Sitecore.Configuration;
using Sitecore.Data.Items;
using Sitecore.Data.Proxies;
using Sitecore.Diagnostics;
using Sitecore.Resources.Media;
using Sitecore.SecurityModel;

namespace CustomMediaProvider
{
  public class Media : Sitecore.Resources.Media.Media
  {
    public override Sitecore.Resources.Media.Media Clone()
    {
      return new Media();
    }

    public override void UpdateMetaData(MediaStream mediaStream)
    {
      Assert.ArgumentNotNull(mediaStream, "mediaStream");
      var innerItem = MediaData.MediaItem.InnerItem;
      if (!innerItem.Paths.IsMediaItem) return;
      using (new EditContext(innerItem, SecurityCheck.Disable))
      {
        innerItem["extension"] = mediaStream.Extension;
        innerItem["mime type"] = mediaStream.MimeType;
        innerItem["size"] = mediaStream.Length.ToString();
        if (Settings.Media.AutoSetAlt)
        {
          innerItem["Alt"] = innerItem.Name;
        }
        var shellOptions = MediaUrlOptions.GetShellOptions();
        shellOptions.Thumbnail = true;
        shellOptions.Height = 0x10;
        shellOptions.Width = 0x10;
        var realItem = ProxyManager.GetRealItem(innerItem, true);
        // CODE ADDED TO REMOVE LANGUAGE FROM URL WHILE UPLOADING
        var iconUrl = MediaManager.GetMediaUrl(realItem, shellOptions);
        var prefix = "/" + Context.Language.Name + "/";
        if (iconUrl.StartsWith(prefix)) iconUrl = iconUrl.Replace(prefix, "");
        innerItem.Appearance.Icon = iconUrl;
      }
    }
  }
}

Step 2 – Register the Media class to all Media Types

This is the painful part: we will need to register our class to every Media Type we want to cover. There is no easy way for that other than adding a node for each Media Type, paying attention to the correct way for patching that specific node (some nodes may have a Media class already defined and thus needs correct patching, others don’t).

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
 <sitecore>
 <mediaLibrary>
 <!-- This is our Custom MediaProvider causing all the trouble -->
 <mediaProvider>
 <patch:attribute name="type">CustomMediaProvider.MediaProvider,CustomMediaProvider</patch:attribute>
 </mediaProvider>
 <!-- And here goes our custom Media class being attached to several Media Types -->
 <mediaTypes>
 <mediaType name="Any" extensions="*">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider" patch:instead="*[@type='Sitecore.Resources.Media.Media, Sitecore.Kernel']"/>
 </prototypes>
 </mediaType>
 <mediaType name="AVI video" extensions="avi">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="Windows Bitmap image" extensions="bmp, dib">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider" patch:instead="*[@type='Sitecore.Resources.Media.ImageMedia, Sitecore.Kernel']"/>
 </prototypes>
 </mediaType>
 <mediaType name="Flash" extensions="swf">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider" patch:instead="*[@type='Sitecore.Resources.Media.SwfMedia, Sitecore.Kernel']"/>
 </prototypes>
 </mediaType>
 <mediaType name="Flash video" extensions="flv">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="GIF image" extensions="gif">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider" patch:instead="*[@type='Sitecore.Resources.Media.ImageMedia, Sitecore.Kernel']"/>
 </prototypes>
 </mediaType>
 <mediaType name="HTML" extensions="htm,html,stm">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="JPEG image" extensions="jpg, jpeg, jpe, jfif">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider" patch:instead="*[@type='Sitecore.Resources.Media.ImageThumbnailGenerator, Sitecore.Kernel']"/>
 </prototypes>
 </mediaType>
 <mediaType name="MP3" extensions="mp3">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider" patch:instead="*[@type='Sitecore.Resources.Media.Mp3Media, Sitecore.Kernel']"/>
 </prototypes>
 </mediaType>
 <mediaType name="MP4 video" extensions="mp4">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="PDF file" extensions="pdf">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="PNG image" extensions="png">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider" patch:instead="*[@type='Sitecore.Resources.Media.ImageMedia, Sitecore.Kernel']"/>
 </prototypes>
 </mediaType>
 <mediaType name="SVG image" extensions="svg">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider" patch:instead="*[@type='Sitecore.Resources.Media.SvgMedia, Sitecore.Kernel']"/>
 </prototypes>
 </mediaType>
 <mediaType name="QuickTime movie" extensions="mov, qt">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="TIFF image" extensions="tiff, tif">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider" patch:instead="*[@type='Sitecore.Resources.Media.ImageMedia, Sitecore.Kernel']"/>
 </prototypes>
 </mediaType>
 <mediaType name="Zip file" extensions="zip">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider" patch:instead="*[@type='Sitecore.Resources.Media.ZipMedia, Sitecore.Kernel']"/>
 </prototypes>
 </mediaType>
 <mediaType name="Video for Adobe Flash Player" extensions="f4v">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="Windows Media video" extensions="WMV">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="Word document" extensions="doc">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="Word 2007 document" extensions="docx">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="PPTX File" extensions="pptx">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="PPT File" extensions="ppt">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="XLSX File" extensions="xlsx">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 <mediaType name="XLS File" extensions="xls">
 <prototypes>
 <media type="CustomMediaProvider.Media, CustomMediaProvider"/>
 </prototypes>
 </mediaType>
 </mediaTypes>
 </mediaLibrary>
 </sitecore>
</configuration>
Posted in Bug fixing, Development, Support Ticket, Upgrades

ERROR [Content Testing]: Cannot find PhantomJS executable at ' (...) /data/tools/phantomjs/phantomjs.exe'. Aborting screenshot generation.

In Sitecore 8 and above, while starting a Multivariate Tests, you may see broken thumbnails and popups with warnings such as below, and your Sitecore Log showing the error at this blog title.

Error Running Test - PhantomJS - 1
Error Running Test - PhantomJS - 2
According to what Sitecore Development Team announced at this community post for why Sitecore adopted PhantomJS:

Content Testing in Sitecore uses the Phantom JS tool for generation of the screenshot image files. In case you didn’t know, Sitecore has had screenshot generation features for quite some time. We use it to generate icons for items that you’ll be listing in the UI like renderings. These icon generation features are based on the System.Windows.Forms.WebBrowser control built into .net. So why did we not use the existing screenshot features for Content Testing? In early testing we found discrepancies with the WebBrowser control. WebBrowser uses Internet Explorer installed on the server where Sitecore is running. But the specific version that it uses isn’t always the latest. There are registry updates one can make to force the appropriate IE version, but this seemed like a big ask of users. This was one of the reasons we chose to use Phantom JS instead.

Cause and Solution

The error in question will show the full path to where Sitecore is looking for PhantomJS. This path is defined at Sitecore.ContentTesting.config, at the entry “ContentTesting.PhantomJS.ExecutablePath”. Start your investigation by checking where your setting is currently pointing to.

By default this setting points to “$(dataFolder)/tools/phantomjs/phantomjs.exe”. If that is your case, check if your Data folder has the PhantomJS executable under that path. If this file is lacking, take a copy from a fresh Sitecore installation (make sure to use the very same revision your site is using).

See also: PhantomJS and security hardening

Posted in Uncategorized Tagged with: ,

Workflow not starting (state field is empty) when Versions.AddVersion after Sitecore upgrade from 6.5 to 8.1

UPDATE: A ticket at Sitecore Helpdesk demonstrated the issue happened to be something else. During the upgrade, the site definition we’re using got the attribute “enableWorkflow” removed, which in turn made the default value “false” take over, and that was causing ALL WORKFLOWS to be simply ignored.

If your site does have enableWorlflow=”true” then you don’t need to manually start workflows as shown in this article

After an upgrade, a certain code was simply not working as expected anymore. When you create a new version of an item (item.Versions.AddVersion), whose _Standard Values had a Default Workflow setup, the workflow simply was not starting, and thus the “State” field gets empty. The very same code, at the non-upgraded version, works perfectly.

Looking at the code, it was never touching Workflows, suggesting that Sitecore would be automatically handling the thing. Since it was an upgrade, I lost myself investigating for things I could have left behind: Pipelines, Scheduled Tasks, etc.

And I then discovered it was Sitecore that changed the way how the API acts… a change that made me struggle in for almost a week!  And I just learned that when I decided to create a small script to run the same code on both 6.5 and 8.1 instances.

The code was getting an item by it’s ID, adding a version and printing the number of the newly created version:

Workflow not starting - 1

Sitecore 6.5

When I run that on 6.5, which has 5 versions before:

Workflow not starting - 2

The new version is created:

Workflow not starting - 3

And the State is correctly filled:

Workflow not starting - 4

Sitecore 8.1

However when I run it on 8.1, which previously had 39 versions:

Workflow not starting - 5

The 40th version is created, but the State field is empty!

Workflow not starting - 6

Solution

So what I had to do, to make the same code run correctly on 8.1, was to force the item thru the workflow after a new version is created. That screw hard to find, easy to twist:

Item newVersion = myItem.Versions.AddVersion();

var defaultWorkflow = newVersion.Fields[FieldIDs.DefaultWorkflow].Value;
ID defaultWorkflowId;
if (!ID.TryParse(defaultWorkflow, out defaultWorkflowId))
    return;

var workflow = newVersion.Database.WorkflowProvider.GetWorkflow(defaultWorkflowId.ToString());
if (workflow!=null)
    workflow.Start(newVersion);
Posted in Sitecore API, Upgrades, Workflow

Developers working in parallel with Continuous Integration builds for Dev and QA

When multiple developers are working in parallel (at the same build, but working in different parts) it is very common to use branches at source control, to keep each developer working in isolation and eliminating any chance that one developer’s work interferes with the work of another. Of course one day their work will have to be integrated, and that is the moment when things can go wrong. The following article proposes one of many possible setups to enable that isolation.

Creating a branch from another branch is very simple and easy. When working with a Continuous Integration system, however, your branch is supposed to be monitored, built and deployed to a place people can access and test it.

A branch and its servers with Continuous Integration

The following image illustrates a scenario where multiple developers are working at the same branch, and a second branch is used as a basis to deployments. The blue boxes at the right represents servers with specific purposes. Being now a Sitecore good practice to develop and test in scaled environment, all environments will have two nodes, one for Content Management (CM) and another for Content Delivery (CD).

01 - All in the same branch

The Continuous Integration server monitors the branch “All developers” and deploys changes to DEV (at every change) and QA (manually). When the time for a deployment comes, the work is merged to Deploy Branch. The Continuous Integration server will then make the last integration to the servers “Deploy-CM” and “Deploy-CD”, where things can be tested for the last time before the actual deployment.

The Problem with this approach is that all developers are pushing their modifications to the very same branch, and thus deployments must carry everything at that branch (except of course, if very granular merges are made, but that can also be tricky and error-prone). To enable a better safe separation, the following setup is proposed.

Multiple Branches

The image below now shows a second branch, with its respective DEV and QA servers. Now each developer can make their work at their own branch, keeping a physical separation that guarantees one developer’s work won’t interfere with the other.
02 - Two DEV branches

The image also shows that branch “Dev1” (the older branch) were used to create branch “Dev2”, which guarantees that both branches are identical at that point in time. Obviously, each branch will have their modifications from that time on, and the code integration is now a bit harder to be executed. As time passes, each branch will be more and more different, bringing the need to a more careful exercise during the integration, executed at the “Deploy” branch.

Deployments and Reverse Merges

When all developers were working at the same branch, the act of checking in their changes were the integration itself. Continuous Integration server would push it to the servers, and integrations were continuously made due to the same code base. But now with physically separated branches, developers must communicate and keep their branches as much in sync as possible. The following images shows a sequence of deployments and reverse merges, that would keep both Dev branches in sync while sustaining their separations.

Image 1 – First Deployment (by Dev1) and Reverse Merge (by Dev2)

03 - First Deploy

Image 2 – Second Deployment (by Dev1) and Reverse Merge (by Dev2)

04 - Second Deploy

Image 3 – First Deployment (by Dev2) and Reverse Merge (by Dev1)

05 - Third Deploy

 

Posted in Architecture, Continuous Integration, Dev, Environments, QA

Health Check builds with Continuous Integration - How big and how often these needs to be?

Background

While refactoring the TFS and Continuous Integration struture of a couple projects, one thing toke my attention: Health Check builds were stealing most of the building times, which for me was representing long build queues to wait. The reason they were crowding the queue is because they were taking a long time to complete (about 10 to 15 minutes), and building too often (every 5 minutes when changes are pending).

Strength or kindness?

Since our CI tool has only two build agents, the obvious answer is to increase this number to properly serve the whole company. But does this tells the whole story? More build agents means more resources (disk space, licenses, etc) and unfortunately we still live in a world of limited resources. This also means we rarely will have resources enough to properly serve everyone at peak times.

Take as comparison the limited space most cities have to make their roads. Most of the time traffic can be ok, but when everyone goes out at same time, like in rushy times, there are no room for all and we have car jams.

Fortunately it’s easier to interfere and positively affect our Continuous Integration systems, so why not try a thing or two as a sign of civility, just like we do in a Drive Thru not taking too much to order your fast food?

Health Check builds, how big?

Depending on the Solution Architect who setup the project you can have different things, but most cases what I see is Health Checks doing everything but deployments. Since we use TDS in our projects, you can make it build a package of the whole deployment, along with some meta data. But is it really necessary? It’s a question I made myself, when decided to make it different.

My Health Check ended up being minimalist: just a compilation is executed, while TDS is kept totally off. No packages, nothing. Ended up with builds taking 45 to 90 secs to finish, much better!

Health Check builds: how often?

Being tiny means it can build more often. In my case I have it building after changes are detected, with a 5 minutes latency to avoid subsequent check-ins to causing multiple builds in sequence.

Full builds and Health Checks working together

No doubt the resulting Health Check builds are weaker, as it’s doing just a tiny part of the integration process. To fill the gaps, I also setup full builds working in conjunction with them. These are also set to run automatically when changes are pending, but with a much longer latency. Having it building each 4 or 8 hours will ensure a full integration is made once or twice in a normal day work.

Let’s also keep in mind these deploys can and must be manually triggered by developers, no matter the latency, as soon as they finish the user story they are working on, so they can test what they did at the integration servers before considering their work done.

In short: deployments will be made when developers finishes their work, or at minimum once or twice a day if pending changes are to be deployed.

Full builds with Asynchronous HTTP calls

Another improvement I made was replacing at full builds some Synchronous HTTP calls by Asynchronous. They were mainly used to wake up instances after deployments and publishing from CM to CD. Most cases, The build agent doesn’t really need to wait for these calls to respond before going to the next step, so we can save it’s precious time to another teams.

Your impressions?

What about your experiences, do you agree and disagree? What other factors are left behind at this analysis? Let me know your thoughts!

Posted in Continuous Integration, Health Check builds, Team City

Automatic check-up of Sitecore Server Roles with Powershell

This week I had a boring demand of checking up a Sitecore installation against the Excel spreadsheet provided by Sitecore, to make sure all configs are enabled or disabled according to the Server Role.

You can get cockeyed by doing this manually, so of course I looked for a way to automatize it. Ended up discovering this simple powershell script written by Michael West. The only issue with this script is it does touches the files while checking, something I’d never dare doing in production.

I have modified his script so that, instead of touching the files, it takes notes of files that are different than specified by Sitecore’s spreadsheet so one can compare it manually afterwards. Resulting script can be seen at this link.

Multiple roles

But what if you wish to have the same Sitecore installation handling multiple roles? According to Martina Welander, the general recommendation for mixed roles is that, if something must be enabled anywhere, that config must be enabled. Well, while this is not a rule we can trust 100%, still the script is not changing anything and thus it won’t hurt if it does this check.

You can check for multiple roles by adding them separated by commas (Eg: CM, RPT)

Posted in Uncategorized Tagged with: , ,

Less than a week to the Sitecore Symposium 2016 in New Orleans!

Countdown to SymposiumI’m totally excited for the days to come – starts next week the Sitecore Symposium and MVP Summit 2016, in New Orleans.

Will be my first experience in an official Sitecore event. Great opportunity to in person meetings with representatives of Sitecore, another partners and services providers. Particularly exciting is the chance to get together and taste what another Sitecore MVPs are thinking and saying.

I’m reaching the ground in New Orleans along with my colleagues from Nonlinear Creations, will be a week full of social and technical experiences.

When I come from this experience, expect to find articles and another materials with the inspirations I’ll bring!

Posted in MVP Summit, Sitecore Symposium

Slowness with Custom Properties on .NET Security Provider

If you ever used the standard .NET Security Provider with Sitecore, you may love how easy it is to create and use Custom Profile Properties, where data can be easily saved at user profiles. But a huge issue can emerge if you attempt to retrieve users at your database by one of these custom properties.

The Problem

Let’s say you have a custom property called Document ID, and you wish for some reason to retrieve the user that has a certain number on it – for instance, if your users can login to your website both using their Logins or Document IDs – then you may have something like this at your code:

var userFound = UserManager.GetUsers().FirstOrDefault(f => f.Profile["Document ID"] == inputDocumentId);

This simple code can bring you a big headache, because of the way .NET Security Provider builds the SQL Query responsible for executing your LINQ expression. Since all Custom Properties are stored as meta data, it is simply not directly queriable. Then what Security Provider does is one SELECT query for each user at your database, deserializing the Custom Properties on memory so it can be compared to the user input.

If you have few users at your database, which is usually the case when you are in development phase, you’ll probably not notice any issue. But after your go-live, as your user base starts growing, it will gradually get slower and slower. At the project that inspired this blog post, we had a sudden data load of more than 20k users, and as a consequence the system got impracticably slow overnight.

The Solution

One of the possible technical solutions, the one we used at the project in question, was to extend the table aspnet_Users at our Core database. That is the main table used by .NET Security Provider to store users. What we do is to create a new column Document ID, where this data will be stored in a clean format:

aspnet_Users

After that, we need to attach some code to both “user:created” and “user:updated” pipelines. This code will be responsible for updating the Document ID column when users are created or updated.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
 <sitecore>
 <events>
 <event name="user:created">
 <handler type="MyProject.Pipelines.UpdateUserProfileCache, MyProject" method="OnUserCreatedUpdated" />
 </event>
 <event name="user:updated">
 <handler type="MyProject.Pipelines.UpdateUserProfileCache, MyProject" method="OnUserCreatedUpdated" />
 </event>
 </events>
 </sitecore>
</configuration>

Then your code will probably look like the following. The class CustomProfileIndexManager, responsible for doing the actual Update and Select at the database is not there, but you can easily guess how to build yours.

namespace MyProject.Pipelines 
{
 public class UpdateUserProfileCache
 {
   public void OnUserCreatedUpdated(object sender, EventArgs args)
   {
     var scArgs = (SitecoreEventArgs)args;
     if (!scArgs.Parameters.Any())
       return;

     var user = (System.Web.Security.MembershipUser)scArgs.Parameters.First();

     // Will only act for "extranet" users
     var username = user.UserName;
     if (!username.StartsWith("extranet"))
       return;

     var scUser = Sitecore.Security.Accounts.User.FromName(username,false);

     // Following method is responsible for saving "Document ID" 
     // into the respective column at the database
     CustomProfileIndexManager.SaveUserToCache(scUser);
   }
 }
}

So your problematic code would be replaced by something like this:

// Will do a simple "SELECT * FROM aspnet_Users WHERE Document ID = 'xxxx'" 
var userFound = CustomProfileIndexManager.GetUserByDocumentId(inputDocumentId);

 

What about the existent base of users?

Of course that will only cover users that are new or updated. If you already have a certain base of users, you can build a simple script to “touch” all your users, such as this:

 var startProcess = DateTime.Now;
 Response.Write("<font color=\"red\">Loading users...</font>");
 Response.Flush();

 // Get all users
 var users = DomainManager.GetDomain("extranet").GetUsers();
 Response.Write(string.Format("OK! ({0}ms)<hr>", DateTime.Now.Subtract(startProcess).TotalMilliseconds));
 Response.Flush();

 // Loop into all users
 var counter = 0;
 foreach (var user in users)
 {
   var startUserProcess = DateTime.Now;

   counter++;
   if ((counter % 10) == 0)
   {
     Response.Write(string.Format("--- Total time: {0} minutes<br>", DateTime.Now.Subtract(startProcess).TotalMinutes));
     Response.Flush();
   }
   Response.Write(string.Format("User #{0} - Email: {1} - Processing...", counter, user.Profile.Email));

   // Following method is responsible for saving "Document ID" 
   // into the respective column at the database
   CustomProfileIndexManager.SaveUserToCache(user);

   Response.Write(string.Format(" OK! ({0}ms)<br/>", DateTime.Now.Subtract(startUserProcess).TotalMilliseconds));
 }
 Response.Write(string.Format("<h3>TOTAL TIME: {0} minutes</h3>", DateTime.Now.Subtract(startProcess).TotalMinutes));
 Response.Flush();
 Response.End();

 

Posted in Development, Security Provider
Social Media Auto Publish Powered By : XYZScripts.com