How to crawl items in non-SXA locations

We have created import job to import product data into Sitecore. As initial location we have chosen current SXA Site. With this setup all search components were giving us correct search results.

Problems started when we moved this data / items outside of Site. Suddenly no results started to appear. We have investigated in Solr Admin and indeed we couldn’t find any expected items in SXA indexes. Same query was giving us results for regular Sitecore master index but not in SXA master index.

I have asked about this problem on Sitecore Slack in #sxa channel and Vincenzo Luigi correctly pointed me to Sitecore Support.

We have received solution within minutes from them 🙂

Solution

We had to expand our SXA indexes (master and web) with one more crawler (default Sitecore crawler) and aim it to the necessary location.

We have used similar patch for both indexes:

<index id="sitecore_sxa_web_index" type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex, Sitecore.ContentSearch.SolrProvider" patch:source="Sitecore.XA.Foundation.Search.Solr.config">
...
<locations hint="list:AddCrawler">
<crawler type="Sitecore.XA.Foundation.Search.Crawlers.SxaItemCrawler, Sitecore.XA.Foundation.Search">
<Database>web</Database>
<Root>/sitecore</Root>
</crawler>
<crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
<Database>web</Database>
<Root>/sitecore/content/Settings</Root>
</crawler>
</locations>
...
</index>

Mind the type of crawler which is different to default sxa item crawler. Also mind the Root parameter that needs to be customized to your Item root which you would like to index.

Thanks Sitecore Support for the help! This blog post is to limit similar questions heading towards you so you can focus on other things 😉

Update for SXA 9.3:

Mohamed Cherif gave me information that in SXA 9.3 there is a new interface template _SearchableWithoutRelatedItems. If your custom template inferits from it, items based on your template will be crawled even if they are in an excluded location. The code doesn’t look for it to be an SXA page anymore if it inherits from that interface. So you don’t need to add crawler anymore 🙂

SXA all the things!