Lucene Search and Bidirectional Relationships
Introduction
The term bidirectional refers to navigating the one to many relationships that are common in relational databases. Sitecore often has difficulty finding content items in both directions in a bidirectional relationship, particularly when it has to find a large number of items.
In Sitecore there are currently three possible ways to solve the problem of bidirectional relationships:
- Links Database
- Sitecore queries
- Lucene Search
Lucene Search is by far the most efficient way to handle bidirectional relationships in Sitecore. It can quickly find items in relationships where there are a large number of items without having a significant effect on performance. Ivan Sharamok wrote an interesting blog on the most efficient ways to extract data from Sitecore: Working with Lucene Search Index in Sitecore 6.
Sitecore implements a wrapper for Lucene called Sitecore.Search which encapsulates the search functionality in Lucene.
Some of the benefits of using the Sitecore.Search API:
- Closely integrated with Sitecore
- Safe and more secure to use
- Enforces good programming practices
- Contains classes specifically related to search
Blog Objectives
- To provide Sitecore customers and partners with a starting point when attempting to tackle the problem of bidirectional relationships
- To explain the functionality of the main classes used in the bidirectional shared source module
- To introduce the Sitecore.Search API and to give practical examples of how it can be used to solve a common problem
Prerequisites
- Sitecore CMS 6.0 or later
- Sitecore Starter Kit – Training Sample Site
- Microsoft Visual Studio 2010
Solution Components
The bidirectional example described in this blog is also available as a shared source module. The module consists of a Sitecore update package and a Visual Studio solution.
Click here to download the module: Bidirectional Shared Source Module
The Visual Studio Solution includes:
The update package includes:
- RelatedItems.cs – Web control that adds related items to selected product pages.
- SearchResults.cs – This includes an update to the existing Search Results Web control to include related items and refinements in the search results.
- Facet.cs – The facet Web control that adds refinements to the search results.
- Facet items for Brand Name and Category added to the Meta-Data folder in the content tree:
- The package adds a new field called Brand to the Product Category template:
This blog does not cover how to create all the component parts of this solution. I aim to focus on the more complex components such as the Searcher and the Query Runner (Searcher.cs and Query Runner.cs) and will break the code in these classes down into smaller chunks. It is in these classes that you can see the Sitecore.Search API being used to solve the problem of bidirectional relationships.
The example in this blog is just one solution and there are many other ways to solve the same problem. For example, in your own solution you may want to create your own Database Crawler or create your own config include file settings. The settings contained in these files can vary greatly from one solution to another and it is possible to use the Sitecore.Search API to create your own crawler and config include files.
Steps
To create this solution, first download the shared source module and then follow these steps:
1. Implement a Searcher Class
2. Implement a Query Runner Class
3. Create a Rendering Control
4. Create a Facet Control
Next step: Implement a Searcher Class
Hi there,
I was wondering how you would index or query one to many relations when using fields like multilist.
For me this indeed works with a droplink but not with real one to many fields.
What is your opinion about this?
Koen
Hi Koen,
After consulting with some of our developers, I have the following suggestions for you:
1st approach
Override the Sitecore.Search.Crawlers.DatabaseCrawler.AddAllFields method to add a new field to the index with the value of the multilist field you want to search.
To make the value searchable it has to be processed as follows:
var indexedValue = string.Join(“ “, value.Split(‘|’).Select(id => ShortID.Encode(id)).ToArray())
document.Add(this.CreateTextField(“relatedTo”, indexedValue)); // index field name is “relatedTo”, we will use it later.
This will shorten the GUID values convert pipes to spaces, for example:
“{12345678-1234-1234-1234-1234567890AB}|{11111111-1111-1111-1111-111111111111}” will be converted to “123456781234123412341234567890AB 11111111111111111111111111111111“. This format is searchable by Lucene.
When implementing a matching function, you will have to search for ShortID(item.ID) in the field you have added using the following query:
searchContext.Search(“relatedTo:” + ShortID.Encode(item.ID)) // we search in the “relatedTo” field added earlier.
2nd approach
You could use the Advanced Database Crawler.
This has added functionality to support custom field crawlers for the field types you mentioned.
All you will have to do is to write a method that would parse the field value into the desired format (similar to the 1st approach described above).
I hope this helps you to solve the problem.
Chris
I’m just starting out with the Lucene index information in Sitecore 6 (using 6.2.0, planning to upgrade to 6.4.1 soon). However, I can’t follow your examples very far because the Visual Studio solution from trac.sitecore.net doesn’t include the “Sitecore.Training.Index” project in the solution. It wasn’t included in my original Sitecore Foundations training packet, so is it somewhere else to download? I’m basically stuck at this point, since I can’t compile anything in VS. Even the “starterkit” project is giving me build errors. Any extra help to get this going would be very, very appreciated!
Thanks!
Hi Brian,
I think you may have only downloaded the package from TRAC.
To download the “Sitecore.Training.Index” project you first need a subversion client such as Tortoise:
http://tortoisesvn.net/downloads.html
To download the “Sitecore.Training.Index” project:
1. Copy the URL from the TRAC page:
http://svn.sitecore.net/BidirectionalLuceneSearch/
2. Using Tortoise, create a new folder on your Desktop.
3. Right click the folder and select SVN Checkout.
4. In the URL of repository field paste the URL and add Trunk to the end of the path:
http://svn.sitecore.net/BidirectionalLuceneSearch/Trunk
5. In the Check Out dialog box, click OK.
The project files should now appear in your folder.
I hope this helps.