#sqlr

This article was written by Matthew Fitzgerald, Software Engineer, and Mike Zupan, Systems Engineer at OpenSky

The fix will go live next week (Sept 29)

One of the most important ways Shoppers find Sellers is through Search. Recently, we’ve found a hole in our search index. The problem we had was, Sellers would select Products with keywords like “beauty” in the Product Name but that Seller would not show up in the Search results for “beauty”.

The reason for this was our Seller search index only had direct information on the Seller, ie: Seller Username, Business Name, Bio, etc… The Seller index had no knowledge of Products the Seller was promoting. So any keywords the Product might have, wouldn’t return the seller in a search.

The Solution

We needed to modify our Seller index to have knowledge of both Products and Sellers and return the results of distinct sellers. The problem, our current Search technology (SOLR) didn’t support that behavior of Grouping Product Records by Seller and collapsing them into distinct Seller results.

The Tech Part

After some hefty Googling, I found that the feature to Group Search results does exist in a future version of the technology yet to be released. This feature became so popular that the developers of our Search technology released a “patch” to include this feature in the current version of the software. This is how I applied the patch.

Step 1

If you are like me, you hate Subversion and you love GIT. I run Ubuntu and using apt-get I installed a git subversion bridge.

sudo apt-get install git-svn

Step 2

Get some reading material, then start the checkout the latest stable tag of Solr

git-svn clone http://svn.apache.org/repos/asf/lucene/solr/tags/release-1.4.1

Step 3

Download and apply the patch

cd release-1.4.1 wget https://issues.apache.org/jira/secure/attachment/12448216/SOLR-236-1_4_1.patch patch -p 0 -i SOLR-236-1_4_1.patch

Step 4

Recompile it, Baby!

ant compile

Now you want to build the example project so we can test it

ant example

Now it is time to test. So go into the example project and the example data directory

cd example/exampledocs

Now you can import all the xml data

java -jar post.jar *.xml

Now start the server

cd .. java -jar start.jar

Now you should be able to hit the following url and get back grouped data

http://localhost:8983/solr/select? wt=json&indent=true&fl=id,name &q=solr+memory &group=true&group.field=manu_exact &collapse.field=manu_exact

Are you into PHP? This is how to implement it with the native PECL Driver.

$query = new SolrQuery('beauty'); $query->addParam('group','true'); $query->addParam('group.field','sellerId'); $query->addParam('collapse.field','sellerId'); $query->addParam('rows', 1000);