“Best Bets” functionality for search systems
I was working on a Best Bets system this week, which is essentially what I did 8 years ago on my first BBC project . It is nice to working on something straightforward but I’ve had to do a lot of explaining of the concept. What follows is my advice if you are think about adding Best Bets to your search.
What are Best Bets?
Best Bets are essentially editorial picks that appear at the top of the search results. They are a manual intervention for use when the search engine isn’t developing the best results for the users. Some sites use them to fix just a couple of problematic queries but others have built up extensive databases of thousands of best bets.
You can see examples in Peter Morville’s Best Bets collection on Flickr:
http://www.flickr.com/photos/morville/collections/72157603790587909/
Some search systems have Best Bets functionality as standard (surprisingly SharePoint is one of these) or you can have something bespoke added. The first system I ever worked with was just a basic text file that I edited and uploaded to server – you should be able to get something better than that!
A Bad Idea?
Kas Thomas thinks that we shouldn’t do best bets:
“In point of fact, the search software should do all this for you. After all, that’s its job: to return relevant results (automatically) in response to queries. Why would you sink tens (or hundreds) of thousands of dollars into an enterprise search system only to override it with a manually assembled collection of point-hacks? Sure, search is a hard problem. But if your search system is so poor at delivering relevant results that it can’t figure out what your users need without someone in IT explicitly telling it the answer, maybe you should search for a new search vendor.”
http://www.cmswatch.com/Trends/1286-Best-bets:-a-worst-practice
This is the sort of language I expect from the vendors but it is a bit surprising from industry analysts. Yes, the search systems should be good enough. But they’re not. They’re certainly not good enough without a lot of work. A lot of expensive work. If your supplier says “the search is really good, you don’t need to worry about it” then you definitely need to worry about it.
As James Robertson says “No amount of tweaking of metadata or search configuration will… ensure that the most relevant results always appear at the beginning of the list.”
http://www.steptwo.com.au/papers/cmb_bestbets
Oh and IT shouldn’t be managing the Best Bets anyway. The teams I’ve worked with it has always been an editorial or product management role. After all why would you build a simple tool to allow editorial intervention and then ask IT to put the content in?
A simple best bets solution, that can be maintained by editorial/product teams rather than scarce technical experts (or worse expensive consultants) is often a better business solution than battling with the search algorithm to try and get it right for all the scenarios. Particularly on a tight budget.
Other pros for Best Bets:
- Just fixes that problem. It doesn’t change any other results. There’s no mysterious black box that has you banging your head against the desk about why when you changed Property X to fix the results for Query Y the results for Query Z changed like that.
- Fixes the problem straight away. You don’t have to wait for the next crawl or even for an emergency crawl to finish. Sometimes it really is that important. Other times someone else thinks it really is that important and you want them to leave you alone now.
- Buys you time whilst you improve the algorithm.
Managing Best Bets
The critics are however correct that Best Bets have some drawbacks. You have to create and maintain them. If you let the links break then you’ve created a worse user experience than the one you set out to fix.
- Don’t go overboard. Only create them where there are clear problems
- Plan for maintenance time. Who is going to add Best Bets and when? Do they have time to check existing Best Bets?
- Make sure you have access to search logs so you can see what terms users might be having difficulties with
- If possible, set up a broken and redirected link checker to run over the Best Bets
And yes, do look at what your Best Bets tell you about the weakness of your search system. If you have the permissions and the skills you may be able to put that knowledge to use in improving the algorithm. But even if you can’t make the changes yourself and there’s no budget for incremental changes (which there often isn’t) then you can at least start building a business case for a search improvement project.
Designing the display
It is tempting to strongly highlight the Best Bets to draw attention to them but this is one area where usability testing tells us a different story.
Users demonstrate a very strong preference for the first ‘ordinary’ looking search result, which is presumably a behaviour they have learnt from web search engines. With search engines any result that is styled slightly differently is probably an ad. Some users didn’t even notice the existence of best bets when we had tried to draw attention to them. This may be a similar situation to banner blindness.
So don’t make a song and dance about it. We might feel the need to tell the user all the effort we’ve put into helping them but ultimately they just want the right result for their query. And they don’t care how it gets to the top of the results, so long as it is at the top of the results.
(Think about it. You’d never highlight a set of the results with a label saying “Brought to you by the IA tweaking the algorithm to weight page title more heavily”)
3 steps to happy Best Bets
In summary:
- If the system you are buying doesn’t come with a built in Best Bets system, see if you can get a simple one added on.Think of it as safety net for once all the developers and project managers have packed up and left you to your own devices.
- Put them at the top of the search results. If you feel the need to style them differently then keep the styling as minimal as possible
- Don’t get carried away and make sure you maintain those links!
Related posts: