ia play

This rings true for people who are clearly in one camp or another, and probably explains my diary nightmare:

“There are two types of schedule, which I’ll call the manager’s schedule and the maker’s schedule. The manager’s schedule is for bosses. It’s embodied in the traditional appointment book, with each day cut into one hour intervals. You can block off several hours for a single task if you need to, but by default you change what you’re doing every hour.

When you use time that way, it’s merely a practical problem to meet with someone. Find an open slot in your schedule, book them, and you’re done.

Most powerful people are on the manager’s schedule. It’s the schedule of command. But there’s another way of using time that’s common among people who make things, like programmers and writers. They generally prefer to use time in units of half a day at least. You can’t write or program well in units of an hour. That’s barely enough time to get started.

When you’re operating on the maker’s schedule, meetings are a disaster. A single meeting can blow a whole afternoon, by breaking it into two pieces each too small to do anything hard in.”

via Maker’s Schedule, Manager’s Schedule.

IAs are a bit of both, so I have a diary cluttered with lots of ‘short’ decision making meetings but I also need to carve out half day (at least) chunks so I can actually design or write stuff. The real GTD challenge for me is keeping my design work going in weeks where I only have scattered time here and there.

I always like examples of recommendation engines and the like that have got a bit muddled. The WalMart Apes scandal remains the classic. In this case the book is Apocalypses: Prophecies, Cults and Millennial Beliefs Throughout the Ages and the sponsored link reads “Cheap Weber BBQs”.

It would be nice to think that the suggestion that customer interested in a book on apocalypses might also like a BBQ had some sort of ‘burn in hell’ connnection but it appears to just be that the author is called “Weber” which is a BBQ brand.

Which started me thinking about how to improve the recommendation engine with a bit of semantic insight about which fields to match upon. You could just not match on the author field but presumably some of the sponsored links are actually related to the author (I’m thinking the Gillian McKeiths and Deepak Chopras of the world). So you’d need some semantic information about the content of the sponsored link as well. Which could be a bit more challenging…

NTEN continue to share lots of useful information about their redesign process, including this insight into their web analytics:

“Our bounce rate is pretty darn high for folks who find our site through search: about 68%. New visitors also bounce at a high rate: about 67%. Our blog, which gives us the most traffic from search, has a bounce rate above 75%.

Friend of NTEN Avinash Kaushik says that organizations should aim for a bounce rate under 50%. We don’t expect our new visitor bounce rate will get THAT low, but there’s some work we can do to make sure people find MORE great content and stick around our site.”

via Wireframe Testing: Failing Informatively | NTEN: The Nonprofit Technology Network.

I’m probably going to get a new job title. And it won’t be UX-anything, so don’t worry that I’ve had a change of heart on that.

I don’t use my IA title much within the organisation. The web team get it but that’s four people.  I tend to introduce myself by what projects I’m working on. In project kick off meetings and meetings with stakeholders I’ll explain what I’ll be doing on the project but not my title.

A lot of the teams I work with are intimidated by IT projects. And for them the language of user experience design and information architecture is as alienating and terrifying as the language of server architecture and database design. It is all big words from people who get paid more than they do and seem to work in an alternate universe of conferences, social networks and blogging.

So mostly my introductions go something like…”I’m Karen, I’m part of the project team and I’ll be responsible for making sure users can find their way around the new site”. Or “the search actually works this time”. Or “putting your content into the system isn’t such a nightmare”.

So my boss and I are trying to come up with something that both more accurately conveys what I actually do and is also a user friendly one.

Anyone got any examples of doing user research into what their job title should be?

I was working on a Best Bets system this week, which is essentially what I did 8 years ago on my first BBC project .  It is nice to working on something straightforward but I’ve had to do a lot of explaining of the concept.  What follows is  my advice if you are think about adding Best Bets to your search.

What are Best Bets?

Best Bets are essentially editorial picks that appear at the top of the search results. They are a manual intervention for use when the search engine isn’t developing the best results for the users. Some sites use them to fix just a couple of problematic queries but others have built up extensive databases of thousands of best bets.

You can see examples in Peter Morville’s Best Bets collection on Flickr:
http://www.flickr.com/photos/morville/collections/72157603790587909/

Some search systems have Best Bets functionality as standard (surprisingly SharePoint is one of these) or you can have something bespoke added. The first system I ever worked with was just a basic text file that I edited and uploaded to server – you should be able to get something better than that!

A Bad Idea?

Kas Thomas thinks that we shouldn’t do best bets:

“In point of fact, the search software should do all this for you. After all, that’s its job: to return relevant results (automatically) in response to queries. Why would you sink tens (or hundreds) of thousands of dollars into an enterprise search system only to override it with a manually assembled collection of point-hacks? Sure, search is a hard problem. But if your search system is so poor at delivering relevant results that it can’t figure out what your users need without someone in IT explicitly telling it the answer, maybe you should search for a new search vendor.”

http://www.cmswatch.com/Trends/1286-Best-bets:-a-worst-practice

This is the sort of language I expect from the vendors but it is a bit surprising from industry analysts. Yes, the search systems should be good enough. But they’re not. They’re certainly not good enough without a lot of work. A lot of expensive work. If your supplier says “the search is really good, you don’t need to worry about it” then you definitely need to worry about it.

As James Robertson says “No amount of tweaking of metadata or search configuration will… ensure that the most relevant results always appear at the beginning of the list.”
http://www.steptwo.com.au/papers/cmb_bestbets

Oh and IT shouldn’t be managing the Best Bets anyway. The teams I’ve worked with it has always been an editorial or product management role. After all why would you build a simple tool to allow editorial intervention and then ask IT to put the content in?

A simple best bets solution, that can be maintained by editorial/product teams rather than  scarce technical experts (or worse expensive consultants) is often a better business solution than battling with the search algorithm to try and get it right for all the scenarios. Particularly on a tight budget.

Other pros for Best Bets:

• Just fixes that problem. It doesn’t change any other results. There’s no mysterious black box that has you banging your head against the desk about why when you changed Property X to fix the results for Query Y the results for Query Z changed like that.
• Fixes the problem straight away. You don’t have to wait for the next crawl or even for an emergency crawl to finish. Sometimes it really is that important. Other times someone else thinks it really is that important and you want them to leave you alone now.
• Buys you time whilst you improve the algorithm.

Managing Best Bets

The critics are however correct that Best Bets have some drawbacks. You have to create and maintain them. If you let the links break then you’ve created a worse user experience than the one you set out to fix.

• Don’t go overboard. Only create them where there are clear problems
• Plan for maintenance time. Who is going to add Best Bets and when? Do they have time to check existing Best Bets?
• Make sure you have access to search logs so you can see what terms users might be having difficulties with
• If possible, set up a broken and redirected link checker to run over the Best Bets

And yes, do look at what your Best Bets tell you about the weakness of your search system. If you have the permissions and the skills you may be able to put that knowledge to use in improving the algorithm. But even if you can’t make the changes yourself and there’s no budget for incremental changes (which there often isn’t) then you can at least start building a business case for a search improvement project.

Designing the display

It is tempting to strongly highlight the Best Bets to draw attention to them but this is one area where usability testing tells us a different story.

Users demonstrate a very strong preference for the first ‘ordinary’ looking search result, which is presumably a behaviour they have learnt from web search engines. With search engines any result that is styled slightly differently is probably an ad. Some users didn’t even notice the existence of best bets when we had tried to draw attention to them. This may be a similar situation to banner blindness.

So don’t make a song and dance about it. We might feel the need to tell the user all the effort we’ve put into helping them but ultimately they just want the right result for their query. And they don’t care how it gets to the top of the results, so long as it is at the top of the results.

(Think about it. You’d never highlight a set of the results with a label saying “Brought to you by the IA tweaking the algorithm to weight page title more heavily”)

3 steps to happy Best Bets

In summary:

1. If the system you are buying doesn’t come with a built in Best Bets system, see if you can get a simple one added on.Think of it as safety net for once all the developers and project managers have packed up and left you to your own devices.
2. Put them at the top of the search results. If you feel the need to style them differently then keep the styling as minimal as possible
3. Don’t get carried away and make sure you maintain those links!

Related posts:

• Testing site search
• Search log analysis
• SharePoint search: good or bad?

“Admit it. Ours can be an insular profession. As much as most of us think we communicate simply and effectively, we often don’t. Why? I think it’s because we’re sometimes overly concerned about how we’re coming across to our fellow UXers. You know what? Forget about them. Your real audience is the business stakeholder. When you’re planning a presentation or trying to figure out how to communicate your research or design solution, don’t let your inner Nielsen—or head-Nielsen for fans of the reimagined Battlestar Galactica TV series—prevent you from communicating in terms and concepts that your stakeholders can understand and groove on.

You know what this means, don’t you? You’re not allowed to use the term heuristic evaluation anymore. Banish it from your professional vocabulary! Now, wave goodbye to it, because, if you use it again, I will personally come to your house and punch you in the arm.”

via 8 Things You Should Be Doing in Your UX Practice, but Probably Aren’t :: UXmatters.

Absolutely. Couldn’t agree more. But it is ok to tell them you are user experience designer?

Interesting stuff on the impact of CAPTCHAs:

“From the data you can see that with CAPTCHA on, there was an 88% reduction in SPAM but there were 159 failed conversions. Those failed conversions could be SPAM, but they could also be people who couldn’t figure out the CAPTCHA and finally just gave up. With CAPTCHA’s on, SPAM and failed conversions accounted for 7.3% of all the conversions for the 3 month period. With CAPTCHA’s off, SPAM conversions accounted for 4.1% of all the conversions for the 3 month period. That possibly means when CAPTCHA’s are on, the company could lose out on 3.2% of all their conversions!

Given the fact that many clients count on conversions to make money, not receiving 3.2% of those conversions could put a dent in sales. Personally, I would rather sort through a few SPAM conversions instead of losing out on possible income.”

via SEOmoz | CAPTCHAs’ Effect on Conversion Rates.

Inside the Index and Search Engines is 624 pages of lovely SharePoint search info. It is the sort of book that sets me apart from my colleagues. I was delighted when it arrived, everyone else was sympathetic.

The audience is “administrators” and “developers”. I’m never sure how technical they are imagining when they say “administrators” so I waded in anyway. The book defines topics for administrators as; managing the index file; configuring the end-user experience; managing metadata; search usage reports; configuring BDC applications; monitoring performance; administering protocol handlers and iFilters. I skimmed through the content for developers and found some useful nuggets in there too.

Contents:
1. Introducing Enterprise Search in SharePoint 2007
2. The End-User Search Experience
3. Customizing the Search User Interface
4. Search Usage Reports
5. Search Administration
6. Indexing and Searching Business Data
7. Search Deployment Considerations
8. Search APIs
9. Advanced Search Engine Topics
10. Searching with Windows SharePoint Services 3.0

The book begins by setting the scene, and with lots of fluff about why search matters and some slightly awkward praise for Microsoft’s efforts. It gets much more interesting later, so you can probably skip most of the introduction.

Content I found useful:

Chapter 1. Introducing Enterprise Search in SharePoint 2007

p.28-33 includes a comparison of features for a quick overview of Search Server, Search Server Express and SharePoint Server.

“Queries that are submitted first go through layers of word breakers and stemmers before they are executed against the content index file is available. Word breaking is a technique for isolating the important words out of the content, and stemmers store the variations on a word” p.32

Keyword query syntax p.44

• maximum query length 1024 characters
• by default is not case sensitive
• defaults to AND queries
• phrase searches can be run with quote marks
• wildcard searching is not supported at the level of keyword syntax search queries. Developers could build this functionality using CONTAINS in the SQL query syntax
• exclude words with
• you can search for properties  e.g rnib author:loasby
• property searches can include prefix searches e.g author:loas
• properties are ANDed unless it the same property repeated (which would run as OR search)

Search URL parameters p.50

• k = keyword query
• s = the scope
• v = sort e.g “&v=date”

Chapter 4: The Search Usage Reports

Search queries report contains:

• number of queries
• query origin site collections
• number of queries per scope
• query terms

Search results report contains:

• search result destination pages (which URL was clicked by users)
• queries with zero results
• most clicked best bets
• search results with zero best bets
• queries with low clickthrough

Data can be exported to Excel (useful if I need to share the data in an accessible format).

You cannot view data beyond the 30 day data window. The suggested solution is to export every report!

Chapter 5: Search Administration

Can manage the crawl by:

• create content sources
• define crawl rules : exclude content (can use wildcard patterns), follow/noindex, crawl URLs with query strings
• define crawl schedules
• removed unwanted items with immediate effect
• troubleshoot crawls

There’s a useful but off-topic box about file shares vs. sharepoint on p.225

Crawler can discover metadata from:

• file properties e.g name, extension, date and size
• additional microsoft office properties
• SharePoint list columns
• Meta Tags from in HTML
• Email subject and to fields
• User profile properties

You can view the list of crawled properties via the Metadata Property Mappings link in the Configure Search Settings page. The Included In Index indicates if the property is searchable.

Managed properties can be:

• exposed in advanced search and in query syntax
• displayed in search results
• used in search scope rules
• used in custom relevancy ranking

Adjusting the weight of properties in ranking is not an admin interface task and can only be done via the programming interface.

High Confidence Results: A different (more detailed?) result for results that the search engine believes are an exact match for the query.

Authoritative Pages

• site central to high priority business process should be authoritative
• sites that encourage collaboration and actions should be authoritative
• external sites should not be authoritative

Thesaurus p.291

• an XML file on the server with no admin interface
• no need to include stemming variations
• different lanuage thesauri exist. The one used depends on the language specified by client apps sending requests
• tseng.xml and tsenu.xml

Noise words p.294

• language specific plain text files, in the same directory as the thesaurus
  • for US english the file name is noiseenu.txt

Diacritic-sensitive search

• off by default

Chapter 8 – Search APIs

Mostly too technical but buried in the middle of chapter 8 are the ranking parameters:

• saturation constant for term frequency
• saturation constand for click distance
• weight of click distance for calculating relevance
• saturation constant for URL depth
• weight of URL depth for calculating relevance
• weight for ranking applied to non-default language
• weight of HTML, XML and TXT content type
• weight of document content types (Word, PP, Excel and Outlook)
• weight of list items content types

They’ll come in handy when I’m baffling over some random ranking decisions that SP has made.

Chapter 9 – Advanced Search Engine Topics
Skipped through most of this but it does covers the Codeplex Faceted Search on p.574-585

Overall

A good percentage of the book was valuable to a non-developer, particularly one who is happy to skip over chunks of code. I’ve seen and heard a lot of waffle about what SharePoint search does and doesn’t do, so it was great to get some solid answers.
Inside the Index and Search Engines: Microsoft® Office SharePoint® Server 2007

Related posts
SharePoint search: some ranking factors
SharePoint search: good or bad?

Where search analytics is concerned it appears the RNIB is actually doing what everyone else is doing i.e. using Google Analytics:

“The use of Google Analytics is very much on the increase. Just under a quarter of responding organisations (23%) now use Google Analytics exclusively compared to only 14% a year ago.
A further 57% of respondents are using Google Analytics in conjunction with another tool (up from 52% in 2008), which means that 80% of companies are now using Google for analytics compared to 66% last year…

The majority of responding companies believe that they have set up Google Analytics properly.
There is more doubt among those who do not use Google exclusively, with 23% of these
respondents saying they don’t know if it has been properly configured”

And I’m firmly in the later 46% camp these days:

“since 2008 there has been an increase from 8% to 15% of companies who have two dedicated web analysts and a decrease in the proportion of companies who have one analyst (from 32% to 26%).
But while this is a positive development, it can also be seen that exactly the same proportion of companies (46%) report that they do not have any web analysts.”

Via Online Measurement and Strategy report

The Nonprofit Technology Network have been sharing lots of info about their ongoing site redesign:

We’re going to make sure our site architecture is sound before we worry about making it purty.

The story so far:

* We started with a card sort. Rebecca Sherrill, our Information Architect at Beaconfire, has written a terrific synopsis of that process, with definitions, a walk-through of the process, and an overview of the findings. You should read it.

* Building on the results of the card sort and an Audience Matrix (Excel) we had filled out earlier, Beaconfire produced a draft site map. Holly and I worked with them in a conference call to revise the map (PDF), then brought the entire staff into the process during our weekly staff call.

Beaconfire now has our feedback, which they’ll use to refine the site map, then produce a wireframe version of the site.

via Redesigning NTEN.org: of Card Sorts and Site Maps | NTEN: The Nonprofit Technology Network.