As i was working on tuning the results we get from the MindServer, i became a little worried about how misleading the statement that there were only 99 results on the search for Shakespeare would be.
We were able to fix part of the problem by tuning the relevancy calculation -- we choose to assume that if you're searching you're more likely searching for "aboutness" than anything else. So we've ranked subjects and titles as more important than authorship. (We already knew that this system "looses" at known item searching -- we choose to optimize for our strength.)
The problem remained that we were never going to return more than 500 results -- and that often the 500 results would compress down from 500 editions to fewer, as often more than one edition in a work will be returned as a result. A search on widgets might produce 294 resulting works, a search on whatsits might produce 459 resulting -- but there may be many many more things on widgets in the Union Catalog, including your uncle's dissertation on Widgets -- but The Classic Text on Widgets, which went through twenty editions and is held by every major institution, overwhelmed the 500 editions results.
So, the user who goes through all 294 results on widgets to see how his uncle's dissertation was ranked is terribly disappointed. It's not mentioned at all. And then the user searches on widgets and his uncle's name, and there's the dissertation....
I suggested we change the word "total" to something else -- "recommended" ("recomminded") or "relevant," perhaps. The discussion went on and one of the Information Architects suggested we remove the number altogether. Good enough, i figured, and we had a reviewer lined up who would tell us if this was going to be the usability error others immediately insisted it would be.
Well, who needs a reviewer. At every institution Merrilee visited, the "where's the number of results?" question boomed. It's back, it's back -- without the word that bothered me: "total."
Now remains the question: how do people use this number? It behaves unlike other numbers, mainly because if your ssearch term is at all common, we're retreiving the top 500 editions, identifying them by works (thus reducing the 500 by however many editions of the same wor were returned), and displaying that number.
442 tea
451 tea coffee
449 +tea +coffee
457 tea or coffee
444 tea not coffee
444 tea not (coffee or cafe)
440 tea not (coffee cafe)
I doubt that we'll get questions about this wierd behavior -- which is why i bring it up here.
Then again, i have been impressed by the intelligently critical feedback from users so far. I might just be surprised. (Meanwhile, i'm trying to remember what i bet my boss with respect to feed back from "users." )
Posted by judielaine at October 3, 2003 06:35 PM | TrackBack