We’ve just pushed live a new update to the Online Check add-on module.

When you next run DupeFree Pro you’ll be prompted to install the update (if you have a license to the Online Check module), or you can simply manually run the DFP Updater from the Orb menu item “Update DFP” or via the Windows start menu from the  DupeFree Pro v2 “all programs” folder.

This update includes a fix for Yahoo! search results counts, which were not returning results as of yesterday due to a change Yahoo! made to thier search results pages (SERPs). After installing this update DFP will begin showing correct results counts for Yahoo! again.

Occasionally the search engines will change the way their SERPs are delivered which may prevent DFP from being able to grab the search results counts numbers, however, we constantly monitor this and will always make changes as quickly as possible to rectify the issue in the DFP Online Check module.

The second change in this update is that AOL search results counts have been replaced with the search results counts from Ask/Teoma. The reason for this is as follows…

Firstly, AOL started to become unrelaible with “quotes match” searches. We noticed AOL started also including results for the broad match of the searched phrase, which was making the results impractical for the purpose of finding duplicates of the text selection searches. As a results of this, AOL was also showing hugely exaggerated search results counts due to the search results also including broad matches and not just quote matches.

Another reason for moving to the Ask/Teoma search engine is simply that they are curently recognised as the 4th largest (in terms of usage) search engine after Google, Yahoo! & Bing. The graph below is taken from the ‘Search Engine Analysis‘ graph at Hitwise.com and shows Ask.com as the 4th largest Search Engine…

Also examining Alexa’s rank for Ask.com also shows us they are in the global traffic rank position of 45.

So removing AOL in place of Ask.com makes a lot of sense.

However, there was one major problem…

Ask.com search results do not display a search results count number! This is likely due to the unique ‘Q&A’ approach which Ask.com is taking with it’s search results. They are less about keyword orientated search and more about delivering specific answers to specific questions, so numbers of results could seem an irrelevant bit of information here and detract from their intended purpose.

However, there is a work around because Ask Jeeves (or rather IAC – the parent company who own’s Ask Jeeves) acquired the Teoma.com search engine back in september 2001 from a team of scientists at Rutgers (the State University of New Jersey). This was essentially for the purpose of getting the technology behind Teoma so they could compete better against the likes of Google and Yahoo!.

Since Teoma has been under the IAC ownership, it has been playing a large part in Ask.com search results, forming the backbone of the Ask Jeeves technology. Teoma also actually disappeared in 2006, however, last year in April 2010 it was brought back without any form of grand fanfare. Here’s some comments from IAC about this relaucnh:

“As we announced in November, we are focused on building the best service for answering questions and delivering the best results for searches posed as questions on Ask.com. We re-launched Teoma.com in mid-April to provide a simplified interface for everyday keyword web search. Teoma.com’s organic results will be similar to, but not exactly the same as, those on Ask.com, primarily for testing purposes, but also to provide a more simplified search experience.”

The above quote from IAC indicates that Teoma technology is likely still powering the Ask.com results…

After some testing between Ask and Teoma search results, we can confirm that the search results are extremely similar. In fact it appears that the more long-tail and obscure the search phrase, the increasingly similar the results become. And this is ideal for phrase match, long text section searches, that the DupeFree Pro Online Check Module performs.

Here’s an example comparison for the phrase “what a crazy day it was yesterday” :-

As you can see from this example the results are almost identical across Ask and Teoma, with the main difference being the positions. Also, what you can’t see in the above example, is that both search engines also delivered the exact same number of results, which is 12 in this case. All of this really helps concrete the fact that Ask and Teoma are both being driven off the same index. This is the key aspect we need to know in order to safely move forward with using Teoma as the search engine to act as Ask’s search data.

So, until a better search engine comes along, or ask begins display search counts, we will continue with Teoma for the Onilne Check module’s 3rd search engine data source.

Oh, and for a little fun-fact, teoma means “expert” in Gaelic!

Cheers,
Mike