Search this site:
Enterprise Search Blog
« NIE Newsletter

Mismatched Search Engine Woes

Last Updated Dec 2008

By: Mark Bennett, New Idea Engineering, Inc., Volume 2 - Number 6 - June 2005


Since our earliest days, New Idea Engineering has been in the business of helping companies implement enterprise search. Usually we get involved when the installation didn't go quite right, or when there was a need for advanced features that are difficult to implement. Usually the search solution already selected was right for the job; our customers just needed some help up the learning curve. Lately we've seen a change. We still get called in to implement advanced capabilities, or to help with installation and set-up; but it seems that more and more of the people we see have selected the wrong search technology for their requirements.

In the 1990s there were 'leading' vendors who understood search and who 'got' the internet; and there were other vendors whose technology wasn't quite up to speed and who sometimes didn't 'get it' about the 'net. But here in the new millennium, we're living the good life: most search engine vendors are offering pretty good products, though some customers still experience sticker shock when they see the final cost.

We still hear complaints about vendors and technology, some of which might be deserved. But seriously folks, if you've never tried to install and configure a 1995 vintage web-based search engine, then you may not appreciate how far the vendors have come.

The Wrong Stuff

To us, the number one reason that companies are dissatisfied with their search technology is that, as we start to work with them, it's clear they have bought the wrong search engine for the job. We're still pondering the cause of all this: it doesn't seem to be from intentional or malicious misrepresentation by the vendors sales teams. Nor is it from lack of effort by the companies: most have gone through at least some level of due diligence, and found the software to be promising... at first. But when the implementation starts, it becomes clear that the solution was either more powerful (and probably more expensive) than needed; or worse, that the solution was just not powerful enough for the task at hand.

Instead what we see is that vendor X returned phone calls promptly, and vendor Y blew it. Or that vendor Z did a drive-by analysis, and the customer probably should have asked for an on-site evaluation

We've also seen miscommunication between customers and vendors, particularly when it comes to the word 'can'. A company might ask a vendor ’Can your product index database content?” Vendors are optimistic, and the sales staff usually has a great understanding of features - and a list provided by their marketing staff - so if the feature list says the software can access database content, the answer is 'yes'.

But just as the word 'is' might mean different things to different people, the affirmative answer to the question 'can it...' might not be a complete. The product might be able to index database content as soon as you install it - probably the original intent of the question. But it could mean 'yes, when you buy the expensive extra add-in for database access'; or it could mean 'yes, if our consulting staff comes on-site for a six month assignment to write the code'. By the way, Jolt cola may be required as well.

Prospective buyers would do well to ask better questions: Does your product index database content in the standard configuration? If not, how much customization would be involved to enable database access? Do I need to use the API to do it, or is it a configuration option? And most importantly, 'Can I speak to a reference customer who has recently done the same thing with your software?'.

Another edge case of 'can it' miscommunication involves the simultaneous use of multiple features. For example, consider a company that has database content and web based content, and the sales staff says their product can index both. It might be good for the company to ask 'Can it do both at the same time'. It may be obvious to the company that indexing both in the same application is the desired capability; but when presented with a checklist of requirements, vendors don't always consider them as simultaneous requirements. When you are going to spend a bug chunk of your company's cash on software, be sure the vendor knows exactly how you plan on using it!

Some Examples of Mismatches

Examples make things so much clearer. Those presented here are fictitious, and no real companies or vendors were injured in the making of this newsletter. And the names have been changed to protect the innocent. And in the spirit of full disclosure, we may exaggerate to make the points clearer.

Vocabulary term: “API” means “Application Programming Interface”, which means the vendor will sell you a set of C or Java tools that will let you access their search engine from within your application at a much lower level. Programmers love APIs!

Scenario One:

Symptom: The HR department has bought Product X and is overwhelmed with how complex it is; they don't have an engineering staff!

Diagnosis: They overbought – they could probably have found a much simpler and less expensive system.

Scenario Two:

Symptom: Company B wants to be the next Google! But the technology they selected can't index more than 100,000 documents

Diagnosis: They under-bought – they probably needed an industrial-strength highly scalable engine, which would probably have cost more as well.

Scenario Three:

Symptom: Giant “Fortune 500” company C bought a pretty good search engine and have been using it for a while. Adding documents has always been just a few clicks away. But now they need to implement SSO (Single Sign On), a type of security system that lets users log in once and insures document level security for all searches. And they also need to bring global search to lots of their other systems, including their customer database, their document management system, and even into their proprietary resume tracking system that has a ton of meta data fields. They are frustrated that their search engine doesn't seem to integrate with all of this.

Diagnosis: The solution they bought 5 years ago seems to have been a good fit with the business requirements and mandates they had - 5 years ago. But they are probably going to need a search engine upgrade, possibly to a new vendor, to meet the new requirements. This is probably going to require more in-house technical resources and a more expensive search engine. If the new business requirements are real, then there must be some financial justification for them; so the ROI arguments for search will actually just be a part of that.

Scenario Four:

Symptom: Another big company, Company D, wound up with more than one search engine; it turns out this is the rule, not the exception – most large companies have multiple search engines. They want search activity reporting for all the engines, and document promotion, and a simple way to add in meta data. And they'd even like to have one unified search that crosses all these engines. But each vendor's solution seems to involve a sizable upgrade, and tossing out all the other vendors; so although technically feasible, this is unacceptable to Company D.

Diagnosis: Like the database industry years ago, the search engine industry is maturing and 3rd party vendors are starting to offer cross-platform solutions and custom integration services that work with multiple search engines. If Company D really doesn't want the expense and effort of migrating all their applications to a single engine, then 3rd parties are the logical choice.

Disclosure: Yes, New Idea Engineering, the authors of this newsletter, do offer these cross-vendor products and services. If this scenario happens to fit, we invite you to drop us a line.

Scenario Five:

Symptom: Startup C wants to set the world on fire with their new super email / blogger / CRM system. They bought the upscale product Y, including product Y’s Java API. Things went well at first, but now they want to do things that vendor Y’s API doesn't support, and the licensing fees are killing them.

Diagnosis: They should have considered using an open source toolkit. Since they are an engineering team, they would feel right at home with the relative complexity of an open source library, and since they have the source code, they could (in theory) make it do almost anything. When evaluating an open source library, check the license carefully, not all open source licenses are the same. Generally you want a 'BSD style' or 'L-GPL' license. You probably don't want to a license that requires you to release your source code, or a license that is not free for commercial use.

And if you need support for open source solutions that have been tested and packaged, you might talk with the folks at SpikeSource at

Getting it Right

There are a few simple steps top follow if you are preparing to evaluate different search solutions.

  • Determine the features you need
  • Identify key sources of data to be included in search
  • Identify security requirements
  • Set up a representative test data are
  • Identify potential vendor
  • Describe the full requirements to vendors
  • Ask vendors to show you your representative data in their standard application
  • Insist the vendor spell out any options and the scope and price of any consulting

You may want to secure the assistance of an expert third-party company to assist in the evaluation. There are many companies that have experience dealing with multiple vendors, and they can help you navigate the process from initial specifications to final deployment. You may find it helpful to utilize industry or local associations, analyst tools like the Gartner "Magic Quadrant" for Enterprise Search, or independent tools like the New Idea Engineering Enterprise Search Matrix.