SearchTrack Architecture

Search Analytic Component

SearchTrack’s Search Analytic component provides real-time logging of all search activity along with database-driven activity reports for fast access to the information you need. The Search Analytic component can also present historical data loaded from web server or existing search engine logs.

Activity Reports provide specialized analysis of user queries and allow you to identify queries that return no hits, trends in search terms, and breakouts by frequency, user, domain, and date. Direct links are provided from search analytics to update your user behavior based taxonomy and suggest the best results for any visitor query.

Search Analytics: Logging

As SearchTrack receives an HTML result list from the search engine, it extracts the number of documents found, the number of document searched, and other information regarding the search and results. Once the page is displayed to the visitor, SearchTrack logs the search activity to the defined database.

Some of the information recorded includes:

• Search term(s) (normalized)
• Search terms (as entered)
• IP Address of the visitor making the request
• The sub-section of the site the search was performed over
• The number of results found (if available from the search engine)
• The total number of documents returned from the search (if available from the search engine)

Search Analytics: A Behavior-Based Taxonomy

As SearchTrack logs user searches over time, it creates a special taxonomy of the search terms your site visitors actually use: a Behavior Based Taxonomy. You can use this Behavior Based Taxonomy to immediately define the best results for your top queries using Directed Results. You can also use this Behavior Based Taxonomy as a starting point to create a full site taxonomy.

SearchTrack stores the behavior based taxonomy for directed results in memory so that lookups don’t require a database lookup (a persistent version of the taxonomy is read and written from common database applications but only on server startup and with changes to the taxonomy). Since the taxonomy typically only holds maps for at most a few hundred common search times (which typically comprise 70-80% of all searches adjusted for frequency) memory requirements are small.

Most Popular Searches

Knowing the top user queries lets content owners see what site visitors are looking for, how often they are searching for the term, and how many documents come back in response to a given search. This provides the knowledge needed to tune web content so deliver better answers and can provide valuable feedback for support staff. It can also help marketing staff understand the effectiveness of product advertising and positioning.

Top Visitors

Knowing which companies are visiting your site and what search terms they are using can provide important information to help sales and support staff better anticipate customer needs. It can also tell you what interest your competitors have in your content.

Most Frequent Visitors

The visitor report also allows you to drill down on individual visitors specific queries.

Trend Reports

In addition to activity reports, SearchTrack provides a trend report to help identify trends in search activity over time. This can help spot product problems, market interest, and response to marketing campaigns.

Search Activity Trends

The ability to see trends emerge over time lets you anticipate customer needs and help insure timely response to customers and prospects.

No Hits Report

Perhaps nothing is more frustrating than searching on a web site and not finding any content, especially when you know there must be content available. Sometimes the problem is a simple spelling mistake; other times it can indicate a disconnect between user vocabulary and site vocabulary. The No Hits report lets you identify what queries return no content, and lets you take action to improve your site.

No Hits Report

Reports are viewed via the administrative console. SearchTrack supports multiple levels of access via password. The read-only password allows administrators to view reports which can provide valuable feedback for support staff. The read-write password provides full report access and allows creation, editing, and deletion of directed results.

Search Analytics: Summary

SearchTrack can be used as a tool to perform historical search log analysis, or as licensed software to generate live search activity reports over time. As licensed software, it can also improve search results by permitting content owners to identify the best answers for common queries. In either case, the knowledge gained from evaluating search activity is a vital first step to understanding what web site visitors really want.

Directed Result Component

Directed Results lets a privileged user define the right document to appear at the top of the result list, to suggested related terms or misspellings, or to provide guidance if the search engine returns no results for a given query without any need to tag documents or change search engine parameters.

Once a directed result is defined for a visitor search, SearchTrack adds the directed results to search engine results in a header or side bar to promote the best results.

Directed Results: Entry

From the Most Popular Searches and No Hits reports, a read-write administrator sees links to define the best directed result for any search. Clicking on ‘Create’ from a Most Popular Query report, for example, brings up a Create Directed Results screen.

Create Directed Results

In defining a directed result, you can define not only the URL, but you can override the existing document title and description. You can also defined related terms and searches.

Directed Results: Result Mark-Up

Once you have created a directed result for a search term, subsequent searches for the term immediately produce the right result.

When SearchTrack receives a result list from your search engine, it consults its behavior-based taxonomy to determine whether any directed results are defined. If so, SearchTrack inserts the appropriate HTML for the specific documents or related content additional search terms, and related content. SearchTrack allows directed results to be displayed as suggestions across the top of results lists as shown, or as related links along the side of the result list.

Immediate Directed Results

Directed Result Summary

Directed Results are the best way for you to provide site visitors with the right results for your most popular searches without the need to modify your documents or to change your search engine configuration. The improvement is immediate, significant, and measurable.

When you decide to start using SearchTrack, we have a proven methodology that will get you going quickly, so you can see better search results in days.

K2 – (Recommended) Check-Up

Because Verity K2 is so powerful and has so many configuration options, we strongly encourage that we start our engagement with a Verity K2 Check-Up. The Check-Up looks at dozens of parameters that can cause K2 to not deliver the best results possible results, and while SearchTrack can improve any search it’s best to have your search engine optimized first. Learn about the K2 Check-Up at http://www.ideaeng.com/ds/k2_datasheet.html.

Initial Test and Configuration Installation

When we install SearchTrack, we need to identify a number of configuration options specific to your search engine and your customized search result. We start by creating a test environment, typically on the same server that hosts your search engine server.

We work with your information technology staff to set up the test area, and to identify how and where SearchTrack will be installed, and what server port SearchTrack will use to proxy the search engine. The parameters we check include:

• Set Up Test Environment
• Copy Search form(s) to test area
• Analyze Environment
• System Environment
• Verify Java installation and path
• Identify ports for ST server
• Identify privileged and non-privileged SearchTrack passwords
• Identify any load balancing or fail-over requirements

Database

SearchTrack maintains search analytics in a relational database to provide high speed logging that will not slow your visitor search experience, and will provide you with high performance reporting. During installation, we will:

• Identify which Database Server to use
• Create database user/password
• Create Database Tables
• Test the Database connection

Search Forms and Results

Because search engine forms and results are typically customized to each site, we next need to examine both your search forms and results lists. This will allow SearchTrack to emulate the user search connection to your search engine, to extract search statistics from the result list, and insert directed results ahead or alongside your existing search results. We will:

• Analyze your search forms
• Identify search action parameters
• Identify required hidden fields and values
• Analyze your result list layout
• Identify the pattern where directed results will go
• Identify the result list patterns that indicate no hits were found
• Identify the result count/total count patterns

Create Configuration

With the information gathered by our configuration tools and analysis, we will create and test the initial SearchTrack configuration files including:

• Configuration file parameters
• Test database logging with search form(s)
• Test no hits logging and pattern
• Verify reporting is working properly for all forms and options
• Test Directed Results

Depending on the extent of the test environment, we may also work with your content staff to identify preliminary candidate searches and directed results for the top queries discovered during the test period.

Update installation for production

Once the test site is working properly, SearchTrack is ready to move into production. At this point we set all production options and have the system ready to go live. Preparing for production includes:

• On Unix systems, insert SearchTrack in system start-up files
• On Windows servers, install SearchTrack as service
• Schedule DNS update job to report visitor domain names as well as IP addresses
• Test final configuration and logging
• Verify auto start by rebooting server if possible
• Verify DNS schedule
• Reset database content

Go live

Once SearchTrack is installed and tested in its production configuration, it’s time to update your search forms and begin using SearchTrack. This involves:

• Updating your web search form
• Verify search activity and directed results are working

Once SearchTrack is live, New Idea Engineering will work to train your content owners in the use of both search analytics directed results to improve your search engine results in minutes.

This section details the sequence of events to complete the historical analysis project.

1. Mutual Non-Disclosure Agreement

Because New Idea Engineering will be accessing your systems and data, and because you will have access to proprietary NIE tools and utilities, we suggest that we each execute a mutual non disclosure agreement. We have a standard form, or we can use your company’s mutual NDA form.

2. Identify Primary Contacts

Both parties will identify primary project managers for business and for technical matters during the project.

3. Arrange for Access

The check-up and installation requires command-line and HTTP access to your search server. Access can be handled in three different ways, depending on your needs and policies: o New Idea staff comes on-site to perform the installation; or o New Idea staff uses remote access via VPN/Terminal Services/SSH Telnet

4. Issue a Purchase Order for the Project

Your company purchase order can be faxed or emailed to our facilities, as well as any other forms your company may require such as contractor policies, tax id numbers and other administrative paperwork. Contact us by fax at 408-446-0241, by email at scarlet@ideaeng.com.

5. Schedule the Start Time

Once the purchase order is issued, the project contacts will select a mutually acceptable date to start the project. Depending on availability, the lead time can be as little as a week once all other dependencies are resolved.

6. Provide Search Forms and Results List

Part of the installation requires understanding the search forms and result layout you currently use. Providing New Idea Engineering with your search forms and typical results lists prior to the installation makes the process go faster and gets you up and running much faster.

7. Perform the K2 Check-Up

If you are using Verity K2, we strongly recommend you start with a K2 Check-Up to insure your search engine installation is running as efficiently as possible. The check-up takes days, and produces a full inventory of your collections, forms, and scripts; it may also recommend steps you can take to resolve any problems the check-up might identify.

8. Install SearchTrack Software

Because the dynamic search analytics requires capturing of live search activity in addition to historical analysis, we would install SearchTrack on a system at your site and provide your technical staff with the information needed to make the search capture live. Once installed, all search activity in your site will be logged, along with document counts.

9. Train Your Staff on SearchTrack

New Idea Engineering will train your IT staff, content owners, and users on how to manage, use, and benefit from SearchTrack.

10. Deliver Reports and Recommendations

Once we have loaded the data and updated document counts, we will create reports that show top queries, top visitors, no hits, and trend reports for the period of the logs. The format will include a written report with recommendations, as well as a brief presentation of findings.