Paco Underhill's Why We Buy: The Science Of Shopping (Touchstone Books, 2000) illuminated the mysterious behavior of shoppers wandering around in retail stores.

The realization comes to marketers in a flash that the behavior of a website visitor is immeasurably more measurable. It's obvious. It's intuitive. It's exciting.

Then comes the hard part.

The marketing manager heads over to the IT department or the Web manager and asks, "What can you tell me about what's happening on my site?"

Web manager: "What do you need?"

Marketing manager: "What have you got?"

Web manager: "Whatdda ya need?"

Marketing manager: ""Whadda ya got?"

It an attempt to halt this circular madness, I'd like to start with, "What have you got?"

There are a variety of ways to collect data about what people are up to on a Web site and here they are in the briefest terms possible.

• Performance Monitors: Is your server serving quickly? Is it overloaded? Are your e-commerce applications running smoothly? Is your server sending out "404, File Not Found" messages? Performance monitors watch the technical side of things and give you a holler if something goes bump in the site.
  
• Server Logs: Every Web server makes a log entry of every page it sends out. By using log file analysis tools, you can glean such interesting things as number of visits and pageviews, entry & exit pages, hourly usage, browser & operating system used, and usage by country. All of which come with the caveat: sort of. Each of these measurements has some inherent irregularity so take them all with a good sized grain of salt.
  
• Referer (sic) Logs: If the visitor comes to your site via a link on another page, your Web server will also record the URL of that previous page.
  This is enormously informative if you're buying banner ad space or tracking the success of a pay-per-click campaign. Did banner ad A outperform B? Did the press release cause a stir? Are people finding you while searching for a word you didn't expect? Interesting stuff.
  
• Tagged URLs: Server logs have trouble confidently revealing what an individual did during a single visit, so tagged URLs were designed for session or clickpath analysis.

When a home page is served, all of the links on the page are tagged with a sequential ID number. A link that normally looks like this: www.company.com/page2.html is tagged by the server on the way out the door to look like this: www.company.com/page2.html?127 indicating that I'm the 127th person to visit today. When I click on that link, the server records that page2.html was requested by visitor number 127 and tags the links on the requested page with 127 as well.

• Cookies--Server logs can rarely recognize individuals from one day to the next, so along came cookies. Wander over to https://www.cookiecentral.com with your knowledge of these delectable morsels. For our purposes, it's enough to know that you can track an individual over time using a cookie, and you can correlate their traffic patterns with any personal information they may have divulged while on your site.
  
• Packet Sniffing--Information moves from here to there online inside small data packets. Your home page is divided up into capsules of about 1,000 bytes of data and off they go, finding their way to your visitor's screen. Should the visitor type something in a form and hit the submit button, the entered data get packetized and sent back to you. By sniffing the packet on the way back, you not only know which submit button they clicked (top of the page, middle or bottom) but you also capture the entered data. Now you have a record of the click as well as the information that would normally go straight into an application database and require a good deal of extract, transfer and load effort to match up with the server activity information.
  
• Web Beacon (cleardot.gif)--Web pages are often stored in temporary cache files on the visitor's computer and in cache farms at ISP's to minimize the time it takes to go back to a previously viewed page. This minimizes the traffic on the Internet overall and improves customer experience on your site. It also means that your server doesn't always know when a page is viewed.
  Embedding a transparent, one pixel by one pixel image that is designated as "no-cache" can help identify when each page is viewed. If such a graphic is served from a third party server, the aggregate information about cross-server and cross-site traffic can be useful.
  
• Client Side Surveillance--A small Java program watches every move you make. Every mouseover. Every keystroke. This method can collect the most complete picture of what an individual does on your site. The data can be used to playback a visit and can capture data entered into a form even if the form is completed or the submit or clear buttons are not used. Very powerful, somewhat spooky. Full disclosure and full opt-in are highly recommended.
  
• Panel Observation--If you'd like to look over peoples' shoulders and ask them personal questions at the same time, then the panel research method is the way to go. Each panelist agrees to have client side surveillance software on their computer and is willing to respond to the occasional pop-up asking them why they made certain click choices or how they feel about the color of the site's background, etc. You can also learn how they feel about the competition.
  
• Eye Tracking--If you want to get up close and personal, there are a number of firms measuring what individuals actually look at on individual pages of your site. You'd be surprised at what people simply do not see online.
  
• Gross Traffic Patterns--A handful of companies are learning what people do online by watching from the vantage point of the ISP. Millions of people looking at millions of sites per day reveals which sites are the most popular.

For the price of admission, you can learn where people (in general) were before they came to your site and where they went thereafter.

Now you know what the IT department is so quick to respond with, "What do you need?"

There is so much data available (granted some of it is rather expensive to gather), that we must stop asking for anything and figure out what we want in particular.

As a professional consultant, I can tell you that the correct answer to "What do you need?" is always, "It depends. When would you like to set up an appointment?"