The Internet is home to billions of websites. Google claims to index over 8 billion web pages, and MSN announced 5 billion not long ago. Yahoo! is probably close, but upon query, a spokesperson said, "…while we don't disclose the size of the index, our top priority is to provide the most comprehensive and relevant search experience to consumers."

How many users visit these websites? In the U.S. alone, search engine users totaled over 130 million in 2004 and will go beyond 150 million in 2006. If you’re in business to sell, there’s a captive audience out there.

Many companies have a website, but the majority of these websites are not optimized to maximize search engine traffic. Research indicates that 80 to 90 percent of corporate websites are not well optimized. It also shows that properly optimized websites enjoy fantastic gains in unique visitors and conversions. These huge gains have been reported a number of times in research studies and in the media. So why aren’t more companies reaping the rewards of search engine traffic?

Many companies are neglecting the important back-end issues that can make their websites search-engine friendly. Sure, users visit these website and click to download white papers. The graphics are sweet, and some of these sites rock. They may not have the best navigation and usability, but landing pages are in place – it all looks first-class. When we surf the web, we see a lot of this same functionality. But the problem with the average website is that it’s not optimized to gain top rankings on the major search engines.

While all appears to be smooth on the front end, potential problems lurk on the back end. To quote Dan Thies of SEO Research Labs, "We routinely conduct technical reviews for our SEO partners, and find duplicate content or other server-related issues on over half of the sites we see."

There are a plethora of back-end technical connections going on behind the scenes, and if you’re lulled into complacency, your site could be missing the road to top rankings that increase your conversions.

When it comes to communicating with search engine spiders, many of America’s best-known company websites are not properly optimized to gain top rankings. That’s because the back-end is shorting-out and not connecting with the stealthy search engine robots that index and rank websites for major search engines like Google, Yahoo! and MSN.

Following is a list of back-end technical items related to search engine robots that must be addressed if you want your company website to perform well and outperform your competitors in the major search engines. As part of your Organic Search Engine Optimization Program, the items below should be thoroughly reviewed, and appropriate recommendations made, in a professional SEO Site Review.

1. Site Architecture and Applications -- This deals with your web server (e.g., Apache, Microsoft IIS, Sun) and the program languages used on your site (e.g., HTML, asp, php, jsp). A look into your web hosting (e.g., dedicated or shared) and your database technology (e.g., MySQL, Oracle, PostgreSQL). It also covers investigation of your web applications such as content management systems, site search, directories, and shopping carts. Consideration for how you set-up your cookies and sessions (human vs. spider). A check of your HTTP response headers looking at proper use of 404 error pages. Complete review of basics such as your contact information, FAQ, privacy policy, various resources, site map and number of links on a site map page.
2. Site Navigation -- A look at your image maps and image links, javascript and DHTML and how they are organized on the page. Consideration for your text links and form based navigation, redirection 301, 302 pages and META refresh. A look at the effectiveness and placement of your global navigation and functional sections such as news, forums, shopping, etc. A review of site search functionality, usability and logs.
3. Document Structure -- A review of your title tags, META description and META keywords. A look at your page layout CSS and tables. Proper headings (H1, H2 and page structure). The effect of pop-ups and browser compatibility such as AOL, IE, Mozilla, Netscape, etc. The effect of any plug-Ins, Flash or Java, scripts and stylesheets.
4. Keyword Strategy & Copywriting -- An analysis of your primary themes and search terms, your supporting keywords and their optimization and effectiveness in your current content. How you are using landing pages and their URLS. A look at your internal links (cross-linking) also related to the use of keywords and their placement. A look at keywords and placement in outbound links.
5. Search Engine Positioning -- Index saturation: the number of pages found and/or indexed in Google and the crawlability of the site. Identify potential duplicate content and review link popularity, link quality and relevance. Check PageRank on Google toolbar and directory listings such as DMOZ and Yahoo.

All search engine spiders must first be able to digest your content before they can post your high-ranking results. That’s why a technical site review is the first step in Search Engine Optimization. Once the above back-end items have been analyzed by a qualified SEO technician, your IT people can begin the process of making the necessary adjustments to your website. When adjustments have been made, you are on the road to capturing all of that delicious organic traffic. Happy traveling! Remember, it’s a journey not a destination.

Paul J. Bruemmer is founder of trademarkSEO, a search engine optimization firm serving clients nationwide. His companies have provided search engine marketing services to over 10,000 websites, including many of the most prominent names in American business. His articles have appeared in ClickZ, MarketingProfs, WebProNews, SitePoint, Pandia and Search Engine Guide.