Crawling Dead Ends

Search engine dead ends are something you have to remind site owners about, as they do not think about the web in terms of how a search engine navigates a site, just how people do, particularly themselves. If they can navigate to said page, then everybody can is their thinking.

Google recently announced they are experimenting crawling through web forms on a small, select group of high quality web sites. Is this notice that soon any site owner will be able to hide content behind web forms and expect it to be crawled? No, I don’t think so. First, there are other search engines to think about; Google is not the other engine. Second, the coverage from Googlebot crawling through web forms certainly will not be as extensive as link crawling.

While this search advancement is welcome, I don’t think it should change how good, accessible sites are designed. The advice that Matt gives in his fictional site review still stands. If you need to place a selection of some kind in front of content that you need crawled (like your whole site), it is best to do this with links rather than a web form. There are other reasons for maximizing the content on the site’s main page rather than it being a simple portal with an image and region dropdown.