« DNS lords expose netizens to 'poisoning' | Main | Man hacks video game to propose to girlfriend »

Google bots now submit forms in effort to find new pages

"Google's search bots, which scour the web constantly for new pages, have begun a new, more active phase of their indexing jobs.

In a blog post last week, Jayant Madhavan and Alon Halevy of Google's crawling and indexing team said the company has begun an experiment in which its indexing software experimentally enters text in website forms to see what previously undiscovered pages may appear.

The post said: "In the past few months, we have been exploring some HTML forms to try to discover new web pages and URLs that we otherwise couldn't find and index for users who search on Google. This experiment is part of Google's broader effort to increase its coverage of the web. In fact, HTML forms have long been thought to be the gateway to large volumes of data beyond the normal scope of search engines.""

Article: http://googlewebmastercentral.blogspot.com/2008/04/crawling-through-html-forms.html


Feed You can follow this conversation by subscribing to the comment feed for this post.

All Comments are Moderated and will be delayed!