Yahoo recently introduced the ‘robot-no content’ tag which allows users to specify what content they do not want the search engine spiders to crawl due to irrelevance. This ensures that the search engine crawler will only look at relevant information. This will help bridge the gap of having a website that caters to users or search engines. With this content filtering, it could be possible to have a website that does both.  This new concept will help webmasters in optimizing their sites better. Sometimes, there’s content on the site that isn’t quite relevant to the main content of the site; for instance, privacy statements, mission statements, terms and conditions, etc. Instead, you can get the Yahoo search engine crawler to only look at the content in which you would want them to by applying this ‘robot-no content’ tag to certain parts of the page that indicate to Yahoo’s search engine crawler what parts of the page are unrelated to the main content and are only useful to visitors.
(more…)

Recently, while performing a fairly routine search on Google, something caught my eye.  My results were being fragmented into what are being called “Google categories.”  As other blogs have pointed out, these results are prompted by broad keyterms like DVD players and lawn care.  Our example shows these results are also prompted by a product model number. 

(more…)