View Image
|
Web
Education
iSEEK™bot
iSEEKbot is the name of iSEEK's indexing agent often referred to as a search engine web crawler. It is a program that moves from web page to page indexing the web content it finds. We respect and obey the robots.txt standard, so if you do not want iSEEKbot to index your site you can tell it not to. The robots.txt is a file that you can place within your web server's root html directory. It tells iSEEKbot and other search engines what pages you do not want the spider to crawl and index.
Meta Tags Supported by iSEEKbot
iSEEKbot will exclude a page that has this metatag:
<meta name="robots" content="noindex">
iSEEKbot will not follow links on a page that has this metatag:
<meta name="robots" content="nofollow">
iSEEKbot will not cache a page that has this metatag:
<meta name="robots" content="noarchive">
|