Print version (in a new window)

WebWarper / Frequently Asked Questions (F.A.Q.) / Web-publishers' questions

I don't want WebWarper-version of my site to be indexed in search systems. Why don't you disable such indexing by robots.txt file?

We consider it normal and correct for users to find both usual link to your site and the version of this link with http://webwarper.net/ww/~av/ prefix, that allows viewing the site with optimization. Users should be able to choose a way they prefer to view your site.

Moreover, indexing of WebWarper-versions of sites by search robots is absolutely necessary for Web publishers who decided to optimize their resources using our service (see our offer for Web publishers on the main WebWarper page). Some sites are always viewed via WebWarper (Variant A of optimization). If we block indexing optimized pages by using standard robots.txt file or other means, then the normal indexing of these sites will become impossible.

However, you can disable indexing WebWarper-version of your site yourself. You need to use META tags that block pages indexing: please see the document http://www.robotstxt.org/wc/exclusion.html#meta. These META tags should be added in a way that they are "visible" only when the site is viewed through WebWarper. To do so, please use the special "pseudo-comment":

<ww><!--<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">--></ww>

For additional details please refer to the question "Is it possible to show different fragments of an HTML-page, depending upon the presence of WebWarper?"

Also see the question "I don't want people to view my site through WebWarper".

To contents