amnesty.org
robots.txt

Robots Exclusion Standard data for amnesty.org

Resource Scan

Scan Details

Site Domain amnesty.org
Base Domain amnesty.org
Scan Status Ok
Last Scan2024-05-08T13:05:55+00:00
Next Scan 2024-06-07T13:05:55+00:00

Last Scan

Scanned2024-05-08T13:05:55+00:00
URL https://amnesty.org/robots.txt
Redirect https://www.amnesty.org/robots.txt
Redirect Domain www.amnesty.org
Redirect Base amnesty.org
Domain IPs 141.193.213.20, 141.193.213.21
Redirect IPs 141.193.213.20, 141.193.213.21
Response IP 141.193.213.20
Found Yes
Hash 764fd5d79ae2c41711170046bd257ca3fdb6af0c0d1c5b9c8ee97821870d44e4
SimHash 6728da17e3b2

Groups

*

Rule Path
Disallow /ar/search/
Disallow /fr/search/
Disallow /es/search/
Disallow /en/search/
Disallow /ar/wp-admin/
Disallow /en/wp-admin/
Disallow /es/wp-admin/
Disallow /fr/wp-admin/
Allow /ar/wp-admin/admin-ajax.php
Allow /en/wp-admin/admin-ajax.php
Allow /es/wp-admin/admin-ajax.php
Allow /fr/wp-admin/admin-ajax.php

ccbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5