fao.org
robots.txt
Robots Exclusion Standard data for fao.org
Resource Scan
Scan Details
| Site Domain | fao.org |
| Base Domain | fao.org |
| Scan Status | Ok |
| Last Scan | 2024-10-20T04:21:40+00:00 |
| Next Scan | 2024-11-19T04:21:40+00:00 |
Last Scan
| Scanned | 2024-10-20T04:21:40+00:00 |
| URL | https://fao.org/robots.txt |
| Redirect | https://www.fao.org/robots.txt |
| Redirect Domain | www.fao.org |
| Redirect Base | fao.org |
| Domain IPs | 104.18.22.5, 104.18.23.5 |
| Redirect IPs | 104.18.10.41, 104.18.11.41, 2606:4700::6812:a29, 2606:4700::6812:b29 |
| Response IP | 104.18.11.41 |
| Found | Yes |
| Hash | 4813213a20ec84cce0d745b196d12cfc0f68988d591b5dd4ead1c61f16cec8a9 |
| SimHash | 700f1f1afdf5 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | /index.php | - |
| Disallow | /t3lib/ | Nothing to see here |
| Disallow | /typo3/ | Nothing to see here |
| Disallow | /*?id=* | Disable non-realurl - re-instated 10/Oct/2013 |
| Disallow | /*%26type%3D98 | - specified in Google webmaster tools for the Google exclusion - re-instated 10/Oct/2013 |
| Disallow | /fileadmin/user_upload/PermRep/ | don't need to be indexed (23/06/2014 - nw) |
| Disallow | /fileadmin/user_upload/en/ | don't need to be indexed (11/07/2014 - permreps) |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.fao.org/newsroom/sitemap/sitemap.gz |
| sitemap | https://www.fao.org/in-action/ectad/sitemap/sitemap.gz |
| sitemap | https://www.fao.org/americas/sitemap/sitemap.gz |
| sitemap | https://www.fao.org/director-general/sitemap/sitemap-index.xml |
| sitemap | https://www.fao.org/europe/sitemap/sitemap.gz |
| sitemap | https://www.fao.org/world-food-day/sitemap/sitemap.gz |
Comments