agilest.org
robots.txt
Robots Exclusion Standard data for agilest.org
Resource Scan
Scan Details
| Site Domain | agilest.org |
| Base Domain | agilest.org |
| Scan Status | Ok |
| Last Scan | 2025-11-04T04:22:41+00:00 |
| Next Scan | 2025-12-04T04:22:41+00:00 |
Last Scan
| Scanned | 2025-11-04T04:22:41+00:00 |
| URL | https://agilest.org/robots.txt |
| Domain IPs | 104.21.18.136, 172.67.182.37, 2606:4700:3035::ac43:b625, 2606:4700:3036::6815:1288 |
| Response IP | 104.21.18.136 |
| Found | Yes |
| Hash | 378ad888aaf6e2d05eeef611cf2b64093194c563e27629a834f6136e77eb2111 |
| SimHash | 0530d0c88629 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin | папка на Ñ |
| Disallow | /? | вÑе паÑамеÑÑÑ Ð·Ð°Ð¿ÑоÑа на главной |
| Disallow | /wp- | вÑе ÑÐ°Ð¹Ð»Ñ WP: /wp-json/, /wp-includes, /wp-content/plugins |
| Disallow | *?s= | поиÑк |
| Disallow | *%26s%3D | поиÑк |
| Disallow | /search/ | поиÑк |
| Disallow | /author/ | аÑÑ |
| Disallow | /users/ | аÑÑ |
| Disallow | */trackback | ÑÑекбеки, ÑÐ²ÐµÐ´Ð¾Ð¼Ð»ÐµÐ½Ð¸Ñ Ð² комменÑаÑиÑÑ |
| Disallow | */feed | вÑе ÑÐ¸Ð´Ñ |
| Disallow | */rss | rss Ñид |
| Disallow | */embed | вÑе вÑÑÑÐ°Ð¸Ð²Ð°Ð½Ð¸Ñ |
| Disallow | */wlwmanifest.xml | xml-Ñайл маниÑеÑÑа Windows Live Writer (еÑли не иÑполÑзÑеÑе, |
| Disallow | /xmlrpc.php | Ñайл WordPress API |
| Disallow | *utm*%3D | ÑÑÑлки Ñ utm-меÑками |
| Disallow | *openstat%3D | ÑÑÑлки Ñ Ð¼ÐµÑками openstat |
| Disallow | /shop/ | ÑÑандаÑÑÐ½Ð°Ñ ÐÑ ÑÑÑаниÑа |
| Disallow | /*?ical=1 | - |
| Allow | */uploads | оÑкÑÑваем Ð¿Ð°Ð¿ÐºÑ Ñ Ñайлами uploads |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Warnings
- 4 invalid lines.
Comments