amnesty.or.th
robots.txt
Robots Exclusion Standard data for amnesty.or.th
Resource Scan
Scan Details
Site Domain | amnesty.or.th |
Base Domain | amnesty.or.th |
Scan Status | Ok |
Last Scan | 2024-09-22T22:14:52+00:00 |
Next Scan | 2024-10-22T22:14:52+00:00 |
Last Scan
Scanned | 2024-09-22T22:14:52+00:00 |
URL | https://amnesty.or.th/robots.txt |
Domain IPs | 104.21.42.112, 172.67.161.165, 2606:4700:3030::ac43:a1a5, 2606:4700:3031::6815:2a70 |
Response IP | 172.67.161.165 |
Found | Yes |
Hash | 9b3592a18d66240f6ed4ef0f87b26e1029f134d17de0a6bd99cfeec962869f0b |
SimHash | 00787450217e |
Groups
*
Rule | Path |
---|---|
Disallow | /config |
Disallow | /controllers |
Disallow | /elements |
Disallow | /helpers |
Disallow | /jobs |
Disallow | /languages |
Disallow | /libraries |
Disallow | |
Disallow | /models |
Disallow | /page_types |
Disallow | /single_pages |
Disallow | /tools |
Disallow | /index.php/tools |
Disallow | /blocks/*.php$ |
Disallow | /blocks/*.xml$ |
Disallow | /concrete/*.php$ |
Disallow | /concrete/*.xml$ |
Disallow | /packages/*.php$ |
Disallow | /packages/*.xml$ |
Disallow | /updates/*.php$ |
Disallow | /updates/*.xml$ |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
Warnings
- 2 invalid lines.
Comments