checkurl.org
robots.txt
Robots Exclusion Standard data for checkurl.org
Resource Scan
Scan Details
| Site Domain | checkurl.org |
| Base Domain | checkurl.org |
| Scan Status | Ok |
| Last Scan | 2024-11-09T20:31:32+00:00 |
| Next Scan | 2024-11-16T20:31:32+00:00 |
Last Scan
| Scanned | 2024-11-09T20:31:32+00:00 |
| URL | https://checkurl.org/robots.txt |
| Domain IPs | 104.21.25.130, 172.67.134.64, 2606:4700:3030::6815:1982, 2606:4700:3033::ac43:8640 |
| Response IP | 104.21.25.130 |
| Found | Yes |
| Hash | 454218cc10ae61816d20095b0f99e57664a39436a9178939a7975144670b43c5 |
| SimHash | 58314e244f31 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /result.php |
| Disallow | /cgi-bin |
| Disallow | /admin |
| Disallow | /includes |
| Disallow | /user |
| Disallow | /pages |
| Disallow | /*sample |
| Disallow | /index |
| Disallow | /LICENSE |
| Disallow | /readme |
| Disallow | /README |
| Disallow | /yourl |
| Disallow | /CONTRIBUTING |
| Disallow | /composer |
| Disallow | /CHANGELOG |
| Disallow | /?nop%2F |
| Disallow | /_ |
| Disallow | /%26 |
| Disallow | /frontend/ |
| Allow | /frontend/assets/ |
| Allow | */img |
| Allow | */image |
| Allow | */*css |
| Allow | */upload |
| Allow | */cookie |
| Allow | */*js |
| Disallow | */template |
Other Records
| Field | Value |
|---|---|
| sitemap | https://checkurl.org/sitemap.xml |