theurnist.com
robots.txt
Robots Exclusion Standard data for theurnist.com
Resource Scan
Scan Details
| Site Domain | theurnist.com |
| Base Domain | theurnist.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-05-30T06:13:49+00:00 |
| Next Scan | 2024-08-28T06:13:49+00:00 |
Last Successful Scan
| Scanned | 2023-08-05T01:50:51+00:00 |
| URL | https://theurnist.com/robots.txt |
| Domain IPs | 104.21.58.252, 172.67.166.133, 2606:4700:3031::ac43:a685, 2606:4700:3035::6815:3afc |
| Response IP | 172.67.166.133 |
| Found | Yes |
| Hash | 31efb237d6824f6355fb78fefc4a0c1f1b0fc527139ba9b7619735e02027be8e |
| SimHash | 75505250c3a1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /404 |
| Disallow | /data-deletion |
| Disallow | /logout |
| Disallow | /goto |
| Disallow | /goto/ |
| Disallow | /search-article |
| Disallow | /search |
| Disallow | /tim-kiem/ |
| Disallow | /tim-kiem-truyen |
| Disallow | /w/ |
| Disallow | /sw.js |