diez.md
robots.txt
Robots Exclusion Standard data for diez.md
Resource Scan
Scan Details
| Site Domain | diez.md |
| Base Domain | diez.md |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-11-03T20:34:56+00:00 |
| Next Scan | 2024-11-10T20:34:56+00:00 |
Last Successful Scan
| Scanned | 2024-10-26T06:50:48+00:00 |
| URL | https://diez.md/robots.txt |
| Domain IPs | 104.22.32.165, 104.22.33.165, 172.67.25.194, 2606:4700:10::6816:20a5, 2606:4700:10::6816:21a5, 2606:4700:10::ac43:19c2 |
| Response IP | 104.22.32.165 |
| Found | Yes |
| Hash | 4b2a0faa92e0f686a13e0d9807462a56f82e05dc939307d7617002b9c705c0bd |
| SimHash | e53bd2684fc3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | |
| Disallow | /cgi-bin/ |
| Disallow | /search |
| Disallow | /page |
| Disallow | /s?= |
| Disallow | /*paged |
| Disallow | /cauta |
| Disallow | /wp-json |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 120 |
Comments