urlit.ru
robots.txt
Robots Exclusion Standard data for urlit.ru
Resource Scan
Scan Details
| Site Domain | urlit.ru |
| Base Domain | urlit.ru |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-10-07T16:32:05+00:00 |
| Next Scan | 2025-01-05T16:32:05+00:00 |
Last Successful Scan
| Scanned | 2024-05-18T16:29:55+00:00 |
| URL | http://urlit.ru/robots.txt |
| Redirect | https://urlit-id.ru/robots.txt |
| Redirect Domain | urlit-id.ru |
| Redirect Base | urlit-id.ru |
| Domain IPs | 2a00:15f8:a000:5:1:11:0:46ef, 2a00:15f8:a000:5:1:12:0:46ef, 2a00:15f8:a000:5:1:13:0:46ef, 2a00:15f8:a000:5:1:14:0:46ef, 90.156.201.106, 90.156.201.47, 90.156.201.83, 90.156.201.84 |
| Redirect IPs | 2a00:15f8:a000:5:1:11:8:37d2, 2a00:15f8:a000:5:1:12:8:37d2, 2a00:15f8:a000:5:1:13:8:37d2, 2a00:15f8:a000:5:1:14:8:37d2, 90.156.201.13, 90.156.201.49, 90.156.201.94, 90.156.201.97 |
| Response IP | 90.156.201.94 |
| Found | Yes |
| Hash | e051536332d1a8e678ee86b793dc0b279d8eb02b75f8a7599bee0304e2a0acbc |
| SimHash | 28d54b06c1d1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /forbidden/ |
| Disallow | /notfound/ |
| Disallow | /files/ |
| Disallow | /cart/ |
| Disallow | /order/ |
| Disallow | /eorder/ |