av4u.site
robots.txt
Robots Exclusion Standard data for av4u.site
Resource Scan
Scan Details
| Site Domain | av4u.site |
| Base Domain | av4u.site |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2026-01-22T11:09:52+00:00 |
| Next Scan | 2026-02-21T11:09:52+00:00 |
Last Successful Scan
| Scanned | 2025-12-01T00:26:34+00:00 |
| URL | https://av4u.site/robots.txt |
| Domain IPs | 104.21.82.152, 172.67.158.237, 2606:4700:3030::6815:5298, 2606:4700:3032::ac43:9eed |
| Response IP | 172.67.158.237 |
| Found | Yes |
| Hash | b93624985b9b1f65df3b8542f1b9b913d30e7dee93d52c3fc537802af1950e9d |
| SimHash | 46b70b53cdd4 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
googlebot
| Rule | Path |
|---|---|
| Disallow | /tag* |
| Disallow | /reembed/ |
| Disallow | /index.php* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 40 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /tag* |
| Disallow | /reembed/ |
| Disallow | /index.php* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
*
| Rule | Path |
|---|---|
| Disallow | /tag* |
| Disallow | /reembed/ |
| Disallow | /index.php* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |
Warnings
- `content-signal` is not a known field.
Comments