sparrou.net
robots.txt
Robots Exclusion Standard data for sparrou.net
Resource Scan
Scan Details
| Site Domain | sparrou.net |
| Base Domain | sparrou.net |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-09-15T00:34:36+00:00 |
| Next Scan | 2025-11-14T00:34:36+00:00 |
Last Successful Scan
| Scanned | 2025-06-23T12:12:56+00:00 |
| URL | https://sparrou.net/robots.txt |
| Domain IPs | 104.21.79.235, 172.67.150.8, 2606:4700:3036::ac43:9608, 2606:4700:3037::6815:4feb |
| Response IP | 104.21.79.235 |
| Found | Yes |
| Hash | ba2e881833e43c81e2bd4147f7611e4e7292af1a6698644a9d7c6e1877c28a87 |
| SimHash | 60d858c1c0a5 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin |
| Disallow | /wp-admin/ |
| Allow | /wp-admin/admin-ajax.php |
| Disallow | /wp-login |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-includes/ |
| Disallow | /*/attachment/ |
| Disallow | /?attachment_id* |
| Disallow | /comments/ |
| Disallow | /xmlrpc.php |
| Disallow | /trackback |
| Disallow | /*trackback |
| Disallow | /*trackback* |
| Disallow | /*/trackback |
| Allow | /feed/$ |
| Disallow | /feed/ |
| Disallow | /comments/feed/ |
| Disallow | /*/feed/$ |
| Disallow | /*/feed/rss/$ |
| Disallow | /*/trackback/$ |
| Disallow | /*/*/feed/$ |
| Disallow | /*/*/feed/rss/$ |
| Disallow | /*/*/trackback/$ |
| Disallow | /*/*/*/feed/$ |
| Disallow | /*/*/*/feed/rss/$ |
| Disallow | /*/*/*/trackback/$ |
| Allow | /*.js$ |
| Allow | /*.css$ |
| Disallow | /*.pdf$ |
| Disallow | /*blackhole |
| Disallow | /?blackhole |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://sparrou.net/sitemap.xml |
Warnings
- `clean-param` is not a known field.
Comments