bigdatawire.com
robots.txt
Robots Exclusion Standard data for bigdatawire.com
Resource Scan
Scan Details
| Site Domain | bigdatawire.com |
| Base Domain | bigdatawire.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-12-13T07:01:48+00:00 |
| Next Scan | 2025-12-27T07:01:48+00:00 |
Last Successful Scan
| Scanned | 2025-11-04T10:51:05+00:00 |
| URL | https://bigdatawire.com/robots.txt |
| Domain IPs | 104.21.51.162, 172.67.182.132, 2606:4700:3031::ac43:b684, 2606:4700:3034::6815:33a2 |
| Response IP | 104.21.51.162 |
| Found | Yes |
| Hash | 13fc2e1f3476e4193e1deea6e701102d4c701c354840a2bcca5a4dd69968232c |
| SimHash | 4920cc20c3a2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /?blackhole |
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |