moates.net
robots.txt
Robots Exclusion Standard data for moates.net
Resource Scan
Scan Details
| Site Domain | moates.net |
| Base Domain | moates.net |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-09-07T22:53:07+00:00 |
| Next Scan | 2024-12-06T22:53:07+00:00 |
Last Successful Scan
| Scanned | 2023-01-19T12:23:19+00:00 |
| URL | https://moates.net/robots.txt |
| Domain IPs | 104.254.53.130 |
| Response IP | 104.254.53.130 |
| Found | Yes |
| Hash | fec4b7d9ee3e6e611c056a0a49802dedb5f69c7c8e1db0ad405c00903248b5ed |
| SimHash | 0366da12c1b3 |
Groups
mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | / |
googlebot
| Rule | Path |
|---|---|
| Disallow | /*.PDF$ |
| Disallow | /*.pdf$ |
| Disallow | /*.MOV |
| Disallow | /*.mov |
| Disallow | /*.AVI |
| Disallow | /*.avi |
| Disallow | /DSCN*.htm |
exabot
| Rule | Path |
|---|---|
| Disallow | /*.PDF$ |
| Disallow | /*.pdf$ |
| Disallow | /*.MOV$ |
| Disallow | /*.mov$ |
| Disallow | /*.AVI$ |
| Disallow | /*.avi$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3600 |
*
| Rule | Path |
|---|---|
| Disallow | |
| Disallow |
Warnings
- 7 invalid lines.
Comments