obituaries.pahrumpvalleytimes.com
robots.txt
Robots Exclusion Standard data for obituaries.pahrumpvalleytimes.com
Resource Scan
Scan Details
Site Domain | obituaries.pahrumpvalleytimes.com |
Base Domain | pahrumpvalleytimes.com |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Couldn't connect to server. |
Last Scan | 2024-07-13T11:58:13+00:00 |
Next Scan | 2024-10-11T11:58:13+00:00 |
Last Successful Scan
Scanned | 2023-02-27T10:53:31+00:00 |
URL | https://obituaries.pahrumpvalleytimes.com/robots.txt |
Domain IPs | 35.165.100.130, 52.88.72.170, 54.186.221.41 |
Response IP | 54.186.221.41 |
Found | Yes |
Hash | 3ac1f42d9bd99b105bf87670c07571c3f725ddcbc51e17b2a111c99cff062c45 |
SimHash | 6b05b7730be7 |
Groups
*
No rules defined. All paths allowed.
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
*
Rule | Path |
---|---|
Disallow | /*?*search_type* |
*
Rule | Path |
---|---|
Disallow | /ajax/post_form/ |
*
Rule | Path |
---|---|
Disallow | /wp-admin/ |
Allow | /wp-admin/admin-ajax.php |
Warnings
- 8 invalid lines.
Comments