beavertonoregon.gov
robots.txt

Robots Exclusion Standard data for beavertonoregon.gov

Resource Scan

Scan Details

Site Domain beavertonoregon.gov
Base Domain beavertonoregon.gov
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-08-09T07:26:12+00:00
Next Scan 2024-11-07T07:26:12+00:00

Last Successful Scan

Scanned2024-03-20T07:16:15+00:00
URL https://beavertonoregon.gov/robots.txt
Domain IPs 40.89.242.76
Response IP 40.89.242.76
Found Yes
Hash 5b070e93aa21b3328307a100a4e880ccaedbbc5fbd7954d490086c9614860156
SimHash 045ada52e1a1

Groups

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

*
siteimprove

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

siteimprovebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

siteimprovebot-crawler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20