46job.jp
robots.txt

Robots Exclusion Standard data for 46job.jp

Resource Scan

Scan Details

Site Domain 46job.jp
Base Domain 46job.jp
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-05-10T13:00:52+00:00
Next Scan 2024-08-08T13:00:52+00:00

Last Successful Scan

Scanned2023-09-21T10:06:57+00:00
URL https://46job.jp/robots.txt
Domain IPs 13.33.88.21, 13.33.88.51, 13.33.88.65, 13.33.88.91
Response IP 13.33.88.51
Found Yes
Hash 9ac7e7d85217fae4841a948a5ea015ee19b98a00984798f6e4685217889a01d4
SimHash 535ecc52709a

Groups

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 110

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60