Log In Sign Up

job-draft.jp
robots.txt

Robots Exclusion Standard data for job-draft.jp

Resource Scan

Scan Details

Site Domain job-draft.jp
Base Domain job-draft.jp
Scan Status Ok
Last Scan2024-11-08T20:00:30+00:00
Next Scan 2024-12-08T20:00:30+00:00

Last Scan

Scanned2024-11-08T20:00:30+00:00
URL https://job-draft.jp/robots.txt
Domain IPs 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132
Response IP 151.101.130.132
Found Yes
Hash 60515c403b1e38c90d727a3517bf0080c1d433f973c1cc552210dce7f09e88ad
SimHash 92946d8f5540

Groups

archive.org_bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

special_archiver

Rule Path
Disallow /

megalodon

Rule Path
Disallow /
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top