jobcase.com
robots.txt

Robots Exclusion Standard data for jobcase.com

Resource Scan

Scan Details

Site Domain jobcase.com
Base Domain jobcase.com
Scan Status Ok
Last Scan2024-09-19T18:16:30+00:00
Next Scan 2024-09-26T18:16:30+00:00

Last Scan

Scanned2024-09-19T18:16:30+00:00
URL https://jobcase.com/robots.txt
Redirect https://www.jobcase.com:443/robots.txt
Redirect Domain www.jobcase.com
Redirect Base jobcase.com
Domain IPs 34.236.89.244, 44.213.243.92
Redirect IPs 104.18.26.244, 104.18.27.244, 2606:4700::6812:1af4, 2606:4700::6812:1bf4
Response IP 104.18.26.244
Found Yes
Hash 5507f902abcd5afc7549c2e4d64e4f2ba8a00c1b195477c5ff32a14a40115c99
SimHash fadcadf3d844

Groups

piplbot

Rule Path
Disallow /

*

Rule Path
Disallow /admin/
Disallow /activities/
Disallow /redirect/
Disallow /resumes/
Disallow /login
Disallow /pub/dir/jobs/
Disallow /pub/dir/profiles/
Disallow /job-postings/
Disallow /jobs/*/listing-check
Disallow /r/jobs/landing_page
Disallow /companies/
Disallow /locations/
Disallow /conversations/*/comments/new
Disallow /conversations/*/upvote
Disallow /conversations/*/report
Disallow /conversations/comments/*/report
Disallow /conversations/comments/guest/
Disallow /groups/*/members
Disallow /groups/*/details
Disallow /groups/*/report
Disallow /profiles/
Disallow /profile
Disallow /profile/*
Disallow /p/*
Disallow /registration/new
Disallow /praises/new
Disallow /reviews/new_review
Disallow /topics?q=
Disallow /groups?q=
Disallow /hiring_fairs/
Disallow /forms/*
Disallow /causal-reg
Disallow /career-coach

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
  • .well-known/apple-app-site-association should not be disallowed to support universal links for iOS 9