Log In Sign Up

hk.jobsdb.com
robots.txt

Robots Exclusion Standard data for hk.jobsdb.com

Resource Scan

Scan Details

Site Domain hk.jobsdb.com
Base Domain jobsdb.com
Scan Status Ok
Last Scan2024-11-03T23:41:28+00:00
Next Scan 2024-11-17T23:41:28+00:00

Last Scan

Scanned2024-11-03T23:41:28+00:00
URL https://hk.jobsdb.com/robots.txt
Domain IPs 104.18.39.136, 172.64.148.120, 2606:4700:4400::6812:2788, 2606:4700:4400::ac40:9478
Response IP 104.18.39.136
Found Yes
Hash 1790e463984e76586c336a0a73f524a6244802832e7091beee80f158e6e6d35e
SimHash 8216c9c0dc55

Groups

mediapartners-google
adidxbot

Rule Path
Disallow

*

Rule Path
Disallow */job/
Disallow *?returnUrl=
Disallow *?page=

linkedinbot
baiduspider
petalbot

Rule Path
Disallow /

anthropic-ai
bytespider
ccbot
diffbot
google-extended
omgili
gptbot

Rule Path
Disallow /companies
Disallow */job/

linkedinbot

Rule Path
Allow */job/
Back to top

Comments

  • robots.txt file for hk.jobsdb.com
  • Unrestricted access
  • Default directives
  • Disallowed bots
  • Exceptions
Back to top