pl.jobtome.com
robots.txt

Robots Exclusion Standard data for pl.jobtome.com

Resource Scan

Scan Details

Site Domain pl.jobtome.com
Base Domain jobtome.com
Scan Status Ok
Last Scan2024-04-20T12:52:25+00:00
Next Scan 2024-05-20T12:52:25+00:00

Last Scan

Scanned2024-04-20T12:52:25+00:00
URL https://pl.jobtome.com/robots.txt
Domain IPs 104.26.14.161, 104.26.15.161, 172.67.69.217
Response IP 172.67.69.217
Found Yes
Hash de957cf7b4a3df95dda12b04b1b13c04738956d9bcd22e7278663cc9d295cc6a
SimHash c302d2d4e231

Groups

*

Rule Path
Disallow /search/getjob
Disallow /i/*
Disallow /cd/*
Disallow /private/search
Disallow *fo%3D1
Disallow /feed-wel/
Disallow /ads/
Disallow *%26orgn%3D113
Disallow /api/*

mediapartners-google
adidxbot

Rule Path
Allow *

grapeshot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

baiduspider
yisouspider

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

Comments

  • &&&&&
  • &&&&&
  • @@@
  • @@@
  • @@@
  • @@@
  • @@@
  • @@@
  • @@@
  • @@@
  • @@@
  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449

Warnings

  • 4 invalid lines.