jober.pl
robots.txt

Robots Exclusion Standard data for jober.pl

Resource Scan

Scan Details

Site Domain jober.pl
Base Domain jober.pl
Scan Status Ok
Last Scan2026-02-04T13:18:56+00:00
Next Scan 2026-02-11T13:18:56+00:00

Last Scan

Scanned2026-02-04T13:18:56+00:00
URL https://jober.pl/robots.txt
Redirect https://www.jober.pl/robots.txt
Redirect Domain www.jober.pl
Redirect Base jober.pl
Domain IPs 104.26.0.17, 104.26.1.17, 172.67.70.170, 2606:4700:20::681a:11, 2606:4700:20::681a:111, 2606:4700:20::ac43:46aa
Redirect IPs 104.26.0.17, 104.26.1.17, 172.67.70.170, 2606:4700:20::681a:11, 2606:4700:20::681a:111, 2606:4700:20::ac43:46aa
Response IP 104.26.1.17
Found Yes
Hash b6971f47102e8f430f23693a852272cd6c46a9c0c9bc6662eb1e40d8ae0fc1d8
SimHash c942da82c093

Groups

*

Rule Path
Allow /
Disallow /jobs?*
Disallow /?*
Disallow /en/
Disallow /de/
Disallow /pl/
Disallow /pt/
Disallow /nl/
Disallow /apply/

stepstone

Rule Path
Disallow /

Comments

  • Block StepStone crawler
  • https://www.thestepstonegroup.com/english/crawler/