archijob.co.il
robots.txt

Robots Exclusion Standard data for archijob.co.il

Resource Scan

Scan Details

Site Domain archijob.co.il
Base Domain archijob.co.il
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-02-08T21:13:31+00:00
Next Scan 2026-05-09T21:13:31+00:00

Last Successful Scan

Scanned2024-07-18T17:21:31+00:00
URL https://archijob.co.il/robots.txt
Redirect https://www.archijob.co.il/robots.txt
Redirect Domain www.archijob.co.il
Redirect Base archijob.co.il
Domain IPs 104.26.2.189, 104.26.3.189, 172.67.68.145, 2606:4700:20::681a:2bd, 2606:4700:20::681a:3bd, 2606:4700:20::ac43:4491
Redirect IPs 104.26.2.189, 104.26.3.189, 172.67.68.145, 2606:4700:20::681a:2bd, 2606:4700:20::681a:3bd, 2606:4700:20::ac43:4491
Response IP 172.67.68.145
Found Yes
Hash 8648c544ec94607e0599d1e529c833c49f3899b98e1f27fb41250be8f169db53
SimHash 4f154a65e3d9

Groups

googlebot-image

Rule Path
Disallow /archilist/up/board/

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

serpstatbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

Comments

  • http://www.apple.com/go/applebot
  • Disallow: /
  • Yandex
  • Disallow: /
  • Yandexbot
  • Disallow: /
  • BLEXBot
  • MJ12bot
  • serpstatbot
  • Bingbot
  • BingPreview
  • AdIdxBot