dajar.pl
robots.txt

Robots Exclusion Standard data for dajar.pl

Resource Scan

Scan Details

Site Domain dajar.pl
Base Domain dajar.pl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-11T21:18:46+00:00
Next Scan 2024-11-09T21:18:46+00:00

Last Successful Scan

Scanned2023-03-27T08:05:37+00:00
URL https://dajar.pl/robots.txt
Domain IPs 104.26.6.62, 104.26.7.62, 172.67.69.146, 2606:4700:20::681a:63e, 2606:4700:20::681a:73e, 2606:4700:20::ac43:4592
Response IP 104.26.6.62
Found Yes
Hash 5b41530bf6c8fc32101b83885fc7ab129a153c11a7c63ecacb8705ffc8904521
SimHash d644cc098293

Groups

*

Rule Path
Disallow /product-search/
Disallow /searchengine/
Disallow /rest/
Disallow /c/1
Disallow /c/2

*

Rule Path
Disallow /*manyfilters%3Dtrue
Disallow /*limit%3D
Disallow /*dir%3D
Disallow /*order%3D

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

jobboersebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

adscanner

Rule Path
Disallow /

Comments

  • .__________________________.
  • | .___________________. |==|
  • | | ................. | | |
  • | | ::[ Dear robot ]: | | |
  • | | ::::[ be nice ]:: | | |
  • | | ::::::::::::::::: | | |
  • | | ::::::::::::::::: | | |
  • | | ::::::::::::::::: | | |
  • | | ::::::::::::::::: | | ,|
  • | !___________________! |(c|
  • !_______________________!__!
  • / \
  • / [][][][][][][][][][][][][] \
  • / [][][][][][][][][][][][][][] \
  • ( [][][][][____________][][][][] )
  • \ ------------------------------ /
  • \______________________________/