spa24.pl
robots.txt

Robots Exclusion Standard data for spa24.pl

Resource Scan

Scan Details

Site Domain spa24.pl
Base Domain spa24.pl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-04-08T20:42:04+00:00
Next Scan 2024-07-07T20:42:04+00:00

Last Successful Scan

Scanned2021-09-19T00:20:59+00:00
URL https://spa24.pl/robots.txt
Found Yes
Hash 7818a7c38498b34cc8f8af8200a22b7e5e36fd475d41bf6c486a40e5d36865b2
SimHash a25ada4263b8

Groups

*

Rule Path
Disallow /files/ajax/ajax_loader.js

*

Rule Path
Disallow /photo.php

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

twiceler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

gigabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /