noclegi-kamienica-lublin.pl
robots.txt

Robots Exclusion Standard data for noclegi-kamienica-lublin.pl

Resource Scan

Scan Details

Site Domain noclegi-kamienica-lublin.pl
Base Domain noclegi-kamienica-lublin.pl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-04-04T12:20:59+00:00
Next Scan 2024-07-03T12:20:59+00:00

Last Successful Scan

Scanned2023-06-05T16:29:11+00:00
URL http://noclegi-kamienica-lublin.pl/robots.txt
Domain IPs 193.222.135.84
Response IP 193.222.135.84
Found Yes
Hash c8994e31ee38ffcb5289c2013a2ccddb856c14d1248bb3f785c47d00697f8d35
SimHash ec164a5026d8

Groups

*

Rule Path
Disallow /files/ajax/ajax_loader.js
Disallow /*?geo=
Disallow /*?language=
Disallow /*?command=
Disallow /plugin/Szukaj/
Disallow /plugin/_async_Szukaj/
Disallow /plugin/_async_Rezerwacja/
Disallow /plugin/WspolnyRequest/
Disallow /plugin/tracking/
Disallow /plugin/_obiekty_Datepicker/
Disallow /plugin/_obiekty_Trendy/
Disallow /plugin/_obiekty_NajnowszeOferty/
Disallow /plugin/SocialMedia/
Disallow /rezerwuj/*/opinie/

*

Rule Path
Disallow /photo.php

twiceler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

gigabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

pustkibot

Rule Path
Disallow /

crawler_eb_germany_2

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

whatsapp

Rule Path
Disallow /

sirdatabot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

konturbot

Rule Path
Disallow /

uipbot

Rule Path
Disallow /

gg peekbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

webgains-bot

Rule Path
Disallow /

mindupbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

gozlebot

Rule Path
Disallow /

hbbtv

Rule Path
Disallow /