Log In Sign Up

godzinyotwarcia24.pl
robots.txt

Robots Exclusion Standard data for godzinyotwarcia24.pl

Resource Scan

Scan Details

Site Domain godzinyotwarcia24.pl
Base Domain godzinyotwarcia24.pl
Scan Status Ok
Last Scan2024-11-16T10:06:26+00:00
Next Scan 2024-11-23T10:06:26+00:00

Last Scan

Scanned2024-11-16T10:06:26+00:00
URL https://godzinyotwarcia24.pl/robots.txt
Domain IPs 49.12.73.94
Response IP 49.12.73.94
Found Yes
Hash e830e22de3a09c2737fd9c295e26b2689807e6a2dc48221f8002cce236f7c210
SimHash 625ec941b640

Groups

*

Rule Path
Disallow /zglos-blad/
Disallow /dodaj-opinie/

slurp

Rule Path
Disallow /edytuj-firme/

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Disallow /edytuj-firme/

Other Records

Field Value
crawl-delay 10

semrushbot-sa

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

semrush

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

seoscanners

Rule Path
Disallow /

xenu link sleuth

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

wget

Rule Path
Disallow /

curl

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /
Back to top

Comments

  • block AI crawlers
Back to top