Log In Sign Up

cremonacasa.it
robots.txt

Robots Exclusion Standard data for cremonacasa.it

Resource Scan

Scan Details

Site Domain cremonacasa.it
Base Domain cremonacasa.it
Scan Status Ok
Last Scan2024-09-27T07:12:21+00:00
Next Scan 2024-10-04T07:12:21+00:00

Last Scan

Scanned2024-09-27T07:12:21+00:00
URL https://www.cremonacasa.it/robots.txt
Domain IPs 52.51.232.43, 54.247.116.255
Response IP 52.51.232.43
Found Yes
Hash c52d0d255a36e2aca0be487e44cbea0df51b0cad2018556937f8c6eea52c75fe
SimHash 0a1a5547c321

Groups

gohome

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

applebot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexadnet

Rule Path
Disallow /

yandexblogs

Rule Path
Disallow /

yandexfordomain

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandeximageresizer

Rule Path
Disallow /

yandexmarket

Rule Path
Disallow /

yandexnews

Rule Path
Disallow /

yandexontodb

Rule Path
Disallow /

yandexsitelinks

Rule Path
Disallow /

yandexspravbot

Rule Path
Disallow /

yandexturbo

Rule Path
Disallow /

yandexvertis

Rule Path
Disallow /

yandexverticals

Rule Path
Disallow /

yandexvideo

Rule Path
Disallow /

yandexwebmaster

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

*

Rule Path
Allow /
Back to top

Comments

  • Blocco Yandex
  • Fine blocco Yandex
  • Blocco Semrush
  • Fine Blocco Semrush
Back to top