Log In Sign Up

tugimnasio.es
robots.txt

Robots Exclusion Standard data for tugimnasio.es

Resource Scan

Scan Details

Site Domain tugimnasio.es
Base Domain tugimnasio.es
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-11-13T13:37:41+00:00
Next Scan 2024-11-20T13:37:41+00:00

Last Successful Scan

Scanned2024-11-05T13:37:00+00:00
URL https://tugimnasio.es/robots.txt
Domain IPs 85.208.102.203
Response IP 85.208.102.203
Found Yes
Hash 9076b85c14bbb501a36fd8d11dc3b1aa3078bc98a93e4349ae0bed3cb4900752
SimHash 3a47e062c693

Groups

*

Rule Path
Disallow /?q=*
Disallow /listing/*
Disallow /region/*
Disallow /lp-reviews/*

baiduspider

Rule Path
Disallow /

yahoo! slurp china

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

semrushbot
yandeximages

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

url_spider_pro

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

web image collector

Rule Path
Disallow /

xenu's link sleuth 1.1c

Rule Path
Disallow /

xenu's

Rule Path
Disallow /

zeus

Rule Path
Disallow /

zeus link scout

Rule Path
Disallow /

erocrawler

Rule Path
Disallow /

linkscan/8.1a unix

Rule Path
Disallow /

keyword density/0.9

Rule Path
Disallow /

webcopier v3.2a

Rule Path
Disallow /

webcapture 2.0

Rule Path
Disallow /

webcopier v.2.2

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

black hole

Rule Path
Disallow /

xenu\\\'s link sleuth 1.1c

Rule Path
Disallow /

xenu\\\'s

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.tugimnasio.es/sitemap.xml
Back to top

Comments

  • SPECIFIC BOTS SETTINGS
Back to top

Warnings

  • 1 invalid line.
  • `visit-time` is not a known field.
Back to top