Log In Sign Up

amaseguros.com
robots.txt

Robots Exclusion Standard data for amaseguros.com

Resource Scan

Scan Details

Site Domain amaseguros.com
Base Domain amaseguros.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-18T10:07:45+00:00
Next Scan 2024-11-16T10:07:45+00:00

Last Successful Scan

Scanned2023-04-04T05:28:57+00:00
URL https://amaseguros.com/robots.txt
Domain IPs 45.60.1.125, 45.60.5.125
Response IP 45.60.1.125
Found Yes
Hash 31b53320a0615849cd22fe770845fc2665d01497713425858f5d4d304f69502d
SimHash 3a07c062d6b3

Groups

*

Rule Path
Allow /

baiduspider

Rule Path
Disallow /

yahoo! slurp china

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

archive.org_bot
semrushbot
yandeximages

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

url_spider_pro

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

web image collector

Rule Path
Disallow /

xenu's link sleuth 1.1c

Rule Path
Disallow /

xenu's

Rule Path
Disallow /

zeus

Rule Path
Disallow /

zeus link scout

Rule Path
Disallow /

erocrawler

Rule Path
Disallow /

linkscan/8.1a unix

Rule Path
Disallow /

keyword density/0.9

Rule Path
Disallow /

webcopier v3.2a

Rule Path
Disallow /

webcapture 2.0

Rule Path
Disallow /

webcopier v.2.2

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

black hole

Rule Path
Disallow /

xenu\\\'s link sleuth 1.1c

Rule Path
Disallow /

xenu\\\'s

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://amaseguros.com/sitemap.xml
sitemap https://amaseguros.com/sitemap-antiguo.xml
Back to top

Comments

  • Tratamiento de User-agents especificos
Back to top

Warnings

  • 1 invalid line.
  • `visit-time` is not a known field.
Back to top