Log In Sign Up

demotores.com
robots.txt

Robots Exclusion Standard data for demotores.com

Resource Scan

Scan Details

Site Domain demotores.com
Base Domain demotores.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-05-23T19:17:07+00:00
Next Scan 2024-07-22T19:17:07+00:00

Last Successful Scan

Scanned2024-03-02T17:13:52+00:00
URL https://demotores.com/robots.txt
Redirect https://www.chileautos.cl/robots.txt
Redirect Domain www.chileautos.cl
Redirect Base chileautos.cl
Domain IPs 13.227.254.10, 13.227.254.119, 13.227.254.120, 13.227.254.39
Redirect IPs 108.156.133.117, 108.156.133.19, 108.156.133.53, 108.156.133.96
Response IP 108.156.133.117
Found Yes
Hash 96f59d40d3a7ae1b8750e2f6911c4c36f0162abd2a5d877d285a11621e07274e
SimHash 03575252bb11

Groups

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

adsbot-google-mobile-apps

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

trovitbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

nutch

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

admantx

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

carsnipbot

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

gptbot

Rule Path
Allow /noticias/
Disallow /vehiculos/
Disallow /resultados?*
Disallow /resultados/?*
Disallow /vehiculos/?*
Disallow /vehiculos?*
Disallow /vehiculos/autos?*
Disallow /vehiculos/autos/?*
Disallow /vehiculos/autos-veh%C3%AF%C2%BF%C2%BDculo?*
Disallow /vehiculos/autos-veh%C3%AF%C2%BF%C2%BDculo/?*
Disallow /vehiculos/resultados?*
Disallow /vehiculos/resultados/?*
Disallow /vehiculos/detalles?*
Disallow /vehiculos/detalles/*
Disallow /_details/*
Disallow /noticias/resultados?*
Disallow /noticias/resultados/?*
Disallow /*?sort
Disallow /*?Sort
Disallow /*?q*
Disallow /*?s*
Back to top

Other Records

Field Value
sitemap https://chileautos.cl/sitemaps/chileautos/latest-editorial-details.xml
sitemap https://chileautos.cl/sitemaps/chileautos/editorial-details.xml
sitemap https://chileautos.cl/sitemaps/chileautos/editorial-listing.xml
sitemap https://chileautos.cl/sitemaps/chileautos/stock-listings.xml
sitemap https://chileautos.cl/sitemaps/chileautos/profile-pages.xml
Back to top