autopos.es
robots.txt

Robots Exclusion Standard data for autopos.es

Resource Scan

Scan Details

Site Domain autopos.es
Base Domain autopos.es
Scan Status Ok
Last Scan2024-11-03T11:01:28+00:00
Next Scan 2024-12-03T11:01:28+00:00

Last Scan

Scanned2024-11-03T11:01:28+00:00
URL https://autopos.es/robots.txt
Domain IPs 162.159.136.54, 162.159.137.54
Response IP 162.159.137.54
Found Yes
Hash 40c490f5f2f6c86f337713db8c07cf467afa8cafb9ae2d8284caad396c1b3917
SimHash 719c9043dea4

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Allow /wp-includes/*.js
Allow /wp-includes/*.css
Disallow *trackback
Disallow /feed/
Disallow */feed/
Disallow /?s=
Disallow /search
Disallow /tag/*
Disallow /tag/
Disallow /login/
Disallow /acceso-autopos/
Disallow /talleres/robots.txt
Disallow /posventa-camion/feeds

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path
Disallow /

googlebot-image

Rule Path
Allow /

seokicks

Rule Path
Disallow /

mj12bot
ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

adsbot/3.1

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

cncdialer

Rule Path
Disallow /

maxthon

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Disallow /sites/default/files/

Other Records

Field Value
crawl-delay 10

rogerbot/1.2

Rule Path
Disallow /sites/default/files/

Other Records

Field Value
crawl-delay 10

rogerbot/1.2

Rule Path
Disallow /sites/default/files/

rogerbot

Rule Path
Disallow /sites/default/files/
Disallow /*/keywords/*

Other Records

Field Value
crawl-delay 10

semrushbot
semrushbot-ba
dotbot

Rule Path
Disallow /sites/default/files/

Other Records

Field Value
crawl-delay 30

bomborabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

y!j-asr/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

gigabot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

spbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 40

the knowledge ai

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

g-i-g-a-b-o-t

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

wget

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

npbot

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

gsa-crawler

Rule Path
Disallow /

panscient

Rule Path
Disallow /

coccocbot-web/1.0

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

bubing

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

ccbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

Other Records

Field Value
sitemap https://autopos.es/sitemap_index.xml
sitemap https://autopos.es/news-sitemap.xml

Comments

  • Google Images
  • agrego SEOkicks por alto consumo de peticiones por indexación.

Warnings

  • `crawl-limit` is not a known field.