todopisos.es
robots.txt

Robots Exclusion Standard data for todopisos.es

Resource Scan

Scan Details

Site Domain todopisos.es
Base Domain todopisos.es
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-10-21T02:56:01+00:00
Next Scan 2024-12-20T02:56:01+00:00

Last Successful Scan

Scanned2024-08-22T21:25:31+00:00
URL https://todopisos.es/robots.txt
Redirect https://www.todopisos.es/robots.txt
Redirect Domain www.todopisos.es
Redirect Base todopisos.es
Domain IPs 5.135.40.62
Redirect IPs 5.135.40.62
Response IP 5.135.40.62
Found Yes
Hash daccc9dcbbe6879437b6fc188262fa8cd0336f144a0a74bacc7069de390bf14c
SimHash d01e5362c621

Groups

*

Rule Path
Allow /*.css*
Allow /*.png*
Allow /*.jpg*
Allow /*.gif*
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /tmp/
Disallow /*?

Other Records

Field Value
crawl-delay 10

gptbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

woorank

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

sp_auditbot

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /

oncrawl

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

cocolyzebot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

xenu

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

wget

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /

orthogaffe

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

ruxitsynthetic

Rule Path
Disallow /

bytespider

Rule Path
Disallow /