ceresciudad.com
robots.txt

Robots Exclusion Standard data for ceresciudad.com

Resource Scan

Scan Details

Site Domain ceresciudad.com
Base Domain ceresciudad.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-11T06:10:35+00:00
Next Scan 2026-02-09T06:10:35+00:00

Last Successful Scan

Scanned2025-09-18T08:13:02+00:00
URL https://ceresciudad.com/robots.txt
Domain IPs 2a02:4780:15:377c:1b25:ff99:a207:e6d4, 2a02:4780:39:5a56:ad1:3403:4916:8102, 84.32.84.154, 84.32.84.183
Response IP 77.37.66.39
Found Yes
Hash ede481cfd7f7d684504ce98ebca1e64c5188db4cb6d7bae955c24cf4a6c700c6
SimHash e1d87d9acaf5

Groups

*

Rule Path
Allow /wp-content/uploads/
Allow /feed/$
Disallow /wp-
Disallow /wp-content/
Disallow /trackback/
Disallow /wp-admin/
Disallow /?s=
Disallow /search
Disallow /archives/
Disallow /index.php
Disallow /*?
Disallow /*.php$
Disallow /*.js$
Disallow /*.inc$
Disallow /*.css$
Disallow */feed/
Disallow */trackback/
Disallow /page/
Disallow /tag/
Disallow /category/
Disallow /music/
Disallow /maximafm/
Disallow /gilberto/
Disallow /*.sql$
Disallow /*.tgz$
Disallow /*.gz$
Disallow /*.tar$
Disallow /*.svn$
Disallow /*.cgi$
Disallow /*.inc$
Disallow /*.wmv$
Disallow /*.xhtml$

googlebot
googlebot-image

Rule Path
Disallow /wp-includes/
Disallow /music/
Disallow /maximafm/
Disallow /gilberto/
Disallow /wp-admin/
Allow /wp-content/uploads/

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

msiecrawler

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

libwww

Rule Path
Disallow /

orthogaffe

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

Other Records

Field Value
sitemap https://ceresciudad.com/sitemap_index.xml

Comments

  • No indexar copias de seguridad
  • Reglas para bots conocidos
  • Ralentizamos algunos bots que se suelen volver locos
  • Bloqueo de bots y crawlers poco utiles