lectorhub.j5z.xyz
robots.txt

Robots Exclusion Standard data for lectorhub.j5z.xyz

Resource Scan

Scan Details

Site Domain lectorhub.j5z.xyz
Base Domain j5z.xyz
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-25T05:28:15+00:00
Next Scan 2025-10-09T05:28:15+00:00

Last Successful Scan

Scanned2025-09-16T04:51:16+00:00
URL https://lectorhub.j5z.xyz/robots.txt
Domain IPs 104.21.49.21, 172.67.188.199, 2606:4700:3031::6815:3115, 2606:4700:3036::ac43:bcc7
Response IP 104.21.49.21
Found Yes
Hash c7b2d5d4767524bde61f57bf51f314a9a4992bc02a7e7ae8eb1f7d68cff05607
SimHash d4925c716d88

Groups

*

Rule Path
Allow /
Allow /manga/
Allow /manhwa/
Allow /series/
Allow /chapter/
Allow /search/
Allow /genre/
Allow /category/
Allow /latest/
Allow /popular/
Allow /completed/
Allow /ongoing/
Allow /css/
Allow /js/
Allow /images/
Allow /img/
Allow /assets/
Allow /fonts/
Allow /static/
Allow /media/
Allow /uploads/
Allow /covers/
Allow /thumbnails/
Disallow /admin/
Disallow /administrator/
Disallow /panel/
Disallow /dashboard/
Disallow /login/
Disallow /auth/
Disallow /register/
Disallow /signup/
Disallow /user/
Disallow /profile/
Disallow /account/
Disallow /settings/
Disallow /api/
Disallow /config/
Disallow /database/
Disallow /db/
Disallow /backup/
Disallow /backups/
Disallow /logs/
Disallow /log/
Disallow /temp/
Disallow /tmp/
Disallow /cache/
Disallow /vendor/
Disallow /node_modules/
Disallow /.git/
Disallow /.env
Disallow /composer.json
Disallow /composer.lock
Disallow /package.json
Disallow /package-lock.json
Disallow /webpack.config.js
Disallow /gulpfile.js
Disallow /Gruntfile.js
Disallow /*.log$
Disallow /*.sql$
Disallow /*.zip$
Disallow /*.tar$
Disallow /*.gz$
Disallow /*.bak$
Disallow /error_log
Disallow /access_log
Disallow /*?print=*
Disallow /*?share=*
Disallow /*?utm_*
Disallow /*?ref=*
Disallow /*?source=*
Disallow /*?fbclid=*
Disallow /*?gclid=*
Disallow /*?sort=date
Disallow /*?page=*&sort=*
Disallow /search?*&*
Disallow /*?action=*

facebookexternalhit

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

twitterbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

discordbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

whatsapp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

telegrambot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

slurp

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

duckduckbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

seznambot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
crawl-delay 1

Other Records

Field Value
sitemap https://lectorhub.j5z.xyz/sitemap_index.xml
sitemap https://lectorhub.j5z.xyz/page-sitemap.xml
sitemap https://lectorhub.j5z.xyz/wp-manga-sitemap.xml
sitemap https://lectorhub.j5z.xyz/sitemap-manhwa.xml
sitemap https://lectorhub.j5z.xyz/sitemap-chapters.xml

Comments

  • robots.txt para LectorHub - Plataforma de Lectura JeazScans
  • Configuración optimizada para sitio de manga/manhwa
  • Permitir indexación de contenido público importante
  • Permitir recursos estáticos
  • Bloquear áreas administrativas y privadas
  • Bloquear archivos y directorios del sistema
  • Bloquear archivos específicos
  • Bloquear URLs con parámetros problemáticos para SEO
  • Permitir bots específicos de redes sociales para vista previa
  • Configuración para bots principales
  • Bloquear bots agresivos o no deseados
  • Configuración general de crawl-delay
  • Ubicación del sitemap