bubuku.com
robots.txt

Robots Exclusion Standard data for bubuku.com

Resource Scan

Scan Details

Site Domain bubuku.com
Base Domain bubuku.com
Scan Status Ok
Last Scan2024-06-01T17:52:37+00:00
Next Scan 2024-07-01T17:52:37+00:00

Last Scan

Scanned2024-06-01T17:52:37+00:00
URL https://bubuku.com/robots.txt
Domain IPs 35.204.44.59
Response IP 35.204.44.59
Found Yes
Hash 5a2a53b91cdc20a6dc765cbcd430560586d36621773cdae92d9af14f354cb480
SimHash a3560250c830

Groups

*

Rule Path
Allow /*.js$
Allow /*.css$
Allow /wp-includes/js/
Allow /wp-content/*.js
Allow /wp-includes/*.js
Allow /wp-includes/*.css
Allow /wp-content/*.css
Allow /wp-includes/js/jquery/jquery.min.js
Allow /wp-content/uploads/*
Allow /wp-admin/admin-ajax.php
Allow /wp-content/*.jpg
Allow /wp-content/*.png
Allow /wp-content/*.gif
Allow /wp-content/*.svg
Allow /wp-content/*.woff
Allow /wp-content/*.woff2
Allow /wp-content/*.font
Disallow /wp-admin/
Disallow /xmlrpc.php
Disallow /wp-login
Disallow /staging.bubuku.com/
Disallow /?utm*
Disallow /*/?s=*
Disallow /?s=*

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

compspybot
curious george
cybeye.com
docomo
exb language crawler
ezooms
flamingo_searchengine
genieo
genio
lwnutch
lexxebot
openwebindex
rediffnewsbot
seoengworldbot
scanmine
screaming frog seo spider
shopwiki
showyoubot
sosospider
wocbot
yeti
yeti
youdaobot
daumoa
gsa-crawler
libcrawl
linkdex
magpie-crawler
repparser
rogerbot
sindice-site-manager
sogou spider
sogou
woriobot
yacybot
yolinkbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

openai

Rule Path
Disallow /

chatgpt

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.bubuku.com/sitemap_index.xml

Comments

  • En condiciones normales este es el sitemap
  • Bloqueo básico para todos los bots y crawlers
  • Lista de bots que deberías permitir.
  • Lista de bots bloqueados
  • Bloquear IAs