Log In Sign Up

themesharebd.blogspot.com
robots.txt

Robots Exclusion Standard data for themesharebd.blogspot.com

Resource Scan

Scan Details

Site Domain themesharebd.blogspot.com
Base Domain themesharebd.blogspot.com
Scan Status Ok
Last Scan2025-11-15T11:29:07+00:00
Next Scan 2025-11-22T11:29:07+00:00

Last Scan

Scanned2025-11-15T11:29:07+00:00
URL https://themesharebd.blogspot.com/robots.txt
Domain IPs 2404:6800:4003:c0f::84, 64.233.170.132
Response IP 74.125.200.132
Found Yes
Hash fecabb3a4fd80a4b5029d508363886c48dfcef705596d0e1303818263cac2551
SimHash 60109f02e520

Groups

*

Rule Path
Allow /wp-admin/admin-ajax.php
Allow /*.css$
Allow /*.js$
Allow /*.png$
Allow /*.jpg$
Allow /*.jpeg$
Allow /*.gif$
Allow /*.webp$

googlebot-image

Rule Path
Allow /wp-content/uploads/

msnbot-media

Rule Path
Allow /wp-content/uploads/

applebot

Rule Path
Allow /wp-content/uploads/

yandeximages

Rule Path
Allow /wp-content/uploads/

pinterest

Rule Path
Allow /wp-content/uploads/

baidu-image

Rule Path
Allow /wp-content/uploads/

mediapartners-google*

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

google-extended

Rule Path
Allow /

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

qwantify

Rule Path
Allow /

baiduspider

Rule Path
Allow /

sogou

Rule Path
Allow /

yandex

Rule Path
Allow /

exabot

Rule Path
Allow /

facebot

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

seznambot

Rule Path
Allow /

mojeekbot

Rule Path
Allow /

youdaobot

Rule Path
Allow /

gigabot

Rule Path
Allow /

ccbot

Rule Path
Allow /

blexbot

Rule Path
Allow /

mj12bot

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

semrushbot

Rule Path
Allow /

dotbot

Rule Path
Allow /

rogerbot

Rule Path
Allow /

baidu-image

Rule Path
Allow /

pinterest

Rule Path
Allow /

gptbot

Rule Path
Allow /

llmbot

Rule Path
Allow /

openai

Rule Path
Allow /

chatgptbot

Rule Path
Allow /

anthropic

Rule Path
Allow /

huggingface

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

neevabot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://themesharebd.blogspot.com/atom.xml
Back to top

Comments

  • Allow essential resources for proper rendering
  • ------------------------------------------
  • Images and media
  • ------------------------------------------
  • ------------------------------------------
  • Advertising and monetization bots
  • ------------------------------------------
  • ------------------------------------------
  • Global and regional search engine bots
  • ------------------------------------------
  • ------------------------------------------
  • AI and modern bots (LLM / Chatbot crawlers)
  • ------------------------------------------
  • ------------------------------------------
  • Sitemap references for all bots
  • ------------------------------------------
Back to top