Log In Sign Up

store.drjosephadel.com
robots.txt

Robots Exclusion Standard data for store.drjosephadel.com

Resource Scan

Scan Details

Site Domain store.drjosephadel.com
Base Domain drjosephadel.com
Scan Status Ok
Last Scan2026-03-24T01:39:10+00:00
Next Scan 2026-04-23T01:39:10+00:00

Last Scan

Scanned2026-03-24T01:39:10+00:00
URL https://store.drjosephadel.com/robots.txt
Domain IPs 18.156.88.174, 3.124.161.162, 3.74.190.245
Response IP 3.124.161.162
Found Yes
Hash 7af479887217fdfb2d46803437163d2344587ea714d6faf7f5c5ed3e2d2b57ee
SimHash 381d9e0369cb

Groups

*

Rule Path
Allow /
Disallow */login
Disallow */registration
Disallow *page%3D-*
Disallow *page%3D0*
Disallow *?page=-*
Disallow *?page=0*

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /
Disallow *page%3D-*
Disallow *page%3D0*

googlebot-image

Rule Path
Allow /
Disallow *page%3D-*
Disallow *page%3D0*

googlebot-mobile

Rule Path
Allow /
Disallow *page%3D-*
Disallow *page%3D0*

bingbot

Rule Path
Allow /
Disallow *page%3D-*
Disallow *page%3D0*

applebot

Rule Path
Allow /
Disallow *page%3D-*
Disallow *page%3D0*

yandexbot

Rule Path
Allow /
Disallow *page%3D-*
Disallow *page%3D0*

duckduckbot

Rule Path
Allow /
Disallow *page%3D-*
Disallow *page%3D0*
Back to top

Other Records

Field Value
sitemap https://store.drjosephadel.com/sitemap.xml
Back to top

Comments

  • High-volume SEO crawlers - Rate limit
  • AI Bot Control - Restrict training on content
  • Social Media Bots
  • Always welcome Google, Bing, and other mainstream search engines
Back to top