Log In Sign Up

dalbelloboots.com
robots.txt

Robots Exclusion Standard data for dalbelloboots.com

Resource Scan

Scan Details

Site Domain dalbelloboots.com
Base Domain dalbelloboots.com
Scan Status Ok
Last Scan2026-01-13T19:57:12+00:00
Next Scan 2026-02-12T19:57:12+00:00

Last Scan

Scanned2026-01-13T19:57:12+00:00
URL https://dalbelloboots.com/robots.txt
Domain IPs 216.24.57.251, 216.24.57.7
Response IP 216.24.57.251
Found Yes
Hash 7ca6a86dfc3e6fe897b6d02f9a999855c7cc40968798285629d66ebf1628ac3f
SimHash 666e9f7004f1

Groups

*

Rule Path
Disallow *.php$
Disallow *.php?*
Disallow /*/video
Disallow /video
Disallow /*/wp-content
Disallow /wp-content
Disallow /*/api
Disallow /api
Disallow /en-ru
Disallow /ru-ru
Disallow /en-sa
Disallow /en-ue
Disallow /en-kg
Disallow /en-mx

Other Records

Field Value
crawl-delay 20

dotbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

springbot

Rule Path
Disallow /

mozilla/5.0 (compatible; mj12bot/v1.4.8; http://mj12bot.com/)

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://dalbelloboots.com/sitemap.xml
Back to top

Comments

  • Dalbello - robots.txt
  • Rules below should apply to all user-agents
  • Don't allow not existing files
  • Don't allow unused directories
  • Don't allow unused langs
  • Crawl Delay - <num> sec delay
  • Sitemaps
  • Block Bots
Back to top