azadhindtimes.com
robots.txt

Robots Exclusion Standard data for azadhindtimes.com

Resource Scan

Scan Details

Site Domain azadhindtimes.com
Base Domain azadhindtimes.com
Scan Status Ok
Last Scan2026-01-29T07:48:05+00:00
Next Scan 2026-02-28T07:48:05+00:00

Last Scan

Scanned2026-01-29T07:48:05+00:00
URL https://azadhindtimes.com/robots.txt
Domain IPs 104.21.19.100, 172.67.185.190, 2606:4700:3031::ac43:b9be, 2606:4700:3033::6815:1364
Response IP 172.67.185.190
Found Yes
Hash d58a78c4bd745fead94d29a77a6f3d09f6560fd1524226e7429256a9aede88e1
SimHash 403ccc4066b8

Groups

*

Rule Path
Disallow /application/
Disallow /system/
Disallow /cache/
Disallow /logs/
Disallow /tmp/
Disallow /backup/
Disallow /cdn/
Disallow /vendor/

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Comments

  • =====================================================
  • 🤖 Global Rules (All Bots)
  • =====================================================
  • =====================================================
  • 🚫 Block Heavy SEO & Scraper Bots
  • =====================================================
  • =====================================================
  • 🐢 Slow Down Friendly Bots (Optional)
  • =====================================================