bdnews24.com
robots.txt

Robots Exclusion Standard data for bdnews24.com

Resource Scan

Scan Details

Site Domain bdnews24.com
Base Domain bdnews24.com
Scan Status Ok
Last Scan4/8/2025, 3:36:45 AM
Next Scan 4/15/2025, 3:36:45 AM

Last Scan

Scanned4/8/2025, 3:36:45 AM
URL https://bdnews24.com/robots.txt
Domain IPs 104.21.57.238, 172.67.151.211, 2606:4700:3037::6815:39ee, 2606:4700:3037::ac43:97d3
Response IP 172.67.151.211
Found Yes
Hash 38c5a86ad7966e785ad0dd6640ad404e7e44272c8f85d8863eae5ba7958e69fb
SimHash 807ed452a3a2

Groups

*

Rule Path
Disallow

semrush

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

claude

Rule Path
Disallow /

open ai

Rule Path
Disallow /

Comments

  • User-agent: Ahref
  • Disallow: /