beardeddragon.org
robots.txt

Robots Exclusion Standard data for beardeddragon.org

Resource Scan

Scan Details

Site Domain beardeddragon.org
Base Domain beardeddragon.org
Scan Status Ok
Last Scan2024-06-16T04:03:37+00:00
Next Scan 2024-06-23T04:03:37+00:00

Last Scan

Scanned2024-06-16T04:03:37+00:00
URL https://beardeddragon.org/robots.txt
Redirect https://www.beardeddragon.org/robots.txt
Redirect Domain www.beardeddragon.org
Redirect Base beardeddragon.org
Domain IPs 104.26.10.46, 104.26.11.46, 172.67.75.62, 2606:4700:20::681a:a2e, 2606:4700:20::681a:b2e, 2606:4700:20::ac43:4b3e
Redirect IPs 104.26.10.46, 104.26.11.46, 172.67.75.62, 2606:4700:20::681a:a2e, 2606:4700:20::681a:b2e, 2606:4700:20::ac43:4b3e
Response IP 172.67.75.62
Found Yes
Hash ab4c20be6dbf4a2ab3705ffc78d6b2f6ae971ceb33030d5c0afe92866c8926b6
SimHash 5a3cdd40a602

Groups

omgili

Rule Path
Disallow /

*

Rule Path
Disallow /home/

Comments

  • User-agent: AhrefsBot
  • Disallow: /
  • User-agent: SemrushBot
  • Disallow: /
  • User-agent: SemrushBot-SA
  • Disallow: /
  • User-agent: PetalBot
  • Disallow: /
  • User-agent: MJ12Bot
  • Disallow: /
  • User-agent: BLEXBot
  • Disallow: /
  • User-agent: MegaIndex
  • Disallow: /
  • User-agent: Amazonbot # Amazon's user agent
  • Disallow: /
  • User-agent: Screaming Frog SEO Spider
  • Disallow: /
  • Crawl-delay: 10