Log In Sign Up

amigavibes.lepodcast.fr
robots.txt

Robots Exclusion Standard data for amigavibes.lepodcast.fr

Resource Scan

Scan Details

Site Domain amigavibes.lepodcast.fr
Base Domain lepodcast.fr
Scan Status Ok
Last Scan2025-06-17T18:17:17+00:00
Next Scan 2025-07-17T18:17:17+00:00

Last Scan

Scanned2025-06-17T18:17:17+00:00
URL https://amigavibes.lepodcast.fr/robots.txt
Domain IPs 104.26.4.225, 104.26.5.225, 172.67.71.74, 2606:4700:20::681a:4e1, 2606:4700:20::681a:5e1, 2606:4700:20::ac43:474a
Response IP 104.26.5.225
Found Yes
Hash 080cdd99611baff113e7e7ab00b519fa32afcd1eef36e0733cc05d7cca524120
SimHash b6954d8d6454

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top