Log In Sign Up

15mins.today
robots.txt

Robots Exclusion Standard data for 15mins.today

Resource Scan

Scan Details

Site Domain 15mins.today
Base Domain 15mins.today
Scan Status Ok
Last Scan5/23/2025, 8:54:04 AM
Next Scan 5/30/2025, 8:54:04 AM

Last Scan

Scanned5/23/2025, 8:54:04 AM
URL https://15mins.today/robots.txt
Redirect https://www.15mins.today/robots.txt
Redirect Domain www.15mins.today
Redirect Base 15mins.today
Domain IPs 104.21.30.102, 172.67.172.189, 2606:4700:3032::6815:1e66, 2606:4700:3033::ac43:acbd
Redirect IPs 104.21.30.102, 172.67.172.189, 2606:4700:3032::6815:1e66, 2606:4700:3033::ac43:acbd
Response IP 172.67.172.189
Found Yes
Hash c195246e3e463621f1802429d7a79ffe5be1d4dc64aa41fec9625bd6770711e0
SimHash ba8d6d856450

Groups

semrushbot

Rule Path
Disallow /

blackwidow

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://www.15mins.today/sitemap.xml
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /
Back to top