revistametal.com
robots.txt

Robots Exclusion Standard data for revistametal.com

Resource Scan

Scan Details

Site Domain revistametal.com
Base Domain revistametal.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-11-10T01:03:15+00:00
Next Scan 2025-02-08T01:03:15+00:00

Last Successful Scan

Scanned2023-04-19T20:41:15+00:00
URL http://revistametal.com/robots.txt
Redirect https://metalmagazine.eu/robots.txt
Redirect Domain metalmagazine.eu
Redirect Base metalmagazine.eu
Domain IPs 51.254.111.161
Redirect IPs 104.21.20.168, 172.67.193.57, 2606:4700:3030::ac43:c139, 2606:4700:3031::6815:14a8
Response IP 104.21.20.168
Found Yes
Hash 3e0aa71a1d2ac5b34f3e06a16c2a25ef155ec1613df9e6fba8daf9190af8356e
SimHash ba850d857770

Groups

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://metalmagazine.eu/sitemaps/sitemap.xml.gz

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /