Log In Sign Up

metalmagazine.eu
robots.txt

Robots Exclusion Standard data for metalmagazine.eu

Resource Scan

Scan Details

Site Domain metalmagazine.eu
Base Domain metalmagazine.eu
Scan Status Ok
Last Scan2024-11-14T22:12:04+00:00
Next Scan 2024-11-21T22:12:04+00:00

Last Scan

Scanned2024-11-14T22:12:04+00:00
URL https://metalmagazine.eu/robots.txt
Domain IPs 104.21.20.168, 172.67.193.57, 2606:4700:3030::ac43:c139, 2606:4700:3031::6815:14a8
Response IP 104.21.20.168
Found Yes
Hash 6d8fa87697919762906df6795ab722de895e3100d007de11b2654e5aeaea066b
SimHash aaa50d2de4d0

Groups

*

Rule Path
Disallow /assets/
Disallow /files/
Disallow /*?id=
Disallow /*?fbclid=
Disallow /search
Disallow /es/search
Disallow /en/search
Back to top

Other Records

Field Value
sitemap https://metalmagazine.eu/sitemap.xml.gz
sitemap https://metalmagazine.eu/posts.en.sitemap.xml.gz
sitemap https://metalmagazine.eu/posts.es.sitemap.xml.gz
sitemap https://metalmagazine.eu/categories.es.sitemap.xml.gz
sitemap https://metalmagazine.eu/categories.en.sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top