testgroup.be
robots.txt

Robots Exclusion Standard data for testgroup.be

Resource Scan

Scan Details

Site Domain testgroup.be
Base Domain testgroup.be
Scan Status Ok
Last Scan2025-10-24T17:51:19+00:00
Next Scan 2025-11-07T17:51:19+00:00

Last Scan

Scanned2025-10-24T17:51:19+00:00
URL https://testgroup.be/robots.txt
Redirect https://www.testgroup.be/robots.txt
Redirect Domain www.testgroup.be
Redirect Base testgroup.be
Domain IPs 168.119.106.127, 2a01:4f8:c013:116e::1
Redirect IPs 168.119.106.127, 2a01:4f8:c013:116e::1
Response IP 168.119.106.127
Found Yes
Hash bad606b8e46e6212c5e85a95f9820de04c16f8f580984bd13d9383e3d3dbcbd6
SimHash 2065915767f2

Groups

*

Rule Path
Disallow /~/*
Disallow /admin/*
Disallow /csrf
Disallow /edit/*
Disallow /forms/*
Disallow /honeypot

Other Records

Field Value
sitemap https://www.testgroup.be/sitemap.xml

Comments

  • Robots file for TestGroup
  • For all user agents out there (shout out)
  • Path of static version of previous website which should not be accessible for search engines
  • Exclude admin, edit and forms
  • Sitemaps