agrafresh.be
robots.txt

Robots Exclusion Standard data for agrafresh.be

Resource Scan

Scan Details

Site Domain agrafresh.be
Base Domain agrafresh.be
Scan Status Ok
Last Scan2026-02-28T02:00:35+00:00
Next Scan 2026-03-07T02:00:35+00:00

Last Scan

Scanned2026-02-28T02:00:35+00:00
URL https://agrafresh.be/robots.txt
Redirect https://www.agrafresh.be/robots.txt
Redirect Domain www.agrafresh.be
Redirect Base agrafresh.be
Domain IPs 84.199.55.102
Redirect IPs 84.199.55.102
Response IP 84.199.55.102
Found Yes
Hash f72d5323beffa1b44d2e6485e8f08a9efdf7442fc2241594b0ae1f6f2782cdcd
SimHash 09681d722d9b

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/

Other Records

Field Value
sitemap https://www.agrafresh.be/sitemaps-1-sitemap.xml
sitemap https://d54c73766.access.telenet.be/en/sitemaps-1-sitemap.xml
sitemap https://d54c73766.access.telenet.be/sitemaps-1-sitemap.xml
sitemap https://d54c73766.access.telenet.be/fr/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.agrafresh.be/
  • live - don't allow web crawlers to index cpresources/ or vendor/