mijnrecepten.nl
robots.txt

Robots Exclusion Standard data for mijnrecepten.nl

Resource Scan

Scan Details

Site Domain mijnrecepten.nl
Base Domain mijnrecepten.nl
Scan Status Ok
Last Scan2024-11-11T04:25:02+00:00
Next Scan 2024-11-18T04:25:02+00:00

Last Scan

Scanned2024-11-11T04:25:02+00:00
URL https://mijnrecepten.nl/robots.txt
Domain IPs 2a01:7c8:fffd:1f9:5054:ff:fedd:94a, 37.97.226.117
Response IP 37.97.226.117
Found Yes
Hash f1795dc29a90a7ddd345853f9fbc26b3439b0ead9ea338a9591a5cf97ccca352
SimHash 5220dc40c197

Groups

*

Rule Path
Disallow /cms/
Disallow /dev/
Disallow /dev2/
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://mijnrecepten.nl/sitemap.xml

Comments

  • Slow down bots
  • User-agent: *
  • Crawl-delay: 10
  • AI