Log In Sign Up

reisleven.nl
robots.txt

Robots Exclusion Standard data for reisleven.nl

Resource Scan

Scan Details

Site Domain reisleven.nl
Base Domain reisleven.nl
Scan Status Ok
Last Scan2024-11-15T09:16:45+00:00
Next Scan 2024-11-22T09:16:45+00:00

Last Scan

Scanned2024-11-15T09:16:45+00:00
URL https://reisleven.nl/robots.txt
Domain IPs 104.21.46.92, 172.67.137.82, 2606:4700:3030::6815:2e5c, 2606:4700:3031::ac43:8952
Response IP 104.21.46.92
Found Yes
Hash 886bc52a7ffb835f4c2671233c7b652751e1f2bfe7eadf252f76059faf450942
SimHash 4969d8f2a132

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-content/cache/
Disallow /*?*infinite_scroll=
Disallow /?s=
Disallow /page/*/?s=
Disallow /search/
Disallow *?attachment_id*
Disallow /*.pdf

gptbot
chatgpt-user
ccbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://reisleven.nl/sitemap_index.xml
Back to top

Comments

  • Block WP endpoints
  • ---------------------
  • Block params
  • ---------------------
  • Block internal search
  • ---------------------
  • Block others
  • ---------------------
  • Block AI/Scrapers
  • ---------------------
Back to top