Log In Sign Up

womanly.nl
robots.txt

Robots Exclusion Standard data for womanly.nl

Resource Scan

Scan Details

Site Domain womanly.nl
Base Domain womanly.nl
Scan Status Ok
Last Scan2024-09-27T18:00:57+00:00
Next Scan 2024-10-04T18:00:57+00:00

Last Scan

Scanned2024-09-27T18:00:57+00:00
URL https://womanly.nl/robots.txt
Domain IPs 104.21.36.224, 172.67.200.6, 2606:4700:3031::6815:24e0, 2606:4700:3036::ac43:c806
Response IP 172.67.200.6
Found Yes
Hash e720c522171bd8c34411ae6ab2d10161e35fe925ae6206e85394a3f9a6c8ee25
SimHash 4969d8f2a533

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-content/cache/
Disallow /*?*infinite_scroll=
Disallow /*?*p=
Disallow /?s=
Disallow /page/*/?s=
Disallow /search/
Disallow *?attachment_id*
Disallow /*.pdf

gptbot
chatgpt-user
ccbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://womanly.nl/sitemap_index.xml
Back to top

Comments

  • Block WP endpoints
  • ---------------------
  • Block params
  • ---------------------
  • Block internal search
  • ---------------------
  • Block others
  • ---------------------
  • Block AI/Scrapers
  • ---------------------
Back to top