Log In Sign Up

ajanvaraus.terveystalo.com
robots.txt

Robots Exclusion Standard data for ajanvaraus.terveystalo.com

Resource Scan

Scan Details

Site Domain ajanvaraus.terveystalo.com
Base Domain terveystalo.com
Scan Status Ok
Last Scan2024-11-03T08:41:22+00:00
Next Scan 2024-12-03T08:41:22+00:00

Last Scan

Scanned2024-11-03T08:41:22+00:00
URL https://ajanvaraus.terveystalo.com/robots.txt
Domain IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash ea42938947e296f31ee363be0498e6dd785a2c058c7c74f5325f45a4b85b7ab8
SimHash ac02a913cd75

Groups

*

Rule Path
Disallow /
Allow /$
Allow /fi/$
Allow /sv/$
Allow /en/$
Back to top

Other Records

Field Value
sitemap https://ajanvaraus.terveystalo.com/sitemap.xml
Back to top

Comments

  • These rules apply to all crawlers
  • By default, disallow all URL's
  • But then DO allow the root URL
  • https://developers.google.com/search/reference/robots_txt#order-of-precedence-for-group-member-lines
  • Then white-list URL's we specifically want indexed
  • Link to sitemap
Back to top