Log In Sign Up

astronauts.id
robots.txt

Robots Exclusion Standard data for astronauts.id

Resource Scan

Scan Details

Site Domain astronauts.id
Base Domain astronauts.id
Scan Status Ok
Last Scan2025-09-13T15:13:17+00:00
Next Scan 2025-09-20T15:13:17+00:00

Last Scan

Scanned2025-09-13T15:13:17+00:00
URL https://astronauts.id/robots.txt
Redirect https://www.astronauts.id/robots.txt
Redirect Domain www.astronauts.id
Redirect Base astronauts.id
Domain IPs 34.36.17.255
Redirect IPs 104.20.44.43, 172.66.175.18, 2606:4700:10::6814:2c2b, 2606:4700:10::ac42:af12
Response IP 104.20.44.43
Found Yes
Hash a2c6eb7466b7d06a7d9a06f8f4e85fab8fb88ca2a81fc6dd4e15820f4a42b41a
SimHash cc6bcb00cc53

Groups

*

Rule Path
Disallow */_next/*
Disallow /blog/wp-admin/
Disallow /blog/wp-admin/*
Disallow /account
Disallow /account/*
Disallow /application/health
Disallow /cart
Disallow /d/*
Disallow /login
Disallow /login/*
Disallow /order
Disallow /order/*
Disallow /payment/*
Disallow /register
Disallow /search
Disallow /search?*

timpibot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /
Disallow /blog/*/feed
Disallow /blog/*/*/feed
Disallow /blog/*/*/feed
Disallow /blog/*/*/*/feed
Disallow /blog/*/*/?register=1
Disallow /ads.txt
Back to top

Other Records

Field Value
sitemap https://www.astronauts.id/blog/sitemap_index.xml
Back to top

Comments

  • Block all crawlers for wp admin and next public folder
Back to top

Warnings

  • 2 invalid lines.
Back to top