Log In Sign Up

maxi.host
robots.txt

Robots Exclusion Standard data for maxi.host

Resource Scan

Scan Details

Site Domain maxi.host
Base Domain maxi.host
Scan Status Ok
Last Scan2024-09-19T01:23:56+00:00
Next Scan 2024-10-19T01:23:56+00:00

Last Scan

Scanned2024-09-19T01:23:56+00:00
URL https://maxi.host/robots.txt
Redirect https://www.latitude.sh/robots.txt
Redirect Domain www.latitude.sh
Redirect Base latitude.sh
Domain IPs 104.26.6.183, 104.26.7.183, 172.67.68.34, 2606:4700:20::681a:6b7, 2606:4700:20::681a:7b7, 2606:4700:20::ac43:4422
Redirect IPs 76.76.21.241, 76.76.21.93
Response IP 76.76.21.22
Found Yes
Hash ffeeb940548c70637d5536abf6c6d520785b8b685227c23592d70010f4c85a42
SimHash 69e88e99e520

Groups

*

Rule Path
Allow /api/og
Allow /dashboard$
Allow /dashboard/signup$
Disallow /dashboard/
Disallow /api/
Disallow /ip-geo-location.csv
Allow /

twitterbot

Rule Path
Allow /api/og
Back to top

Other Records

Field Value
sitemap https://www.latitude.sh/sitemap.xml
Back to top

Comments

  • Allow dynamic open graph images
  • Allow auth pages
  • Allow all other pages not disallowed above
  • Host
  • Sitemaps
  • Twitterbot doesn't seem to follow Googles robot.txt spec
  • This rule has been added to allow twitterbot to crawl images for sharing on twitter
Back to top

Warnings

  • `host` is not a known field.
Back to top