Log In Sign Up

maxihost.com
robots.txt

Robots Exclusion Standard data for maxihost.com

Resource Scan

Scan Details

Site Domain maxihost.com
Base Domain maxihost.com
Scan Status Ok
Last Scan2024-09-18T20:46:51+00:00
Next Scan 2024-10-18T20:46:51+00:00

Last Scan

Scanned2024-09-18T20:46:51+00:00
URL https://maxihost.com/robots.txt
Redirect https://www.latitude.sh/robots.txt
Redirect Domain www.latitude.sh
Redirect Base latitude.sh
Domain IPs 104.26.14.11, 104.26.15.11, 172.67.70.93, 2606:4700:20::681a:e0b, 2606:4700:20::681a:f0b, 2606:4700:20::ac43:465d
Redirect IPs 76.76.21.164, 76.76.21.9
Response IP 76.76.21.22
Found Yes
Hash ffeeb940548c70637d5536abf6c6d520785b8b685227c23592d70010f4c85a42
SimHash 69e88e99e520

Groups

*

Rule Path
Allow /api/og
Allow /dashboard$
Allow /dashboard/signup$
Disallow /dashboard/
Disallow /api/
Disallow /ip-geo-location.csv
Allow /

twitterbot

Rule Path
Allow /api/og
Back to top

Other Records

Field Value
sitemap https://www.latitude.sh/sitemap.xml
Back to top

Comments

  • Allow dynamic open graph images
  • Allow auth pages
  • Allow all other pages not disallowed above
  • Host
  • Sitemaps
  • Twitterbot doesn't seem to follow Googles robot.txt spec
  • This rule has been added to allow twitterbot to crawl images for sharing on twitter
Back to top

Warnings

  • `host` is not a known field.
Back to top