Log In Sign Up

gn-app.com
robots.txt

Robots Exclusion Standard data for gn-app.com

Resource Scan

Scan Details

Site Domain gn-app.com
Base Domain gn-app.com
Scan Status Ok
Last Scan2025-11-20T12:26:47+00:00
Next Scan 2025-11-27T12:26:47+00:00

Last Scan

Scanned2025-11-20T12:26:47+00:00
URL https://gn-app.com/robots.txt
Redirect https://www.gn-app.com/robots.txt
Redirect Domain www.gn-app.com
Redirect Base gn-app.com
Domain IPs 216.150.1.1, 216.150.1.193
Redirect IPs 216.150.1.65, 216.150.16.1
Response IP 216.150.1.129
Found Yes
Hash 81caa26bf86e1763d1aecdcc79d235917ad598e74a39078b25ee579e959d1dd1
SimHash 24a53f1601e7

Groups

*

Rule Path
Allow /
Allow /api/sitemap
Allow /api/robots
Disallow /api/
Disallow /_next/
Disallow /__/auth/
Disallow /sw.js
Disallow /firebase-messaging-sw.js
Disallow /workbox-*.js
Back to top

Other Records

Field Value
sitemap https://www.gn-app.com/api/sitemap
Back to top

Comments

  • Robots.txt
  • Allow all crawlers to access the site
  • Sitemap location - helps search engines find all pages efficiently
  • This includes hreflang information for international SEO
  • Allow sitemap and robots endpoint to be crawled
  • Note: /robots.txt is rewritten to /api/robots internally, but this is transparent to crawlers
  • Disallow other API endpoints to reduce unnecessary crawl load
  • Disallow crawling of internal Next.js files
  • Disallow crawling of service worker files
Back to top