gravitas.ai
robots.txt

Robots Exclusion Standard data for gravitas.ai

Resource Scan

Scan Details

Site Domain gravitas.ai
Base Domain gravitas.ai
Scan Status Ok
Last Scan2025-05-28T18:35:36+00:00
Next Scan 2025-06-27T18:35:36+00:00

Last Scan

Scanned2025-05-28T18:35:36+00:00
URL https://gravitas.ai/robots.txt
Domain IPs 2001:8d8:100f:f000::234, 217.160.0.81
Response IP 217.160.0.81
Found Yes
Hash 9250a550806b509dab0c7fc11f307f7b888b5c3906255dbef5212f7afe0cb2ae
SimHash 50b4a3576ff2

Groups

*

Rule Path
Disallow /api/
Disallow /cgi-bin/
Disallow /images/

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://gravitas.ai/sitemap.xml

Comments

  • Allow all robots to visit all files
  • Disallow specific directories
  • Crawl-delay directive for specific bots (in seconds)
  • Sitemap