startup.ai
robots.txt
Robots Exclusion Standard data for startup.ai
Resource Scan
Scan Details
| Site Domain | startup.ai |
| Base Domain | startup.ai |
| Scan Status | Ok |
| Last Scan | 2025-10-12T00:44:51+00:00 |
| Next Scan | 2025-11-11T00:44:51+00:00 |
Last Scan
| Scanned | 2025-10-12T00:44:51+00:00 |
| URL | https://startup.ai/robots.txt |
| Domain IPs | 104.21.39.166, 172.67.146.197, 2606:4700:3030::6815:27a6, 2606:4700:3032::ac43:92c5 |
| Response IP | 172.67.146.197 |
| Found | Yes |
| Hash | e2c8c27b33f38456399575e42093108a15716b64a8d13cf0e9b0a2a5cb8ab07b |
| SimHash | 6b28ca368491 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /admin/ |
| Disallow | /api/ |
| Allow | / |
| Allow | /how-it-works |
| Allow | /startup-ideas |
| Allow | /domains |
| Allow | /zero-code-ai-startup |
| Allow | /no-code-startup-builder |
| Allow | /ai-startup-idea-generator |
| Allow | /investor-inquiries |
| Allow | /startup-idea/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://startup.ai/sitemap.xml |
Comments