pangea.ai
robots.txt
Robots Exclusion Standard data for pangea.ai
Resource Scan
Scan Details
| Site Domain | pangea.ai |
| Base Domain | pangea.ai |
| Scan Status | Ok |
| Last Scan | 2025-10-24T01:03:13+00:00 |
| Next Scan | 2025-11-23T01:03:13+00:00 |
Last Scan
| Scanned | 2025-10-24T01:03:13+00:00 |
| URL | https://pangea.ai/robots.txt |
| Domain IPs | 104.21.4.247, 172.67.187.65, 2606:4700:3033::6815:4f7, 2606:4700:3033::ac43:bb41 |
| Response IP | 104.21.4.247 |
| Found | Yes |
| Hash | f3e5c37e08f1d5d64f9a6b259cc66374004851495ae749e7f049e8fcc66dd78d |
| SimHash | 08145912c437 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /api/auth/me |
| Disallow | /cdn-cgi/speculation |
| Disallow | /*?Programming-Languages |
| Disallow | /*?field |
| Disallow | /api/proxy/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://pangea.ai/sitemap.xml |