unit21.ai
robots.txt
Robots Exclusion Standard data for unit21.ai
Resource Scan
Scan Details
| Site Domain | unit21.ai |
| Base Domain | unit21.ai |
| Scan Status | Ok |
| Last Scan | 2024-08-27T00:20:20+00:00 |
| Next Scan | 2024-09-26T00:20:20+00:00 |
Last Scan
| Scanned | 2024-08-27T00:20:20+00:00 |
| URL | https://unit21.ai/robots.txt |
| Redirect | https://www.unit21.ai/robots.txt |
| Redirect Domain | www.unit21.ai |
| Redirect Base | unit21.ai |
| Domain IPs | 75.2.70.75, 99.83.190.102 |
| Redirect IPs | 52.197.0.54, 52.199.221.217, 54.178.223.218 |
| Response IP | 54.178.223.218 |
| Found | Yes |
| Hash | 7d2c06119904f95fd17b088b87c62e77f71b5c1ca60d7c77c7d8d91224046359 |
| SimHash | 6b01f941cd93 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /styleguide |
| Disallow | /thank-you |
| Disallow | /meeting-thank-you |
| Disallow | /zdeleted-sections/ |
| Disallow | /api-dev-careers |
| Disallow | /api-dev-job |
| Disallow | /terms-of-service/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.unit21.ai/sitemap.xml |
Comments