auth0.com
robots.txt
Robots Exclusion Standard data for auth0.com
Resource Scan
Scan Details
| Site Domain | auth0.com |
| Base Domain | auth0.com |
| Scan Status | Ok |
| Last Scan | 2024-11-04T23:24:28+00:00 |
| Next Scan | 2024-11-18T23:24:28+00:00 |
Last Scan
| Scanned | 2024-11-04T23:24:28+00:00 |
| URL | https://auth0.com/robots.txt |
| Domain IPs | 104.17.254.182, 104.17.255.182, 2606:4700::6811:feb6, 2606:4700::6811:ffb6 |
| Response IP | 104.17.254.182 |
| Found | Yes |
| Hash | 6762d12841916f793b9d5c3b8c1a2655f3d6ca21289087ceace6681ae9b80f45 |
| SimHash | e889481729d2 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /docs/addons/* |
| Disallow | /terms* |
| Disallow | /docs/email-wall/* |
| Disallow | /docs/video-series/* |
| Disallow | /*wp-includes* |
| Disallow | /login/* |
| Disallow | /connect/* |
| Disallow | /docs*?*utm_source=* |
| Disallow | /learn/lp-auth0-vs-stormpath* |
| Disallow | /legal* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://auth0.com/sitemap.xml |