deepsource.io
robots.txt
Robots Exclusion Standard data for deepsource.io
Resource Scan
Scan Details
| Site Domain | deepsource.io |
| Base Domain | deepsource.io |
| Scan Status | Ok |
| Last Scan | 2024-06-20T12:14:00+00:00 |
| Next Scan | 2024-07-04T12:14:00+00:00 |
Last Scan
| Scanned | 2024-06-20T12:14:00+00:00 |
| URL | https://deepsource.io/robots.txt |
| Redirect | https://app.deepsource.com/robots.txt |
| Redirect Domain | app.deepsource.com |
| Redirect Base | deepsource.com |
| Domain IPs | 172.66.40.212, 172.66.43.44, 2606:4700:3108::ac42:28d4, 2606:4700:3108::ac42:2b2c |
| Redirect IPs | 104.26.4.129, 104.26.5.129, 172.67.74.108, 2606:4700:20::681a:481, 2606:4700:20::681a:581, 2606:4700:20::ac43:4a6c |
| Response IP | 104.26.4.129 |
| Found | Yes |
| Hash | 40d8d309ab5ae149e43b7fa143a6f917c132a73237d97c3f759c2ef2a83f769d |
| SimHash | e93c8d45c315 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /services |
| Disallow | /lp/* |
| Disallow | /callback/* |
| Disallow | /control-panel/* |
| Disallow | /report/* |
| Disallow | /health/* |
| Disallow | /request-pilot-license/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |