deepsource.io
robots.txt
Robots Exclusion Standard data for deepsource.io
Resource Scan
Scan Details
Site Domain | deepsource.io |
Base Domain | deepsource.io |
Scan Status | Ok |
Last Scan | 2024-06-06T12:13:52+00:00 |
Next Scan | 2024-06-20T12:13:52+00:00 |
Last Scan
Scanned | 2024-06-06T12:13:52+00:00 |
URL | https://deepsource.io/robots.txt |
Redirect | https://app.deepsource.com/robots.txt |
Redirect Domain | app.deepsource.com |
Redirect Base | deepsource.com |
Domain IPs | 172.66.40.212, 172.66.43.44, 2606:4700:3108::ac42:28d4, 2606:4700:3108::ac42:2b2c |
Redirect IPs | 104.26.4.129, 104.26.5.129, 172.67.74.108, 2606:4700:20::681a:481, 2606:4700:20::681a:581, 2606:4700:20::ac43:4a6c |
Response IP | 104.26.4.129 |
Found | Yes |
Hash | 40d8d309ab5ae149e43b7fa143a6f917c132a73237d97c3f759c2ef2a83f769d |
SimHash | e93c8d45c315 |
Groups
*
Rule | Path |
---|---|
Disallow | /services |
Disallow | /lp/* |
Disallow | /callback/* |
Disallow | /control-panel/* |
Disallow | /report/* |
Disallow | /health/* |
Disallow | /request-pilot-license/ |
Other Records
Field | Value |
---|---|
crawl-delay | 5 |