login.techcrunch.com
robots.txt
Robots Exclusion Standard data for login.techcrunch.com
Resource Scan
Scan Details
| Site Domain | login.techcrunch.com |
| Base Domain | techcrunch.com |
| Scan Status | Ok |
| Last Scan | 2024-06-21T12:12:26+00:00 |
| Next Scan | 2024-07-05T12:12:26+00:00 |
Last Scan
| Scanned | 2024-06-21T12:12:26+00:00 |
| URL | https://login.techcrunch.com/robots.txt |
| Domain IPs | 106.10.248.157, 2406:2000:e4:1504::2000 |
| Response IP | 106.10.248.157 |
| Found | Yes |
| Hash | dbbe251fc22a452bb7312888d439b9d8082ecf8afa174061c51d1fb3fd6a22f7 |
| SimHash | 1989aa422bf2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /config/isp_verify_user |
| Disallow | /config/pwtoken_get |
| Disallow | /config/pwtoken_login |
| Disallow | /auth/ |
| Disallow | /myaccount/ |
| Disallow | /m |