softpedia.com
robots.txt
Robots Exclusion Standard data for softpedia.com
Resource Scan
Scan Details
| Site Domain | softpedia.com |
| Base Domain | softpedia.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-05-01T15:14:34+00:00 |
| Next Scan | 2024-05-08T15:14:34+00:00 |
Last Successful Scan
| Scanned | 2024-04-23T15:14:08+00:00 |
| URL | https://softpedia.com/robots.txt |
| Redirect | https://www.softpedia.com/robots.txt |
| Redirect Domain | www.softpedia.com |
| Redirect Base | softpedia.com |
| Domain IPs | 64.77.16.71, 64.77.16.72 |
| Redirect IPs | 104.22.12.228, 104.22.13.228, 172.67.5.104, 2606:4700:10::6816:ce4, 2606:4700:10::6816:de4, 2606:4700:10::ac43:568 |
| Response IP | 104.22.12.228 |
| Found | Yes |
| Hash | 8b15ea484c1141c4999a47ee6ff2f84aa9622bff612fc215991a16580582c389 |
| SimHash | 64dd43e8d491 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /_xaja/dltrk.php |
| Disallow | /_xaja/nrtrk.php |
| Disallow | /zd.php |
| Disallow | /yd.php |
| Disallow | /cgi-bin/ |
| Disallow | /addineyeV2.html |
| Disallow | /DARTIframe.html |
| Disallow | /doubleclick |