naspers.com
robots.txt
Robots Exclusion Standard data for naspers.com
Resource Scan
Scan Details
| Site Domain | naspers.com |
| Base Domain | naspers.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-07-04T16:31:45+00:00 |
| Next Scan | 2024-10-02T16:31:45+00:00 |
Last Successful Scan
| Scanned | 2023-05-10T13:31:39+00:00 |
| URL | https://www.naspers.com/robots.txt |
| Domain IPs | 104.20.33.28, 104.20.34.28, 172.67.1.48, 2606:4700:10::6814:211c, 2606:4700:10::6814:221c, 2606:4700:10::ac43:130 |
| Response IP | 172.67.1.48 |
| Found | Yes |
| Hash | 1918311e97e3597af2464aa39f32c868077ee6bbe909b9438f0a08d924453b6c |
| SimHash | f8118409afd2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /Development |
| Disallow | /annualreport2017 |
| Disallow | /annualresultsfy2017 |
| Disallow | /Investors-New |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.naspers.com/site-map |
Comments