cariloker.net
robots.txt
Robots Exclusion Standard data for cariloker.net
Resource Scan
Scan Details
| Site Domain | cariloker.net |
| Base Domain | cariloker.net |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-10-11T12:27:18+00:00 |
| Next Scan | 2025-01-09T12:27:18+00:00 |
Last Successful Scan
| Scanned | 2023-09-18T08:14:47+00:00 |
| URL | https://cariloker.net/robots.txt |
| Domain IPs | 104.21.86.68, 172.67.216.91, 2606:4700:3031::ac43:d85b, 2606:4700:3033::6815:5644 |
| Response IP | 104.21.86.68 |
| Found | Yes |
| Hash | fbf59709fddefa3490ccfaab2aaac9f7a2351d7375eae030fd55bdc98ab6d660 |
| SimHash | 6244d708d540 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | |
| Disallow | /aset/ |
| Disallow | /assets/ |
| Disallow | /page/ |
| Disallow | /proses/ |
| Disallow | /txt/ |
| Disallow | /cgi-bin/ |
| Disallow | /admin |
| Disallow | /scripts/ |
| Disallow | /styles/ |
| Disallow | /vendor/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
googlebot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /cgi-bin/ |
| Disallow | /aset/ |
| Disallow | /assets/ |
| Disallow | /page/ |
| Disallow | /proses/ |
| Disallow | /txt/ |
| Disallow | /admin |
| Disallow | /scripts/ |
| Disallow | /styles/ |
| Disallow | /vendor/ |
| Disallow | /?s=* |
| Disallow | /*.js$ |
| Disallow | /*.inc$ |
| Disallow | /*.asp$ |
| Disallow | /*.css$ |
| Disallow | /*.wmv$ |
| Disallow | /*.cgi$ |
| Disallow | /*.xhtml$ |
| Disallow | /*rurl%3D* |
| Disallow | /?MD |
| Disallow | /?DA |
| Disallow | /?SA |
| Disallow | /?NA |
Comments