fremantle.co.za
robots.txt
Robots Exclusion Standard data for fremantle.co.za
Resource Scan
Scan Details
| Site Domain | fremantle.co.za |
| Base Domain | fremantle.co.za |
| Scan Status | Ok |
| Last Scan | 2025-04-29T19:56:36+00:00 |
| Next Scan | 2025-05-06T19:56:36+00:00 |
Last Scan
| Scanned | 2025-04-29T19:56:36+00:00 |
| URL | http://fremantle.co.za/robots.txt |
| Domain IPs | 108.167.146.59 |
| Response IP | 108.167.146.59 |
| Found | Yes |
| Hash | dccc1ef233775543d0e2d1ed02ba80411d68ebd98be34503526eddcde4b72c8b |
| SimHash | 095459804793 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | / |
| Allow | /wp-content/uploads/ |
| Allow | /tag/ |
| Allow | /pages/ |
| Allow | /category/ |
| Allow | /listings/ |
| Allow | /places/ |
| Disallow | /cgi-bin/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-admin/ |
| Disallow | /readme.html |
| Disallow | /refer/ |
| Disallow | /author/ |
Other Records
| Field | Value |
|---|---|
| sitemap | /sitemap.xml |