commontimes.com.au
robots.txt
Robots Exclusion Standard data for commontimes.com.au
Resource Scan
Scan Details
| Site Domain | commontimes.com.au |
| Base Domain | commontimes.com.au |
| Scan Status | Ok |
| Last Scan | 2024-11-01T13:22:51+00:00 |
| Next Scan | 2024-11-08T13:22:51+00:00 |
Last Scan
| Scanned | 2024-11-01T13:22:51+00:00 |
| URL | https://commontimes.com.au/robots.txt |
| Domain IPs | 108.156.133.102, 108.156.133.111, 108.156.133.112, 108.156.133.74 |
| Response IP | 108.156.133.112 |
| Found | Yes |
| Hash | 2f52a42fd712ee097eeb615c4d167fde047f5747d1c710a0624dfe5591b883c2 |
| SimHash | c528ee6e6792 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /assets |
| Disallow | /cache |
| Disallow | /sources |
| Disallow | /api |
| Disallow | /script_backups |
| Disallow | /updates |
| Disallow | /install |
| Disallow | /admincp |
| Disallow | /admin-panel |
| Disallow | *?link1=timeline&u=*&type=car-shows |
| Disallow | *?link1=timeline&u=*&type=products |
| Disallow | *?link1=timeline&u=*&type=properties |
| Disallow | *?link1=timeline&u=*&type=news |
| Disallow | *?link1=timeline&u=blog |