caravansforsale.co.uk
robots.txt
Robots Exclusion Standard data for caravansforsale.co.uk
Resource Scan
Scan Details
| Site Domain | caravansforsale.co.uk |
| Base Domain | caravansforsale.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-06-02T02:38:01+00:00 |
| Next Scan | 2024-06-09T02:38:01+00:00 |
Last Scan
| Scanned | 2024-06-02T02:38:01+00:00 |
| URL | https://caravansforsale.co.uk/robots.txt |
| Redirect | https://www.caravansforsale.co.uk/robots.txt |
| Redirect Domain | www.caravansforsale.co.uk |
| Redirect Base | caravansforsale.co.uk |
| Domain IPs | 104.26.12.137, 104.26.13.137, 172.67.71.182, 2606:4700:20::681a:c89, 2606:4700:20::681a:d89, 2606:4700:20::ac43:47b6 |
| Redirect IPs | 104.26.12.137, 104.26.13.137, 172.67.71.182, 2606:4700:20::681a:c89, 2606:4700:20::681a:d89, 2606:4700:20::ac43:47b6 |
| Response IP | 172.67.71.182 |
| Found | Yes |
| Hash | 33618f5a0c5fd1f5d784433dc0b56dc7b5d913ffe19b8e6c7f6525236bc9c113 |
| SimHash | 3c2e185ce732 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*?sstr=* |
| Disallow | /*?ads_pa=* |
| Disallow | /print_ad.php |
| Disallow | /ads_print.php |
| Disallow | /print_ad_payment.php |
| Disallow | /print_voucher.php |
| Disallow | /print_terms.php |
| Disallow | /*?infinity=* |
| Disallow | /*report.php?* |
| Disallow | /*ads_y_* |
| Disallow | /*?ads_pid_max=* |
| Disallow | /*?perpage=* |
| Disallow | /*?tag_id=* |
| Disallow | /*?fbclid=* |
| Disallow | /*?infinity=* |
| Disallow | /*?page=ads&id* |
| Disallow | /*?ads_berth_min=0&ads_berth_max=0&page=* |
| Disallow | /*?tag=* |
| Disallow | /*?gs=* |
| Disallow | /*gclid%3D* |
| Disallow | /*?amp%3B* |
| Disallow | /*?ref=* |
| Disallow | /*op%3D* |
| Disallow | /members.php* |
| Disallow | /member/* |
| Disallow | /member.php* |
| Disallow | /*ads_i%3D* |
| Disallow | /*ads_v%3D* |
| Disallow | /*ads_pid%3D* |
| Disallow | /*list_type%3D* |
| Disallow | /*ads_tp%3D* |
| Disallow | /*ads_y%3D* |
| Disallow | /*cmonth%3D* |
| Disallow | /*cyear%3D* |
| Disallow | /*view_mode%3D* |
| Disallow | /*order_advice%3D* |
| Disallow | /*ads_maid%3D* |
| Disallow | /*ads_c%3D* |
| Disallow | /*ads_ctid%3D* |
| Disallow | /*url%3Dmaxdim* |
| Disallow | /*ads_nw%3D* |
| Disallow | /*cat_id%3D* |
| Disallow | /*sec_id%3D* |
| Disallow | /*ads_berth%3D* |
| Disallow | /*order%3D* |
| Disallow | /*qf_cat%3D* |
| Disallow | /*accfnw%3D* |
| Disallow | /*accfrl%3D* |
| Disallow | /*accfpp%3D* |
| Disallow | /*radius%3D* |
| Disallow | /*?dm_* |
| Disallow | /*?tag=* |
| Disallow | /*_test_email_lang* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.caravansforsale.co.uk/map/sitemap_index.xml |