jobsite.co.uk
robots.txt
Robots Exclusion Standard data for jobsite.co.uk
Resource Scan
Scan Details
| Site Domain | jobsite.co.uk |
| Base Domain | jobsite.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-06-09T00:52:00+00:00 |
| Next Scan | 2024-06-16T00:52:00+00:00 |
Last Scan
| Scanned | 2024-06-09T00:52:00+00:00 |
| URL | https://www.jobsite.co.uk/robots.txt |
| Domain IPs | 104.69.172.62 |
| Response IP | 104.69.172.62 |
| Found | Yes |
| Hash | ea7def5b82e40a9b52293f035e7d6ec53b41e183f9eeaad8893ec230a65f5d71 |
| SimHash | ae85210cec7a |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | /*?page=* | - |
| Disallow | /job/ | - |
| Disallow | /tp-out | - |
| Disallow | /*%26page%3D* | this is where there is a page parameter on a facited link - its a catch all |
| Disallow | /*?WT.mc_id=* | stop aggregated jobs |
| Disallow | /*%26WT.mc_id%3D* | stop aggregated jobs |
| Disallow | /*?similarjobsmodal=* | - |
| Disallow | /*%26similarjobsmodal%3D* | - |
| Disallow | /*entryUrl%3D* | 5/10/18 RD |
| Disallow | /*entryurl%3D* | 5/10/18 RD |
| Disallow | *%3D%3D$ | - |
| Disallow | /*%3Daggregator* | - |
| Disallow | /vacancies | - |
| Disallow | /surveys/ | - |
| Disallow | /cgi-bin/ | - |
| Disallow | /trk.gif | - |
| Disallow | /*%26jbe_* | - |
| Disallow | /*?jbe_* | - |
| Disallow | /vacancy/ | - |
| Disallow | /insider/wp-content | - |
| Disallow | /home | - |
| Disallow | /social-media | - |
| Disallow | /mediacentre | - |
| Disallow | /privacy | - |
| Disallow | /faq | - |
| Disallow | /help | - |
| Disallow | /ovp_client.html | - |
| Disallow | /remind | - |
| Disallow | */sortby_* | - |
| Disallow | */demo_display_advert.cgi | - |
| Disallow | /advancedsearch | - |
| Disallow | /*?src=* | - |
| Disallow | /*%26src%3D* | - |
| Disallow | /*%26Visitor-Source%3D* | - |
| Disallow | /*?Visitor-Source=* | - |
| Disallow | /jobs/seafire | - |
| Disallow | /seafire | - |
| Disallow | /trace | - |
| Disallow | /candidate/ | - |
| Disallow | /recent_search | - |
| Disallow | /skillwordbanner | - |
| Disallow | /sectorpages/ | - |
| Disallow | /channelpages/ | - |
| Disallow | /corpages/ | - |
| Disallow | /remind | - |
| Disallow | /*salary%3D* | - |
| Disallow | /*Salary%3D* | - |
| Disallow | /*salarytypeid%3D* | - |
| Disallow | /*SalaryTypeid%3D* | - |
| Disallow | /*companytype%3D* | - |
| Disallow | /*postedwithin%3D* | - |
| Disallow | /*PostedWithin%3D* | - |
| Disallow | /*radius%3D* | - |
| Disallow | /*Radius%3D* | - |
| Disallow | /*%26q%3D* | - |
| Disallow | /*?r=* | - |
| Disallow | /*%26r%3D* | - |
| Disallow | /*?v=* | removed 30/8/17 RD |
| Disallow | /*%26v%3D* | - |
| Disallow | /*?s=* | - |
| Disallow | /*Sort%3D* | - |
| Disallow | /*sort%3D* | - |
| Disallow | /*jobtitle%3D* | - |
| Disallow | /*page%3D* | - |
| Disallow | /*Page%3D* | - |
| Disallow | /jsd/ | - |
| Disallow | /savedjobs | - |
| Disallow | /personalisation/ | - |
| Disallow | /Accessible/ | - |
| Disallow | /Authenticated/ | - |
| Disallow | /authenticated/ | - |
| Disallow | /jobs-at/jobs/in- | - |
| Disallow | /jobs-at/*/jobs/part-time/ | - |
| Disallow | /jobs-at/*/jobs/permanent/ | - |
| Disallow | /jobs-at/*/jobs/contract/ | - |
| Disallow | /jobs-at/*/jobs/temporary/ | - |
| Disallow | /jobs/permanent/ | - |
| Disallow | /account/ | - |
| Disallow | /Account/ | - |
| Disallow | /*/similarjobs/* | - |
| Disallow | /JobSeeking/ | - |
| Disallow | /*?action= | - |
| Disallow | /*%26action%3D | - |
| Disallow | /jobs/work-from-home/*/in-* | - |
| Disallow | /jobs-at/*/jobs/work-from-home/ | - |
| Disallow | /*?action= | - |
| Disallow | /*%26action%3D | - |
| Disallow | /jobs/work-from-home/*/in-* | - |
| Disallow | /membersarea/ | - |
| Disallow | /membersarea | - |
| Disallow | /headerfooter/ | - |
| Disallow | /mya/ | - |
| Disallow | /optimizely-edge/ | - |
Comments