bidstats.uk
robots.txt
Robots Exclusion Standard data for bidstats.uk
Resource Scan
Scan Details
| Site Domain | bidstats.uk |
| Base Domain | bidstats.uk |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-12-14T01:06:12+00:00 |
| Next Scan | 2026-03-14T01:06:12+00:00 |
Last Successful Scan
| Scanned | 2025-05-11T17:10:57+00:00 |
| URL | https://bidstats.uk/robots.txt |
| Domain IPs | 20.117.224.35 |
| Response IP | 20.117.224.35 |
| Found | Yes |
| Hash | 317f1b9bffcc368572df3687f63f2ad27ed52be7405ba5721572e71329ada37a |
| SimHash | 791a5021c1b1 |
Groups
googlebot
bingbot
slurp
duckduckbot
facebot
ia_archiver
msnbot
applebot
facebookexternalhit
linkedinbot
pinterestbot
telegrambot
twitterbot
| Rule | Path |
|---|---|
| Disallow | *?*&*& |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
*
| Rule | Path |
|---|---|
| Disallow | /tenders/2020/ |
| Disallow | /tenders/2019/ |
| Disallow | /tenders/2018/ |
| Disallow | /tenders/2017/ |
| Disallow | /tenders/2016/ |
| Disallow | /tenders/2015/ |
| Disallow | *? |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Comments