pfjones.co.uk
robots.txt

Robots Exclusion Standard data for pfjones.co.uk

Resource Scan

Scan Details

Site Domain pfjones.co.uk
Base Domain pfjones.co.uk
Scan Status Ok
Last Scan2025-08-08T05:53:28+00:00
Next Scan 2025-09-07T05:53:28+00:00

Last Scan

Scanned2025-08-08T05:53:28+00:00
URL https://pfjones.co.uk/robots.txt
Domain IPs 192.124.249.15
Response IP 192.124.249.15
Found Yes
Hash 68a270d2da1150d6a9c9f53f602cdae9fd3a49dcca089fe2566b636dcba856a3
SimHash c00ffaf7c657

Groups

*

Rule Path
Disallow /app/
Disallow /support/server.php?*
Disallow /store_closed.html
Disallow /?search_performed
Disallow /*dispatch%3Dproducts.search*
Allow /js/

googlebot

Rule Path
Allow .js
Allow .css

googlebot
slurp
msnbot
mediapartners-google*
googlebot-image
yahoo-mmcrawler
ia_archiver

Rule Path
Disallow

bingbot

Rule Path
Disallow /*?*items_per_page=
Disallow /*?*sort_order=
Disallow /*?*sort_by=
Disallow /*?*layout=
Disallow /*?*target=
Disallow /*?*page_id=

Other Records

Field Value
crawl-delay 15

sistrix crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

Comments

  • Block SISTRIX