Log In Sign Up

reading.cataloxy.co.uk
robots.txt

Robots Exclusion Standard data for reading.cataloxy.co.uk

Resource Scan

Scan Details

Site Domain reading.cataloxy.co.uk
Base Domain cataloxy.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-10-04T22:49:09+00:00
Next Scan 2025-10-11T22:49:09+00:00

Last Successful Scan

Scanned2025-09-03T22:48:32+00:00
URL https://reading.cataloxy.co.uk/robots.txt
Domain IPs 104.21.91.17, 172.67.207.234, 2606:4700:3031::6815:5b11, 2606:4700:3032::ac43:cfea
Response IP 104.21.91.17
Found Yes
Hash 72dd72b0c5c2ff2579aa52c6523fbbf8743b618d05161e9c1904b02532d18b9c
SimHash 4a54dc63f150

Groups

yandexdirect
mediapartners-google
yadirectfetcher

Rule Path
Allow /

yandex

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*
Disallow

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*
Disallow

Other Records

Field Value
crawl-delay 1

oai-searchbot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 3

gptbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

chatgpt-user

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 1

yahoo

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

bingbot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

msnbot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

mail.ru

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 15

*

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 3
Back to top

Other Records

Field Value
sitemap https://reading.cataloxy.co.uk/sitemaps/co.uk/reading/gen_sitemap_index.xml
Back to top

Warnings

  • `clean-param` is not a known field.
Back to top