Log In Sign Up

wolverhampton.cataloxy.co.uk
robots.txt

Robots Exclusion Standard data for wolverhampton.cataloxy.co.uk

Resource Scan

Scan Details

Site Domain wolverhampton.cataloxy.co.uk
Base Domain cataloxy.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-10-14T06:10:59+00:00
Next Scan 2025-10-28T06:10:59+00:00

Last Successful Scan

Scanned2025-09-06T02:26:59+00:00
URL https://wolverhampton.cataloxy.co.uk/robots.txt
Domain IPs 104.21.91.17, 172.67.207.234, 2606:4700:3031::6815:5b11, 2606:4700:3032::ac43:cfea
Response IP 172.67.207.234
Found Yes
Hash 0a11db00fc03d245a52cee617f8543a20a232607db2d9004ce8e5d596aa14d41
SimHash 4a54dd63f150

Groups

yandexdirect
mediapartners-google
yadirectfetcher

Rule Path
Allow /

yandex

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*
Disallow

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*
Disallow

Other Records

Field Value
crawl-delay 1

oai-searchbot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 3

gptbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

chatgpt-user

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 1

yahoo

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

bingbot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

msnbot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

mail.ru

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 15

*

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 3
Back to top

Other Records

Field Value
sitemap https://wolverhampton.cataloxy.co.uk/sitemaps/co.uk/wolverhampton/gen_sitemap_index.xml
Back to top

Warnings

  • `clean-param` is not a known field.
Back to top