Log In Sign Up

cataloxy-ar.com
robots.txt

Robots Exclusion Standard data for cataloxy-ar.com

Resource Scan

Scan Details

Site Domain cataloxy-ar.com
Base Domain cataloxy-ar.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2026-01-30T16:20:43+00:00
Next Scan 2026-04-30T16:20:43+00:00

Last Successful Scan

Scanned2025-10-03T02:34:24+00:00
URL https://www.cataloxy-ar.com/robots.txt
Domain IPs 104.21.14.184, 172.67.160.36, 2606:4700:3036::ac43:a024, 2606:4700:3037::6815:eb8
Response IP 172.67.160.36
Found Yes
Hash 9975ec6f1ecef5357a910034384953a6d2b5857f8e6513be82a1b40d51aa09c1
SimHash 0a56cf63f170

Groups

yandexdirect
mediapartners-google
yadirectfetcher

Rule Path
Allow /

yandex

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*
Disallow

Other Records

Field Value
crawl-delay 1

yahoo

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

msnbot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

mail.ru

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 10

applebot

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 1

applebot-extended

Rule Path
Disallow /

*

Rule Path
Disallow /*comments.htm?*aa
Disallow /*comments.htm*comment_id%3D
Disallow /*comments.htm*del_comment%3D
Disallow /*comments.htm*confirm_send%3D
Disallow /*agreement.htm?o
Disallow /*?logout=
Disallow /*regions*?cr=
Disallow /setsubscribe
Disallow *confirmedok%3D
Disallow /board*?extend=
Disallow /*?print=
Disallow /*?*%40
Disallow /firms_rand_gold.htm*
Disallow /services_b.php*
Disallow /*%26df%5B
Disallow /*%26df%5B
Disallow /*?df%5B
Disallow /*?df%5B
Disallow /cabinet/*
Disallow /informers/?
Disallow /*?*payment
Disallow /*?change_city
Disallow /*?add=
Disallow /*%26add%3D
Disallow /end.php
Disallow /decode-encoded
Disallow /*?cataloxy
Disallow /*%26cataloxy
Disallow /feedback/?qT
Disallow *setlist%5Bdelivery%5D*
Disallow *setlist%5Bdelivery*
Disallow *setlist%5Bpriority%5D*
Disallow *setlist%5Bpriority*
Disallow /favorite.htm$
Disallow /rabota/*?pr=
Disallow /*keyword%3D
Disallow *%26txt
Disallow *?txt
Disallow /url/
Disallow /robox_notification.htm*

Other Records

Field Value
crawl-delay 3
Back to top

Warnings

  • `clean-param` is not a known field.
Back to top