sportrx.com
robots.txt
Robots Exclusion Standard data for sportrx.com
Resource Scan
Scan Details
Site Domain | sportrx.com |
Base Domain | sportrx.com |
Scan Status | Ok |
Last Scan | 2024-09-25T07:07:43+00:00 |
Next Scan | 2024-10-25T07:07:43+00:00 |
Last Scan
Scanned | 2024-09-25T07:07:43+00:00 |
URL | https://sportrx.com/robots.txt |
Redirect | https://www.sportrx.com/robots.txt |
Redirect Domain | www.sportrx.com |
Redirect Base | sportrx.com |
Domain IPs | 151.101.130.216, 151.101.194.216, 151.101.2.216, 151.101.66.216 |
Redirect IPs | 151.101.130.216, 151.101.194.216, 151.101.2.216, 151.101.66.216 |
Response IP | 151.101.194.216 |
Found | Yes |
Hash | d13edcd3a082ee981128743198b5f2d99c3d7b854f3b98e55c5988a63fa671eb |
SimHash | ad356a2523b3 |
Groups
yandex
ichiro
moget
naverbot
baiduspider
baidu
sogou
youdao
mj12bot
goodzer
istellabot
gigabot
dotbot
seznambot
ltx71
spiderbot
opensiteexplorer
openlinkprofiler
majestic12
datagnion
sogou
ahrefs
scoutjet
changedetection
istellabot
napoveda.seznam
linkfluence
smarter
mj12bot
mojeek
naver
deusu
slack
aihitdata
wotbox
kazbt
mediatoolkit
safedns
aboundex
webmeup-crawler
orangebot
baiduspider-video
baiduspider-image
Rule | Path |
---|---|
Disallow | / |
bingbot
*
Rule | Path |
---|---|
Disallow | /404/ |
Disallow | /app/ |
Disallow | /cgi-bin/ |
Disallow | /downloader/ |
Disallow | /errors/ |
Disallow | /includes/ |
Disallow | /lib/ |
Disallow | /magento/ |
Disallow | /pkginfo/ |
Disallow | /report/ |
Disallow | /scripts/ |
Disallow | /shell/ |
Disallow | /stats/ |
Disallow | /var/ |
Disallow | /blog/search/ |
Disallow | /disabled_product/ |
Disallow | /giftvoucher/index/check |
Disallow | /privacy-policy-cookie-restriction-mode |
Disallow | /index.php/ |
Disallow | /catalog/product_compare/ |
Disallow | /catalog/category/view/ |
Disallow | /catalog/product/view/ |
Disallow | /catalogsearch/ |
Disallow | /checkout/ |
Disallow | /control/ |
Disallow | /contacts/ |
Disallow | /customer/ |
Disallow | /customize/ |
Disallow | /newsletter/ |
Disallow | /poll/ |
Disallow | /review/ |
Disallow | /sendfriend/ |
Disallow | /tag/ |
Disallow | /wishlist/ |
Disallow | /catalog/product/gallery/ |
Disallow | /search-by-prescription/results/ |
Disallow | /ccpay |
Disallow | /ccpay/ |
Disallow | /cron.php |
Disallow | /cron.sh |
Disallow | /error_log |
Disallow | /install.php |
Disallow | /LICENSE.html |
Disallow | /LICENSE.txt |
Disallow | /LICENSE_AFL.txt |
Disallow | /STATUS.txt |
Disallow | /*.php$ |
Disallow | /cdn-cgi/ |
Disallow | /shopby*?product_list_limit=* |
Disallow | /shopby*?show_full_attribute=* |
Disallow | /shopby*?product_list_dir=* |
Disallow | /shopby*?product_list_order=* |
Disallow | /shopby*?is_eligible=* |
Disallow | /shopby*%26product_list_limit%3D* |
Disallow | /shopby*%26show_full_attribute%3D* |
Disallow | /shopby*%26product_list_dir%3D* |
Disallow | /shopby*%26product_list_order%3D* |
Disallow | /shopby*%26is_eligible%3D* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
Other Records
Field | Value |
---|---|
sitemap | https://www.sportrx.com/sitemap.xml |
sitemap | https://www.sportrx.com/sportrx-blog/index.php/sitemap_index.xml |
Comments