laptopmarket.ie
robots.txt

Robots Exclusion Standard data for laptopmarket.ie

Resource Scan

Scan Details

Site Domain laptopmarket.ie
Base Domain laptopmarket.ie
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-18T10:36:21+00:00
Next Scan 2024-11-16T10:36:21+00:00

Last Successful Scan

Scanned2024-03-29T08:34:43+00:00
URL https://laptopmarket.ie/robots.txt
Domain IPs 104.21.18.95, 172.67.181.139, 2606:4700:3034::6815:125f, 2606:4700:3036::ac43:b58b
Response IP 172.67.181.139
Found Yes
Hash d0dd2b50c328e347ed16e27facac2f1d3a54c5c61275dafc1c8d7996fd705b62
SimHash 1138d6222b1a

Groups

*

Rule Path
Disallow /?blackhole

Other Records

Field Value
crawl-delay 90

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /?s=
Disallow /search/
Disallow /wp-json/
Disallow /cart/
Disallow /wishlist/
Disallow /checkout/
Disallow /my-account/
Disallow *?filter_color=*
Disallow *?filter_size=*
Disallow *?min_price=*
Disallow *?max_price=*
Disallow /*add-to-cart%3D*

discobot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

lcc

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

mauibot

Rule Path
Disallow /

crawl.amazonbot.amazon

Rule Path
Disallow /