finderscheapers.com
robots.txt

Robots Exclusion Standard data for finderscheapers.com

Resource Scan

Scan Details

Site Domain finderscheapers.com
Base Domain finderscheapers.com
Scan Status Ok
Last Scan2024-09-22T16:03:50+00:00
Next Scan 2024-09-29T16:03:50+00:00

Last Scan

Scanned2024-09-22T16:03:50+00:00
URL https://finderscheapers.com/robots.txt
Domain IPs 169.55.89.60
Response IP 169.55.89.60
Found Yes
Hash f7de1c6dcbc32261bf6866a0323f3bc720a532ca7f1bf3dd88dacce6520ce642
SimHash 0ab8db20ea09

Groups

*

Rule Path
Disallow /ProductPrices.aspx
Disallow /productprices.aspx
Disallow /AdsenseFrame.aspx
Disallow /adsenseframe.aspx
Disallow /Redirect.aspx
Disallow /redirect.aspx
Disallow /Email.aspx
Disallow /email.aspx
Disallow /Tracking.aspx
Disallow /SearchPrices.aspx
Disallow /cdnimages/image.aspx
Disallow /fc/cdnimages/image.aspx
Disallow /cdnimages/
Disallow /fc/cdnimages/
Disallow /OfferInfo
Disallow /offerinfo
Disallow /MobileHelper/

etaospider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

accelobot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

adbeat_bot

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-favo

Rule Path
Disallow /

baiduspider-cpro

Rule Path
Disallow /

baiduspider-ads

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

expo9

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexmedia

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandexcatalog

Rule Path
Disallow /

yandexdirect

Rule Path
Disallow /

yandexblogs

Rule Path
Disallow /

yandexnews

Rule Path
Disallow /

yandexpagechecker

Rule Path
Disallow /

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

yahoo ad monitoring

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1