bidlink.net
robots.txt

Robots Exclusion Standard data for bidlink.net

Resource Scan

Scan Details

Site Domain bidlink.net
Base Domain bidlink.net
Scan Status Ok
Last Scan2024-05-15T03:27:16+00:00
Next Scan 2024-06-14T03:27:16+00:00

Last Scan

Scanned2024-05-15T03:27:16+00:00
URL https://bidlink.net/robots.txt
Redirect https://www.bidlink.net//robots.txt
Redirect Domain www.bidlink.net
Redirect Base bidlink.net
Domain IPs 35.185.67.187
Redirect IPs 35.185.67.187
Response IP 35.185.67.187
Found Yes
Hash 825ed71a9789ac83d1683076901d2d660b1c679cd644a7f6dd7bfe6d4e87e699
SimHash de164482c6a0

Groups

yandex

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /defense_contract/

Other Records

Field Value
crawl-delay 3

coccocbot-image

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

spbot

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dataforseobot/1.0

Rule Path
Disallow /

facebot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

seekrbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

my-tiny-bot

Rule Path
Disallow /