lantalk.net
robots.txt

Robots Exclusion Standard data for lantalk.net

Resource Scan

Scan Details

Site Domain lantalk.net
Base Domain lantalk.net
Scan Status Ok
Last Scan2024-11-12T05:08:31+00:00
Next Scan 2024-11-19T05:08:31+00:00

Last Scan

Scanned2024-11-12T05:08:31+00:00
URL http://lantalk.net/robots.txt
Redirect http://www.lantalk.net/robots.txt
Redirect Domain www.lantalk.net
Redirect Base lantalk.net
Domain IPs 18.169.55.149
Redirect IPs 18.169.55.149
Response IP 18.169.55.149
Found Yes
Hash c7aa83182542da614d4a618cde418e3c551b38314be249bdbb5303b454f49bf1
SimHash 06568b91e237

Groups

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

seoengbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

seomoz

Rule Path
Disallow /

geliyoobot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

acoon

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-favo

Rule Path
Disallow /

baiduspider-cpro

Rule Path
Disallow /

baiduspider-ads

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /buy.php
Disallow /redirect.php
Disallow /files/*
Disallow /downloads/confirmation/*
Disallow /*.exe$
Disallow /*.msi$
Disallow /*.cab$
Disallow /*.zip$

Warnings

  • `host` is not a known field.