dong-fan.com
robots.txt

Robots Exclusion Standard data for dong-fan.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	dong-fan.com
Base Domain	dong-fan.com
Scan Status	Ok
Last Scan	2024-10-30T16:26:10+00:00
Next Scan	2024-11-29T16:26:10+00:00

Last Scan

Scanned	2024-10-30T16:26:10+00:00
URL	https://dong-fan.com/robots.txt
Redirect	https://www.dong-fan.com/robots.txt
Redirect Domain	www.dong-fan.com
Redirect Base	dong-fan.com
Domain IPs	221.248.96.228
Redirect IPs	221.248.96.228
Response IP	221.248.96.228
Found	Yes
Hash	d2ffa914343f56ddaef75c2f8c1d87f4952832f6219fc7ea3bc27c874a1df7c8
SimHash	d2365440caa9

Groups

chatgpt-user

Rule	Path
Disallow	/

Rule

Path

Disallow

oai-searchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

facebookexternalhit

Rule	Path
Disallow	/

Rule

Path

Disallow

awariobot

Rule	Path
Disallow	/

Rule

Path

Disallow

bdbot

Rule	Path
Disallow	/

Rule

Path

Disallow

tinytestbot

Rule	Path
Disallow	/

Rule

Path

Disallow

y!j-wsc

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

yak

Rule	Path
Disallow	/

Rule

Path

Disallow

seekportbot

Rule	Path
Disallow	/

Rule

Path

Disallow

netestate ne crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

y!j-brw

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/

Rule

Path

Disallow

serpstatbot

Rule	Path
Disallow	/

Rule

Path

Disallow

seolytbot

Rule	Path
Disallow	/

Rule

Path

Disallow

barkrowler

Rule	Path
Disallow	/

Rule

Path

Disallow

cincraw

Rule	Path
Disallow	/

Rule

Path

Disallow

coccocbot

Rule	Path
Disallow	/

Rule

Path

Disallow

dataforseobot

Rule	Path
Disallow	/

Rule

Path

Disallow

linespider

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex.ru

Rule	Path
Disallow	/

Rule

Path

Disallow

screaming frog seo spider

Rule	Path
Disallow	/

Rule

Path

Disallow

seekport crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

twitterbot

Rule	Path
Disallow	/

Rule

Path

Disallow

y!j-asr

Rule	Path
Disallow	/

Rule

Path

Disallow

amazonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mauibot

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

yandex

Rule	Path
Disallow	/

Rule

Path

Disallow

ltx71

Rule	Path
Disallow	/

Rule

Path

Disallow

smrjbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider

Rule	Path
Disallow	/

Rule

Path

Disallow

bubing

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot

Rule	Path
Disallow	/

Rule

Path

Disallow

icc-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bingbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	5

Field

Value

crawl-delay

dong-fan.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

chatgpt-user

oai-searchbot

facebookexternalhit

awariobot

bdbot

tinytestbot

y!j-wsc

gptbot

yak

seekportbot

netestate ne crawler

y!j-brw

bytespider

serpstatbot

seolytbot

barkrowler

cincraw

coccocbot

dataforseobot

linespider

megaindex.ru

screaming frog seo spider

seekport crawler

twitterbot

y!j-asr

amazonbot

ccbot

mauibot

dotbot

yandex

ltx71

smrjbot

semrushbot

ahrefsbot

baiduspider

bubing

mj12bot

blexbot

applebot

icc-crawler

petalbot

bingbot

Other Records

dong-fan.com
robots.txt