dexport.com
robots.txt

Robots Exclusion Standard data for dexport.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	dexport.com
Base Domain	dexport.com
Scan Status	Ok
Last Scan	2025-12-11T12:33:19+00:00
Next Scan	2025-12-25T12:33:19+00:00

Last Scan

Scanned	2025-12-11T12:33:19+00:00
URL	https://dexport.com/robots.txt
Domain IPs	45.152.250.91
Response IP	45.152.250.91
Found	Yes
Hash	afc8904f88133feb7ffc3f2ca69ed10c73e730f08e8129d04613299beb23e775
SimHash	4b9d654260e0

Groups

xovibot

Rule	Path
Disallow	/

Rule

Path

Disallow

wotbox

Rule	Path
Disallow	/

Rule

Path

Disallow

wbsearchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

unisterbot

Rule	Path
Disallow	/

Rule

Path

Disallow

turnitinbot

Rule	Path
Disallow	/

Rule

Path

Disallow

spbot

Rule	Path
Disallow	/

Rule

Path

Disallow

sogou spider

Rule	Path
Disallow	/

Rule

Path

Disallow

smtbot

Rule	Path
Disallow	/

Rule

Path

Disallow

siteexplorer

Rule	Path
Disallow	/

Rule

Path

Disallow

seznambot

Rule	Path
Disallow	/

Rule

Path

Disallow

seokicks-robot

Rule	Path
Disallow	/

Rule

Path

Disallow

sentibot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

searchmetricsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

screenerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

publiclibraryarchive.org

Rule	Path
Disallow	/

Rule

Path

Disallow

panscient.com

Rule	Path
Disallow	/

Rule

Path

Disallow

nutch

Rule	Path
Disallow	/

Rule

Path

Disallow

netestate ne crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/wp-admin/
Disallow	/product/*

Rule

Path

Disallow

/wp-admin/

Disallow

/product/*

Other Records

Field	Value
crawl-delay	10

Field

Value

crawl-delay

mixbot

Rule	Path
Disallow	/

Rule

Path

Disallow

memorybot

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex.ru/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex.ru

Rule	Path
Disallow	/

Rule

Path

Disallow

meanpathbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mail.ru

Rule	Path
Disallow	/

Rule

Path

Disallow

lipperhey

Rule	Path
Disallow	/

Rule

Path

Disallow

linkpadbot

Rule	Path
Disallow	/

Rule

Path

Disallow

istellabot

Rule	Path
Disallow	/

Rule

Path

Disallow

heritrix

Rule	Path
Disallow	/

Rule

Path

Disallow

googlebot

Rule	Path
Disallow	/wp-admin/

Rule

Path

Disallow

/wp-admin/

feedbooster

Rule	Path
Disallow	/

Rule

Path

Disallow

ezooms

Rule	Path
Disallow	/

Rule

Path

Disallow

exb language crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

easouspider

Rule	Path
Disallow	/

Rule

Path

Disallow

bubing

Rule	Path
Disallow	/

Rule

Path

Disallow

bpimagewalker/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

advbot

Rule	Path
Disallow	/

Rule

Path

Disallow

aboundexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

abonti

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Disallow	/wp-admin/
Allow	/wp-admin/admin-ajax.php

Rule

Path

Disallow

/wp-admin/

Allow

/wp-admin/admin-ajax.php

Other Records

Field	Value
crawl-delay	10

Field

Value

crawl-delay

Other Records

Field	Value
sitemap	https://dexport.com/sitemap_index.xml

Field

Value

sitemap

https://dexport.com/sitemap_index.xml

dexport.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

xovibot

wotbox

wbsearchbot

unisterbot

turnitinbot

spbot

sogou spider

smtbot

siteexplorer

seznambot

seokicks-robot

sentibot

semrushbot

searchmetricsbot

screenerbot

publiclibraryarchive.org

panscient.com

nutch

netestate ne crawler

mj12bot

Other Records

mixbot

memorybot

megaindex.ru/2.0

megaindex.ru

meanpathbot

mail.ru

lipperhey

linkpadbot

istellabot

heritrix

googlebot

feedbooster

ezooms

exb language crawler

easouspider

bubing

bpimagewalker/2.0

blexbot

advbot

aboundexbot

abonti

*

Other Records

Other Records

dexport.com
robots.txt