gunvor-deutschland.de
robots.txt

Robots Exclusion Standard data for gunvor-deutschland.de

Archived Snapshots

Resource Scan

Scan Details

Site Domain	gunvor-deutschland.de
Base Domain	gunvor-deutschland.de
Scan Status	Ok
Last Scan	2024-10-20T23:06:05+00:00
Next Scan	2024-11-19T23:06:05+00:00

Last Scan

Scanned	2024-10-20T23:06:05+00:00
URL	https://gunvor-deutschland.de/robots.txt
Domain IPs	193.9.117.130
Response IP	193.9.117.130
Found	Yes
Hash	0b09a1e49adb1a37e6e74249a39528ede2f93df2525cae87a78e95c826522f87
SimHash	6a5d5443c611

Groups

googlebot

Rule	Path
Allow	/*.css$
Allow	/*.js$
Allow	/*.png$
Allow	/*.jpg$
Allow	/*.svg$
Allow	/*.gif$
Allow	/*.ttf$
Allow	/*.otf$
Allow	/*.eot$
Allow	/*.woff$

Rule

Path

Allow

/*.css$

Allow

/*.js$

Allow

/*.png$

Allow

/*.jpg$

Allow

/*.svg$

Allow

/*.gif$

Allow

/*.ttf$

Allow

/*.otf$

Allow

/*.eot$

Allow

/*.woff$

msnbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	3

Field

Value

crawl-delay

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	3

Field

Value

crawl-delay

yandex

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	3

Field

Value

crawl-delay

applebot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	3

Field

Value

crawl-delay

blexbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	30

Field

Value

crawl-delay

a6-indexer

Rule	Path
Disallow	/

Rule

Path

Disallow

alphaseobot

Rule	Path
Disallow	/

Rule

Path

Disallow

alphaseobot-sa

Rule	Path
Disallow	/

Rule

Path

Disallow

aspiegelbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blackboard safeassign

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/

Rule

Path

Disallow

crawler4j

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

gigabot

Rule	Path
Disallow	/

Rule

Path

Disallow

liebaofast

Rule	Path
Disallow	/

Rule

Path

Disallow

mauibot

Rule	Path
Disallow	/

Rule

Path

Disallow

mauibot (crawler.feedback+wc@gmail.com)

Rule	Path
Disallow	/

Rule

Path

Disallow

megaindex.ru/2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

mqqbrowser

Rule	Path
Disallow	/

Rule

Path

Disallow

nimbostratus-bot/v1.3.2

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

seekport crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-sa

Rule	Path
Disallow	/

Rule

Path

Disallow

serpstatbot

Rule	Path
Disallow	/

Rule

Path

Disallow

seznambot

Rule	Path
Disallow	/

Rule

Path

Disallow

sputnikbot/2.3

Rule	Path
Disallow	/

Rule

Path

Disallow

the knowledge ai

Rule	Path
Disallow	/

Rule

Path

Disallow

turnitinbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ucbrowser

Rule	Path
Disallow	/

Rule

Path

Disallow

yacybot

Rule	Path
Disallow	/

Rule

Path

Disallow

yeti

Rule	Path
Disallow	/

Rule

Path

Disallow

yisouspider

Rule	Path
Disallow	/

Rule

Path

Disallow

Other Records

Field	Value
sitemap	https://gunvor-deutschland.de/gunvor-deutschland.de.index.sitemap

Field

Value

sitemap

https://gunvor-deutschland.de/gunvor-deutschland.de.index.sitemap

gunvor-deutschland.derobots.txt

Resource Scan

Scan Details

Last Scan

Groups

googlebot

msnbot

Other Records

ahrefsbot

Other Records

yandex

Other Records

applebot

Other Records

blexbot

Other Records

a6-indexer

alphaseobot

alphaseobot-sa

aspiegelbot

blackboard safeassign

bytespider

crawler4j

dotbot

gigabot

liebaofast

mauibot

mauibot (crawler.feedback+wc@gmail.com)

megaindex.ru/2.0

mqqbrowser

nimbostratus-bot/v1.3.2

petalbot

seekport crawler

semrushbot

semrushbot-sa

serpstatbot

seznambot

sputnikbot/2.3

the knowledge ai

turnitinbot

ucbrowser

yacybot

yeti

yisouspider

Other Records

gunvor-deutschland.de
robots.txt