reklama-crimea.com
robots.txt

Robots Exclusion Standard data for reklama-crimea.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	reklama-crimea.com
Base Domain	reklama-crimea.com
Scan Status	Ok
Last Scan	2024-10-28T14:13:43+00:00
Next Scan	2024-11-27T14:13:43+00:00

Last Scan

Scanned	2024-10-28T14:13:43+00:00
URL	https://reklama-crimea.com/robots.txt
Domain IPs	104.21.62.161, 172.67.137.21, 2606:4700:3030::ac43:8915, 2606:4700:3037::6815:3ea1
Response IP	172.67.137.21
Found	Yes
Hash	014954e8d890464a8a120aacb4821ceca71e479138b2ec879e3971edf1b3220f
SimHash	465c49e0e712

Groups

*

Rule	Path
Disallow	/?
Disallow	/my/
Disallow	/users/
Disallow	/notebook
Disallow	/invitations/
Disallow	/orders/
Disallow	/redirects/

Rule

Path

Disallow

/my/

Disallow

/users/

Disallow

/notebook

Disallow

/invitations/

Disallow

/orders/

Disallow

/redirects/

yandex

Rule	Path
Disallow	/?
Disallow	/my/
Disallow	/users/
Disallow	/notebook
Disallow	/invitations/
Disallow	/orders/
Disallow	/redirects/

Rule

Path

Disallow

/my/

Disallow

/users/

Disallow

/notebook

Disallow

/invitations/

Disallow

/orders/

Disallow

/redirects/

exabot

Rule	Path
Disallow	/

Rule

Path

Disallow

idbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bingbot

Rule	Path
Disallow	/

Rule

Path

Disallow

wotbox

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider-image

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider-video

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider-news

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider-favo

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider-cpro

Rule	Path
Disallow	/

Rule

Path

Disallow

baiduspider-ads

Rule	Path
Disallow	/

Rule

Path

Disallow

baidu

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

riddler

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

linkpadbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

flipboardproxy

Rule	Path
Disallow	/

Rule

Path

Disallow

aihitbot

Rule	Path
Disallow	/

Rule

Path

Disallow

trovitbot

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

anthropic-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

claude-web

Rule	Path
Disallow	/

Rule

Path

Disallow

google-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

amazonbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

Other Records

Field	Value
sitemap	https://reklama-crimea.com/sitemap.xml
sitemap	https://reklama-crimea.com/sitemap.xml

Field

Value

sitemap

https://reklama-crimea.com/sitemap.xml

sitemap

https://reklama-crimea.com/sitemap.xml

Warnings

`host` is not a known field.

reklama-crimea.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

yandex

exabot

idbot

bingbot

wotbox

baiduspider

baiduspider-image

baiduspider-video

baiduspider-news

baiduspider-favo

baiduspider-cpro

baiduspider-ads

baidu

mj12bot

ahrefsbot

dotbot

riddler

semrushbot

linkpadbot

blexbot

flipboardproxy

aihitbot

trovitbot

gptbot

ccbot

anthropic-ai

claude-web

google-extended

claudebot

claudebot

amazonbot

ia_archiver

Other Records

Warnings

reklama-crimea.com
robots.txt