areadvd.com
robots.txt

Robots Exclusion Standard data for areadvd.com

Resource Scan

Scan Details

Site Domain areadvd.com
Base Domain areadvd.com
Scan Status Ok
Last Scan2024-06-10T09:58:08+00:00
Next Scan 2024-07-10T09:58:08+00:00

Last Scan

Scanned2024-06-10T09:58:08+00:00
URL https://areadvd.com/robots.txt
Domain IPs 195.227.159.6
Response IP 195.227.159.6
Found Yes
Hash 46510ce59f8973363b472c6cd2b33ad46537a8b02e171a067b24d5620d96376f
SimHash 58af4de7c102

Groups

*

Rule Path
Disallow /impressum/
Disallow /datenschutz/
Disallow /cookies/
Disallow /cgi-bin/
Disallow /rev/
Disallow /template/
Disallow /template_2-0/
Disallow /vb/
Disallow /werbebanner/
Disallow /wpt/
Disallow /wp-admin/
Disallow /wp-snapshots/
Disallow /wp-json/
Disallow /images/
Disallow /images/2006/
Disallow /images/2007/
Disallow /images/2008/
Disallow /images/2009/
Disallow /images/2010/
Disallow /images/2011/
Disallow /images/2012/
Disallow /images/2013/
Disallow /images/2013/
Disallow /images/2014/
Disallow /images/2015/
Disallow /images/2016/
Disallow /images/2017/
Disallow /images/2018/
Disallow /images/2019/
Disallow /images/2020/
Disallow /images/2021/
Disallow /images/2022/
Disallow /images/2023/

Other Records

Field Value
crawl-delay 4

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

wellknownbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.areadvd.de/sitemap.xml

Warnings

  • 4 invalid lines.