documentonews.gr
robots.txt

Robots Exclusion Standard data for documentonews.gr

Resource Scan

Scan Details

Site Domain documentonews.gr
Base Domain documentonews.gr
Scan Status Ok
Last Scan2026-02-13T12:30:33+00:00
Next Scan 2026-02-20T12:30:33+00:00

Last Scan

Scanned2026-02-13T12:30:33+00:00
URL https://documentonews.gr/robots.txt
Redirect https://www.documentonews.gr/robots.txt
Redirect Domain www.documentonews.gr
Redirect Base documentonews.gr
Domain IPs 104.26.10.38, 104.26.11.38, 172.67.74.159, 2606:4700:20::681a:a26, 2606:4700:20::681a:b26, 2606:4700:20::ac43:4a9f
Redirect IPs 104.26.10.38, 104.26.11.38, 172.67.74.159, 2606:4700:20::681a:a26, 2606:4700:20::681a:b26, 2606:4700:20::ac43:4a9f
Response IP 104.26.11.38
Found Yes
Hash 3ad54e00d01001654eb8148c5eaac431dad95e27482f3eac60a1e8ff0dc40ac5
SimHash 420dc8c1eab1

Groups

*

Rule Path
Allow /wp-content/uploads/
Disallow /wp-content/plugins/
Disallow /wp-admin/
Disallow /readme.html
Allow /author/

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Allow /author/

Other Records

Field Value
crawl-delay 20

semrushbot

Rule Path
Allow /author/

Other Records

Field Value
crawl-delay 20

petalbot

Rule Path
Allow /author/

Other Records

Field Value
crawl-delay 20

bingbot

Rule Path
Allow /author/

Other Records

Field Value
crawl-delay 10

yandex

Rule Path
Allow /author/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.documentonews.gr/sitemap.xml
sitemap https://www.documentonews.gr/news-sitemap.xml