designblognews.com
robots.txt

Robots Exclusion Standard data for designblognews.com

Resource Scan

Scan Details

Site Domain designblognews.com
Base Domain designblognews.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan3/13/2025, 2:18:18 AM
Next Scan 6/11/2025, 2:18:18 AM

Last Successful Scan

Scanned11/20/2023, 12:36:07 AM
URL https://designblognews.com/robots.txt
Domain IPs 104.21.56.35, 172.67.176.105, 2606:4700:3033::6815:3823, 2606:4700:3034::ac43:b069
Response IP 172.67.176.105
Found Yes
Hash 5ffd64f26b3677cdcc43c03a9bf5e5ab5d6a3e68809ad244826c6640f77f9d54
SimHash 981d48c48892

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /pin/tag/
Disallow /wp-admin/
Disallow /login/*

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.designblognews.com/sitemap.xml
sitemap https://www.designblognews.com/sitemap-image.xml

Comments

  • Disallow: /author*