arun.blog
robots.txt

Robots Exclusion Standard data for arun.blog

Resource Scan

Scan Details

Site Domain arun.blog
Base Domain arun.blog
Scan Status Ok
Last Scan2025-12-13T08:53:07+00:00
Next Scan 2025-12-20T08:53:07+00:00

Last Scan

Scanned2025-12-13T08:53:07+00:00
URL https://arun.blog/robots.txt
Redirect https://www.arun.blog/robots.txt
Redirect Domain www.arun.blog
Redirect Base arun.blog
Domain IPs 104.21.43.125, 172.67.179.133, 2606:4700:3030::6815:2b7d, 2606:4700:3033::ac43:b385
Redirect IPs 104.21.43.125, 172.67.179.133, 2606:4700:3030::6815:2b7d, 2606:4700:3033::ac43:b385
Response IP 172.67.179.133
Found Yes
Hash b8d3cfeab6a54e304a885fb3e1c823f7a64e40846d7702881ac8a7be748ad7db
SimHash 60145960a013

Groups

*

Rule Path
Allow /
Allow /rss.xml

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

youbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.arun.blog/sitemap-index.xml
sitemap https://www.arun.blog/rss.xml