site-on.net
robots.txt

Robots Exclusion Standard data for site-on.net

Resource Scan

Scan Details

Site Domain site-on.net
Base Domain site-on.net
Scan Status Ok
Last Scan2025-09-10T20:46:20+00:00
Next Scan 2025-10-10T20:46:20+00:00

Last Scan

Scanned2025-09-10T20:46:20+00:00
URL https://site-on.net/robots.txt
Redirect http://site-on.net/robots.txt
Domain IPs 185.68.16.163, 2a00:7a60:0:10a3::1
Response IP 185.68.16.163
Found Yes
Hash 32caee3c421c07e54e64ce1de5798f568448a30b59cabd9fcf5728cd8d727a03
SimHash 0d288440c3b0

Groups

*

Rule Path
Disallow /blog/
Allow /blog/*.css$
Allow /blog/*.js$
Allow /blog/*.png$
Allow /blog/*.gif$
Allow /blog/*.jpg$
Allow /blog/*.jpeg$
Allow /blog/*.ttf$
Allow /blog/*.eot$
Allow /blog/*.svg$
Allow /blog/*.woff$

yandex

Rule Path
Disallow /blog/
Allow /blog/*.css$
Allow /blog/*.js$
Allow /blog/*.png$
Allow /blog/*.gif$
Allow /blog/*.jpg$
Allow /blog/*.jpeg$
Allow /blog/*.ttf$
Allow /blog/*.eot$
Allow /blog/*.svg$
Allow /blog/*.woff$

Other Records

Field Value
crawl-delay 0.1

Other Records

Field Value
sitemap http://site-on.net/sitemap.xml

Warnings

  • `host` is not a known field.