linusblog.org
robots.txt
Robots Exclusion Standard data for linusblog.org
Resource Scan
Scan Details
| Site Domain | linusblog.org |
| Base Domain | linusblog.org |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a server error. |
| Last Scan | 2025-10-30T17:49:08+00:00 |
| Next Scan | 2025-11-29T17:49:08+00:00 |
Last Successful Scan
| Scanned | 2025-10-07T09:59:59+00:00 |
| URL | https://linusblog.org/robots.txt |
| Domain IPs | 46.8.141.129 |
| Response IP | 46.8.141.129 |
| Found | Yes |
| Hash | 2b11943a42a3a2eca79899194412fa01aca2f43406116cc74b4842fe20b32b3f |
| SimHash | 41545263c2f5 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin |
| Disallow | /wp-admin |
| Disallow | /wp-login.php |
| Disallow | /wp-register.php |
| Disallow | /wp-includes |
| Disallow | /wp-content/cache |
| Disallow | /trackback |
| Disallow | /feed |
| Disallow | /comments |
| Disallow | /category/*/* |
| Disallow | */trackback |
| Disallow | */feed |
| Disallow | /tag |
| Allow | /wp-content/uploads |
| Allow | /wp-includes/js |
| Allow | /eot/js |
| Allow | /eot/js/*/*.js* |
| Allow | /*/*/*/*/*.js* |
| Allow | /wp-content/cache/minify/000000/*.css* |
| Allow | /wp-content/cache/minify/000000/*.css* |
| Allow | /wp-content/plugins/*/*/*/*.js* |
| Allow | /wp-content/themes/hueman-master/js/*.js* |
yandex
| Rule | Path |
|---|---|
| Disallow | /tag |
| Disallow | /cgi-bin/ |
| Disallow | /wp-login.php |
| Disallow | /wp-register.php |
| Disallow | /wp-includes/ |
| Disallow | /xmlrpc.php |
| Disallow | /wp-admin |
| Disallow | /wp-content/plugins |
| Disallow | /wp-content/themes |
| Disallow | /tag/ |
| Disallow | /page/ |
| Disallow | /trackback/ |
| Disallow | /feed/ |
| Disallow | /comments/ |
| Disallow | */trackback/ |
| Disallow | */feed/ |
| Disallow | */comments/ |
| Disallow | /?feed= |
| Disallow | /?s= |
| Disallow | /*?* |
| Disallow | /*? |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://linusblog.org/sitemap.xml |
Warnings
- `host` is not a known field.
Comments