craft.do
robots.txt
Robots Exclusion Standard data for craft.do
Resource Scan
Scan Details
| Site Domain | craft.do |
| Base Domain | craft.do |
| Scan Status | Ok |
| Last Scan | 2024-04-29T11:29:55+00:00 |
| Next Scan | 2024-05-29T11:29:55+00:00 |
Last Scan
| Scanned | 2024-04-29T11:29:55+00:00 |
| URL | https://craft.do/robots.txt |
| Redirect | https://www.craft.do/robots.txt |
| Redirect Domain | www.craft.do |
| Redirect Base | craft.do |
| Domain IPs | 18.67.181.101, 18.67.181.118, 18.67.181.129, 18.67.181.46 |
| Redirect IPs | 18.161.6.14, 18.161.6.52, 18.161.6.72, 18.161.6.8 |
| Response IP | 18.165.171.77 |
| Found | Yes |
| Hash | 474e2f1a408a6adf8384ba14c36318d928e81c27313f2cc25eab06364eab58c9 |
| SimHash | 6545de530a51 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
*
| Rule | Path |
|---|---|
| Disallow | /preview/* |
| Disallow | /payment |
| Disallow | /404 |
| Disallow | /500 |
| Disallow | /api/draft/* |
| Disallow | /.well-known/* |
*
| Rule | Path |
|---|---|
| Disallow | /facelift/* |
*
| Rule | Path |
|---|---|
| Disallow | /do-not-crawl/* |
*
| Rule | Path |
|---|---|
| Disallow | /download/desktop/Craft%20Installer.exe |
| Disallow | /downloads/craft-logo-pack.zip |
*
| Rule | Path |
|---|---|
| Disallow | /ab/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.craft.do/sitemap.xml |
| sitemap | https://www.craft.do/server-sitemap/templates-sitemap.xml |
| sitemap | https://www.craft.do/server-sitemap/blog-sitemap.xml |
| sitemap | https://www.craft.do/server-sitemap/author-sitemap.xml |
| sitemap | https://www.craft.do/server-sitemap/template-gallery-sitemap.xml |
| sitemap | https://www.craft.do/server-sitemap/blog-categories-sitemap.xml |
Warnings
- `host` is not a known field.
Comments