findsubscriptionboxes.com
robots.txt
Robots Exclusion Standard data for findsubscriptionboxes.com
Resource Scan
Scan Details
| Site Domain | findsubscriptionboxes.com |
| Base Domain | findsubscriptionboxes.com |
| Scan Status | Ok |
| Last Scan | 2025-11-05T09:04:15+00:00 |
| Next Scan | 2025-11-12T09:04:15+00:00 |
Last Scan
| Scanned | 2025-11-05T09:04:15+00:00 |
| URL | https://findsubscriptionboxes.com/robots.txt |
| Redirect | https://www.findsubscriptionboxes.com/robots.txt |
| Redirect Domain | www.findsubscriptionboxes.com |
| Redirect Base | findsubscriptionboxes.com |
| Domain IPs | 146.88.235.94 |
| Redirect IPs | 146.88.235.94 |
| Response IP | 146.88.235.94 |
| Found | Yes |
| Hash | ccad9ea7bbced93e93fc05fa241495fd5230de427eb77b2082f7608d54951905 |
| SimHash | 2444dc135622 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin |
| Allow | /*.js$ |
| Allow | /*.css$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
googlebot
| Rule | Path | Comment |
|---|---|---|
| Allow | .js | - |
| Allow | .css | - |
| Allow | /wp-includes | - |
| Allow | /wp-content | - |
| Disallow | /cgi-bin | - |
| Disallow | /wp-admin | - |
| Disallow | /wp-includes | - |
| Disallow | /wp-content | - |
| Disallow | /feed | - |
| Disallow | /*/feed | - |
| Allow | /blog/comment-lancer-une-box-mensuelle-tous-nos-conseils.html | - |
| Disallow | /comments | - |
| Disallow | */comment-* | Blocks the Comments Permalinks and Comment Pages |
alexabot
mediapartners-google
adsbot-google
googlebot-image
googlebot-mobile
ia_archiver-web.archive.org
googlebot
googlebot-image
googlebot-mobile
msnbot
slurp
teoma
twiceler
gigabot
scrubby
robozilla
nutch
ia_archiver
baiduspider
naverbot
yeti
yahoo-mmcrawler
psbot
asterias
yahoo-blogs/v3.9
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.findsubscriptionboxes.com/sitemap_index.xml |
Comments