starluky.com
robots.txt
Robots Exclusion Standard data for starluky.com
Resource Scan
Scan Details
| Site Domain | starluky.com |
| Base Domain | starluky.com |
| Scan Status | Ok |
| Last Scan | 2024-11-16T16:48:23+00:00 |
| Next Scan | 2024-11-23T16:48:23+00:00 |
Last Scan
| Scanned | 2024-11-16T16:48:23+00:00 |
| URL | https://starluky.com/robots.txt |
| Redirect | https://www.starluky.com/robots.txt |
| Redirect Domain | www.starluky.com |
| Redirect Base | starluky.com |
| Domain IPs | 104.26.10.149, 104.26.11.149, 172.67.70.13, 2606:4700:20::681a:a95, 2606:4700:20::681a:b95, 2606:4700:20::ac43:460d |
| Redirect IPs | 104.26.10.149, 104.26.11.149, 172.67.70.13, 2606:4700:20::681a:a95, 2606:4700:20::681a:b95, 2606:4700:20::ac43:460d |
| Response IP | 104.26.10.149 |
| Found | Yes |
| Hash | caf5b9d2d544291082caeac28f63fbbf01e28cf8b2e1330535915006e5c37b5b |
| SimHash | c234ffe945f2 |
Groups
youdaobot/1.0
| Rule | Path | Comment |
|---|---|---|
| Allow | / | Block Bad Bots. "AI recommended setting" by ChatGPTUser-agent: ia_archiver |
| Disallow | / | - |
wget
| Rule | Path | Comment |
|---|---|---|
| Disallow | / | ChatGPT Bot Blocker - Block ChatGPT Bot from scrapping your contentUser-agent: GPTBot |
| Disallow | / | Spam Backlink BlockerDisallow: /feed/ |
| Disallow | /feed/$ | - |
| Disallow | /comments/feed | - |
| Disallow | /trackback/ | - |
| Disallow | */?author=* | - |
| Disallow | */author/* | - |
| Disallow | /author* | - |
| Disallow | /author/ | - |
| Disallow | */comments$ | - |
| Disallow | */feed | - |
| Disallow | */feed$ | - |
| Disallow | */trackback | - |
| Disallow | */trackback$ | - |
| Disallow | /?feed= | - |
| Disallow | /wp-comments | - |
| Disallow | /wp-feed | - |
| Disallow | /wp-trackback | - |
| Disallow | */replytocom%3D | Block Bad Bots. Powered by Better Robots.txt ProUser-agent: GiftGhostBot |
| Disallow | / | - |
dataprovider site explorer
| Rule | Path | Comment |
|---|---|---|
| Disallow | / | Loading Performance for WoocommerceDisallow: /cart/ |
| Disallow | /checkout/ | - |
| Disallow | /my-account/ | - |
| Disallow | /*?orderby=price | - |
| Disallow | /*?orderby=rating | - |
| Disallow | /*?orderby=date | - |
| Disallow | /*?orderby=price-desc | - |
| Disallow | /*?orderby=popularity | - |
| Disallow | /*?filter | - |
| Disallow | /*?orderby=title | - |
| Disallow | /*?orderby=desc | - |
| Disallow | /*?filter | - |
| Disallow | /*add-to-cart%3D* | - |
| Disallow | /*add_to_wishlist%3D* | - |
| Disallow | /*?paged=&count=* | - |
| Disallow | /*?count=* | Image Crawlability by search enginesUser-agent: * |
| Allow | /*.png* | - |
| Allow | /*.jpg* | - |
| Allow | /*.gif* | - |
| Allow | /*.webp* | Avoid crawler traps causing crawl budget issuesDisallow: /search/ |
| Disallow | *?s=* | - |
| Disallow | *?p=* | - |
| Disallow | *%26p%3D* | - |
| Disallow | *%26preview%3D* | - |
| Disallow | /search | Social Media CrawlingUser-agent: facebookexternalhit/1.0 |
| Allow | / | - |
Warnings
- 25 invalid lines.