starluky.com
robots.txt
Robots Exclusion Standard data for starluky.com
Resource Scan
Scan Details
| Site Domain | starluky.com |
| Base Domain | starluky.com |
| Scan Status | Ok |
| Last Scan | 2024-09-28T16:43:14+00:00 |
| Next Scan | 2024-10-05T16:43:14+00:00 |
Last Scan
| Scanned | 2024-09-28T16:43:14+00:00 |
| URL | https://starluky.com/robots.txt |
| Redirect | https://www.starluky.com/robots.txt |
| Redirect Domain | www.starluky.com |
| Redirect Base | starluky.com |
| Domain IPs | 104.21.96.6, 172.67.150.28, 2606:4700:3031::6815:6006, 2606:4700:3036::ac43:961c |
| Redirect IPs | 104.21.96.6, 172.67.150.28, 2606:4700:3031::6815:6006, 2606:4700:3036::ac43:961c |
| Response IP | 172.67.150.28 |
| Found | Yes |
| Hash | f8dc4c1ae3e38e3e2dcf61a2139a883e77b336dddf5f004859a1174ddf9f8f7c |
| SimHash | c234ffe945f2 |
Groups
ia_archiver
| Rule | Path | Comment |
|---|---|---|
| Allow | / | Popular chinese search enginesUser-agent: Baiduspider |
| Allow | / | - |
youdaobot/1.0
| Rule | Path | Comment |
|---|---|---|
| Allow | / | Block Bad Bots. "AI recommended setting" by ChatGPTUser-agent: ia_archiver |
| Disallow | / | - |
wget
| Rule | Path | Comment |
|---|---|---|
| Disallow | / | ChatGPT Bot Blocker - Block ChatGPT Bot from scrapping your contentUser-agent: GPTBot |
| Disallow | / | Spam Backlink BlockerDisallow: /feed/ |
| Disallow | /feed/$ | - |
| Disallow | /comments/feed | - |
| Disallow | /trackback/ | - |
| Disallow | */?author=* | - |
| Disallow | */author/* | - |
| Disallow | /author* | - |
| Disallow | /author/ | - |
| Disallow | */comments$ | - |
| Disallow | */feed | - |
| Disallow | */feed$ | - |
| Disallow | */trackback | - |
| Disallow | */trackback$ | - |
| Disallow | /?feed= | - |
| Disallow | /wp-comments | - |
| Disallow | /wp-feed | - |
| Disallow | /wp-trackback | - |
| Disallow | */replytocom%3D | Block Bad Bots. Powered by Better Robots.txt ProUser-agent: GiftGhostBot |
| Disallow | / | - |
dataprovider site explorer
| Rule | Path | Comment |
|---|---|---|
| Disallow | / | Loading Performance for WoocommerceDisallow: /cart/ |
| Disallow | /checkout/ | - |
| Disallow | /my-account/ | - |
| Disallow | /*?orderby=price | - |
| Disallow | /*?orderby=rating | - |
| Disallow | /*?orderby=date | - |
| Disallow | /*?orderby=price-desc | - |
| Disallow | /*?orderby=popularity | - |
| Disallow | /*?filter | - |
| Disallow | /*?orderby=title | - |
| Disallow | /*?orderby=desc | - |
| Disallow | /*?filter | - |
| Disallow | /*add-to-cart%3D* | - |
| Disallow | /*add_to_wishlist%3D* | - |
| Disallow | /*?paged=&count=* | - |
| Disallow | /*?count=* | Image Crawlability by search enginesUser-agent: * |
| Allow | /*.png* | - |
| Allow | /*.jpg* | - |
| Allow | /*.gif* | - |
| Allow | /*.webp* | Avoid crawler traps causing crawl budget issuesDisallow: /search/ |
| Disallow | *?s=* | - |
| Disallow | *?p=* | - |
| Disallow | *%26p%3D* | - |
| Disallow | *%26preview%3D* | - |
| Disallow | /search | Social Media CrawlingUser-agent: facebookexternalhit/1.0 |
| Allow | / | - |
Warnings
- 26 invalid lines.