lustery.com
robots.txt
Robots Exclusion Standard data for lustery.com
Resource Scan
Scan Details
| Site Domain | lustery.com |
| Base Domain | lustery.com |
| Scan Status | Ok |
| Last Scan | 2025-07-29T19:39:50+00:00 |
| Next Scan | 2025-08-28T19:39:50+00:00 |
Last Scan
| Scanned | 2025-07-29T19:39:50+00:00 |
| URL | https://lustery.com/robots.txt |
| Domain IPs | 104.18.20.101, 104.18.21.101, 2606:4700::6812:1465, 2606:4700::6812:1565 |
| Response IP | 104.18.21.101 |
| Found | Yes |
| Hash | 9472ee933d45714617087bf532759d37521c6aa3c96a9d2e422ab63c06beada6 |
| SimHash | 6c10fc5441f7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admin/* |
| Disallow | /testing/* |
| Disallow | /*webhooks |
| Disallow | /*epoch |
| Disallow | /*payments |
| Disallow | /*ccbill |
| Disallow | /api/user/* |
| Disallow | /api/security/* |
| Disallow | /apply |
| Disallow | /join |
| Disallow | /login |
| Disallow | /upgrade |
| Disallow | /verify-your-email |
| Disallow | /password-reset |
| Disallow | /forgot-password |
| Disallow | /maintenance |
| Disallow | /terms-and-conditions |
| Disallow | /content-removal-request |
| Disallow | /dmca |
| Disallow | /prohibited-content |
| Disallow | /profile |
| Disallow | /settings |
| Disallow | /notification-settings |
| Disallow | /collections |
| Disallow | /favorite-articles |
| Disallow | /favorite-videos |
| Disallow | /followed-couples |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
googlebot
| Rule | Path |
|---|---|
| Disallow | /admin/* |
| Disallow | /testing/* |
| Disallow | /*webhooks |
| Disallow | /*epoch |
| Disallow | /*payments |
| Disallow | /*ccbill |
| Disallow | /api/user/* |
| Disallow | /api/security/* |
| Disallow | /apply |
| Disallow | /join |
| Disallow | /login |
| Disallow | /upgrade |
| Disallow | /verify-your-email |
| Disallow | /password-reset |
| Disallow | /forgot-password |
| Disallow | /maintenance |
| Disallow | /terms-and-conditions |
| Disallow | /content-removal-request |
| Disallow | /dmca |
| Disallow | /prohibited-content |
| Disallow | /profile |
| Disallow | /settings |
| Disallow | /notification-settings |
| Disallow | /collections |
| Disallow | /favorite-articles |
| Disallow | /favorite-videos |
| Disallow | /followed-couples |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://lustery.com/sitemap.xml |
| sitemap | https://lustery.com/sitemap-videos.xml |
Warnings
- `host` is not a known field.