lightingtheweb.com
robots.txt
Robots Exclusion Standard data for lightingtheweb.com
Resource Scan
Scan Details
| Site Domain | lightingtheweb.com |
| Base Domain | lightingtheweb.com |
| Scan Status | Ok |
| Last Scan | 2025-10-11T04:40:19+00:00 |
| Next Scan | 2025-11-10T04:40:19+00:00 |
Last Scan
| Scanned | 2025-10-11T04:40:19+00:00 |
| URL | https://lightingtheweb.com/robots.txt |
| Domain IPs | 104.21.70.87, 172.67.222.41, 2606:4700:3031::6815:4657, 2606:4700:3036::ac43:de29 |
| Response IP | 172.67.222.41 |
| Found | Yes |
| Hash | 6e8c30780edeb1e350c770650d71efcecc65c86da50c098b9904117c6dc0d22d |
| SimHash | 2316f0165952 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*/list.aspx?* |
| Disallow | /search.aspx |
| Allow | /*/list.aspx?bid |
| Allow | / |
*
| Rule | Path |
|---|---|
| Disallow | /ProductSheet.aspx |
| Disallow | /secure/ |
| Disallow | /product.aspx |
| Disallow | /more-info.aspx |
| Disallow | /cart.aspx |
| Disallow | /search.aspx |
| Disallow | /*/product.aspx |
| Disallow | /Secure/* |
| Disallow | /Project/* |
| Disallow | /Product/* |
| Disallow | /Search/* |
| Disallow | /Cart/* |
| Disallow | /Error/* |
| Disallow | /User/* |
| Disallow | /Media/* |
| Disallow | /Survey/* |
| Disallow | /Supplier/* |
| Disallow | /User/* |
| Disallow | /*.php$ |
| Disallow | /*first_answer%3D |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.lightingtheweb.com/sitemaplightingtheweb.xml |
Warnings
- 1 invalid line.
Comments