manythings.online
robots.txt
Robots Exclusion Standard data for manythings.online
Resource Scan
Scan Details
| Site Domain | manythings.online |
| Base Domain | manythings.online |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-06-28T09:06:23+00:00 |
| Next Scan | 2025-09-26T09:06:23+00:00 |
Last Successful Scan
| Scanned | 2025-03-01T08:31:09+00:00 |
| URL | https://manythings.online/robots.txt |
| Redirect | https://www.manythings.online/robots.txt |
| Redirect Domain | www.manythings.online |
| Redirect Base | manythings.online |
| Domain IPs | 104.21.81.209, 172.67.164.155, 2606:4700:3030::ac43:a49b, 2606:4700:3032::6815:51d1 |
| Redirect IPs | 104.21.81.209, 172.67.164.155, 2606:4700:3030::ac43:a49b, 2606:4700:3032::6815:51d1 |
| Response IP | 104.21.81.209 |
| Found | Yes |
| Hash | 4e8f72b6649fdc6bbb5c7c0bfdfbfac45c9e2ee4cf389fcf8cf942e4da4bb85d |
| SimHash | a6acec6236bd |
Groups
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 230 |
*
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /download/ |
| Disallow | /export/ |
| Disallow | /system/ |
| Disallow | /wishlist/ |
| Disallow | /login/ |
| Disallow | /srch%26srtch%3D* |
| Disallow | /product_walmart_get_url* |
| Disallow | /product_ebay_get_url* |
| Disallow | /product_get_discount* |
| Disallow | /product_amazon_get_url* |
| Disallow | /product_amazon_get_price* |
| Disallow | /product_overstock_get_url* |
| Disallow | /product_sears_get_url* |
| Disallow | /product_api* |