nataliproducts.com
robots.txt
Robots Exclusion Standard data for nataliproducts.com
Resource Scan
Scan Details
| Site Domain | nataliproducts.com |
| Base Domain | nataliproducts.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-07-12T02:38:45+00:00 |
| Next Scan | 2024-10-10T02:38:45+00:00 |
Last Successful Scan
| Scanned | 2023-11-22T19:20:51+00:00 |
| URL | https://nataliproducts.com/robots.txt |
| Domain IPs | 100.25.76.171, 3.225.180.121, 3.234.71.255 |
| Response IP | 3.234.71.255 |
| Found | Yes |
| Hash | 173faff2824f8c5c379799aa3cf39a903d46376d9928e170555c5cd678cd2a08 |
| SimHash | 0c823d73c6b1 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /media/customer/ |
| Disallow | /phpMyAdmin-R563GFRSW/ |
| Allow | /*?p= |
| Disallow | /404/ |
| Disallow | /app/ |
| Disallow | /cgi-bin/ |
| Disallow | /downloader/ |
| Disallow | /errors/ |
| Disallow | /includes/ |
| Disallow | /magento/ |
| Disallow | /media/captcha/ |
| Disallow | /media/customer/ |
| Disallow | /media/dhl/ |
| Disallow | /media/downloadable/ |
| Disallow | /media/import/ |
| Disallow | /media/pdf/ |
| Disallow | /media/sales/ |
| Disallow | /media/tmp/ |
| Disallow | /media/xmlconnect/ |
| Disallow | /pkginfo/ |
| Disallow | /report/ |
| Disallow | /scripts/ |
| Disallow | /shell/ |
| Disallow | /stats/ |
| Disallow | /var/ |
| Disallow | */index.php/ |
| Disallow | */catalog/product_compare/ |
| Disallow | */catalog/category/view/ |
| Disallow | */catalog/product/view/ |
| Disallow | */catalog/product/gallery/ |
| Disallow | */catalogsearch/ |
| Disallow | */control/ |
| Disallow | */contacts/ |
| Disallow | */customer/ |
| Disallow | */customize/ |
| Disallow | */newsletter/ |
| Disallow | */poll/ |
| Disallow | */review/ |
| Disallow | */sendfriend/ |
| Disallow | */tag/ |
| Disallow | */wishlist/ |
| Disallow | */checkout/ |
| Disallow | */onestepcheckout/ |
| Disallow | /cron.php |
| Disallow | /cron.sh |
| Disallow | /error_log |
| Disallow | /install.php |
| Disallow | /LICENSE.html |
| Disallow | /LICENSE.txt |
| Disallow | /LICENSE_AFL.txt |
| Disallow | /STATUS.txt |
| Disallow | /*?dir* |
| Disallow | /*?limit* |
| Disallow | /*?mode* |
| Disallow | /*?___from_store=* |
| Disallow | /*?___store=* |
| Disallow | /*?cat=* |
| Disallow | /*?q=* |
| Disallow | /*?price=* |
| Disallow | /*?availability=* |
| Disallow | /*?brand=* |
| Disallow | /*?p=*& |
| Disallow | /*.php$ |
| Disallow | /*?SID= |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Comments