products.mtbr.com
robots.txt
Robots Exclusion Standard data for products.mtbr.com
Resource Scan
Scan Details
| Site Domain | products.mtbr.com |
| Base Domain | mtbr.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-05-14T14:04:58+00:00 |
| Next Scan | 2024-08-12T14:04:58+00:00 |
Last Successful Scan
| Scanned | 2023-09-23T10:16:58+00:00 |
| URL | https://products.mtbr.com/robots.txt |
| Redirect | https://storage.googleapis.com/vs-mtbr-rbr-robotstxt/mtbr/sparkle/robots.txt |
| Redirect Domain | storage.googleapis.com |
| Redirect Base | storage.googleapis.com |
| Domain IPs | 104.17.97.5, 104.17.98.5, 2606:4700::6811:6105, 2606:4700::6811:6205 |
| Redirect IPs | 142.251.10.207, 142.251.12.207, 142.251.175.207, 172.217.194.207, 172.253.118.207, 2404:6800:4003:c03::cf, 2404:6800:4003:c04::cf, 2404:6800:4003:c0f::cf, 2404:6800:4003:c11::cf, 64.233.170.207, 74.125.130.207, 74.125.200.207, 74.125.24.207, 74.125.68.207 |
| Response IP | 142.250.4.207 |
| Found | Yes |
| Hash | f3bcc35a4b65dcc86902b083c02c886f74c6fa63689a8d53dff584d2bdc71ec9 |
| SimHash | f8397c92a782 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /ajaxform/ |
| Disallow | /ajaxsearch/ |
| Disallow | /ajaxsearch/ |
| Disallow | /search.php |
| Disallow | /search1.php |
| Disallow | /userresendcode.php |
| Disallow | /checkuser.php |
| Disallow | /curl.php |
| Disallow | /curl1.php |
| Disallow | /process.php |
| Disallow | /wp-admin/ |
| Disallow | /commerceredirect.html |
| Disallow | /user-login.html |
| Disallow | /user-registration.html |
| Disallow | /user-profile.html |
| Disallow | /user-profile-1.html |
| Disallow | /user-profile-2.html |
| Disallow | /user-profile-3.html |
| Disallow | /user-profile-4.html |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |