smartcat.com
robots.txt
Robots Exclusion Standard data for smartcat.com
Resource Scan
Scan Details
| Site Domain | smartcat.com |
| Base Domain | smartcat.com |
| Scan Status | Ok |
| Last Scan | 2024-11-10T01:06:04+00:00 |
| Next Scan | 2024-12-10T01:06:04+00:00 |
Last Scan
| Scanned | 2024-11-10T01:06:04+00:00 |
| URL | https://smartcat.com/robots.txt |
| Redirect | https://www.smartcat.com/robots.txt |
| Redirect Domain | www.smartcat.com |
| Redirect Base | smartcat.com |
| Domain IPs | 34.255.74.192, 52.30.217.156, 54.72.182.107 |
| Redirect IPs | 13.227.254.129, 13.227.254.42, 13.227.254.82, 13.227.254.98, 2600:9000:200a:4e00:e:fdf4:5d80:93a1, 2600:9000:200a:6800:e:fdf4:5d80:93a1, 2600:9000:200a:7800:e:fdf4:5d80:93a1, 2600:9000:200a:8000:e:fdf4:5d80:93a1, 2600:9000:200a:8600:e:fdf4:5d80:93a1, 2600:9000:200a:ac00:e:fdf4:5d80:93a1, 2600:9000:200a:b400:e:fdf4:5d80:93a1, 2600:9000:200a:d400:e:fdf4:5d80:93a1 |
| Response IP | 13.227.254.98 |
| Found | Yes |
| Hash | 65fd0d64603a4b6cbab0d2daab6d3575a532ac057ce6c9cfad52fd0ebe0a3dbe |
| SimHash | 6c0d59441391 |
Groups
*
| Rule | Path |
|---|---|
| Disallow |
*
| Rule | Path |
|---|---|
| Disallow | /lp/ |
*
| Rule | Path |
|---|---|
| Disallow | /c/ |
*
| Rule | Path |
|---|---|
| Disallow | /report-of-the-illegal-content/ |
*
| Rule | Path |
|---|---|
| Disallow | /ai-chatgpt-automation-translation-whitepaper/ |
*
| Rule | Path |
|---|---|
| Disallow | /contact-our-legal-support/ |
*
| Rule | Path |
|---|---|
| Disallow | /contact-our-support-team/ |
*
| Rule | Path |
|---|---|
| Disallow | /contact-support/ |
*
| Rule | Path |
|---|---|
| Disallow | /marketpalce/user/ |
*
| Rule | Path |
|---|---|
| Disallow | /users/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.smartcat.com/sitemap.xml |