cdata.com
robots.txt
Robots Exclusion Standard data for cdata.com
Resource Scan
Scan Details
| Site Domain | cdata.com |
| Base Domain | cdata.com |
| Scan Status | Ok |
| Last Scan | 2024-09-25T10:21:51+00:00 |
| Next Scan | 2024-10-25T10:21:51+00:00 |
Last Scan
| Scanned | 2024-09-25T10:21:51+00:00 |
| URL | https://cdata.com/robots.txt |
| Redirect | https://www.cdata.com/robots.txt |
| Redirect Domain | www.cdata.com |
| Redirect Base | cdata.com |
| Domain IPs | 104.22.40.174, 104.22.41.174, 172.67.8.209, 2606:4700:10::6816:28ae, 2606:4700:10::6816:29ae, 2606:4700:10::ac43:8d1 |
| Redirect IPs | 104.22.40.174, 104.22.41.174, 172.67.8.209, 2606:4700:10::6816:28ae, 2606:4700:10::6816:29ae, 2606:4700:10::ac43:8d1 |
| Response IP | 104.22.40.174 |
| Found | Yes |
| Hash | 8d52d6c466e63ec1341325158a5031011c380dc2f4764df6e8d5604f5d1a682e |
| SimHash | cc8cca00c4a7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /lic/ |
| Disallow | /forms/request/ |
| Disallow | /login/ |
| Disallow | /news/*.htm$ |
| Disallow | /news/*.rst$ |
| Disallow | /support/submit.aspx |
| Disallow | /order |
| Disallow | /download/*/ |
| Disallow | /jp/lic/ |
| Disallow | /jp/support/submit.aspx |
| Disallow | /jp/order |
| Disallow | /jp/download/*/ |
ahrefsbot
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /ui/ |
| Disallow | /lic/ |
| Disallow | /forms/request/ |
| Disallow | /login/ |
| Disallow | /news/*.htm$ |
| Disallow | /news/*.rst$ |
| Disallow | /support/submit.aspx |
| Disallow | /order |
| Disallow | /download/*/ |
| Disallow | /jp/lic/ |
| Disallow | /jp/support/submit.aspx |
| Disallow | /jp/order |
| Disallow | /jp/download/*/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
dotbot
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /ui/ |
| Disallow | /lic/ |
| Disallow | /forms/request/ |
| Disallow | /login/ |
| Disallow | /news/*.htm$ |
| Disallow | /news/*.rst$ |
| Disallow | /support/submit.aspx |
| Disallow | /order |
| Disallow | /download/*/ |
| Disallow | /jp/lic/ |
| Disallow | /jp/support/submit.aspx |
| Disallow | /jp/order |
| Disallow | /jp/download/*/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
semrushbot
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /ui/ |
| Disallow | /lic/ |
| Disallow | /forms/request/ |
| Disallow | /login/ |
| Disallow | /news/*.htm$ |
| Disallow | /news/*.rst$ |
| Disallow | /support/submit.aspx |
| Disallow | /order |
| Disallow | /download/*/ |
| Disallow | /jp/lic/ |
| Disallow | /jp/support/submit.aspx |
| Disallow | /jp/order |
| Disallow | /jp/download/*/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
mappy
| Rule | Path |
|---|---|
| Disallow | / |
| Disallow | /cgi-bin/ |
| Disallow | /ui/ |
| Disallow | /lic/ |
| Disallow | /forms/request/ |
| Disallow | /login/ |
| Disallow | /news/*.htm$ |
| Disallow | /news/*.rst$ |
| Disallow | /support/submit.aspx |
| Disallow | /order |
| Disallow | /download/*/ |
| Disallow | /jp/lic/ |
| Disallow | /jp/support/submit.aspx |
| Disallow | /jp/order |
| Disallow | /jp/download/*/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
yandex
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /ui/ |
| Disallow | /lic/ |
| Disallow | /forms/request/ |
| Disallow | /login/ |
| Disallow | /news/*.htm$ |
| Disallow | /news/*.rst$ |
| Disallow | /support/submit.aspx |
| Disallow | /order |
| Disallow | /download/*/ |
| Disallow | /jp/lic/ |
| Disallow | /jp/support/submit.aspx |
| Disallow | /jp/order |
| Disallow | /jp/download/*/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2.0 |
seznambot
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /ui/ |
| Disallow | /lic/ |
| Disallow | /forms/request/ |
| Disallow | /login/ |
| Disallow | /news/*.htm$ |
| Disallow | /news/*.rst$ |
| Disallow | /support/submit.aspx |
| Disallow | /order |
| Disallow | /download/*/ |
| Disallow | /jp/lic/ |
| Disallow | /jp/support/submit.aspx |
| Disallow | /jp/order |
| Disallow | /jp/download/*/ |
Warnings
- `host` is not a known field.
- `request-rate` is not a known field.