tajik-gateway.org
robots.txt
Robots Exclusion Standard data for tajik-gateway.org
Resource Scan
Scan Details
| Site Domain | tajik-gateway.org |
| Base Domain | tajik-gateway.org |
| Scan Status | Ok |
| Last Scan | 2025-12-13T19:31:15+00:00 |
| Next Scan | 2025-12-20T19:31:15+00:00 |
Last Scan
| Scanned | 2025-12-13T19:31:15+00:00 |
| URL | https://tajik-gateway.org/robots.txt |
| Domain IPs | 51.178.200.187 |
| Response IP | 51.178.200.187 |
| Found | Yes |
| Hash | abcc9eb078c43b463938a82a3fba1181056a579e44b2c4fec851f0fb065232c5 |
| SimHash | 0c34f8430593 |
Groups
mail.ru
| Rule | Path |
|---|---|
| Disallow | /index.phtml?dt=* |
| Disallow | /*dt%3D* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 100 |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /documents/ |
| Disallow | /download/ |
| Disallow | /galery/ |
| Disallow | /templates/ |
| Disallow | /*dt%3D* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 100 |
googlebot
| Rule | Path |
|---|---|
| Disallow | /documents/ |
| Disallow | /download/ |
| Disallow | /galery/ |
| Disallow | /templates/ |
| Disallow | /*dt%3D* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 100 |
adsbot-google
| Rule | Path |
|---|---|
| Disallow | /documents/ |
| Disallow | /download/ |
| Disallow | /galery/ |
| Disallow | /templates/ |
| Disallow | /*dt%3D* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 100 |
yandexbot
| Rule | Path |
|---|---|
| Disallow | /documents/ |
| Disallow | /download/ |
| Disallow | /galery/ |
| Disallow | /templates/ |
| Disallow | /*dt%3D* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 100 |
*
| Rule | Path |
|---|---|
| Disallow | /*dt%3D* |
| Disallow | /music |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 200 |
Warnings
- 2 invalid lines.
- `host` is not a known field.