cyber-gateway.net
robots.txt
Robots Exclusion Standard data for cyber-gateway.net
Resource Scan
Scan Details
| Site Domain | cyber-gateway.net |
| Base Domain | cyber-gateway.net |
| Scan Status | Ok |
| Last Scan | 2025-10-01T07:38:23+00:00 |
| Next Scan | 2025-10-31T07:38:23+00:00 |
Last Scan
| Scanned | 2025-10-01T07:38:23+00:00 |
| URL | https://cyber-gateway.net/robots.txt |
| Domain IPs | 104.21.54.201, 172.67.169.3 |
| Response IP | 104.21.54.201 |
| Found | Yes |
| Hash | 90125dd136e25796bf264c7e8b1a7d8e23e909a82724450dc5da377e151e8f90 |
| SimHash | da78daf2ec01 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /financial-information |
| Disallow | /bin/ |
| Disallow | /layouts/ |
| Disallow | /itemlist/ |
| Disallow | /category/ |
| Disallow | */tag/* |
| Disallow | /cache/ |
| Disallow | /cli/ |
| Disallow | /includes/ |
| Disallow | /language/ |
| Disallow | /libraries/ |
| Disallow | /logs/ |
| Disallow | /tmp/ |
googlebot
| Rule | Path |
|---|---|
| Disallow | /financial-information |
| Disallow | /bin/ |
| Disallow | /layouts/ |
| Disallow | /itemlist/ |
| Disallow | /category/ |
| Disallow | */tag/* |
| Disallow | /cache/ |
| Disallow | /cli/ |
| Disallow | /includes/ |
| Disallow | /language/ |
| Disallow | /libraries/ |
| Disallow | /logs/ |
| Disallow | /tmp/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
yandex
| Rule | Path |
|---|---|
| Disallow | /financial-information |
| Disallow | /bin/ |
| Disallow | /layouts/ |
| Disallow | /itemlist/ |
| Disallow | /category/ |
| Disallow | */tag/* |
| Disallow | /cache/ |
| Disallow | /cli/ |
| Disallow | /includes/ |
| Disallow | /language/ |
| Disallow | /libraries/ |
| Disallow | /logs/ |
| Disallow | /tmp/ |