travelgay.ru
robots.txt
Robots Exclusion Standard data for travelgay.ru
Resource Scan
Scan Details
| Site Domain | travelgay.ru |
| Base Domain | travelgay.ru |
| Scan Status | Ok |
| Last Scan | 2024-11-12T21:22:03+00:00 |
| Next Scan | 2024-11-19T21:22:03+00:00 |
Last Scan
| Scanned | 2024-11-12T21:22:03+00:00 |
| URL | https://travelgay.ru/robots.txt |
| Redirect | https://ru.travelgay.com/robots.txt |
| Redirect Domain | ru.travelgay.com |
| Redirect Base | travelgay.com |
| Domain IPs | 104.21.44.236, 172.67.205.63, 2606:4700:3031::6815:2cec, 2606:4700:3037::ac43:cd3f |
| Redirect IPs | 104.18.18.125, 104.18.19.125, 2606:4700::6812:127d, 2606:4700::6812:137d |
| Response IP | 104.18.18.125 |
| Found | Yes |
| Hash | f0851bf025032b4658d6ee5b8edb2e86cde2c840a184ce4b6a9361e12b35cf10 |
| SimHash | 6f0c9844e793 |
Groups
*
| Rule | Path |
|---|---|
| Disallow |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://ru.travelgay.com/sitemap.xml |