lowcostroutes.com
robots.txt
Robots Exclusion Standard data for lowcostroutes.com
Resource Scan
Scan Details
| Site Domain | lowcostroutes.com |
| Base Domain | lowcostroutes.com |
| Scan Status | Ok |
| Last Scan | 2024-09-30T02:35:58+00:00 |
| Next Scan | 2024-10-07T02:35:58+00:00 |
Last Scan
| Scanned | 2024-09-30T02:35:58+00:00 |
| URL | https://lowcostroutes.com/robots.txt |
| Domain IPs | 104.21.17.53, 172.67.222.83, 2606:4700:3032::6815:1135, 2606:4700:3034::ac43:de53 |
| Response IP | 104.21.17.53 |
| Found | Yes |
| Hash | 59f6552532b502eac5053e3e9e38ad76f24494e5b305adb018fd866ca9fccec8 |
| SimHash | 44c6d584f5d6 |
Groups
panscient.com
mj12bot
unisterbot
netseer
discoverybot
expo9
ahrefsbot*
pixray-seeker
proximic
seokicks*
applebot
| Product | Comment |
|---|---|
| applebot | temporarily block apple |
| Rule | Path |
|---|---|
| Disallow | / |
googlebot
No rules defined. All paths allowed.
Other Records
| Field | Value | Comment |
|---|---|---|
| crawl-delay | 2 | 1800 per hour |
yandex
No rules defined. All paths allowed.
Other Records
| Field | Value | Comment |
|---|---|---|
| crawl-delay | 20 | 180 per hour OK |
yahoo
No rules defined. All paths allowed.
Other Records
| Field | Value | Comment |
|---|---|---|
| crawl-delay | 10 | 360 per hour |
bingbot
No rules defined. All paths allowed.
Other Records
| Field | Value | Comment |
|---|---|---|
| crawl-delay | 6 | 600 per hour |
baiduspider*
No rules defined. All paths allowed.
Other Records
| Field | Value | Comment |
|---|---|---|
| crawl-delay | 20 | 180 per hour KO |
jikespider*
No rules defined. All paths allowed.
Other Records
| Field | Value | Comment |
|---|---|---|
| crawl-delay | 60 | 60 per hour KO but low traffic. |
*
| Rule | Path |
|---|---|
| Disallow | /*/*set_version |
| Disallow | */adm |
| Disallow | */csr |
| Disallow | */user |
| Disallow | */rating_box |
| Disallow | /_log |
| Disallow | /_script |
| Disallow | /_test |
| Disallow |
Comments