it.exchange-rates.org
robots.txt
Robots Exclusion Standard data for it.exchange-rates.org
Resource Scan
Scan Details
Site Domain | it.exchange-rates.org |
Base Domain | exchange-rates.org |
Scan Status | Ok |
Last Scan | 2024-09-23T16:18:29+00:00 |
Next Scan | 2024-10-23T16:18:29+00:00 |
Last Scan
Scanned | 2024-09-23T16:18:29+00:00 |
URL | https://it.exchange-rates.org/robots.txt |
Domain IPs | 104.21.55.127, 172.67.148.30, 2606:4700:3031::ac43:941e, 2606:4700:3035::6815:377f |
Response IP | 104.21.55.127 |
Found | Yes |
Hash | f94f353256103659e2ebe4691bd392d06d1e5860d56c089757e1e873c3cc15bd |
SimHash | db1e14e26241 |
Groups
*
Rule | Path |
---|---|
Disallow | /?layout=v2 |
Disallow | /?layout=1 |
Disallow | /GetCustomContent.aspx* |
Disallow | /HistoryExchangeRatesReportDownload.aspx* |
Disallow | /*.aspx$ |
Disallow | /*.aspx* |
amazonbot
Rule | Path |
---|---|
Disallow | /id/ |
Disallow | /my/ |
Disallow | /vn/ |
Disallow | /gr/ |
Disallow | /hu/ |
Disallow | /no/ |
Disallow | /pl/ |
Disallow | /ro/ |
Disallow | /ru/ |
Disallow | /sv/ |
Disallow | /tr/ |
Disallow | /de/wechselkursverlauf/ |
Disallow | /es/historial/ |
Disallow | /fr/historique/ |
Disallow | /it/storico-tassi-cambio/ |
Disallow | /ja/exchange-rate-history/ |
Disallow | /ko/exchange-rate-history/ |
Disallow | /nl/geschiedenis-van-wisselkoersen/ |
Disallow | /pt/historico/ |
Disallow | /th/exchange-rate-history/ |
Disallow | /zh/exchange-rate-history/ |
Disallow | /zh-hant/exchange-rate-history/ |
Disallow | /exchange-rate-history/ |
Allow | /exchange-rate-history/usd* |
Allow | /exchange-rate-history/eur* |
Allow | /exchange-rate-history/gbp* |
Allow | /exchange-rate-history/cad* |
Allow | /exchange-rate-history/aud* |
semrushbot-sa
semrushbot
ahrefsbot
dotbot
mj12bot
mauibot
mauibot (crawler.feedback+wc@gmail.com)
megaindex.ru/2.0
seekport crawler
sputnikbot/2.3
the knowledge ai
turnitinbot
ucbrowser
yacybot
yeti
seznambot
coccocbot-web
petalbot
Rule | Path |
---|---|
Disallow | / |
baiduspider
Rule | Path |
---|---|
Disallow | / |
Allow | /zh |
Allow | /zh-hant |
Disallow | /zh/*.aspx$ |
Disallow | /zh/api* |
Disallow | /zh-hant/*.aspx$ |
Disallow | /zh-hant/api* |
Disallow | /*.aspx$ |
Disallow | /*.aspx* |
baiduspider-image
Rule | Path |
---|---|
Disallow | / |
Allow | /zh |
Allow | /zh-hant |
Disallow | /zh/*.aspx$ |
Disallow | /zh-hant/*.aspx$ |
Disallow | /*.aspx$ |
Disallow | /*.aspx* |
youdaobot
Rule | Path |
---|---|
Disallow | / |
Allow | /zh |
Allow | /zh-hant |
Disallow | /zh/*.aspx$ |
Disallow | /zh-hant/*.aspx$ |
Disallow | /*.aspx$ |
Disallow | /*.aspx* |
sogou web spider
Rule | Path |
---|---|
Disallow | / |
Allow | /zh |
Allow | /zh-hant |
Disallow | /zh/*.aspx$ |
Disallow | /zh-hant/*.aspx$ |
Disallow | /*.aspx$ |
Disallow | /*.aspx* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
sogou inst spider
Rule | Path |
---|---|
Disallow | / |
Allow | /zh |
Allow | /zh-hant |
Disallow | /zh/*.aspx$ |
Disallow | /zh-hant/*.aspx$ |
Disallow | /*.aspx$ |
Disallow | /*.aspx* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
sogou spider2
Rule | Path |
---|---|
Disallow | / |
Allow | /zh |
Allow | /zh-hant |
Disallow | /zh/*.aspx$ |
Disallow | /zh-hant/*.aspx$ |
Disallow | /*.aspx$ |
Disallow | /*.aspx* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
Comments