il.fi
robots.txt

Robots Exclusion Standard data for il.fi

Resource Scan

Scan Details

Site Domain il.fi
Base Domain il.fi
Scan Status Ok
Last Scan2024-06-05T00:33:50+00:00
Next Scan 2024-06-12T00:33:50+00:00

Last Scan

Scanned2024-06-05T00:33:50+00:00
URL https://il.fi/robots.txt
Redirect https://www.iltalehti.fi/robots.txt
Redirect Domain www.iltalehti.fi
Redirect Base iltalehti.fi
Domain IPs 13.35.122.105, 13.35.122.109, 13.35.122.38, 13.35.122.69, 2600:9000:246b:1600:7:e7de:8c80:93a1, 2600:9000:246b:2400:7:e7de:8c80:93a1, 2600:9000:246b:6000:7:e7de:8c80:93a1, 2600:9000:246b:8c00:7:e7de:8c80:93a1, 2600:9000:246b:aa00:7:e7de:8c80:93a1, 2600:9000:246b:c600:7:e7de:8c80:93a1, 2600:9000:246b:e200:7:e7de:8c80:93a1, 2600:9000:246b:ea00:7:e7de:8c80:93a1
Redirect IPs 2600:9000:2024:0:3:3f58:f680:93a1, 2600:9000:2024:3600:3:3f58:f680:93a1, 2600:9000:2024:7400:3:3f58:f680:93a1, 2600:9000:2024:9800:3:3f58:f680:93a1, 2600:9000:2024:c600:3:3f58:f680:93a1, 2600:9000:2024:e000:3:3f58:f680:93a1, 2600:9000:2024:e800:3:3f58:f680:93a1, 2600:9000:2024:f800:3:3f58:f680:93a1, 65.9.112.31, 65.9.112.74, 65.9.112.81, 65.9.112.90
Response IP 18.165.171.47
Found Yes
Hash bbab5a8b0f3dfd4b654aedbb8cf5538dd8bf1b200994dd8e77f440b60561ad6d
SimHash 00164848a0b5

Groups

*

Rule Path
Disallow /preview

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgili

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.iltalehti.fi/sitemap/iltalehti.xml
sitemap https://www.iltalehti.fi/sitemap/categories.xml
sitemap https://www.iltalehti.fi/alekoodit/sitemap.xml
sitemap https://www.iltalehti.fi/sitemap/weather/index.xml
sitemap https://www.iltalehti.fi/telkku/assets/sitemap.xml
sitemap https://www.iltalehti.fi/telkku/sitemap/sitemap.xml