Log In Sign Up

keskustelut.iltasanomat.fi
robots.txt

Robots Exclusion Standard data for keskustelut.iltasanomat.fi

Resource Scan

Scan Details

Site Domain keskustelut.iltasanomat.fi
Base Domain iltasanomat.fi
Scan Status Ok
Last Scan2024-09-21T16:16:39+00:00
Next Scan 2024-09-28T16:16:39+00:00

Last Scan

Scanned2024-09-21T16:16:39+00:00
URL https://keskustelut.iltasanomat.fi/robots.txt
Redirect https://www.is.fi/robots.txt
Redirect Domain www.is.fi
Redirect Base is.fi
Domain IPs 2600:9000:2024:6600:10:27c5:9480:93a1, 2600:9000:2024:6800:10:27c5:9480:93a1, 2600:9000:2024:7a00:10:27c5:9480:93a1, 2600:9000:2024:7e00:10:27c5:9480:93a1, 2600:9000:2024:800:10:27c5:9480:93a1, 2600:9000:2024:aa00:10:27c5:9480:93a1, 2600:9000:2024:d400:10:27c5:9480:93a1, 2600:9000:2024:da00:10:27c5:9480:93a1, 65.9.112.124, 65.9.112.76, 65.9.112.80, 65.9.112.94
Redirect IPs 18.161.97.24, 18.161.97.29, 18.161.97.69, 18.161.97.82, 2600:9000:2024:1600:1b:b70c:39c0:93a1, 2600:9000:2024:4200:1b:b70c:39c0:93a1, 2600:9000:2024:6600:1b:b70c:39c0:93a1, 2600:9000:2024:8000:1b:b70c:39c0:93a1, 2600:9000:2024:8e00:1b:b70c:39c0:93a1, 2600:9000:2024:cc00:1b:b70c:39c0:93a1, 2600:9000:2024:de00:1b:b70c:39c0:93a1, 2600:9000:2024:e800:1b:b70c:39c0:93a1
Response IP 65.9.112.54
Found Yes
Hash ff5bd712367397e0394b17cfe38c7498f9cda51ac0d1315965b99c9d38550ee3
SimHash 491cc160c693

Groups

*

Rule Path
Disallow /promo/
Disallow /sivulaskuri
Disallow /api/
Disallow /rest/
Disallow /public-transit-screen/
Allow /api/paid-article/

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.is.fi/sitemap/html/is/sitemapindex.xml
sitemap https://www.is.fi/rss/custom/news-sitemap.xml
sitemap https://www.is.fi/supersaa/assets/sitemap-index.xml
Back to top