rtvoost.nl
robots.txt

Robots Exclusion Standard data for rtvoost.nl

Resource Scan

Scan Details

Site Domain rtvoost.nl
Base Domain rtvoost.nl
Scan Status Ok
Last Scan2024-06-19T23:31:39+00:00
Next Scan 2024-06-26T23:31:39+00:00

Last Scan

Scanned2024-06-19T23:31:39+00:00
URL https://rtvoost.nl/robots.txt
Redirect https://www.rtvoost.nl/robots.txt
Redirect Domain www.rtvoost.nl
Redirect Base rtvoost.nl
Domain IPs 2001:1690:100:1201::20, 62.165.73.20
Redirect IPs 18.244.214.50, 18.244.214.51, 18.244.214.68, 18.244.214.89, 2600:9000:24ba:2c00:13:781e:4740:93a1, 2600:9000:24ba:4600:13:781e:4740:93a1, 2600:9000:24ba:5000:13:781e:4740:93a1, 2600:9000:24ba:6200:13:781e:4740:93a1, 2600:9000:24ba:7800:13:781e:4740:93a1, 2600:9000:24ba:e00:13:781e:4740:93a1, 2600:9000:24ba:e800:13:781e:4740:93a1, 2600:9000:24ba:f800:13:781e:4740:93a1
Response IP 3.160.246.22
Found Yes
Hash 8342ddceb49dc82b7f2063eec1c0b478557f3280e43d3861046896c496bd2ef7
SimHash 6418ba58a400

Groups

*

Rule Path
Allow /
Disallow /content/
Disallow /embedded/
Disallow /data/
Disallow /inc/
Disallow /*/-
Disallow /nos/
Disallow /zoeken?q=*

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

peer39_crawler

Rule Path
Disallow /

peer39_crawler/1.0

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.rtvoost.nl/sitemap/sitemap.xml.gz