allcharts.info
robots.txt
Robots Exclusion Standard data for allcharts.info
Resource Scan
Scan Details
| Site Domain | allcharts.info |
| Base Domain | allcharts.info |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-11-23T20:10:18+00:00 |
| Next Scan | 2025-12-07T20:10:18+00:00 |
Last Successful Scan
| Scanned | 2025-11-08T11:17:32+00:00 |
| URL | https://allcharts.info/robots.txt |
| Domain IPs | 143.204.160.119, 143.204.160.122, 143.204.160.22, 143.204.160.65, 2600:9000:21f8:3600:16:e06a:7680:93a1, 2600:9000:21f8:7600:16:e06a:7680:93a1, 2600:9000:21f8:a800:16:e06a:7680:93a1, 2600:9000:21f8:cc00:16:e06a:7680:93a1, 2600:9000:21f8:d000:16:e06a:7680:93a1, 2600:9000:21f8:e000:16:e06a:7680:93a1, 2600:9000:21f8:e400:16:e06a:7680:93a1, 2600:9000:21f8:f600:16:e06a:7680:93a1 |
| Response IP | 13.226.2.66 |
| Found | Yes |
| Hash | a1a13863f6a0ddf5f821943f639037e9d09c6c0a812d276fe6cbcd2481000946 |
| SimHash | 10115bf0c6e9 |
Groups
googlebot
googlebot-image
googlebot-video
googlebot-news
google-extended
bingbot
msnbot
duckduckbot
qwantify
applebot
applebot-extended
yandexbot
yandeximages
petalbot
| Rule | Path |
|---|---|
| Disallow | /geodata_postcode/ |
| Disallow | /geodata_gebieden/ |
| Disallow | /geo_onderwijs/ |
| Allow | / |
*
| Rule | Path |
|---|---|
| Disallow | /geodata_postcode/ |
| Disallow | /geodata_gebieden/ |
| Disallow | /geo_onderwijs/ |
| Disallow | /afbeeldingen/ |
| Disallow | /images/ |
| Disallow | /kaarten/ |
| Disallow | /maps/ |
| Allow | / |
ai2bot
ahrefsbot
amazonbot
anthropic-ai
arquivo-web-crawler
archive.org_bot
baiduspider
baiduspider-image
blexbot
bytespider
ccbot
chatgpt-user
claude-web
claudebot
cohere-ai
cohere-training-data-crawler
cognitiveseo bot
dataforseo
diffbot
dotbot
duckassistbot
exabot
facebookbot
friendlycrawler
gptbot
gptbot-image
gptbot-video
gptcrawler
google-cloudvertexbot
ia_archiver
ia_archiver-web.archive.org
imagesiftbot
img2dataset
kangaroo-llm
linkpadbot
lipperhey
lrtbot
magpie-crawler
meta-externalagent
meta-externalfetcher
mj12bot
netpeakspiderbot
nibbler
oai-searchbot
oncrawl
openai-searchbot
omgili
omgilibot
operator
pangubot
peer39_crawler
perplexity-user
perplexitybot
petalbot
proximic
researchbot
rogerbot
seokicks-robot
semrushbot
semrushbot-ocob
semrushbot-swa
silktidebot
spbot
timpibot
webzio-extended
youbot
yandexadditional
yandexadditionalbot
| Rule | Path |
|---|---|
| Disallow | /geodata_postcode/ |
| Disallow | /geodata_gebieden/ |
| Disallow | /geo_onderwijs/ |
| Disallow | /afbeeldingen/ |
| Disallow | /images/ |
| Disallow | /kaarten/ |
| Disallow | /maps/ |
| Disallow | /postcode/ |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://allecijfers.nl/sitemap.xml |
Comments