flevopost.nl
robots.txt

Robots Exclusion Standard data for flevopost.nl

Resource Scan

Scan Details

Site Domain flevopost.nl
Base Domain flevopost.nl
Scan Status Ok
Last Scan2024-11-05T12:18:48+00:00
Next Scan 2024-11-12T12:18:48+00:00

Last Scan

Scanned2024-11-05T12:18:48+00:00
URL https://flevopost.nl/robots.txt
Domain IPs 104.18.42.31, 172.64.145.225, 2606:4700:4400::6812:2a1f, 2606:4700:4400::ac40:91e1
Response IP 172.64.145.225
Found Yes
Hash d9017c0846221a6bf3d8a6697fef5b3d764c1c9979b7dac7138fb404380604ea
SimHash 721c9b68a6a0

Groups

*

Rule Path
Disallow /most-read
Disallow /tag
Disallow /search
Allow /

*

Rule Path
Disallow /economie/Truc-kost-personeel-uitleenbedrijf-geld-21251470.html

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://static.flevopost.nl/sitemap/sitemap.xml.gz
sitemap https://static.flevopost.nl/sitemap/sitemap_news.xml.gz
sitemap https://static.flevopost.nl/sitemap/sitemap_sections.xml.gz