ecowijs.nl
robots.txt

Robots Exclusion Standard data for ecowijs.nl

Resource Scan

Scan Details

Site Domain ecowijs.nl
Base Domain ecowijs.nl
Scan Status Ok
Last Scan2024-09-27T01:28:03+00:00
Next Scan 2024-10-04T01:28:03+00:00

Last Scan

Scanned2024-09-27T01:28:03+00:00
URL https://ecowijs.nl/robots.txt
Domain IPs 162.159.152.23, 162.159.153.241, 2606:4700:7::a29f:9817, 2606:4700:7::a29f:99f1
Response IP 162.159.153.241
Found Yes
Hash 0d381934da769ac4f6c9b244ab08619fd8b5cb99b27fb84369367bea85015643
SimHash de274d20beb0

Groups

*

Rule Path
Disallow /wp-json/
Disallow */feed/
Disallow /beheer-abonnementen*
Disallow */?replytocom*
Disallow /wp-json/
Disallow /?rest_route=
Disallow /wp-admin/
Disallow /wp-content/cache/
Disallow /wp-content/plugins/
Disallow /xmlrpc.php
Disallow /wp-includes/
Allow /wp-includes/css/
Allow /wp-includes/js/
Disallow /cdn-cgi/bm/cv/
Disallow /cdn-cgi/challenge-platform/

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler
mj12bot
yandeximages
ahrefsbot
nuclei
wikido
semrushbot
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow *?p=*
Disallow /wp-includes/
Disallow /wp-login.php
Disallow /wp-admin/*
Disallow /recommends/*
Disallow /?logout&redirect_to=
Disallow *?s=*
Disallow *?q=*
Disallow /search/*
Disallow /search*
Disallow /uncategorized/*
Disallow /wprm_print/*

Other Records

Field Value
sitemap https://ecowijs.nl/sitemap_index.xml

Comments

  • Global rules
  • -----------------
  • Prevent crawling CF challenge URLs
  • Sitemap
  • -----------------
  • Ban bots that don't benefit us.
  • --------------------------------