collada.co.nz
robots.txt

Robots Exclusion Standard data for collada.co.nz

Resource Scan

Scan Details

Site Domain collada.co.nz
Base Domain collada.co.nz
Scan Status Ok
Last Scan2024-05-17T19:57:06+00:00
Next Scan 2024-06-16T19:57:06+00:00

Last Scan

Scanned2024-05-17T19:57:06+00:00
URL https://collada.co.nz/robots.txt
Domain IPs 104.21.64.68, 172.67.177.84, 2606:4700:3031::6815:4044, 2606:4700:3035::ac43:b154
Response IP 172.67.177.84
Found Yes
Hash 760357631fd5f9061fc15fb09e4e8d9e8704df310150b4da85eeaa66476819b0
SimHash 8c125823c9a8

Groups

*

Rule Path
Disallow
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Allow /wp-includes/js/
Allow /wp-includes/images/
Disallow /wp-config-sample.php
Disallow /wp-admin/install.php
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /cgi-bin/
Disallow /junk/
Disallow /tmp/
Disallow /.tmp/
Disallow /cdn-cgi/bm/cv/
Disallow /cdn-cgi/challenge-platform/
Disallow /.well-known/acme-challenge/
Disallow /trackback/
Disallow /xmlrpc.php
Disallow /feed/
Disallow /readme.html
Disallow /readme.*.html
Disallow /licence.txt
Disallow /license.txt
Disallow /installer.php
Disallow /install.php
Disallow /phpinfo.php
Disallow /?s=*
Disallow /search/*
Disallow /site.webmanifest
Disallow /wp-json/

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://collada.co.nz/sitemap_index.xml

Comments

  • Allow All Bots
  • Link to sitemap
  • Block the admin area but allow ajax callbacks
  • Block WP Includes
  • Block WP Install
  • Block Regsiter and Login
  • Block cgi-bin & temp
  • Block SSL Challlenge
  • Block Trackback, Feed and XMLrpc
  • Disallow Common
  • Block Search
  • Disallow Bot Spam
  • Block wp-json API
  • Reject zero benefit bots