Log In Sign Up

captaincook.com.au
robots.txt

Robots Exclusion Standard data for captaincook.com.au

Resource Scan

Scan Details

Site Domain captaincook.com.au
Base Domain captaincook.com.au
Scan Status Ok
Last Scan2024-09-27T04:57:24+00:00
Next Scan 2024-10-04T04:57:24+00:00

Last Scan

Scanned2024-09-27T04:57:24+00:00
URL https://captaincook.com.au/robots.txt
Redirect https://www.captaincook.com.au/robots.txt
Redirect Domain www.captaincook.com.au
Redirect Base captaincook.com.au
Domain IPs 104.26.12.14, 104.26.13.14, 172.67.71.161, 2606:4700:20::681a:c0e, 2606:4700:20::681a:d0e, 2606:4700:20::ac43:47a1
Redirect IPs 108.157.254.107, 108.157.254.13, 108.157.254.58, 108.157.254.82, 2600:9000:2753:1600:0:dcf4:abc0:93a1, 2600:9000:2753:5a00:0:dcf4:abc0:93a1, 2600:9000:2753:8e00:0:dcf4:abc0:93a1, 2600:9000:2753:ba00:0:dcf4:abc0:93a1, 2600:9000:2753:d600:0:dcf4:abc0:93a1, 2600:9000:2753:dc00:0:dcf4:abc0:93a1, 2600:9000:2753:e000:0:dcf4:abc0:93a1, 2600:9000:2753:f400:0:dcf4:abc0:93a1
Response IP 108.157.254.13
Found Yes
Hash c48637b3d26fde6e85fa70e35a9eee422c4ab07cb48b076fe2673452aef4a7c9
SimHash c1781d562735

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/
Back to top

Other Records

Field Value
sitemap https://www.captaincook.com.au/sitemaps-3-sitemap.xml
Back to top

Comments

  • robots.txt for https://www.captaincook.com.au/
  • live - don't allow web crawlers to index cpresources/ or vendor/
Back to top