fjcc.com.au
robots.txt

Robots Exclusion Standard data for fjcc.com.au

Resource Scan

Scan Details

Site Domain fjcc.com.au
Base Domain fjcc.com.au
Scan Status Ok
Last Scan2024-11-05T14:40:27+00:00
Next Scan 2024-11-19T14:40:27+00:00

Last Scan

Scanned2024-11-05T14:40:27+00:00
URL https://fjcc.com.au/robots.txt
Domain IPs 103.42.110.14
Response IP 103.42.110.14
Found Yes
Hash bb823801199c48a19802a4591e1b126cae3569bafcd2559f0c62e52541903e8b
SimHash 25004b10c301

Groups

*

Rule Path
Disallow /

googlebot
mediapartners-google
slurp
msnbot

Rule Path
Allow /sitemap.xml$
Allow /robots.txt$
Allow /index.htm$
Disallow /bot-trap/
Disallow /cgi-bin/
Disallow /fjcc.com.au/
Disallow /fjcc_avatars/
Disallow /forum_2015/
Disallow /forum_21B3/
Disallow /forum_fjcc/
Allow /forum_fjcc/index.php$
Allow /forum_fjcc/index.php?topic=*.0$
Allow /forum_fjcc/index.php?topic=*.*0$
Allow /forum_fjcc/index.php?topic=*.*5$
Allow /forum_fjcc/index.php?board=*.0$
Allow /forum_fjcc/index.php?board=*.*0$
Allow /forum_fjcc/index.php?board=*.*5$
Disallow /forum_labs/
Disallow /forum_test/
Disallow /images/
Disallow /test_avatars/

Comments

  • Disallow everything
  • Allow Googlebot, Yahoo and MSN
  • SMF stuff
  • Now allow bits and then disallow bits