Log In Sign Up

crispymoney.com
robots.txt

Robots Exclusion Standard data for crispymoney.com

Resource Scan

Scan Details

Site Domain crispymoney.com
Base Domain crispymoney.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-10-04T12:52:09+00:00
Next Scan 2024-11-03T12:52:09+00:00

Last Successful Scan

Scanned2024-05-22T04:00:45+00:00
URL https://crispymoney.com/robots.txt
Domain IPs 104.21.68.82, 172.67.192.59, 2606:4700:3030::6815:4452, 2606:4700:3035::ac43:c03b
Response IP 172.67.192.59
Found Yes
Hash 394c5b38f83e279cb371d5127773a4655dc9961d2c28583975ffac80e8b9cccf
SimHash a855992145c5

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Disallow /admincp/
Disallow /calendar.php
Disallow /director.php
Disallow /errors.php*
Disallow /external.php
Disallow /feed/
Disallow /for%26
Disallow /media/*sort%3Dcomments
Disallow /faq.php
Disallow /moderator.php
Disallow /newreply.php
Disallow /payments.php
Disallow /printthread.php
Disallow /private.php
Disallow /register541.php
Disallow /register*.php
Disallow /report.php
Disallow /search.php
Disallow /sendmessage.php
Disallow /subscription.php
Disallow /trapbot/
Disallow /forums/calendar*

mediapartners-google

Rule Path
Disallow /register*.php
Disallow

msnbot-media

Rule Path
Disallow /admincp/
Disallow /calendar.php
Disallow /director.php
Disallow /errors.php*
Disallow /external.php
Disallow /feed/
Disallow /for%26
Disallow /media/*sort%3Dcomments
Disallow /faq.php
Disallow /moderator.php
Disallow /newreply.php
Disallow /payments.php
Disallow /printthread.php
Disallow /private.php
Disallow /register541.php
Disallow /register*.php
Disallow /report.php
Disallow /search.php
Disallow /sendmessage.php
Disallow /subscription.php
Disallow /trapbot/
Disallow /forums/calendar*

mediapartners-google

Rule Path
Disallow /register*.php
Disallow

bingbot

Rule Path
Disallow /admincp/
Disallow /calendar.php
Disallow /director.php
Disallow /errors.php*
Disallow /external.php
Disallow /feed/
Disallow /for%26
Disallow /media/*sort%3Dcomments
Disallow /faq.php
Disallow /moderator.php
Disallow /newreply.php
Disallow /payments.php
Disallow /printthread.php
Disallow /private.php
Disallow /register541.php
Disallow /register*.php
Disallow /report.php
Disallow /search.php
Disallow /sendmessage.php
Disallow /subscription.php
Disallow /trapbot/
Disallow /forums/calendar*

mediapartners-google

Rule Path
Disallow /register*.php
Disallow

slurp

Rule Path
Disallow /admincp/
Disallow /calendar.php
Disallow /director.php
Disallow /errors.php*
Disallow /external.php
Disallow /feed/
Disallow /for%26
Disallow /media/*sort%3Dcomments
Disallow /faq.php
Disallow /moderator.php
Disallow /newreply.php
Disallow /payments.php
Disallow /printthread.php
Disallow /private.php
Disallow /register541.php
Disallow /register*.php
Disallow /report.php
Disallow /search.php
Disallow /sendmessage.php
Disallow /subscription.php
Disallow /trapbot/
Disallow /forums/calendar*

mediapartners-google

Rule Path
Disallow /register*.php
Disallow

msnbot

Rule Path
Disallow /admincp/
Disallow /calendar.php
Disallow /director.php
Disallow /errors.php*
Disallow /external.php
Disallow /feed/
Disallow /for%26
Disallow /media/*sort%3Dcomments
Disallow /faq.php
Disallow /moderator.php
Disallow /newreply.php
Disallow /payments.php
Disallow /printthread.php
Disallow /private.php
Disallow /register541.php
Disallow /register*.php
Disallow /report.php
Disallow /search.php
Disallow /sendmessage.php
Disallow /subscription.php
Disallow /trapbot/
Disallow /forums/calendar*

mediapartners-google

Rule Path
Disallow /register*.php
Disallow

applebot

Rule Path
Disallow /admincp/
Disallow /calendar.php
Disallow /css.php
Disallow /director.php
Disallow /errors.php*
Disallow /external.php
Disallow /feed/
Disallow /for%26
Disallow /media/*sort%3Dcomments
Disallow /faq.php
Disallow /misc.php
Disallow /moderator.php
Disallow /newreply.php
Disallow /payments.php
Disallow /printthread.php
Disallow /private.php
Disallow /register.php
Disallow /report.php
Disallow /search.php
Disallow /sendmessage.php
Disallow /subscription.php
Disallow /trapbot/
Back to top

Comments

  • robots.txt for http://crispymoney.com/
  • generated by robots.php VERSION
  • All hits are logged. If your crawler isn't a search engine or other service
  • that will bring us traffic or benefit us, its not welcome here.
  • We do not allow non-authorized robots. Any unauthorized bot running on our
  • will have its IP(s) banned. Honey pots are running.
  • Showcontents 1
  • Mozilla/5.0 (compatible; WellKnownBot/0.1; +https://well-known.dev/about/#bot)
  • restrict all access 2
Back to top