Log In Sign Up

athenafintech.com
robots.txt

Robots Exclusion Standard data for athenafintech.com

Resource Scan

Scan Details

Site Domain athenafintech.com
Base Domain athenafintech.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2026-02-19T17:10:56+00:00
Next Scan 2026-03-21T17:10:56+00:00

Last Successful Scan

Scanned2025-12-29T16:56:06+00:00
URL https://athenafintech.com/robots.txt
Domain IPs 89.116.192.181
Response IP 89.116.192.181
Found Yes
Hash 6bc20cc018c12c6dcd4cd25e3fc169fa3b1432bc678088082af387f15f95b300
SimHash 296472f2c545

Groups

*

Rule Path
Disallow /private/

*

Rule Path
Disallow /admin/

*

Rule Path
Disallow /wp-admin/

*

Rule Path
Disallow /wp-includes/

*

Rule Path
Disallow

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

bingbot

Rule Path
Allow /

bingpreview

Rule Path
Allow /

slurp

Rule Path
Allow /

yandex

Rule Path
Allow /

baiduspider

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

teoma

Rule Path
Allow /

sogou web spider

Rule Path
Allow /

exabot

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

applebot

Rule Path
Allow /

facebot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.athenafintech.com/sitemap.xml
Back to top

Comments

  • Disallow access to private directories
  • Disallow access to admin directories
  • Disallow access to WordPress admin area
  • Disallow access to WordPress includes
  • Allow all bots to crawl the site
  • Google
  • Bing
  • Yahoo
  • Yandex
  • Baidu
  • DuckDuckGo
  • Ask Jeeves/Teoma
  • Sogou
  • Exalead
  • Alexa
  • Applebot
  • Facebook External Hit
  • LinkedIn
  • Specify the location of your sitemap
Back to top