Log In Sign Up

fajs.org
robots.txt

Robots Exclusion Standard data for fajs.org

Resource Scan

Scan Details

Site Domain fajs.org
Base Domain fajs.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-06-03T16:15:12+00:00
Next Scan 2025-06-10T16:15:12+00:00

Last Successful Scan

Scanned2025-05-26T16:14:17+00:00
URL https://www.fajs.org/robots.txt
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.80.1
Found Yes
Hash 2925c4925a59f29c97556811534e39675e5ce236c981077ebf33322c180ea525
SimHash beffcd20bcb0

Groups

*

Rule Path
Disallow /wp-json/
Disallow /cdn-cgi/bm/cv/
Disallow /cdn-cgi/challenge-platform/

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler
gptbot
google-extended
chatgpt-user
ccbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.fajs.org/sitemap_index.xml
Back to top

Comments

  • Prevent crawling CF challenge URLs
  • Sitemap
  • -----------------
  • Ban bots that don't benefit us.
  • --------------------------------
Back to top