Log In Sign Up

donateers.org
robots.txt

Robots Exclusion Standard data for donateers.org

Resource Scan

Scan Details

Site Domain donateers.org
Base Domain donateers.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-25T00:48:04+00:00
Next Scan 2024-12-24T00:48:04+00:00

Last Successful Scan

Scanned2023-11-05T06:55:57+00:00
URL https://donateers.org/robots.txt
Domain IPs 172.66.40.106, 172.66.43.150, 2606:4700:3108::ac42:286a, 2606:4700:3108::ac42:2b96
Response IP 172.66.40.106
Found Yes
Hash 48cade6993ec5db23255d2eb534c317ab73081e3993250a4bd22a8281a580954
SimHash 0ca30e517c90

Groups

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler

Rule Path
Disallow /

*

Rule Path
Disallow /cdn-cgi/
Disallow /cdn-cgi/bm/cv/
Disallow /cdn-cgi/challenge-platform/
Allow *.css
Back to top

Other Records

Field Value
sitemap https://sitemap.donateers.org/text/al.txt
sitemap https://sitemap.donateers.org/text/ar.txt
sitemap https://sitemap.donateers.org/text/az.txt
sitemap https://sitemap.donateers.org/text/ca.txt
sitemap https://sitemap.donateers.org/text/co.txt
sitemap https://sitemap.donateers.org/text/ct.txt
sitemap https://sitemap.donateers.org/text/dc.txt
sitemap https://sitemap.donateers.org/text/de.txt
sitemap https://sitemap.donateers.org/text/fl.txt
sitemap https://sitemap.donateers.org/text/ga.txt
sitemap https://sitemap.donateers.org/text/hi.txt
sitemap https://sitemap.donateers.org/text/ia.txt
sitemap https://sitemap.donateers.org/text/id.txt
sitemap https://sitemap.donateers.org/text/il.txt
sitemap https://sitemap.donateers.org/text/in.txt
sitemap https://sitemap.donateers.org/text/ks.txt
sitemap https://sitemap.donateers.org/text/ky.txt
sitemap https://sitemap.donateers.org/text/la.txt
sitemap https://sitemap.donateers.org/text/ma.txt
sitemap https://sitemap.donateers.org/text/md.txt
sitemap https://sitemap.donateers.org/text/me.txt
sitemap https://sitemap.donateers.org/text/mi.txt
sitemap https://sitemap.donateers.org/text/mn.txt
sitemap https://sitemap.donateers.org/text/mo.txt
sitemap https://sitemap.donateers.org/text/ms.txt
sitemap https://sitemap.donateers.org/text/mt.txt
sitemap https://sitemap.donateers.org/text/nc.txt
sitemap https://sitemap.donateers.org/text/nd.txt
sitemap https://sitemap.donateers.org/text/ne.txt
sitemap https://sitemap.donateers.org/text/nh.txt
sitemap https://sitemap.donateers.org/text/nj.txt
sitemap https://sitemap.donateers.org/text/nm.txt
sitemap https://sitemap.donateers.org/text/nv.txt
sitemap https://sitemap.donateers.org/text/ny.txt
sitemap https://sitemap.donateers.org/text/oh.txt
sitemap https://sitemap.donateers.org/text/ok.txt
sitemap https://sitemap.donateers.org/text/or.txt
sitemap https://sitemap.donateers.org/text/pa.txt
sitemap https://sitemap.donateers.org/text/ri.txt
sitemap https://sitemap.donateers.org/text/sc.txt
sitemap https://sitemap.donateers.org/text/sd.txt
sitemap https://sitemap.donateers.org/text/tn.txt
sitemap https://sitemap.donateers.org/text/tx.txt
sitemap https://sitemap.donateers.org/text/ut.txt
sitemap https://sitemap.donateers.org/text/va.txt
sitemap https://sitemap.donateers.org/text/vt.txt
sitemap https://sitemap.donateers.org/text/wa.txt
sitemap https://sitemap.donateers.org/text/wi.txt
sitemap https://sitemap.donateers.org/text/wv.txt
sitemap https://sitemap.donateers.org/text/wy.txt
Back to top

Comments

  • Ban noisy bots
  • --------------------------------
Back to top