arfc.org
robots.txt

Robots Exclusion Standard data for arfc.org

Resource Scan

Scan Details

Site Domain arfc.org
Base Domain arfc.org
Scan Status Ok
Last Scan2026-01-09T20:13:53+00:00
Next Scan 2026-01-16T20:13:53+00:00

Last Scan

Scanned2026-01-09T20:13:53+00:00
URL https://arfc.org/robots.txt
Domain IPs 104.21.94.231, 172.67.141.76, 2606:4700:3034::ac43:8d4c, 2606:4700:3036::6815:5ee7
Response IP 104.21.94.231
Found Yes
Hash a3a44230560cdcbfb13bdb388c9026fda263b02f9625629725adf863344bad5b
SimHash 280c8d202990

Groups

*

Rule Path
Disallow /complaints/9999/unknown
Disallow /removed.aspx

grapeshot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

maxpoint bot

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.arfc.org/sitemap.xml

Comments

  • Sitemap: http://www.arfc.org/all.xml
  • Sitemap: http://www.arfc.org/complaints.xml
  • Disallow: /complaints/*/*/*/*/*/*