Log In Sign Up

womensspot.org
robots.txt

Robots Exclusion Standard data for womensspot.org

Resource Scan

Scan Details

Site Domain womensspot.org
Base Domain womensspot.org
Scan Status Ok
Last Scan2026-03-05T08:52:26+00:00
Next Scan 2026-03-12T08:52:26+00:00

Last Scan

Scanned2026-03-05T08:52:26+00:00
URL https://womensspot.org/robots.txt
Redirect https://www.womensspot.org/robots.txt
Redirect Domain www.womensspot.org
Redirect Base womensspot.org
Domain IPs 216.150.1.1, 216.150.16.1
Redirect IPs 216.150.1.1, 216.150.1.129
Response IP 216.150.1.129
Found Yes
Hash bcfa3ada1b14fae610634c95121b9ffa48d0c14dac141198c69a03f1a4f57ad7
SimHash a50bfc77c661

Groups

*

Rule Path
Allow /
Disallow /api/
Disallow /dashboard
Disallow /profile
Disallow /create-article
Disallow /favorites
Disallow /signin
Disallow /signup
Disallow /forgot-password
Disallow /reset-password
Disallow /confirm-email
Disallow /confirm-newsletter
Disallow /unsubscribe
Allow /site-map

Other Records

Field Value
crawl-delay 1
Back to top

Comments

  • robots.txt for Women's Spot
  • https://womensspot.org/robots.txt
  • Disallow admin and private areas
  • Allow site-map page (the UI page, not the XML sitemap)
  • Crawl-delay (optional, adjust as needed)
Back to top