counterapi.dev
robots.txt

Robots Exclusion Standard data for counterapi.dev

Resource Scan

Scan Details

Site Domain counterapi.dev
Base Domain counterapi.dev
Scan Status Ok
Last Scan2025-10-07T07:28:55+00:00
Next Scan 2025-11-06T07:28:55+00:00

Last Scan

Scanned2025-10-07T07:28:55+00:00
URL https://counterapi.dev/robots.txt
Domain IPs 104.21.30.167, 172.67.173.64, 2606:4700:3030::6815:1ea7, 2606:4700:3034::ac43:ad40
Response IP 104.21.30.167
Found Yes
Hash 690506532f1c3a179c7fcfae22e3dcee1717c009bae06619bd350ad435564da4
SimHash 69014032e470

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /app/
Disallow /login
Disallow /register
Disallow /api/
Disallow *.css$
Disallow *.js$
Disallow *.json$
Allow /products/
Allow /pricing
Allow /about-us
Allow /contact

Other Records

Field Value
sitemap https://counterapi.dev/sitemap.xml

Comments

  • Block admin and authentication pages
  • Block file types
  • Allow important pages
  • Sitemap location