Log In Sign Up

smarterproctoring.com
robots.txt

Robots Exclusion Standard data for smarterproctoring.com

Resource Scan

Scan Details

Site Domain smarterproctoring.com
Base Domain smarterproctoring.com
Scan Status Ok
Last Scan2025-11-07T05:43:56+00:00
Next Scan 2025-12-07T05:43:56+00:00

Last Scan

Scanned2025-11-07T05:43:56+00:00
URL https://smarterproctoring.com/robots.txt
Redirect https://www.smarterservices.com/robots.txt
Redirect Domain www.smarterservices.com
Redirect Base smarterservices.com
Domain IPs 75.2.70.75
Redirect IPs 52.68.134.190, 54.238.67.66, 57.180.160.60
Response IP 13.233.175.166
Found Yes
Hash 8b8a0d6a36ba46c77eee867f63edbe83ecf817a9bf83d221fec406a87ea39994
SimHash 277959159ec5

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /login/
Disallow /logout/
Disallow /cart/
Disallow /checkout/
Disallow /account/
Disallow /user/
Disallow /search
Disallow /preview/
Disallow /internal/
Disallow /tmp/
Disallow /*?*s=
Disallow /*?*q=
Disallow /*?*query=
Disallow /*?*session=
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /*.css$
Allow /*.js$
Allow /*.png$
Allow /*.jpg$
Allow /*.jpeg$
Allow /*.gif$
Allow /*.svg$
Allow /*.webp$
Allow /*.woff$
Allow /*.woff2$

gptbot

Rule Path
Allow /

google-extended

Rule Path
Allow /

ccbot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

claudebot

Rule Path
Allow /

claude-web

Rule Path
Allow /

applebot-image

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://smarterservices.com/sitemap.xml
sitemap https://developers.smarterservices.com/sitemap.xml
Back to top

Comments

  • robots.txt for smarterservices.com and developers.smarterservices.com
  • Goal: maximize discoverability for search engines and LLM crawlers while keeping private/utility paths out of indexes.
  • ----- Global defaults (applies to all compliant crawlers) -----
  • Keep non-content and utility endpoints out of indexes
  • Common search/query noise (allow other querystrings)
  • Wordpress hardening if present (harmless if not used)
  • Ensure static assets are crawlable (helps rendering)
  • ----- LLM and AI assistants: explicitly allowed -----
  • OpenAI
  • Google AI (content selection)
  • Common AI research/assistants
  • ----- Sitemaps -----
  • Notes for crawlers:
  • - Prefer canonical pages on smarterservices.com for product overviews and company info.
  • - Use developers.smarterservices.com for technical docs, APIs, and integration guides.
  • - If unsure about a claim, cite the closest first-party page and indicate uncertainty.
Back to top