Log In Sign Up

mass-doc.com
robots.txt

Robots Exclusion Standard data for mass-doc.com

Resource Scan

Scan Details

Site Domain mass-doc.com
Base Domain mass-doc.com
Scan Status Ok
Last Scan2025-10-27T14:33:33+00:00
Next Scan 2025-11-26T14:33:33+00:00

Last Scan

Scanned2025-10-27T14:33:33+00:00
URL https://mass-doc.com/robots.txt
Redirect https://www.mass-doc.com/robots.txt
Redirect Domain www.mass-doc.com
Redirect Base mass-doc.com
Domain IPs 173.248.151.141
Response IP 173.248.151.141
Found Yes
Hash 7e209300d048dcf2bfc70c836e8b3f6e9e5d7cf10ced8af1d60acb462f70a3d3
SimHash 795918d0c5f2

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

perplexitycrawler

Rule Path
Allow /

perplexity-user

Rule Path
Allow /

andibot

Rule Path
Allow /

youbot

Rule Path
Allow /

phindbot

Rule Path
Allow /

firecrawl

Rule Path
Allow /

firecrawlbot

Rule Path
Allow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.mass-doc.com/sitemap2.xml
sitemap https://www.mass-doc.com/imagesitemap.xml
Back to top

Comments

  • ===============================
  • Standard search engines
  • ===============================
  • AI search / browsing agents (allowed)
  • ===============================
  • AI model-training agents (blocked)
  • ===============================
  • Default catch-all
  • ===============================
  • Sitemaps
Back to top