dotcmscloud.com
robots.txt

Robots Exclusion Standard data for dotcmscloud.com

Resource Scan

Scan Details

Site Domain dotcmscloud.com
Base Domain dotcmscloud.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-06-07T21:34:22+00:00
Next Scan 2025-09-05T21:34:22+00:00

Last Successful Scan

Scanned2024-01-22T21:29:12+00:00
URL http://dotcmscloud.com/robots.txt
Redirect https://www.dotcms.com/robots.txt
Redirect Domain www.dotcms.com
Redirect Base dotcms.com
Domain IPs 34.199.105.149, 54.227.235.186
Redirect IPs 138.199.46.68, 2400:52e0:1500::868:1
Response IP 138.199.46.68
Found Yes
Hash 3a841d9f270ce702904882e9ad421d775f9afef90f43b1cbf76e6462c802ade4
SimHash 6f00c973d533

Groups

*

Rule Path
Allow /api/*
Disallow /docs/webdav/autopub/*
Disallow /webdav/*
Disallow /html/*
Disallow /landing-pages/*
Disallow /*/javadocs/*
Disallow /javadocs/*
Disallow /rensai/*
Disallow /docs/1.*
Disallow /docs/2.*
Disallow /docs/3.*
Disallow /docs/4.*
Disallow /docs/5.*
Disallow /servlets/*
Disallow /download/nightly-builds*
Disallow /accelerators/*
Disallow /shop/*
Disallow /doc/dotmarketing/*
Disallow /docs/com/dotmarketing/*
Disallow /browse/*
Disallow /devwiki/*

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 4

Other Records

Field Value
sitemap https://www.dotcms.com/api/vtl/sitemap