Log In Sign Up

cafcspa.com
robots.txt

Robots Exclusion Standard data for cafcspa.com

Resource Scan

Scan Details

Site Domain cafcspa.com
Base Domain cafcspa.com
Scan Status Ok
Last Scan2026-01-04T21:29:59+00:00
Next Scan 2026-02-03T21:29:59+00:00

Last Scan

Scanned2026-01-04T21:29:59+00:00
URL https://cafcspa.com/robots.txt
Domain IPs 51.136.14.31
Response IP 51.136.14.31
Found Yes
Hash 17632c05bdf993a2affb12a994644575262be089825b217b8de4a533845b8955
SimHash 0b3b844d0374

Groups

blp_bbot
businessdbbot
ccbot
covarioids
converacrawler
curl/
discobot
download ninja
email exractor
ezooms
fdm 3.x
flaxcrawler
grabber
grapeshot
gslfbot
heritrix
httrack
intelium_bot
istellabot
java/
larbin
lemurwebcrawler
libwww-perl
metamojicrawler
mj12bot
nutch
openacoon
php/
plukkie
proximic
python-urllib
ruby
seokicks
spbot
turnitinbot
yandexbot
wbsearchbot
weblexbot
wget
wire/0.
zyborg

Rule Path
Disallow /

msnbot/bingbot
bingbot
msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

*

Rule Path
Disallow /*.wgx$
Disallow /AdminCMS
Disallow /admincms
Disallow /BatchCMS
Disallow /batchcms
Disallow /Admin/
Disallow /admin/
Disallow /Auth/
Disallow /Auth$
Disallow /auth/
Disallow /Batch/
Disallow /batch/
Disallow /Custom/
Disallow /custom/
Disallow /Tmp/
Disallow /tmp/
Disallow /LazyLogin
Disallow /lazylogin
Disallow /ShowReel
Disallow /showreel
Disallow /Search/SearchCMS
Disallow /search/searchcms
Disallow /*%3Dprint%3Dtrue
Disallow /*%26ListMode%3D*
Disallow /*%26listmode%3D*
Disallow /*?ListMode=*
Disallow /*?listmode=*
Disallow /*%26SortMode%3D*
Disallow /*%26sortmode%3D*
Disallow /*?SortMode=*
Disallow /*?sortmode=*
Disallow /AreaPersonale
Back to top

Other Records

Field Value
sitemap http://cafc.site/sitemapcms
Back to top

Warnings

  • 1 invalid line.
Back to top