Log In Sign Up

oldwildwest.it
robots.txt

Robots Exclusion Standard data for oldwildwest.it

Resource Scan

Scan Details

Site Domain oldwildwest.it
Base Domain oldwildwest.it
Scan Status Ok
Last Scan2024-09-19T16:28:01+00:00
Next Scan 2024-10-19T16:28:01+00:00

Last Scan

Scanned2024-09-19T16:28:01+00:00
URL https://oldwildwest.it/robots.txt
Redirect https://www.oldwildwest.it/robots.txt
Redirect Domain www.oldwildwest.it
Redirect Base oldwildwest.it
Domain IPs 23.101.66.12
Redirect IPs 23.101.66.12
Response IP 23.101.66.12
Found Yes
Hash f1a6925ec9c70f415c5bcbcc4b8788e279f6efcdca7c5b74d9c994debadbd299
SimHash 0b1b044d0374

Groups

blp_bbot
businessdbbot
ccbot
covarioids
converacrawler
curl/
discobot
download ninja
email exractor
ezooms
fdm 3.x
flaxcrawler
grabber
grapeshot
gslfbot
heritrix
httrack
intelium_bot
istellabot
java/
larbin
lemurwebcrawler
libwww-perl
metamojicrawler
mj12bot
nutch
openacoon
php/
plukkie
proximic
python-urllib
ruby
seokicks
spbot
turnitinbot
yandexbot
wbsearchbot
weblexbot
wget
wire/0.
zyborg

Rule Path
Disallow /

msnbot/bingbot
bingbot
msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

*

Rule Path
Disallow /*.wgx$
Disallow /AdminCMS
Disallow /admincms
Disallow /BatchCMS
Disallow /batchcms
Disallow /Admin/
Disallow /admin/
Disallow /Auth/
Disallow /Auth$
Disallow /auth/
Disallow /Batch/
Disallow /batch/
Disallow /Custom/
Disallow /custom/
Disallow /Tmp/
Disallow /tmp/
Disallow /LazyLogin
Disallow /lazylogin
Disallow /ShowReel
Disallow /showreel
Disallow /Search/SearchCMS
Disallow /search/searchcms
Disallow /*%3Dprint%3Dtrue
Disallow /*%26ListMode%3D*
Disallow /*%26listmode%3D*
Disallow /*?ListMode=*
Disallow /*?listmode=*
Disallow /*%26SortMode%3D*
Disallow /*%26sortmode%3D*
Disallow /*?SortMode=*
Disallow /*?sortmode=*
Disallow /AreaPersonale
Back to top

Other Records

Field Value
sitemap https://www.oldwildwest.it/sitemapcms
Back to top

Warnings

  • 1 invalid line.
Back to top