Log In Sign Up

am-switzerland.ch
robots.txt

Robots Exclusion Standard data for am-switzerland.ch

Resource Scan

Scan Details

Site Domain am-switzerland.ch
Base Domain am-switzerland.ch
Scan Status Ok
Last Scan2024-09-20T01:06:22+00:00
Next Scan 2024-10-04T01:06:22+00:00

Last Scan

Scanned2024-09-20T01:06:22+00:00
URL https://am-switzerland.ch/robots.txt
Redirect https://www.am-switzerland.ch/robots.txt
Redirect Domain www.am-switzerland.ch
Redirect Base am-switzerland.ch
Domain IPs 185.90.37.7
Redirect IPs 185.90.37.7
Response IP 185.90.37.7
Found Yes
Hash 16a7ab0a60184410e67321574b0331c5b8c3b2efad0976932e61707ac4514632
SimHash a26b50e24e34

Groups

psbot
yandex
petalbot
mail.ru_bot
megaindex
baiduspider
yisouspider
bytespider
sogou web spider
sogou inst spider
proximic
admantx
seekport crawler
semrushbot
blexbot
mj12bot
dotbot
bubing
omgili
exabot
blp_bbot/0.1
blp_bbot
sentibot
meltawer
digimind
knowings
sindup
cision
talkwater
turnitinbot
converacrawler
jetbot
newsnow
kbcrawl
amisoftware
newzbin
ask n read
qwam content intelligence
zite
flipboard
youmag
synthesio
trendybuzz
spotter
scoop.it
trendybuzz
linkfluence
spotter
augure
corporama
grub-client
ia_archiver
ia_archiver-web.archive.org
k2spider
libwww
wget
adequat
adequat-systems
auramundi
coexel
ellisphere
leadbox
mention
moreover
mytwip
newsnow
newzbin
opinion-tracker
proxem
score3
trendeo
vecteurplus
verticalsearch
vsw
winello
fetch
infoseek
msiecrawler
offline explorer
sitecheck.internetseer.com
teleport
teleportpro
webcopier
webstripper
zealbot
asknread.com
ellisphere
spotter
omgilibot

Rule Path
Disallow /

*

Rule Path
Allow /
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/
Disallow /assets/
Back to top

Other Records

Field Value
sitemap https://www.am-switzerland.ch/de/sitemaps-1-sitemap.xml
sitemap https://www.am-switzerland.ch/fr/sitemaps-1-sitemap.xml
sitemap https://www.am-switzerland.ch/en/sitemaps-1-sitemap.xml
Back to top

Comments

  • robots.txt for https://www.am-switzerland.ch/de/
  • Disallow commercial bots to prevent ad fraud
  • Allow crawling for other bots
Back to top

Warnings

  • 3 invalid lines.
Back to top