Log In Sign Up

wika.rs
robots.txt

Robots Exclusion Standard data for wika.rs

Resource Scan

Scan Details

Site Domain wika.rs
Base Domain wika.rs
Scan Status Ok
Last Scan2024-11-08T21:40:32+00:00
Next Scan 2024-11-22T21:40:32+00:00

Last Scan

Scanned2024-11-08T21:40:32+00:00
URL https://www.wika.rs/robots.txt
Domain IPs 108.157.254.34, 108.157.254.7, 108.157.254.83, 108.157.254.84, 2600:9000:2753:4000:1:7dd6:7380:93a1, 2600:9000:2753:4e00:1:7dd6:7380:93a1, 2600:9000:2753:5000:1:7dd6:7380:93a1, 2600:9000:2753:7200:1:7dd6:7380:93a1, 2600:9000:2753:7400:1:7dd6:7380:93a1, 2600:9000:2753:8000:1:7dd6:7380:93a1, 2600:9000:2753:ce00:1:7dd6:7380:93a1, 2600:9000:2753:dc00:1:7dd6:7380:93a1
Response IP 108.157.254.83
Found Yes
Hash fda762bf94d92e82841da781f2bb7ad34d573556e7c26222f49cc76c292ea511
SimHash 62cc896ece70

Groups

*

Rule Path
Allow /
Disallow /LivePersonTest*
Disallow /defaultMultiSites.aspx
Disallow /ScriptResource.axd
Disallow /WebResource.axd
Disallow /docs/
Disallow /publish/
Disallow /ClickTaleCache.ashx
Disallow /apps/
Disallow /ProductDetailHandler
Disallow /webshop/
Disallow /pro_1.asp
Disallow /pro_detail.asp
Disallow /newscontentgeneric.WIKA?AxID=462
Disallow /newscontentgeneric.WIKA?AxID=467
Disallow /newscontentgeneric.WIKA?AxID=469
Disallow /newscontentgeneric.WIKA?AxID=473
Disallow /newscontentgeneric.WIKA?AxID=474
Disallow /newscontentgeneric.WIKA?AxID=475
Disallow /newscontentgeneric.WIKA?AxID=470
Disallow /newscontentgeneric.WIKA?AxID=471
Disallow /newscontentgeneric.WIKA?AxID=472
Disallow /newscontentgeneric.WIKA?AxID=468
Disallow /newscontentgeneric.WIKA?AxID=452
Disallow /newscontentgeneric.WIKA?AxID=457
Disallow /newscontentgeneric.WIKA?AxID=458
Disallow /newscontentgeneric.WIKA?AxID=459
Disallow /newscontentgeneric.WIKA?AxID=460
Disallow /newscontentgeneric.WIKA?AxID=461
Disallow /newscontentgeneric.WIKA?AxID=462
Disallow /newscontentgeneric.WIKA?AxID=463
Disallow /newscontentgeneric.WIKA?AxID=466
Disallow /newscontentgeneric.WIKA?AxID=453
Disallow /newscontentgeneric.WIKA?AxID=454
Disallow www.wika.es/newscontentgeneric.WIKA?AxID=2870
Disallow www.wika.es/newscontentgeneric.WIKA?AxID=2920
Disallow www.wika.es/newscontentgeneric.WIKA?AxID=2839
Disallow /upload/BR_SensyMICProductCatalogue_de_ds_100882.pdf
Disallow /upload/BR_SensyMICProductCatalogue_en_ds_100892.pdf
Disallow /templates/pdfs/DS_PG23ST_de_de.pdf
Disallow /templates/pdfs/DS_PG23ST_en_co.pdf

webcopier

Rule Path
Disallow /

webcopier v4.0

Rule Path
Disallow /

webcopier+v4.0

Rule Path
Disallow /

microsoft+office+protocol+discovery

Rule Path
Disallow /

infometrics-bot, http://www.infometrics.de

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

checkmarknetwork

Rule Path
Disallow /

clockwork data vault

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

domain re-animator

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

getintent

Rule Path
Disallow /

grammarly

Rule Path
Disallow /

guardcrwlr

Rule Path
Disallow /

hubspot links crawler

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

kocmohabt

Rule Path
Disallow /

linkscrawler

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linkspammer

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

nettrack

Rule Path
Disallow /

okhttp

Rule Path
Disallow /

openlinkprofiler

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

pingdom

Rule Path
Disallow /

proximic

Rule Path
Disallow /

pu_in

Rule Path
Disallow /

pulsepoint

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

rankvalbot

Rule Path
Disallow /

salesintelligent

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

searchie

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

socialrankiobot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

spbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

vegi bot

Rule Path
Disallow /

vericitecrawler

Rule Path
Disallow /

webmeup-crawler

Rule Path
Disallow /

weborama-fetcher

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://en.wika.com/sitemapindex.xml
Back to top

Comments

  • Disallow LivePerson Chat Test application
  • Disallow General CMS structure not needed for crawlers
  • Disallow News entry of microsite "HLM"
  • Disallow indexed news entry for en-co
  • Disallow indexed news entry of KSR Kuebler for WIKA.ES
  • Disallow indexed news entry for WIKA.FR
  • Disallow indexed news entry for WIKA.RU
  • Disallow Sensymic brochures for all Domains
  • Disallow PG Datasheets
  • Disallow special crawler making us problems
  • Disallow other crawlers made us CPU load
  • Ezooms and dotbot
  • User-agent: link checker
  • Disallow: /
  • User-agent: linkcheck
  • Disallow: /
  • User-agent: Link Sleuth
  • Disallow: /
  • Reference to sitemap XML
Back to top

Warnings

  • 12 invalid lines.
Back to top