Log In Sign Up

trusted.de
robots.txt

Robots Exclusion Standard data for trusted.de

Resource Scan

Scan Details

Site Domain trusted.de
Base Domain trusted.de
Scan Status Ok
Last Scan2024-09-12T11:06:31+00:00
Next Scan 2024-10-12T11:06:31+00:00

Last Scan

Scanned2024-09-12T11:06:31+00:00
URL https://trusted.de/robots.txt
Domain IPs 104.26.2.79, 104.26.3.79, 172.67.75.96, 2606:4700:20::681a:24f, 2606:4700:20::681a:34f, 2606:4700:20::ac43:4b60
Response IP 172.67.75.96
Found Yes
Hash a7de6f46bcdd51dbfc12341a9836859bb75959831e9588d834b8f4706675801b
SimHash 1250c440e911

Groups

*

Rule Path
Allow /
Disallow /partner/
Disallow /datenschutz
Disallow /impressum
Disallow /versand
Disallow /*gclid
Disallow /*msclkid
Disallow /*popup
Disallow /catalog/category
Disallow /*-tarif-vergleich
Disallow /suche

ahrefsbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ezooms robot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

netestate ne crawler (+http://www.website-datenbank.de/)

Rule Path
Disallow /

wiseguys robot

Rule Path
Disallow /

turnitin robot

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

pimonster

Rule Path
Disallow /

pimonster

Rule Path
Disallow /

pi-monster

Rule Path
Disallow /

eccp/1.0 (search@eniro.com)

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

lcc

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /
Back to top

Comments

  • Block Searchmetrics Bot
  • Block MJ12bot as it is just noise
  • Block Sogou
  • Block BlexBot
  • Block Ezooms Robot
  • Block BlexBot
  • Block netEstate NE Crawler (+http://www.website-datenbank.de/)
  • Block WiseGuys Robot
  • Block Turnitin Robot
  • Block Heritrix
  • Block pricepi
  • Block Eniro
  • Block SoGou
  • Block Youdao
Back to top

Warnings

  • 2 invalid lines.
Back to top