autohut.ro
robots.txt

Robots Exclusion Standard data for autohut.ro

Resource Scan

Scan Details

Site Domain autohut.ro
Base Domain autohut.ro
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-04T09:05:04+00:00
Next Scan 2025-12-18T09:05:04+00:00

Last Successful Scan

Scanned2025-10-26T10:41:50+00:00
URL https://autohut.ro/robots.txt
Redirect https://www.autohut.ro/robots.txt
Redirect Domain www.autohut.ro
Redirect Base autohut.ro
Domain IPs 104.21.85.251, 172.67.213.74, 2606:4700:3030::6815:55fb, 2606:4700:3035::ac43:d54a
Redirect IPs 104.21.85.251, 172.67.213.74, 2606:4700:3030::6815:55fb, 2606:4700:3035::ac43:d54a
Response IP 172.67.213.74
Found Yes
Hash e5ba9a8d6b5d8d4151812160f9813c0472662d7617660df3f6ecf77b2c5cb43c
SimHash 71164250f6e0

Groups

*

Rule Path
Disallow /cautare-piesa/
Disallow /filtre-*
Disallow /produse_2018_*/
Disallow /produse_2019_*/
Disallow /produse_2020_*/
Disallow /produse_2022_*/
Disallow /produse_2023_*/
Disallow /produse_2024_*/
Disallow /produse_2025_*/
Disallow /produse_2026_*/
Disallow /produse_2027_*/
Disallow /produse_2028_*/
Disallow /produse_2029_*/
Disallow /produse_2030_*/
Disallow /produse_2031_*/
Disallow /produse_2032_*/
Disallow /produse_2033_*/
Disallow /produse_2034_*/
Disallow /produse_2035_*/
Disallow /produse_2036_*/
Disallow /produse_2037_*/
Disallow /produse_2038_*/
Disallow /produse_2039_*/
Disallow /produse_2040_*/
Disallow /produse_2041_*/
Disallow /produse_2042_*/
Disallow /produse_2043_*/
Disallow /produse_2044_*/
Disallow /produse_2045_*/
Disallow *%3B*
Disallow *%2C*

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

facebot

Rule Path
Disallow /

pinterest

Rule Path
Disallow /

linkedinbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /
Disallow /cautare-piesa/

bingbot

Rule Path
Allow /
Disallow /cautare-piesa/

duckduckbot

Rule Path
Allow /
Disallow /cautare-piesa/

applebot

Rule Path
Allow /
Disallow /cautare-piesa/

Comments

  • ==========================================================
  • ePiesa.ro - Robots.txt (versiune optimizată anti-scraping)
  • Ultima actualizare: Octombrie 2025
  • ==========================================================
  • ----------------------------------------------------------
  • Directive generale (permite indexarea normală, interzice AI training)
  • ----------------------------------------------------------
  • ----------------------------------------------------------
  • Blochează paginile dinamice și de căutare (anti-scraping)
  • ----------------------------------------------------------
  • ----------------------------------------------------------
  • Blocare crawlere / boți inutili sau agresivi
  • ----------------------------------------------------------
  • ----------------------------------------------------------
  • Blocare roboți AI și de conținut automat (Cloudflare Managed list)
  • ----------------------------------------------------------
  • ----------------------------------------------------------
  • Permisiuni pentru motoare de căutare legitime (SEO)
  • ----------------------------------------------------------

Warnings

  • 2 invalid lines.
  • `content-signal` is not a known field.