Log In Sign Up

clujenii.ro
robots.txt

Robots Exclusion Standard data for clujenii.ro

Resource Scan

Scan Details

Site Domain clujenii.ro
Base Domain clujenii.ro
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-25T15:15:00+00:00
Next Scan 2025-10-02T15:15:00+00:00

Last Successful Scan

Scanned2025-09-17T10:45:49+00:00
URL https://clujenii.ro/robots.txt
Domain IPs 89.42.218.117
Response IP 89.42.218.117
Found Yes
Hash e78021252372aa0df39499e530b9d91dde14ca32b312eeb192a08b44796f51e8
SimHash 42261d72ceb3

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /cgi-bin/
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /xmlrpc.php
Disallow /*?replytocom
Disallow /*?amp
Disallow /*?share=*
Disallow /*?fbclid=*

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

googlebot-news

Rule Path
Allow /

googlebot

Rule Path
Allow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://clujenii.ro/sitemap_index.xml
Back to top

Comments

  • Sitemap generat de Yoast SEO
  • Evităm pagini duplicate sau irelevante pentru SEO
  • Crawl-delay pentru roboți agresivi
  • Permitem acces complet pentru Googlebot-News
  • Alte directive pentru Google
  • Blocare totală pentru crawlere spam sau irelevante
Back to top