svet.rs
robots.txt

Robots Exclusion Standard data for svet.rs

Resource Scan

Scan Details

Site Domain svet.rs
Base Domain svet.rs
Scan Status Ok
Last Scan2025-10-24T08:23:17+00:00
Next Scan 2025-10-31T08:23:17+00:00

Last Scan

Scanned2025-10-24T08:23:17+00:00
URL https://svet.rs/robots.txt
Redirect https://svet-scandal.rs/robots.txt
Redirect Domain svet-scandal.rs
Redirect Base svet-scandal.rs
Domain IPs 104.21.9.80, 172.67.189.47, 2606:4700:3035::6815:950, 2606:4700:3036::ac43:bd2f
Redirect IPs 104.21.59.153, 172.67.179.193, 2606:4700:3031::ac43:b3c1, 2606:4700:3033::6815:3b99
Response IP 104.21.59.153
Found Yes
Hash 1a49d048e38cceafd4cc9453eb997cc3a5ab67734fff37028f326088e610a7b9
SimHash e88c120f6d95

Groups

*

Rule Path
Allow /
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /feed/
Disallow */feed/

Other Records

Field Value
sitemap https://svet-scandal.rs/sitemap.xml
sitemap https://svet-scandal.rs/news.xml

Comments

  • Primenjuje se na sve botove
  • Dozvoli pristup svim javnim delovima sajta
  • Blokiraj pristup administratorskom panelu (osim AJAX poziva)
  • Blokiraj feedove (RSS, Atom) jer nisu korisni za SEO
  • Privremena dozvola Googleu da crawl-uje search stranice,
  • kako bi video "410 Gone" i uklonio spam URL-ove
  • (biće vraćeno za ~30-60 dana)
  • Disallow: /?s=
  • Disallow: /*&s=
  • Disallow: /search/
  • Disallow: /pretraga/?rezultati-pretrage=
  • Disallow: /*rezultati-pretrage=
  • XML sitemape