autooglasi.rs
robots.txt

Robots Exclusion Standard data for autooglasi.rs

Resource Scan

Scan Details

Site Domain autooglasi.rs
Base Domain autooglasi.rs
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-01T22:56:47+00:00
Next Scan 2024-07-30T22:56:47+00:00

Last Successful Scan

Scanned2023-06-14T21:24:00+00:00
URL https://autooglasi.rs/robots.txt
Domain IPs 104.21.15.117, 172.67.162.148, 2606:4700:3031::6815:f75, 2606:4700:3031::ac43:a294
Response IP 104.21.15.117
Found Yes
Hash 51cd5cbae68d19993155e6aad9458be7f29973a4fbb0603b241f88e9b08da6a1
SimHash 6917dc368a3b

Groups

*

Rule Path
Disallow
Allow /

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /admin/
Disallow /ajax/
Disallow /assets/
Disallow /css/
Disallow /js/
Disallow /vendor/
Disallow /main.php
Disallow /index.php
Disallow /mix-manifest.json
Disallow /locale/sr
Disallow /locale/en
Disallow /auth/facebook
Disallow /auth/linkedin
Disallow /auth/twitter
Disallow /auth/google

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap http://autooglasi.rs/rs/sitemaps.xml
sitemap http://autooglasi.rs/us/sitemaps.xml