Log In Sign Up

independente.com.br
robots.txt

Robots Exclusion Standard data for independente.com.br

Resource Scan

Scan Details

Site Domain independente.com.br
Base Domain independente.com.br
Scan Status Ok
Last Scan2024-06-09T00:19:50+00:00
Next Scan 2024-06-16T00:19:50+00:00

Last Scan

Scanned2024-06-09T00:19:50+00:00
URL https://independente.com.br/robots.txt
Domain IPs 104.21.80.13, 172.67.172.148, 2606:4700:3030::6815:500d, 2606:4700:3031::ac43:ac94
Response IP 172.67.172.148
Found Yes
Hash 4455a9b6c0fd019550192e29298b2d0e3b2d0e5e101351caa24b8e06b36955c2
SimHash 4bbd5149cc83

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /search
Disallow /search/
Disallow /search/*
Disallow */search/*
Disallow /?s=*
Disallow */?s=*
Disallow /?q=*
Disallow */?q=*
Disallow /?filter_by=
Disallow */?filter_by=
Disallow */feed
Disallow */feed/
Disallow */feed/*

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30
Back to top

Other Records

Field Value
sitemap https://independente.com.br/sitemap_index.xml
sitemap https://stories.independente.com.br/sitemap.xml
Back to top

Comments

  • robots.txt by ServerDo.in -- independente.com.br
Back to top