folha5.com.br
robots.txt

Robots Exclusion Standard data for folha5.com.br

Resource Scan

Scan Details

Site Domain folha5.com.br
Base Domain folha5.com.br
Scan Status Ok
Last Scan2024-11-11T13:33:11+00:00
Next Scan 2024-11-18T13:33:11+00:00

Last Scan

Scanned2024-11-11T13:33:11+00:00
URL https://folha5.com.br/robots.txt
Redirect https://www.folha5.com.br/robots.txt
Redirect Domain www.folha5.com.br
Redirect Base folha5.com.br
Domain IPs 104.21.91.241, 172.67.182.40, 2606:4700:3030::6815:5bf1, 2606:4700:3035::ac43:b628
Redirect IPs 104.21.91.241, 172.67.182.40, 2606:4700:3030::6815:5bf1, 2606:4700:3035::ac43:b628
Response IP 172.67.182.40
Found Yes
Hash c5bde5d9e309a5e750a79c3b91e94159ba2b10e7960c511c1792c1be8b683458
SimHash 6b40dc544793

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.folha5.com.br/sitemaps/1/sitemap-index.xml