arnaldofilho.com.br
robots.txt

Robots Exclusion Standard data for arnaldofilho.com.br

Resource Scan

Scan Details

Site Domain arnaldofilho.com.br
Base Domain arnaldofilho.com.br
Scan Status Ok
Last Scan2024-11-01T15:03:57+00:00
Next Scan 2024-11-08T15:03:57+00:00

Last Scan

Scanned2024-11-01T15:03:57+00:00
URL https://arnaldofilho.com.br/robots.txt
Redirect https://afnoticias.com.br/robots.txt
Redirect Domain afnoticias.com.br
Redirect Base afnoticias.com.br
Domain IPs 96.126.108.196
Redirect IPs 96.126.108.196
Response IP 96.126.108.196
Found Yes
Hash a53e83947f8c5456e3c8685be656b6c16194fb62ec284890f1e3ac7d77498ca3
SimHash e9004ac6fe93

Groups

*

Rule Path
Disallow /banner
Allow /

proximic

Rule Path
Disallow /

starkbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

voluumdsp-content-bot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

*

Rule Path
Disallow /busca?query=*

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

Other Records

Field Value
sitemap https://afnoticias.com.br/sitemap.xml

Warnings

  • 1 invalid line.