Log In Sign Up

peticaopublica.com.br
robots.txt

Robots Exclusion Standard data for peticaopublica.com.br

Resource Scan

Scan Details

Site Domain peticaopublica.com.br
Base Domain peticaopublica.com.br
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-03-21T17:11:55+00:00
Next Scan 2024-06-19T17:11:55+00:00

Last Successful Scan

Scanned2022-11-26T10:19:24+00:00
URL https://peticaopublica.com.br/robots.txt
Response IP 85.17.183.114
Found Yes
Hash b14b0a8d20a1917180780c46ee3f4df6c3accc3e8b3bc2e530da3e2cd168fbbf
SimHash 2b4cc160efb5

Groups

*

Rule Path
Disallow /viewfullsignatures.aspx
Disallow /*.axd$
Disallow /PeticaoListaSignatarios.aspx

mediapartners-google

Rule Path
Allow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandex

Rule Path
Allow /$
Disallow /

semrushbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

ahrefsbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

barkrowler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

rankactivelinkbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

adsbot

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /
Back to top

Comments

  • everything is prohibited except the main page
Back to top