Log In Sign Up

sapo.pt
robots.txt

Robots Exclusion Standard data for sapo.pt

Resource Scan

Scan Details

Site Domain sapo.pt
Base Domain sapo.pt
Scan Status Ok
Last Scan2026-04-05T13:57:53+00:00
Next Scan 2026-04-12T13:57:53+00:00

Last Scan

Scanned2026-04-05T13:57:53+00:00
URL https://sapo.pt/robots.txt
Domain IPs 213.13.145.114
Response IP 213.13.145.114
Found Yes
Hash 2efae8da84c999734a63b79149ee3af068fd8a9bb415ae202231011c81e714d3
SimHash 77d71a414685

Groups

*

Rule Path
Disallow /404
Disallow /500
Disallow /api/v1/
Disallow /assets/
Disallow /pesquisa
Disallow /pesquisar
Disallow /search

addsearchbot

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bigsur.ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt agent

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-searchbot

Rule Path
Disallow /

claude-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cloudvertexbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /

cotoyogi

Rule Path
Disallow /

datenbank crawler

Rule Path
Disallow /

devin

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

gemini-deep-research

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

googleagent-mariner

Rule Path
Disallow /

googleother

Rule Path
Disallow /

googlebot

Rule Path
Disallow

bingbot

Rule Path
Disallow

gptbot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

kangaroo bot

Rule Path
Disallow /

linerbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

mistralai-user

Rule Path
Disallow /

netestate imprint crawler

Rule Path
Disallow /

novaact

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

perplexity-user

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

qualifiedbot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

youbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://sapo.pt/article/sitemap.xml
sitemap https://sapo.pt/blockpage/sitemap.xml
sitemap https://sapo.pt/category/sitemap.xml
sitemap https://sapo.pt/sitemap.xml
Back to top