amp.elperiodico.cat
robots.txt

Robots Exclusion Standard data for amp.elperiodico.cat

Resource Scan

Scan Details

Site Domain amp.elperiodico.cat
Base Domain elperiodico.cat
Scan Status Ok
Last Scan2024-06-29T02:27:23+00:00
Next Scan 2024-07-06T02:27:23+00:00

Last Scan

Scanned2024-06-29T02:27:23+00:00
URL https://amp.elperiodico.cat/robots.txt
Redirect https://www.elperiodico.cat/robots.txt
Redirect Domain www.elperiodico.cat
Redirect Base elperiodico.cat
Domain IPs 199.232.194.133, 199.232.198.133
Redirect IPs 199.232.194.133, 199.232.198.133
Response IP 151.101.42.133
Found Yes
Hash fe7140f244ffd01e91ee0fc6a6e8509c1c96a722e9d3899865998c082f75ade3
SimHash d5bccca40f32

Groups

twitterbot

Rule Path
Allow /

*

Rule Path
Disallow /*/ext_resources/ads/
Disallow /*/ext_resources/portadas/
Disallow /*/noticias/
Disallow /*/component/
Disallow /*/buscador*
Disallow /*/blogscat/
Disallow /swf/
Disallow /component/
Disallow /onbcn/
Disallow /stats/
Disallow /blogs/blogs/
Disallow /comentar.asp/
Disallow /comunes/
Disallow /info/
Disallow /buscador/ca/
Disallow /buscador/es/
Disallow /blogscat/
Disallow /suep/livefyre/
Disallow /valorar.asp
Disallow /comentar.asp
Disallow /r.asp
Disallow /print.asp
Disallow /archivo_titulares.asp
Disallow /envio.asp
Disallow /foros.asp
Disallow /valorada.asp
Disallow /galerias.asp
Disallow /alminuto.asp
Disallow /videos2.asp
Disallow /verpdf.asp
Disallow /alta.asp
Disallow /UpdatedNewsElPeriodico.xml
Disallow /verd-i-blau/
Disallow /buscant-respostes/
Disallow /*/newsletters/
Allow /*/noticias/*.xml