topics.gr
robots.txt

Robots Exclusion Standard data for topics.gr

Resource Scan

Scan Details

Site Domain topics.gr
Base Domain topics.gr
Scan Status Ok
Last Scan2024-09-22T10:15:04+00:00
Next Scan 2024-09-29T10:15:04+00:00

Last Scan

Scanned2024-09-22T10:15:04+00:00
URL https://topics.gr/robots.txt
Redirect https://www.topics.gr/robots.txt
Redirect Domain www.topics.gr
Redirect Base topics.gr
Domain IPs 104.21.30.132, 172.67.172.241, 2606:4700:3033::ac43:acf1, 2606:4700:3036::6815:1e84
Redirect IPs 104.21.30.132, 172.67.172.241, 2606:4700:3033::ac43:acf1, 2606:4700:3036::6815:1e84
Response IP 172.67.172.241
Found Yes
Hash 30daa2e7b3daa9539e3c84e5bf64000c38b9203e3970574c5675f0d88ddbbf10
SimHash 20d49e014e7a

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /rdr.php
Disallow /rdrxml.php
Disallow /rdriframe.php
Disallow /iframe.php
Disallow /forum/post/
Disallow /forum/captcha/
Disallow /user/cp/
Disallow /user/s123/
Disallow /user/filaki.php
Disallow /user/annunci.php
Disallow /user/forum-abuse.php
Disallow /tag/
Disallow /articolo/
Disallow /blogs/
Disallow /crono
Disallow /stories
Disallow /eyeblaster/
Disallow /experimental/
Disallow /argomenti/
Disallow /*%26page%3D
Disallow /notizia/
Disallow /find.php

*

Rule Path
Allow /blogs/vote.php
Allow /blogs/publish.php

dotbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

whitevector crawler

Rule Path
Disallow /

speedy

Rule Path
Disallow /

spbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /