politics.ie
robots.txt

Robots Exclusion Standard data for politics.ie

Resource Scan

Scan Details

Site Domain politics.ie
Base Domain politics.ie
Scan Status Ok
Last Scan2024-10-04T14:10:10+00:00
Next Scan 2024-10-11T14:10:10+00:00

Last Scan

Scanned2024-10-04T14:10:10+00:00
URL https://politics.ie/robots.txt
Domain IPs 209.133.220.18
Response IP 209.133.220.18
Found Yes
Hash db3c635c04f9c263de6bf951e5cd5e089717ef4acf4501dc3dc161ac3c8c0758
SimHash 003a52744f63

Groups

linespider

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

semantic-visions.com crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendictionbot0.5.0

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendictionbot0.5.0

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

ttd-content

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

mediapartners-google

Rule Path
Allow /

*

Rule Path
Disallow /whats-new/
Disallow /account/
Disallow /attachments/
Disallow /goto/
Disallow /login/
Disallow /admin.php
Disallow /tags/
Allow /

Other Records

Field Value
sitemap https://politics.ie/sitemap.xml