Log In Sign Up

megalithic.co.uk
robots.txt

Robots Exclusion Standard data for megalithic.co.uk

Resource Scan

Scan Details

Site Domain megalithic.co.uk
Base Domain megalithic.co.uk
Scan Status Ok
Last Scan2024-09-19T14:53:18+00:00
Next Scan 2024-09-26T14:53:18+00:00

Last Scan

Scanned2024-09-19T14:53:18+00:00
URL https://megalithic.co.uk/robots.txt
Domain IPs 104.21.91.84, 172.67.213.72, 2606:4700:3032::ac43:d548, 2606:4700:3034::6815:5b54
Response IP 172.67.213.72
Found Yes
Hash f9af84a2453a59a14ba3497134946c93517acdc302e3653f4bfaceff013d1b16
SimHash 7e5ed0450525

Groups

mediapartners-google

Rule Path
Disallow /print.php
Disallow /asbmap.php
Disallow /georss.php
Disallow /asb_gps.php
Disallow /asb_kml.php
Disallow /asb_gpx.php
Disallow /a558/a312/public/nmimage.php
Disallow /megp_dev4/
Disallow /article.php/images/*
Disallow %26desktop%3Dtrue*
Disallow /expando_add*

amazonbot

Rule Path
Disallow /

google

Rule Path
Disallow /print.php
Disallow /asbmap.php
Disallow /pollcomments.php
Disallow /pollBooth.php
Disallow /georss.php
Disallow /asb_gps.php
Disallow /asb_kml.php
Disallow /asb_gpx.php
Disallow /friend_ab.php
Disallow /a558/a312/public/nmimage.php
Disallow /comments.php?op=Reply*
Disallow /modules.php?op=modload&name=Members_List*
Disallow /modules.php?op=modload&name=Forum&file=reply*
Disallow /*%26do%3Dupload*
Disallow /*%26orderby%3DtitleA*
Disallow /*%26orderby%3DtitleD*
Disallow /*%26orderby%3DdateA*
Disallow /*%26orderby%3DratingA*
Disallow /*%26orderby%3DratingD*
Disallow /*%26orderby%3DhitsA*
Disallow /*%26orderby%3DhitsD*
Disallow /*%26orderby%3Dname%2BASC*
Disallow /*%26orderby%3Dname%2BDESC*
Disallow /*%26orderby%3Ddate%2BASC*
Disallow /*%26orderby%3Ddate%2BDESC*
Disallow /*%26orderby%3Drate%2BASC*
Disallow /*%26orderby%3Drate%2BDESC*
Disallow /*%26orderby%3Dcounter%2BASC*
Disallow /*%26orderby%3Dcounter%2BDESC*
Disallow /*%26all%3D1*
Disallow /*%26map%3D2*
Disallow /*%26condition%3D*
Disallow /*%26ambience%3D*
Disallow /*%26access%3D*
Disallow /*%26days%3D7*
Disallow /*%26days%3D30*
Disallow /*%26days%3D91*
Disallow /*%26days%3D182*
Disallow /*%26days%3D365*
Disallow /*%26sv%3D1*
Disallow /*%26limit%3D1000*
Disallow /megp_dev4/
Disallow /article.php/images/*
Disallow %26desktop%3Dtrue*
Disallow /expando_add*

Other Records

Field Value
crawl-delay 5

msnbot

Rule Path
Disallow /print.php
Disallow /asbmap.php
Disallow /pollcomments.php
Disallow /pollBooth.php
Disallow /georss.php
Disallow /asb_gps.php
Disallow /asb_kml.php
Disallow /asb_gpx.php
Disallow /friend_ab.php
Disallow /a558/a312/public/nmimage.php
Disallow /comments.php?op=Reply*
Disallow /modules.php?op=modload&name=Members_List*
Disallow /modules.php?op=modload&name=Forum&file=reply*
Disallow /*%26do%3Dupload*
Disallow /*%26orderby%3DtitleA*
Disallow /*%26orderby%3DtitleD*
Disallow /*%26orderby%3DdateA*
Disallow /*%26orderby%3DratingA*
Disallow /*%26orderby%3DratingD*
Disallow /*%26orderby%3DhitsA*
Disallow /*%26orderby%3DhitsD*
Disallow /*%26orderby%3Dname%2BASC*
Disallow /*%26orderby%3Dname%2BDESC*
Disallow /*%26orderby%3Ddate%2BASC*
Disallow /*%26orderby%3Ddate%2BDESC*
Disallow /*%26orderby%3Drate%2BASC*
Disallow /*%26orderby%3Drate%2BDESC*
Disallow /*%26orderby%3Dcounter%2BASC*
Disallow /*%26orderby%3Dcounter%2BDESC*
Disallow /*%26all%3D1*
Disallow /*%26map%3D2*
Disallow /*%26condition%3D*
Disallow /*%26ambience%3D*
Disallow /*%26access%3D*
Disallow /*%26days%3D7*
Disallow /*%26days%3D30*
Disallow /*%26days%3D91*
Disallow /*%26days%3D182*
Disallow /*%26days%3D365*
Disallow /*%26sv%3D1*
Disallow /*%26limit%3D1000*
Disallow /megp_dev4/
Disallow /article.php/images/*
Disallow %26desktop%3Dtrue*
Disallow /expando_add*

Other Records

Field Value
crawl-delay 5

slurp

Rule Path
Disallow /print.php
Disallow /asbmap.php
Disallow /pollcomments.php
Disallow /pollBooth.php
Disallow /georss.php
Disallow /asb_gps.php
Disallow /asb_kml.php
Disallow /asb_gpx.php
Disallow /friend_ab.php
Disallow /a558/a312/public/nmimage.php
Disallow /comments.php?op=Reply*
Disallow /modules.php?op=modload&name=Members_List*
Disallow /modules.php?op=modload&name=Forum&file=reply*
Disallow /*%26do%3Dupload*
Disallow /*%26orderby%3DtitleA*
Disallow /*%26orderby%3DtitleD*
Disallow /*%26orderby%3DdateA*
Disallow /*%26orderby%3DratingA*
Disallow /*%26orderby%3DratingD*
Disallow /*%26orderby%3DhitsA*
Disallow /*%26orderby%3DhitsD*
Disallow /*%26orderby%3Dname%2BASC*
Disallow /*%26orderby%3Dname%2BDESC*
Disallow /*%26orderby%3Ddate%2BASC*
Disallow /*%26orderby%3Ddate%2BDESC*
Disallow /*%26orderby%3Drate%2BASC*
Disallow /*%26orderby%3Drate%2BDESC*
Disallow /*%26orderby%3Dcounter%2BASC*
Disallow /*%26orderby%3Dcounter%2BDESC*
Disallow /*%26all%3D1*
Disallow /*%26map%3D2*
Disallow /*%26condition%3D*
Disallow /*%26ambience%3D*
Disallow /*%26access%3D*
Disallow /*%26days%3D7*
Disallow /*%26days%3D30*
Disallow /*%26days%3D91*
Disallow /*%26days%3D182*
Disallow /*%26days%3D365*
Disallow /*%26sv%3D1*
Disallow /*%26limit%3D1000*
Disallow /megp_dev4/
Disallow /article.php/images/*
Disallow %26desktop%3Dtrue*
Disallow /expando_add*

Other Records

Field Value
crawl-delay 10

yandex

Rule Path
Disallow /print.php
Disallow /asbmap.php
Disallow /pollcomments.php
Disallow /pollBooth.php
Disallow /georss.php
Disallow /asb_gps.php
Disallow /asb_kml.php
Disallow /asb_gpx.php
Disallow /friend_ab.php
Disallow /a558/a312/public/nmimage.php
Disallow /comments.php?op=Reply*
Disallow /modules.php?op=modload&name=Members_List*
Disallow /modules.php?op=modload&name=Forum&file=reply*
Disallow /*%26do%3Dupload*
Disallow /*%26orderby%3DtitleA*
Disallow /*%26orderby%3DtitleD*
Disallow /*%26orderby%3DdateA*
Disallow /*%26orderby%3DratingA*
Disallow /*%26orderby%3DratingD*
Disallow /*%26orderby%3DhitsA*
Disallow /*%26orderby%3DhitsD*
Disallow /*%26orderby%3Dname%2BASC*
Disallow /*%26orderby%3Dname%2BDESC*
Disallow /*%26orderby%3Ddate%2BASC*
Disallow /*%26orderby%3Ddate%2BDESC*
Disallow /*%26orderby%3Drate%2BASC*
Disallow /*%26orderby%3Drate%2BDESC*
Disallow /*%26orderby%3Dcounter%2BASC*
Disallow /*%26orderby%3Dcounter%2BDESC*
Disallow /*%26all%3D1*
Disallow /*%26map%3D2*
Disallow /*%26condition%3D*
Disallow /*%26ambience%3D*
Disallow /*%26access%3D*
Disallow /*%26days%3D7*
Disallow /*%26days%3D30*
Disallow /*%26days%3D91*
Disallow /*%26days%3D182*
Disallow /*%26days%3D365*
Disallow /*%26sv%3D1*
Disallow /*%26limit%3D1000*
Disallow /megp_dev4/
Disallow /article.php/images/*
Disallow %26desktop%3Dtrue*
Disallow /expando_add*

Other Records

Field Value
crawl-delay 10

izsearch.com

Rule Path
Disallow /print.php
Disallow /asbmap.php
Disallow /pollcomments.php
Disallow /pollBooth.php
Disallow /georss.php
Disallow /asb_gps.php
Disallow /asb_kml.php
Disallow /asb_gpx.php
Disallow /friend_ab.php
Disallow /a558/a312/public/nmimage.php
Disallow /comments.php?op=Reply*
Disallow /modules.php?op=modload&name=Members_List*
Disallow /modules.php?op=modload&name=Forum&file=reply*
Disallow /*%26do%3Dupload*
Disallow /*%26orderby%3DtitleA*
Disallow /*%26orderby%3DtitleD*
Disallow /*%26orderby%3DdateA*
Disallow /*%26orderby%3DratingA*
Disallow /*%26orderby%3DratingD*
Disallow /*%26orderby%3DhitsA*
Disallow /*%26orderby%3DhitsD*
Disallow /*%26orderby%3Dname%2BASC*
Disallow /*%26orderby%3Dname%2BDESC*
Disallow /*%26orderby%3Ddate%2BASC*
Disallow /*%26orderby%3Ddate%2BDESC*
Disallow /*%26orderby%3Drate%2BASC*
Disallow /*%26orderby%3Drate%2BDESC*
Disallow /*%26orderby%3Dcounter%2BASC*
Disallow /*%26orderby%3Dcounter%2BDESC*
Disallow /*%26all%3D1*
Disallow /*%26map%3D2*
Disallow /*%26condition%3D*
Disallow /*%26ambience%3D*
Disallow /*%26access%3D*
Disallow /*%26days%3D7*
Disallow /*%26days%3D30*
Disallow /*%26days%3D91*
Disallow /*%26days%3D182*
Disallow /*%26days%3D365*
Disallow /*%26sv%3D1*
Disallow /*%26limit%3D1000*
Disallow /megp_dev4/
Disallow /article.php/images/*
Disallow %26desktop%3Dtrue*
Disallow /expando_add*

Other Records

Field Value
crawl-delay 10

baiduspider

Rule Path
Disallow /print.php
Disallow /asbmap.php
Disallow /pollcomments.php
Disallow /pollBooth.php
Disallow /georss.php
Disallow /asb_gps.php
Disallow /asb_kml.php
Disallow /asb_gpx.php
Disallow /friend_ab.php
Disallow /a558/a312/public/nmimage.php
Disallow /comments.php?op=Reply*
Disallow /modules.php?op=modload&name=Members_List*
Disallow /modules.php?op=modload&name=Forum&file=reply*
Disallow /*%26do%3Dupload*
Disallow /*%26orderby%3DtitleA*
Disallow /*%26orderby%3DtitleD*
Disallow /*%26orderby%3DdateA*
Disallow /*%26orderby%3DratingA*
Disallow /*%26orderby%3DratingD*
Disallow /*%26orderby%3DhitsA*
Disallow /*%26orderby%3DhitsD*
Disallow /*%26orderby%3Dname%2BASC*
Disallow /*%26orderby%3Dname%2BDESC*
Disallow /*%26orderby%3Ddate%2BASC*
Disallow /*%26orderby%3Ddate%2BDESC*
Disallow /*%26orderby%3Drate%2BASC*
Disallow /*%26orderby%3Drate%2BDESC*
Disallow /*%26orderby%3Dcounter%2BASC*
Disallow /*%26orderby%3Dcounter%2BDESC*
Disallow /*%26all%3D1*
Disallow /*%26map%3D2*
Disallow /*%26condition%3D*
Disallow /*%26ambience%3D*
Disallow /*%26access%3D*
Disallow /*%26days%3D7*
Disallow /*%26days%3D30*
Disallow /*%26days%3D91*
Disallow /*%26days%3D182*
Disallow /*%26days%3D365*
Disallow /*%26sv%3D1*
Disallow /*%26limit%3D1000*
Disallow /megp_dev4/
Disallow /article.php/images/*
Disallow %26desktop%3Dtrue*
Disallow /expando_add*

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /print.php
Disallow /asbmap.php
Disallow /pollcomments.php
Disallow /pollBooth.php
Disallow /georss.php
Disallow /asb_gps.php
Disallow /asb_kml.php
Disallow /asb_gpx.php
Disallow /friend_ab.php
Disallow /a558/a312/public/nmimage.php
Disallow /comments.php?op=Reply*
Disallow /modules.php?op=modload&name=Members_List*
Disallow /modules.php?op=modload&name=Forum&file=reply*
Disallow /*%26do%3Dupload*
Disallow /*%26orderby%3DtitleA*
Disallow /*%26orderby%3DtitleD*
Disallow /*%26orderby%3DdateA*
Disallow /*%26orderby%3DratingA*
Disallow /*%26orderby%3DratingD*
Disallow /*%26orderby%3DhitsA*
Disallow /*%26orderby%3DhitsD*
Disallow /*%26orderby%3Dname%2BASC*
Disallow /*%26orderby%3Dname%2BDESC*
Disallow /*%26orderby%3Ddate%2BASC*
Disallow /*%26orderby%3Ddate%2BDESC*
Disallow /*%26orderby%3Drate%2BASC*
Disallow /*%26orderby%3Drate%2BDESC*
Disallow /*%26orderby%3Dcounter%2BASC*
Disallow /*%26orderby%3Dcounter%2BDESC*
Disallow /*%26all%3D1*
Disallow /*%26map%3D2*
Disallow /*%26condition%3D*
Disallow /*%26ambience%3D*
Disallow /*%26access%3D*
Disallow /*%26days%3D7*
Disallow /*%26days%3D30*
Disallow /*%26days%3D91*
Disallow /*%26days%3D182*
Disallow /*%26days%3D365*
Disallow /*%26sv%3D1*
Disallow /*%26limit%3D1000*
Disallow /megp_dev4/
Disallow /article.php/images/*
Disallow %26desktop%3Dtrue*
Disallow /expando_add*

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • Google Ads crawler, we need to allow this on all pages where ads are shown or we may not get paid
Back to top