suite.swiftdigital.com.au
robots.txt

Robots Exclusion Standard data for suite.swiftdigital.com.au

Resource Scan

Scan Details

Site Domain suite.swiftdigital.com.au
Base Domain swiftdigital.com.au
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-08T21:49:20+00:00
Next Scan 2024-12-07T21:49:20+00:00

Last Successful Scan

Scanned2024-04-19T21:05:26+00:00
URL https://suite.swiftdigital.com.au/robots.txt
Domain IPs 13.55.96.102, 54.79.220.28
Response IP 13.55.96.102
Found Yes
Hash 5b7029e07ba98422f49c6000d69327d3db7dc154d01c6151974abc4d7c710b8c
SimHash 62405303d013

Groups

*

Rule Path
Disallow /
Disallow /feed
Disallow /rss

mediapartners-google

Rule Path
Disallow
Disallow /feed
Disallow /rss

googlebot

Rule Path
Disallow
Disallow /feed
Disallow /rss

Other Records

Field Value
crawl-delay 30

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow
Disallow /feed
Disallow /rss

googlebot-mobile

Rule Path
Disallow
Disallow /feed
Disallow /rss

msnbot

Rule Path
Disallow
Disallow /feed
Disallow /rss

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Disallow
Disallow /feed
Disallow /rss

Other Records

Field Value
crawl-delay 10

msnbot

Rule Path
Disallow
Disallow /feed
Disallow /rss

slurp

Rule Path
Disallow
Disallow /feed
Disallow /rss

yahoo! slurp

Rule Path
Disallow
Disallow /feed
Disallow /rss

mj12bot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

moget

Rule Path
Disallow /
Disallow /feed
Disallow /rss

ichiro

Rule Path
Disallow /
Disallow /feed
Disallow /rss

yeti

Rule Path
Disallow /
Disallow /feed
Disallow /rss

naverbot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

sogou spider

Rule Path
Disallow /
Disallow /feed
Disallow /rss

youdaobot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

baiduspider

Rule Path
Disallow /
Disallow /feed
Disallow /rss

baiduspider-video

Rule Path
Disallow /
Disallow /feed
Disallow /rss

baiduspider-image

Rule Path
Disallow /
Disallow /feed
Disallow /rss

yandex

Rule Path
Disallow /
Disallow /feed
Disallow /rss

yandexbot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

spbot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

turnitinbot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

ahrefsbot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

Other Records

Field Value
crawl-delay 10

exabot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

megaindex.ru/2.0

Rule Path
Disallow /
Disallow /feed
Disallow /rss

yandexbot/3.0

Rule Path
Disallow /
Disallow /feed
Disallow /rss

twitterbot

Rule Path
Allow /media
Disallow /feed
Disallow /rss