suitetalking.swiftdigital.com.au
robots.txt

Robots Exclusion Standard data for suitetalking.swiftdigital.com.au

Resource Scan

Scan Details

Site Domain suitetalking.swiftdigital.com.au
Base Domain swiftdigital.com.au
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-07T14:47:55+00:00
Next Scan 2024-11-21T14:47:55+00:00

Last Successful Scan

Scanned2024-09-30T13:55:39+00:00
URL https://suitetalking.swiftdigital.com.au/robots.txt
Domain IPs 52.62.78.10, 52.65.240.201
Response IP 52.62.78.10
Found Yes
Hash ce2a3bbdbb621b512047ab41ff2f900c805a6f5fc93f2308e3f8e043b98cd448
SimHash 62405303d013

Groups

*

Rule Path
Disallow /
Disallow /feed
Disallow /rss

mediapartners-google

Rule Path
Disallow
Disallow /feed
Disallow /rss

googlebot

Rule Path
Disallow
Disallow /feed
Disallow /rss

Other Records

Field Value
crawl-delay 30

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow
Disallow /feed
Disallow /rss

googlebot-mobile

Rule Path
Disallow
Disallow /feed
Disallow /rss

msnbot

Rule Path
Disallow
Disallow /feed
Disallow /rss

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Disallow
Disallow /feed
Disallow /rss

Other Records

Field Value
crawl-delay 10

msnbot

Rule Path
Disallow
Disallow /feed
Disallow /rss

slurp

Rule Path
Disallow
Disallow /feed
Disallow /rss

yahoo! slurp

Rule Path
Disallow
Disallow /feed
Disallow /rss

mj12bot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

moget

Rule Path
Disallow /
Disallow /feed
Disallow /rss

ichiro

Rule Path
Disallow /
Disallow /feed
Disallow /rss

yeti

Rule Path
Disallow /
Disallow /feed
Disallow /rss

naverbot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

sogou spider

Rule Path
Disallow /
Disallow /feed
Disallow /rss

youdaobot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

baiduspider

Rule Path
Disallow /
Disallow /feed
Disallow /rss

baiduspider-video

Rule Path
Disallow /
Disallow /feed
Disallow /rss

baiduspider-image

Rule Path
Disallow /
Disallow /feed
Disallow /rss

yandex

Rule Path
Disallow /
Disallow /feed
Disallow /rss

yandexbot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

spbot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

turnitinbot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

ahrefsbot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

Other Records

Field Value
crawl-delay 10

exabot

Rule Path
Disallow /
Disallow /feed
Disallow /rss

megaindex.ru/2.0

Rule Path
Disallow /
Disallow /feed
Disallow /rss

yandexbot/3.0

Rule Path
Disallow /
Disallow /feed
Disallow /rss

twitterbot

Rule Path
Allow /media
Disallow /feed
Disallow /rss