Log In Sign Up

lifehacker.com.au
robots.txt

Robots Exclusion Standard data for lifehacker.com.au

Resource Scan

Scan Details

Site Domain lifehacker.com.au
Base Domain lifehacker.com.au
Scan Status Ok
Last Scan2025-12-05T00:58:08+00:00
Next Scan 2025-12-12T00:58:08+00:00

Last Scan

Scanned2025-12-05T00:58:08+00:00
URL https://lifehacker.com.au/robots.txt
Redirect https://au.lifehacker.com/robots.txt
Redirect Domain au.lifehacker.com
Redirect Base lifehacker.com
Domain IPs 18.185.24.46, 2a05:d014:803:f30a:4c8d:5530:dac2:179, 2a05:d014:803:f30c:505c:9738:13c:b7a5, 2a05:d014:803:f30e:ffe5:90d1:736a:fcad, 3.126.5.188, 3.71.139.153
Redirect IPs 23.209.46.134, 23.209.46.136
Response IP 23.45.207.72
Found Yes
Hash 2ccf0fda38a8a150105f63479217abee5ed65089fa9249c0f608883bca25fe76
SimHash 75d2131143b7

Groups

*

Rule Path
Disallow /admin/
Disallow /*/admin/
Disallow /se/
Disallow /*.psd$
Disallow /apiproxy/

ai2bot-dolma
anthropic-ai
amazonbot
applebot
applebot-extended
bytespider
ccbot
chatgpt-user
claude-web
claudebot
cohere-ai
ddm*
ddm-dcipher/1.0.7
ddm-dcipher*
diffbot
duckassistbot
facebookbot
gptbot
httrack
meta-externalagent
meta-externalfetcher
nutch
oai-searchbot
offline explorer
omgili
perplexity-user
perplexitybot
scrapy
timpibot
youbot
addsearchbot
ai2bot
ai2bot-dolma
amazonbot
anthropic-ai
applebot
applebot-extended
bigsur.ai
bytespider
ccbot
chatglm-spider
chatgpt-user
claude-searchbot
claude-user
claude-web
claudebot
cohere-ai
cohere-training-data-crawler
cotoyogi
deepseekbot
devin
diffbot
duckassistbot
facebookbot
gptbot
iask
icc-crawler
imagesiftbot
kangaroo bot
kunato
linerbot
meta-externalagent
meta-externalfetcher
mistral.ai
mistralai-user
novaact
oai-searchbot
omgili
pangubot
perplexity-user
perplexitybot
petalbot
qualifiedbot
seekrbot
timpibot
velen crawler
webzio-extended
wrtn
youbot
zanistabot

Rule Path
Disallow /
Back to top

Comments

  • Ziff Davis content is made available for your non-commercial use subject to our
  • Terms of Use here: https://www.ziffdavis.com/terms-of-use
  • Use of any robot, crawler, or other tool to scrape, harvest, extract, or retrieve any content on
  • this website using automated means is prohibited without written permission from Ziff Davis.
  • Prohibited uses include but are not limited to:
  • (1) text and data mining under Art. 4 of the EU Directive on Copyright in the Digital Single
  • Market;
  • (2) development or operation of artificial intelligence or machine learning software or
  • databases, including by training, fine-tuning, embedding, and retrieval-augmented generation;
  • (3) creating data sets containing our content or sharing it with others; and
  • (4) any commercial purposes.
  • Contact licensing@ziffdavis.com for assistance.
Back to top

Warnings

  • 3 invalid lines.
Back to top