jwz.org
robots.txt

Robots Exclusion Standard data for jwz.org

Resource Scan

Scan Details

Site Domain jwz.org
Base Domain jwz.org
Scan Status Ok
Last Scan2024-10-29T10:13:27+00:00
Next Scan 2024-11-28T10:13:27+00:00

Last Scan

Scanned2024-10-29T10:13:27+00:00
URL https://jwz.org/robots.txt
Redirect https://www.jwz.org/robots.txt
Redirect Domain www.jwz.org
Redirect Base jwz.org
Domain IPs 3.16.178.106
Redirect IPs 3.16.178.106
Response IP 3.16.178.106
Found Yes
Hash 06c0006e1672d2c4c4c9785fae2064fe73bfafe3e6f7cb5ec359c7f3b0200027
SimHash d0003bd1c685

Groups

*

Rule Path
Disallow /b/
Disallow /blog/wp-
Disallow /images/scaled/
Disallow /dadadodo/dadadodo.cgi

googlebot-image

Rule Path
Disallow /

twitterbot

Rule Path
Disallow
Allow /images/
Allow /images/scaled/

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

googleother

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

peer39_crawler

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

seekr

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.jwz.org/sitemap.xml

Comments

  • My blog images are only mirrors. Use the primary sources.
  • Necessary for /images/scaled/ to work in in "twitter:image"
  • AI scrapers