gqjapan.jp
robots.txt

Robots Exclusion Standard data for gqjapan.jp

Resource Scan

Scan Details

Site Domain gqjapan.jp
Base Domain gqjapan.jp
Scan Status Ok
Last Scan2024-11-02T09:58:48+00:00
Next Scan 2024-11-09T09:58:48+00:00

Last Scan

Scanned2024-11-02T09:58:48+00:00
URL https://gqjapan.jp/robots.txt
Redirect https://www.gqjapan.jp/robots.txt
Redirect Domain www.gqjapan.jp
Redirect Base gqjapan.jp
Domain IPs 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133
Redirect IPs 13.33.88.18, 13.33.88.3, 13.33.88.44, 13.33.88.5, 2600:9000:2816:1200:17:de95:1200:93a1, 2600:9000:2816:4800:17:de95:1200:93a1, 2600:9000:2816:4c00:17:de95:1200:93a1, 2600:9000:2816:5000:17:de95:1200:93a1, 2600:9000:2816:5e00:17:de95:1200:93a1, 2600:9000:2816:a00:17:de95:1200:93a1, 2600:9000:2816:da00:17:de95:1200:93a1, 2600:9000:2816:f400:17:de95:1200:93a1
Response IP 13.33.88.44
Found Yes
Hash b0d80f5beaf07d451f177730f6d74ae9514651a1551d3307d27038b666381190
SimHash 0414d9d18321

Groups

*

Rule Path
Disallow /magazine/subscription/20170331/subscription_std
Disallow /preview/
Disallow *?image=
Disallow */image/
Disallow /*?redirectURL=
Disallow /*?preview=on
Disallow /*?site=
Disallow /*%7B%7Burl%7D%7D
Disallow /*%7B%7Burl%7D%7D
Disallow /*%7B%7Bsection%7D%7D
Disallow /*%7B%7Bsection%7D%7D
Disallow /auth/*
Disallow /account/*
Disallow /product/
Disallow /user-context?referrer
Disallow /services.min.js?_=
Disallow /com.condenast/yv8*

y!j-wsc

Rule Path
Disallow /magazine/subscription/20170331/subscription_std
Disallow /preview/
Disallow *?image=
Disallow */image/
Disallow /*?redirectURL=
Disallow /*?preview=on
Disallow /*?site=
Disallow /*%7B%7Burl%7D%7D
Disallow /*%7B%7Burl%7D%7D
Disallow /*%7B%7Bsection%7D%7D
Disallow /*%7B%7Bsection%7D%7D
Disallow /auth/*
Disallow /account/*
Disallow /product/
Disallow /user-context?referrer
Disallow /services.min.js?_=

Other Records

Field Value
crawl-delay 10

google-inspectiontool

Rule Path
Allow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.gqjapan.jp/sitemap.xml
sitemap https://www.gqjapan.jp/feed/google-latest-news/sitemap-google-news
sitemap https://www.gqjapan.jp/feed/rss
sitemap https://www.gqjapan.jp/categories-sitemap.xml
sitemap https://www.gqjapan.jp/contributors-sitemap.xml
sitemap https://www.gqjapan.jp/bundles-sitemap.xml

Comments

  • Yahoo JP