j-comi.jp
robots.txt

Robots Exclusion Standard data for j-comi.jp

Resource Scan

Scan Details

Site Domain j-comi.jp
Base Domain j-comi.jp
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-04T07:28:27+00:00
Next Scan 2025-02-02T07:28:27+00:00

Last Successful Scan

Scanned2024-01-10T05:02:05+00:00
URL https://j-comi.jp/robots.txt
Redirect https://www.mangaz.com/robots.txt
Redirect Domain www.mangaz.com
Redirect Base mangaz.com
Domain IPs 2406:da14:ee3:4b01:b839:17eb:f451:1d23, 2406:da14:ee3:4b02:86d7:f457:c651:b475, 35.72.165.246, 54.95.91.224
Redirect IPs 13.115.148.226, 3.115.187.195
Response IP 3.115.187.195
Found Yes
Hash 6f0bf1527062313eb9593ef7b7f5c1f20a7dad28e2b568938b6b1edc503cecd6
SimHash a200d288b7e5

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

mediapartners-google*

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /
Allow /icon.svg

yahoo-mmcrawler

Rule Path
Disallow /

psbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /images/
Disallow /*.jpg$
Disallow /*.jpeg$
Disallow /*.png$
Disallow /*.gif$

Other Records

Field Value
crawl-delay 120

Comments

  • User-agent: Twitterbot
  • Disallow: /
  • GoogleImage
  • YahooImage
  • MSNpicsearch
  • Huawei