Log In Sign Up

comic-walker.com
robots.txt

Robots Exclusion Standard data for comic-walker.com

Resource Scan

Scan Details

Site Domain comic-walker.com
Base Domain comic-walker.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-11T22:09:20+00:00
Next Scan 2024-12-10T22:09:20+00:00

Last Successful Scan

Scanned2024-02-15T21:34:05+00:00
URL https://comic-walker.com/robots.txt
Domain IPs 110.232.194.164
Response IP 110.232.194.164
Found Yes
Hash dc793e89f6b2878a9ea8515c00a0b1a2136cc7a09fa58cb9b00b2c4d469d222a
SimHash 6d3f525141b5

Groups

googlebot-news

Rule Path
Allow /news/
Allow /release/
Allow /contents/calendar/

googlebot

Rule Path
Disallow /member/
Disallow /customer_inquiry/

slurp

Rule Path
Disallow /member/
Disallow /customer_inquiry/

Other Records

Field Value
crawl-delay 3

bingbot

Rule Path
Disallow /member/
Disallow /customer_inquiry/

Other Records

Field Value
crawl-delay 5

yandex

Rule Path
Disallow /member/
Disallow /customer_inquiry/

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /member/
Disallow /customer_inquiry/

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://comic-walker.com/sitemap.xml
Back to top

Comments

  • Google News Robot
  • Google Search Engine Robot
  • Yahoo! Search Engine Robot
  • Microsoft Search Engine Robot
  • Yandex Search Engine Robot
  • Other crawller or bot that might possibly access or crawling respect below
  • Sitemap
Back to top