Log In Sign Up

imagecomics.com
robots.txt

Robots Exclusion Standard data for imagecomics.com

Resource Scan

Scan Details

Site Domain imagecomics.com
Base Domain imagecomics.com
Scan Status Ok
Last Scan2024-11-16T16:40:42+00:00
Next Scan 2024-11-23T16:40:42+00:00

Last Scan

Scanned2024-11-16T16:40:42+00:00
URL https://imagecomics.com/robots.txt
Domain IPs 64.227.108.204
Response IP 64.227.108.204
Found Yes
Hash 949e3f2bcba1056d2d3ea454646d7d1bfc98662af183905011075d782b0ea3ef
SimHash 28201a52e8a0

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env

Other Records

Field Value
crawl-delay 15

petalbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://imagecomics.com/sitemaps-1-sitemap.xml
Back to top

Comments

  • robots.txt for https://imagecomics.com/
  • live - don't allow web crawlers to index cpresources/ or vendor/
  • To block SEMrushBot from crawling your site for different SEO and technical issues:
  • To block SEMrushBot from crawling your site for Backlink Audit tool:
  • To block SEMrushBot from crawling your site for On Page SEO Checker tool and similar tools:
  • To block SEMrushBot from checking URLs your site for SWA tool:
  • To block SEMrushBot from crawling your site for Content Analyzer and Post Tracking tools:
  • To block SEMrushBot from crawling your site for Brand Monitoring:
Back to top