Log In Sign Up

sewcanshe.com
robots.txt

Robots Exclusion Standard data for sewcanshe.com

Resource Scan

Scan Details

Site Domain sewcanshe.com
Base Domain sewcanshe.com
Scan Status Ok
Last Scan2024-11-14T21:29:41+00:00
Next Scan 2024-11-21T21:29:41+00:00

Last Scan

Scanned2024-11-14T21:29:41+00:00
URL https://sewcanshe.com/robots.txt
Domain IPs 104.26.10.35, 104.26.11.35, 172.67.74.111, 2606:4700:20::681a:a23, 2606:4700:20::681a:b23, 2606:4700:20::ac43:4a6f
Response IP 104.26.10.35
Found Yes
Hash d13fcea5facb08a9607e3fcc837fe4ccbfb30db4751803f46ef6b308f5f40cbd
SimHash 38001102e962

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

gptbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://sewcanshe.com/sitemap_index.xml
Back to top

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK
  • to block SEMRUSHBOTS:
  • To block SemrushBot from crawling your site for Backlink Audit tool:
  • To block SemrushBot from crawling your site for On Page SEO Checker tool and similar tools:
  • To block SemrushBot from checking URLs on your site for SWA tool:
  • To block SemrushBot from crawling your site for Content Analyzer and Post Tracking tools:
  • To block SplitSignalBot from crawling your site for SplitSignal tool:
  • To block SemrushBot-COUB from crawling your site for Content Outline Builder tool:
  • AHREFS:
  • MOZ:
  • To block additional AI crawlers:
  • For new training only
  • Not for training, only for user requests
  • Marker for disabling Bard and Vertex AI
Back to top