Log In Sign Up

climendo.com
robots.txt

Robots Exclusion Standard data for climendo.com

Resource Scan

Scan Details

Site Domain climendo.com
Base Domain climendo.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-05T20:25:11+00:00
Next Scan 2026-02-03T20:25:11+00:00

Last Successful Scan

Scanned2025-09-13T00:40:37+00:00
URL https://climendo.com/robots.txt
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.96.1
Found Yes
Hash ef43d1cbc02fe54589254e6a6e1e1da54ba32c71406ef47631f281b2b06ba483
SimHash f21c0cc5e6d0

Groups

*
dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claudebot-web

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claude

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.climendo.com/sitemap.xml
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
Back to top