Log In Sign Up

ibusolutions.com
robots.txt

Robots Exclusion Standard data for ibusolutions.com

Resource Scan

Scan Details

Site Domain ibusolutions.com
Base Domain ibusolutions.com
Scan Status Ok
Last Scan2025-12-30T08:31:43+00:00
Next Scan 2026-01-29T08:31:43+00:00

Last Scan

Scanned2025-12-30T08:31:43+00:00
URL http://ibusolutions.com/robots.txt
Redirect https://heardmarketing.io/robots.txt
Redirect Domain heardmarketing.io
Redirect Base heardmarketing.io
Domain IPs 64.98.135.86
Redirect IPs 104.21.44.93, 172.67.198.61, 2606:4700:3031::6815:2c5d, 2606:4700:3037::ac43:c63d
Response IP 104.21.44.93
Found Yes
Hash c79acf1465acb18113a157287dee8ebb4451312213695c937fc9a20083c6b50c
SimHash 7c16d072a593

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

claude-web

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

google-extended

Rule Path
Allow /

cohere-ai

Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sogou

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

curl

Rule Path
Disallow /

wget

Rule Path
Disallow /

httrack

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

*

Rule Path
Allow /
Disallow /admin/
Disallow /private/

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://heardmarketing.io/sitemap.xml
Back to top

Comments

  • Allow legitimate search engines and social media
  • Allow AI Search Crawlers (AIEO Optimization)
  • AI Crawler Information Files
  • For structured business data, AI crawlers can access:
  • https://heardmarketing.io/llms.txt - Structured business info for LLMs
  • https://heardmarketing.io/sitemap.xml - Full site structure
  • AI-Specific Guidance
  • This website is optimized for AI search engines and answer engines.
  • Key content areas: /services/, /booked-reviewed/, /blog/, /about/
  • Business focus: Digital marketing for local service businesses
  • Block common bad bots and scrapers
  • Block Chinese search engines and crawlers
  • Block aggressive crawlers and scrapers
  • Allow good bots
  • Block admin and private areas
  • Sitemap
  • Crawl delay
Back to top

Warnings

  • 2 invalid lines.
Back to top