Log In Sign Up

musicboard.app
robots.txt

Robots Exclusion Standard data for musicboard.app

Resource Scan

Scan Details

Site Domain musicboard.app
Base Domain musicboard.app
Scan Status Ok
Last Scan2024-09-30T08:39:11+00:00
Next Scan 2024-10-07T08:39:11+00:00

Last Scan

Scanned2024-09-30T08:39:11+00:00
URL https://musicboard.app/robots.txt
Domain IPs 172.66.40.72, 172.66.43.184, 2606:4700:3108::ac42:2848, 2606:4700:3108::ac42:2bb8
Response IP 172.66.43.184
Found Yes
Hash 37f9bd6822ce0f05ac9c467bb14fb677643567349e0f652409a44a284f717084
SimHash fa0958418fd3

Groups

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Back to top

Other Records

Field Value
sitemap https://musicboard.app/sitemap.xml
Back to top

Comments

  • Dark Visitors robots.txt from https://darkvisitors.com/robots-txt-builder
  • AI Data Scraper
  • https://darkvisitors.com/agents/anthropic-ai
  • AI Data Scraper
  • https://darkvisitors.com/agents/bytespider
  • AI Data Scraper
  • https://darkvisitors.com/agents/ccbot
  • AI Data Scraper
  • https://darkvisitors.com/agents/diffbot
  • AI Data Scraper
  • https://darkvisitors.com/agents/facebookbot
  • AI Data Scraper
  • https://darkvisitors.com/agents/google-extended
  • AI Data Scraper
  • https://darkvisitors.com/agents/gptbot
  • AI Data Scraper
  • https://darkvisitors.com/agents/omgili
  • AI Data Scraper
  • https://darkvisitors.com/agents/applebot-extended
  • All other crawlers
Back to top