Log In Sign Up

bachtrack.com
robots.txt

Robots Exclusion Standard data for bachtrack.com

Resource Scan

Scan Details

Site Domain bachtrack.com
Base Domain bachtrack.com
Scan Status Ok
Last Scan2025-11-30T18:58:38+00:00
Next Scan 2025-12-07T18:58:38+00:00

Last Scan

Scanned2025-11-30T18:58:38+00:00
URL https://bachtrack.com/robots.txt
Domain IPs 34.160.136.221
Response IP 34.160.136.221
Found Yes
Hash 52e541875aed0ac6b2e88077666c8e96082a2899838020c9332efe56ba7e6703
SimHash 721cdf50c57c

Groups

yandex

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

makemerrybot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

bytedance

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

perplexitybot:

Rule Path
Disallow /

facebookbot:

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

facebookexternalhit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /22/299/generate-shareable-links*
Disallow /handler/listing/click/
Disallow /de_DE/handler/listing/click/
Disallow /es_ES/handler/listing/click/
Disallow /fr_FR/handler/listing/click/
Disallow /22/275/click/
Disallow /de_DE/22/275/click
Disallow /es_ES/22/275/click
Disallow /fr_FR/22/275/click
Disallow /robocop
Disallow /?q=robocop

a1 sitemap generator*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 4

mediapartners-google*

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://bachtrack.com/sitemap.xml
Back to top

Comments

  • robots.txt
  • This file will be ignored unless it is at the root of your host:
  • Used: http://example.com/robots.txt
  • Ignored: http://example.com/site/robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/wc/robots.html
  • For syntax checking, see:
  • http://www.sxw.org.uk/computing/robots/check.html
  • Unwanted robots
  • Allow baidu to have access, as of 2024-01-05
  • User-agent: baiduspider
  • Disallow: /
  • Disallow AI training bots for various LLMs
  • See https://www.cyberciti.biz/web-developer/block-openai-bard-bing-ai-crawler-bots-using-robots-txt-file/
  • User-agent: GPTBot
  • Disallow: /
  • User-agent: ChatGPT-User
  • Disallow: /
  • User-agent: Google-Extended
  • Disallow: /
  • User-agent: ClaudeBot
  • Disallow: /
  • User-agent: Claude-Web
  • Disallow: /
  • As of 2024-09-01, MSN/Bing are still crawling too fast. It's time to delay them again.
  • Also attempt to slow down the Facebook crawler
  • Ajax items
  • Ad links
  • Robot tracker
  • Comment out these lines if using Google Adsense
Back to top