Log In Sign Up

baseballinfo.net
robots.txt

Robots Exclusion Standard data for baseballinfo.net

Resource Scan

Scan Details

Site Domain baseballinfo.net
Base Domain baseballinfo.net
Scan Status Ok
Last Scan2025-01-15T13:49:32+00:00
Next Scan 2025-01-22T13:49:32+00:00

Last Scan

Scanned2025-01-15T13:49:32+00:00
URL https://baseballinfo.net/robots.txt
Domain IPs 160.16.102.166, 2001:e42:102:1527:160:16:102:166
Response IP 160.16.102.166
Found Yes
Hash af88697831dac28fa3b746e385ac703ec67230fdde8c9ae06dab885bdca3f23c
SimHash 186cc480fe3a

Groups

*

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot/1.1

Rule Path
Disallow /

linguee

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mauibot (crawler.feedback+dc@gmail.com)

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mj12bot/v1.4.8

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot/6~bl

Rule Path
Disallow /
Back to top

Comments

  • https://ahrefs.com/robot
  • http://webmeup-crawler.com/
  • https://moz.com/help/moz-procedures/crawlers/dotbot
  • http://www.linguee.com/bot
  • http://megaindex.com/crawler
  • https://megaindex.com/crawler
  • https://aspiegel.com/petalbot
  • https://www.semrush.com/bot/
Back to top