Log In Sign Up

sonichits.com
robots.txt

Robots Exclusion Standard data for sonichits.com

Resource Scan

Scan Details

Site Domain sonichits.com
Base Domain sonichits.com
Scan Status Ok
Last Scan2024-10-01T06:36:03+00:00
Next Scan 2024-10-08T06:36:03+00:00

Last Scan

Scanned2024-10-01T06:36:03+00:00
URL https://sonichits.com/robots.txt
Domain IPs 104.26.2.44, 104.26.3.44, 172.67.69.184, 2606:4700:20::681a:22c, 2606:4700:20::681a:32c, 2606:4700:20::ac43:45b8
Response IP 104.26.3.44
Found Yes
Hash 7b5e2111936498d1c3ca979526f8f9dda91374dceb572684a7f553807b1484bd
SimHash 5214d563e30f

Groups

spbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

musobot

Rule Path
Disallow /

steeler

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

riddler

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

sogou web spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

sogou inst spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

gumgum bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2
Back to top

Comments

  • User-agent: msnbot
  • Crawl-delay: 1
  • User-agent: *
  • Disallow: /*?track=*
Back to top