Log In Sign Up

everymac.com
robots.txt

Robots Exclusion Standard data for everymac.com

Resource Scan

Scan Details

Site Domain everymac.com
Base Domain everymac.com
Scan Status Ok
Last Scan2024-05-23T20:19:11+00:00
Next Scan 2024-05-30T20:19:11+00:00

Last Scan

Scanned2024-05-23T20:19:11+00:00
URL https://everymac.com/robots.txt
Domain IPs 104.21.36.129, 172.67.194.95, 2606:4700:3030::ac43:c25f, 2606:4700:3035::6815:2481
Response IP 104.21.36.129
Found Yes
Hash 34e1d0d661d2eec25a1343ad1992f57c50034c1a5169cc55fabb5f5862c24db4
SimHash 51ef1506ea2e

Groups

googlebot

Rule Path
Disallow /doubleclick/
Disallow /34666846/
Disallow /app/
Disallow /staging/
Disallow /articles/admin/frames.html
Disallow /404.html

ahrefs

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mbot

Rule Path
Disallow /

java

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

wpws

Rule Path
Disallow /

netestate

Rule Path
Disallow /

xspider

Rule Path
Disallow /

rankactivelinkbot

Rule Path
Disallow /

elmer

Rule Path
Disallow /

linkcheck

Rule Path
Disallow /

winhttp

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

filemaker

Rule Path
Disallow /

frog

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /
Back to top

Warnings

  • 2 invalid lines.
Back to top