Log In Sign Up

taoofmac.com
robots.txt

Robots Exclusion Standard data for taoofmac.com

Resource Scan

Scan Details

Site Domain taoofmac.com
Base Domain taoofmac.com
Scan Status Ok
Last Scan2025-10-20T16:55:35+00:00
Next Scan 2025-10-27T16:55:35+00:00

Last Scan

Scanned2025-10-20T16:55:35+00:00
URL https://taoofmac.com/robots.txt
Domain IPs 104.26.4.187, 104.26.5.187, 172.67.71.149, 2606:4700:20::681a:4bb, 2606:4700:20::681a:5bb, 2606:4700:20::ac43:4795
Response IP 104.26.4.187
Found Yes
Hash 1b30234abdf6afb386627fe2092399c3012246184523931e99a8798e95a33423
SimHash 79551409ffd2

Groups

google

Rule Path
Disallow /js/
Disallow /themes/
Disallow /space/meta/

ia_archiver

Rule Path
Disallow /js/
Disallow /themes/
Disallow /space/meta/

*

Rule Path
Disallow /js/
Disallow /img/
Disallow /themes/

bloo

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

idmarch

Rule Path
Disallow /

fairshare

Rule Path
Disallow /

lightspeedsystems

Rule Path
Disallow /

zmeu

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ias_crawler

Rule Path
Disallow /

copyrightinfringementportal

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

slysearch

Rule Path
Disallow /

npbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

covario-ids

Rule Path
Disallow /

topblogsinfo

Rule Path
Disallow /

spbot

Rule Path
Disallow /

attributor

Rule Path
Disallow /

psbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

sitesucker

Rule Path
Disallow /

scooter

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

slurp

Rule Path
Disallow /

pompos

Rule Path
Disallow /

inetbot

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

zombies

Rule Path
Disallow /braaains
Back to top

Other Records

Field Value
sitemap https://taoofmac.com/sitemap.xml
Back to top

Comments

  • Bandwidth Sucking Nuisances (many of which don't throttle requests):
Back to top

Warnings

  • 2 invalid lines.
Back to top