musclemecca.com
robots.txt

Robots Exclusion Standard data for musclemecca.com

Resource Scan

Scan Details

Site Domain musclemecca.com
Base Domain musclemecca.com
Scan Status Ok
Last Scan2024-09-04T20:53:58+00:00
Next Scan 2024-10-04T20:53:58+00:00

Last Scan

Scanned2024-09-04T20:53:58+00:00
URL https://musclemecca.com/robots.txt
Domain IPs 62.151.181.196
Response IP 62.151.181.196
Found Yes
Hash 86dd9fc8bd46369dbff1658c4944f0bdfdb10d34c17355a22259c293fffb26e1
SimHash 1b3c50435571

Groups

*

Rule Path
Disallow /whats-new/
Disallow /account/
Disallow /goto/
Disallow /login/
Disallow /search/
Disallow /admin.php
Disallow /Xen-Mecca/
Disallow /OldMuscleMecca1/
Disallow /conversations/*
Disallow /conversations/add?to=*
Disallow /bookmarks/*
Disallow /members/*
Disallow /search/*
Disallow /find-new/
Disallow /threads/*/latest
Disallow /watched
Disallow /lost-password
Disallow /forums/king-labs.552/
Disallow /forums/juicepal-uk.595/
Disallow /forums/monsterlabstore.596/
Disallow /forums/us-domestic.583/
Disallow /forums/silverback-compounds.581/
Disallow /forums/ugfreak.572/
Disallow /forums/roidshop.568/
Disallow /forums/dyel.466/
Disallow /forums/1-stop-domestic-shop.577/
Disallow /forums/steroids24.557/
Disallow /forums/roidsource.528/
Disallow /forums/roidshop.568/
Disallow /forums/the-provider.590/
Disallow /cdn-cgi/
Disallow /attachments/
Disallow /misc/
Disallow /payments.php

rogerbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 300

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

wordpress*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

proximic

Rule Path
Disallow /

maxpoint bot

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

httrack

Rule Path
Disallow /
Disallow /training

ravencrawler

Rule Path
Disallow /
Disallow /training

zumbot*

Rule Path
Disallow /
Disallow /training

sputnikbot

Rule Path
Disallow /

sputnik

Rule Path
Disallow /

sputnikbot/2.3

Rule Path
Disallow /

spbot

Rule Path
Disallow /

spbot/4.0.9

Rule Path
Disallow /

ia_archiver

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

blexbot

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

zing-bottabot/2.0

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 90

crawler4j

Rule Path
Disallow /

diffbot/0.1

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 300

caliperbot/1.0

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://musclemecca.com/sitemap.php

Warnings

  • 4 invalid lines.