twostrokesmoke.com
robots.txt
Robots Exclusion Standard data for twostrokesmoke.com
Resource Scan
Scan Details
| Site Domain | twostrokesmoke.com |
| Base Domain | twostrokesmoke.com |
| Scan Status | Ok |
| Last Scan | 2024-09-21T03:33:17+00:00 |
| Next Scan | 2024-09-28T03:33:17+00:00 |
Last Scan
| Scanned | 2024-09-21T03:33:17+00:00 |
| URL | https://twostrokesmoke.com/robots.txt |
| Domain IPs | 209.177.156.219 |
| Response IP | 209.177.156.219 |
| Found | Yes |
| Hash | 709467ee120de3e599a71dcc0b9ccf907ec0c00f7b966752bc5b95cc44d1e99e |
| SimHash | 4a15695053d1 |
Groups
amazonbot
googleother
| Rule | Path |
|---|---|
| Disallow | /member/ |
| Disallow | /modelinfo/ |
| Disallow | /registry/ |
| Disallow | /forum/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
baiduspider
ccbot
cliqzbot
criteobot
crsspxlbot
dataforseobot
discoverybot
domaincrawler
dotbot
extlinksbot
ezooms
grapeshot
istellabot
jikespider
linkpadbot
magpie-crawler
mauibot
megaindex.ru
mj12bot
proximic
seokicks-robot
sistrix
sogou
spbot
velenpublicwebcrawler
wbsearchbot
weborama
weborama-fetcher
wesee
yandex
yyspider
zumbot
| Rule | Path |
|---|---|
| Disallow | /journal/ |
| Disallow | /member/ |
| Disallow | /modelinfo/ |
| Disallow | /phorum/ |
| Disallow | /forum/ |
| Disallow | /registry/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
ahrefsbot
alphabot
alphaseobot
alphaseobot-sa
anthropic-ai
barkrowler
blexbot
bomborabot
careerbot
chatgpt-user
cohere-ai
friendlycrawler
google-extended
gptbot
gumgum
gumgum-bot
linguee
meta-externalagent
ip-web-crawler.com
oai-searchbot
panscient.com
petalbot
semrushbot
semrushbot-ba
semrushbot-sa
seznambot
turnitinbot
trendictionbot
zoominfobot
| Rule | Path |
|---|---|
| Disallow | / |
bingbot
msnbot
msnbot-media
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
seekport
weborama
weborama-fetcher
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
*
| Rule | Path |
|---|---|
| Disallow | /ads/click.php |
| Disallow | /calendar/attend.php |
| Disallow | /desktop |
| Disallow | /forum/newer/* |
| Disallow | /forum/older/* |
| Disallow | /member/activate |
| Disallow | /member/map.php |
| Disallow | /member/password_reset.php |
| Disallow | /member/reset |
| Disallow | /member/pp_result.php |
| Disallow | /modelinfo/ |
| Disallow | /newsletter/ |
| Disallow | /phorum/control.php |
| Disallow | /phorum/feed.php |
| Disallow | /phorum/login.php |
| Disallow | /phorum/moderation.php |
| Disallow | /phorum/pm.php |
| Disallow | /phorum/post.php |
| Disallow | /phorum/posting.php |
| Disallow | /phorum/random.php |
| Disallow | /phorum/read.php?*%2Cprintview%2C* |
| Disallow | /phorum/read.php?*%2Cquote=1 |
| Disallow | /phorum/read.php?*%2Cnewer |
| Disallow | /phorum/read.php?*%2Colder |
| Disallow | /phorum/report.php |
| Disallow | /phorum/search.php |
| Disallow | /registry/imgr_reset_pass.php |
| Disallow | /registry/browse.php |
| Disallow | /registry/transfer.php |
| Disallow | /toggle_comment.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Comments