trabantforums.com
robots.txt
Robots Exclusion Standard data for trabantforums.com
Resource Scan
Scan Details
| Site Domain | trabantforums.com |
| Base Domain | trabantforums.com |
| Scan Status | Failed |
| Failure Reason | Scan timed out. |
| Last Scan | 2024-11-02T02:42:13+00:00 |
| Next Scan | 2024-12-02T02:42:13+00:00 |
Last Successful Scan
| Scanned | 2024-10-04T02:40:48+00:00 |
| URL | https://trabantforums.com/robots.txt |
| Domain IPs | 209.177.156.219 |
| Response IP | 209.177.156.219 |
| Found | Yes |
| Hash | 63b1701e4e562c2b1fcc4a8970070a9e525e0302c8c2077484ad88ec3def974c |
| SimHash | 5a15695053d0 |
Groups
amazonbot
googleother
| Rule | Path |
|---|---|
| Disallow | /member/ |
| Disallow | /modelinfo/ |
| Disallow | /registry/ |
| Disallow | /forum/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
baiduspider
ccbot
cliqzbot
criteobot
crsspxlbot
dataforseobot
discoverybot
domaincrawler
dotbot
extlinksbot
ezooms
grapeshot
istellabot
jikespider
linkpadbot
magpie-crawler
mauibot
megaindex.ru
mj12bot
proximic
seokicks-robot
sistrix
sogou
spbot
velenpublicwebcrawler
wbsearchbot
weborama
weborama-fetcher
wesee
yandex
yyspider
zumbot
| Rule | Path |
|---|---|
| Disallow | /journal/ |
| Disallow | /member/ |
| Disallow | /modelinfo/ |
| Disallow | /phorum/ |
| Disallow | /forum/ |
| Disallow | /registry/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
ahrefsbot
alphabot
alphaseobot
alphaseobot-sa
anthropic-ai
awariorssbot
awariosmartbot
barkrowler
blexbot
bomborabot
bytespider
careerbot
chatgpt-user
cohere-ai
friendlycrawler
google-extended
gptbot
gumgum
gumgum-bot
ip-web-crawler.com
linguee
meta-externalagent
oai-searchbot
perplexitybot
petalbot
semrushbot
semrushbot-ba
semrushbot-sa
seznambot
timpibot
trendictionbot
turnitinbot
zoominfobot
| Rule | Path |
|---|---|
| Disallow | / |
bingbot
msnbot
msnbot-media
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
seekport
weborama
weborama-fetcher
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
*
| Rule | Path |
|---|---|
| Disallow | /ads/click.php |
| Disallow | /calendar/attend.php |
| Disallow | /desktop |
| Disallow | /forum/newer/* |
| Disallow | /forum/older/* |
| Disallow | /member/activate |
| Disallow | /member/map.php |
| Disallow | /member/password_reset.php |
| Disallow | /member/reset |
| Disallow | /member/pp_result.php |
| Disallow | /modelinfo/ |
| Disallow | /newsletter/ |
| Disallow | /phorum/control.php |
| Disallow | /phorum/feed.php |
| Disallow | /phorum/login.php |
| Disallow | /phorum/moderation.php |
| Disallow | /phorum/pm.php |
| Disallow | /phorum/post.php |
| Disallow | /phorum/posting.php |
| Disallow | /phorum/random.php |
| Disallow | /phorum/read.php?*%2Cprintview%2C* |
| Disallow | /phorum/read.php?*%2Cquote=1 |
| Disallow | /phorum/read.php?*%2Cnewer |
| Disallow | /phorum/read.php?*%2Colder |
| Disallow | /phorum/report.php |
| Disallow | /phorum/search.php |
| Disallow | /registry/imgr_reset_pass.php |
| Disallow | /registry/browse.php |
| Disallow | /registry/transfer.php |
| Disallow | /toggle_comment.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Comments