\
  The most prestigious law school admissions discussion board in the world.
BackRefresh Options Favorite

Rate my PC build for large model local LLM

AMD Ryzen 9 9950X GPU1: RTX 4090 GPU2: RTX A6000 128gb RA...
Hyperventilating hell international law enforcement agency
  12/11/24
why not two 4090
cocky indian lodge place of business
  12/11/24
My combo is more optimal for dual GPU since you can't onboar...
Hyperventilating hell international law enforcement agency
  12/11/24
the CPU absolutely does nothing for LLM?
cocky indian lodge place of business
  12/11/24
CPU matters. That's why I got the best one
Hyperventilating hell international law enforcement agency
  12/11/24
There's something called CPU offloading where some processin...
shimmering range multi-billionaire
  12/11/24
Do you think Apple's future M4 max/ultra cpu will be better ...
Hyperventilating hell international law enforcement agency
  12/11/24
no
shimmering range multi-billionaire
  12/11/24
M1 Ultra is still plenty
Heady red rehab
  01/08/25
i have similar but just for poasting
big navy tank
  12/11/24
...
Hyperventilating hell international law enforcement agency
  12/11/24
...
white talented people who are hurt halford
  12/11/24
...
Flatulent hairraiser alpha
  12/11/24
Where are you finding 4090s for sale that aren't jacked up 2...
Copper Generalized Bond
  12/11/24
I got it months ago and just putting together now. Are pr...
Hyperventilating hell international law enforcement agency
  12/11/24
For work?
Curious Wild Rigor Jewess
  12/11/24
Yep
Hyperventilating hell international law enforcement agency
  12/11/24
Is setting up a local LLM pretty easy? Does it need coding s...
galvanic locale
  12/11/24
really easy with ollama, https://ollama.com/
Curious Wild Rigor Jewess
  12/11/24
...
white talented people who are hurt halford
  12/11/24
whats the benefit of running these locally vs just subscribi...
insane toaster
  12/11/24
the ones running locally are not as powerful and don't accep...
Curious Wild Rigor Jewess
  12/11/24
does the hardware above make it practical to train your own?...
insane toaster
  12/11/24
1) Yes the above hardware can easily handle large models and...
Hyperventilating hell international law enforcement agency
  12/11/24
0 chance you will be able to train your own LLM, these compa...
Curious Wild Rigor Jewess
  12/11/24
You don’t need this much power just to take notes/exam...
multi-colored walnut organic girlfriend wrinkle
  12/11/24
163
Hyperventilating hell international law enforcement agency
  12/11/24
...
Flatulent hairraiser alpha
  12/11/24
...
Insanely creepy comical associate
  12/11/24
...
Flatulent hairraiser alpha
  12/12/24
how much did it all come to? will a local llm be capable of ...
amber stimulating mediation point
  12/11/24
Under $5k but I didn't buy a6000 card yet Those fuckers a...
Hyperventilating hell international law enforcement agency
  12/11/24
how many watts does the PSU have?
demanding candlestick maker sex offender
  12/11/24
170 Edit: this is the CPU watts
Hyperventilating hell international law enforcement agency
  12/11/24
that's a bit low, I don't think u actually have built a comp...
demanding candlestick maker sex offender
  12/11/24
Dark Side of the Moon 1350W PSU, Advanced Liquid Cooled CPU ...
Hyperventilating hell international law enforcement agency
  12/11/24
You are using an OEM Alienware part for you PSU? How does t...
demanding candlestick maker sex offender
  12/12/24
OP is growing a beard and changing monikres as we speak
Spectacular mint dog poop
  12/12/24
...
Hyperventilating hell international law enforcement agency
  12/12/24
I really want to put together my own local LLM setup too. Ta...
jade pit coffee pot
  12/11/24
We should start a discord or something
Hyperventilating hell international law enforcement agency
  12/11/24
I'm down, I'm still a total noob at this I haven't even star...
jade pit coffee pot
  12/11/24
If you have an M2 or later MacBook/mini you can install olla...
Hyperventilating hell international law enforcement agency
  12/11/24
can u show us what u can do with ur setup
cocky indian lodge place of business
  12/11/24
What will you use it for? How is it better than just getting...
pink pistol
  12/12/24
Re 1st question: scaling and streamlining CRM and customer s...
Hyperventilating hell international law enforcement agency
  12/12/24
link to wat u can get from ur setup
cocky indian lodge place of business
  12/12/24
automating as much of the client lifecycle as possible t...
Spectacular mint dog poop
  12/12/24
Bump for 5090, how many are you getting OP?
Copper Generalized Bond
  01/08/25
Isn't it just easier to buy a Mac Studio or whatever it's ca...
galvanic locale
  01/08/25
You can use this for DeepSeek now!
emerald histrionic internal respiration
  01/28/25


Poast new message in this thread



Reply Favorite

Date: December 11th, 2024 1:25 AM
Author: Hyperventilating hell international law enforcement agency

AMD Ryzen 9 9950X

GPU1: RTX 4090

GPU2: RTX A6000

128gb RAM

8TB NVMe 4.0

Windows 11 Pro

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48435761)



Reply Favorite

Date: December 11th, 2024 2:03 AM
Author: cocky indian lodge place of business

why not two 4090

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48435788)



Reply Favorite

Date: December 11th, 2024 2:12 AM
Author: Hyperventilating hell international law enforcement agency

My combo is more optimal for dual GPU since you can't onboard dual 4090s on 1 motherboard and eGPU throttles 50-70% of 4090 performance due to inherent bandwidth limitations with thunderbolt connection vs A6000 is optimized for low bandwidth and has 48gb VRAM (vs 24gb for 4090) to handle massive data sets



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48435793)



Reply Favorite

Date: December 11th, 2024 3:34 AM
Author: cocky indian lodge place of business

the CPU absolutely does nothing for LLM?

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48435877)



Reply Favorite

Date: December 11th, 2024 5:14 AM
Author: Hyperventilating hell international law enforcement agency

CPU matters. That's why I got the best one

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48435907)



Reply Favorite

Date: December 11th, 2024 8:22 AM
Author: shimmering range multi-billionaire

There's something called CPU offloading where some processing is handled by the CPU. But the kind of operations used in LLMs is different than the kind of operations at which CPUs excel.

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48436129)



Reply Favorite

Date: December 11th, 2024 1:36 PM
Author: Hyperventilating hell international law enforcement agency

Do you think Apple's future M4 max/ultra cpu will be better at LLM?

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437276)



Reply Favorite

Date: December 11th, 2024 2:57 PM
Author: shimmering range multi-billionaire

no

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437673)



Reply Favorite

Date: January 8th, 2025 9:20 PM
Author: Heady red rehab

M1 Ultra is still plenty

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48533861)



Reply Favorite

Date: December 11th, 2024 1:34 AM
Author: big navy tank

i have similar but just for poasting

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48435773)



Reply Favorite

Date: December 11th, 2024 2:14 AM
Author: Hyperventilating hell international law enforcement agency



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48435795)



Reply Favorite

Date: December 11th, 2024 3:05 PM
Author: white talented people who are hurt halford



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437693)



Reply Favorite

Date: December 11th, 2024 9:07 PM
Author: Flatulent hairraiser alpha



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438902)



Reply Favorite

Date: December 11th, 2024 11:24 AM
Author: Copper Generalized Bond

Where are you finding 4090s for sale that aren't jacked up 2x MSRP by scalpers?

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48436665)



Reply Favorite

Date: December 11th, 2024 1:33 PM
Author: Hyperventilating hell international law enforcement agency

I got it months ago and just putting together now.

Are prices on 4090 still that bad? I thought 5090 and 5080 were about to drop any day now

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437259)



Reply Favorite

Date: December 11th, 2024 1:34 PM
Author: Curious Wild Rigor Jewess

For work?

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437264)



Reply Favorite

Date: December 11th, 2024 8:56 PM
Author: Hyperventilating hell international law enforcement agency

Yep

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438874)



Reply Favorite

Date: December 11th, 2024 1:35 PM
Author: galvanic locale

Is setting up a local LLM pretty easy? Does it need coding savvy?

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437271)



Reply Favorite

Date: December 11th, 2024 1:36 PM
Author: Curious Wild Rigor Jewess

really easy with ollama, https://ollama.com/

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437274)



Reply Favorite

Date: December 11th, 2024 1:38 PM
Author: white talented people who are hurt halford



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437286)



Reply Favorite

Date: December 11th, 2024 3:40 PM
Author: insane toaster

whats the benefit of running these locally vs just subscribing to claude or whatever



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437815)



Reply Favorite

Date: December 11th, 2024 3:45 PM
Author: Curious Wild Rigor Jewess

the ones running locally are not as powerful and don't accept really long context windows. the main benefit is no limits to the number of requests and you can find uncensored models.

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437832)



Reply Favorite

Date: December 11th, 2024 3:55 PM
Author: insane toaster

does the hardware above make it practical to train your own? could you train on just on a giant dump of legal cases and get one superior at legal research at the expense of everything else, or is that even available better commercially at this point?



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437869)



Reply Favorite

Date: December 11th, 2024 4:38 PM
Author: Hyperventilating hell international law enforcement agency

1) Yes the above hardware can easily handle large models and is possibly even overkill for CRM, basic legal queries / prompts about client's file, and drafting complex legal briefs

2) Local llm with win10 Pro (eg, ollama, GPT4All, chat rtx) is the ONLY option for attorneys to do this offline given client data sensitivity

3) if you spent $10k on a fully spec'd out Mac Pro/Studio with 192gb unified ram it would not come close to this setup for local LLM in terms of speed and ability to handle large data and complex writing tasks

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438017)



Reply Favorite

Date: December 11th, 2024 5:23 PM
Author: Curious Wild Rigor Jewess

0 chance you will be able to train your own LLM, these companies spend tens of millions $$$ on gpu hardware to train an LLM.

The best you can do locally is "retrieval augmented generation" that is specific to legal cases/research.

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438197)



Reply Favorite

Date: December 11th, 2024 1:37 PM
Author: multi-colored walnut organic girlfriend wrinkle

You don’t need this much power just to take notes/exams at NYU Law tax classes.

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48437283)



Reply Favorite

Date: December 11th, 2024 8:48 PM
Author: Hyperventilating hell international law enforcement agency

163

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438847)



Reply Favorite

Date: December 11th, 2024 10:29 PM
Author: Flatulent hairraiser alpha



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48439148)



Reply Favorite

Date: December 11th, 2024 10:32 PM
Author: Insanely creepy comical associate



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48439153)



Reply Favorite

Date: December 12th, 2024 12:12 PM
Author: Flatulent hairraiser alpha



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48440746)



Reply Favorite

Date: December 11th, 2024 4:41 PM
Author: amber stimulating mediation point

how much did it all come to? will a local llm be capable of translating from exotic languages?

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438026)



Reply Favorite

Date: December 11th, 2024 8:52 PM
Author: Hyperventilating hell international law enforcement agency

Under $5k but I didn't buy a6000 card yet

Those fuckers are expensive!

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438854)



Reply Favorite

Date: December 11th, 2024 5:25 PM
Author: demanding candlestick maker sex offender

how many watts does the PSU have?

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438199)



Reply Favorite

Date: December 11th, 2024 8:50 PM
Author: Hyperventilating hell international law enforcement agency

170

Edit: this is the CPU watts

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438849)



Reply Favorite

Date: December 11th, 2024 9:06 PM
Author: demanding candlestick maker sex offender

that's a bit low, I don't think u actually have built a computer.

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438901)



Reply Favorite

Date: December 11th, 2024 9:15 PM
Author: Hyperventilating hell international law enforcement agency

Dark Side of the Moon 1350W PSU, Advanced Liquid Cooled CPU & Clear Side Panel

Item number: 321-BIKW

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438939)



Reply Favorite

Date: December 12th, 2024 8:50 AM
Author: demanding candlestick maker sex offender

You are using an OEM Alienware part for you PSU? How does that make sense?

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48440037)



Reply Favorite

Date: December 12th, 2024 8:54 AM
Author: Spectacular mint dog poop

OP is growing a beard and changing monikres as we speak

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48440043)



Reply Favorite

Date: December 12th, 2024 10:17 AM
Author: Hyperventilating hell international law enforcement agency



(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48440299)



Reply Favorite

Date: December 11th, 2024 8:53 PM
Author: jade pit coffee pot

I really want to put together my own local LLM setup too. Tagging this thread for later. 180

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438860)



Reply Favorite

Date: December 11th, 2024 8:53 PM
Author: Hyperventilating hell international law enforcement agency

We should start a discord or something

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438863)



Reply Favorite

Date: December 11th, 2024 8:59 PM
Author: jade pit coffee pot

I'm down, I'm still a total noob at this I haven't even started to do my own research on what hardware I'll need

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438883)



Reply Favorite

Date: December 11th, 2024 9:01 PM
Author: Hyperventilating hell international law enforcement agency

If you have an M2 or later MacBook/mini you can install ollama or lm studio right now on an external ssd and start fucking around

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48438889)



Reply Favorite

Date: December 11th, 2024 10:31 PM
Author: cocky indian lodge place of business

can u show us what u can do with ur setup

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48439150)



Reply Favorite

Date: December 12th, 2024 10:23 AM
Author: pink pistol

What will you use it for? How is it better than just getting a subscription to Claude/ChatGPT?

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48440310)



Reply Favorite

Date: December 12th, 2024 10:58 AM
Author: Hyperventilating hell international law enforcement agency

Re 1st question: scaling and streamlining CRM and customer service; reducing bottlenecks, automating as much of the client lifecycle as possible

Re 2nd question: mainly compliance with HIPAA, SCA, ECPA, CPRA etc

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48440474)



Reply Favorite

Date: December 12th, 2024 11:04 AM
Author: cocky indian lodge place of business

link to wat u can get from ur setup

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48440496)



Reply Favorite

Date: December 12th, 2024 1:06 PM
Author: Spectacular mint dog poop

automating as much of the client lifecycle as possible

tptptptp

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48441000)



Reply Favorite

Date: January 8th, 2025 8:34 PM
Author: Copper Generalized Bond

Bump for 5090, how many are you getting OP?

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48533708)



Reply Favorite

Date: January 8th, 2025 9:08 PM
Author: galvanic locale

Isn't it just easier to buy a Mac Studio or whatever it's called? Supposedly it's very good for private LLM stuff

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48533823)



Reply Favorite

Date: January 28th, 2025 3:12 PM
Author: emerald histrionic internal respiration

You can use this for DeepSeek now!

(http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2:#48598692)