Oh I have so many thoughts on this @harpreet.sahota (nah same machine setup as always - just upgraded my mics hahaha).
But here are my thoughts:
To Cloud or Not To Cloud - that is the question
def i_should_cloud() -> bool:
if (you.discipline == "high") or (you.bank_balance == inf):
return True
else:
return False
(P.S.: Sorry for writing code without unit tests)
Down sides of on-prem
Speed is less the issue than size of your VRAM tbh - SOTA models can be pretty damn huge to fit onto memory - take that into consideration. Either way it is simply not going to live up to the kinds of GPUs you can get on cloud (Tesla A100s 
etc.)
Why I hate laptops
- battery issues
- size & heat management issues for long training runs
- hardware upgrade limitations
Why I hate desktops
- I live in Australia… I really don’t need a space heater.
- Still limited in terms of upgrade path
Down sides of cloud
Cloud costs are genuinely expensive - be wary of compute instances that you just leave running - that’s a time-bomb your wallet will NOT like. Especially if you’re linked up to a powerful GPU.
What’re you trying to do?
- What kind of models are you trying to train?
- Can they fit in your vRAM on a standalone graphics card?
- What CAN and CAN’T you do with a commercial gaming card? make sure it suits your purpose.
Operating System Considerations
MacOS
For on-prem just don’t bother. Usually I would recommend this for developer experience… but since moving to apple silicon basically none of the key packages work reliably (inc. numpy, pandas, etc.) and you’d have to build some dependencies from source. It’s a hassle - wait for that to settle before going for mac again.
If you’re going pure cloud or dev-in-docker then this is still a pretty good option.
Linux
It’s a pain to set up but once it’s set up is the easiest to get up and running with training networks etc. This is great if all you do is ML with this machine.
I’d recommend this as a dual boot option for on-prem.
Windows
If it’s Win11 you’ll have Windows Subsystem Linux with GPU pass through. I’ve found this actually works extremely well post-June 2022. It’s my personal set up of choice because I like having access to other things like CAD software and good pro-grade software like office 365 etc. that just make my life easy.
It also means far less finicky drivers and machine setup.
Hardware Configurations to Consider
Towers/Desktops
- Good: If your intent is a pure home workstation that’s static this is good
- Good: Easy maintenance
- Good: Value for money (no portability premium)
- Bad: upgrades are limited by tower size. You may be able to upgrade ram and storage and CPU, but changing to a higher class GPU sometimes is impossible without a bigger case
Gaming “Laptops”
Let’s be real… these aren’t usually laptops but i’ve used a Dell G7 before and quite enjoyed using it.
- Bad: portability premium for a machine that is best behaved as a desktop anyway
- Bad: little to no upgradability
Laptops + eGPUs
This is a powerful combo if you want full power at home but only need CPU internal power to demo models live in front of people or want to just use the laptop as a regular laptop.
- Good: Extremes of portability (unplug the GPU and tk
- Good: Configurability (you can use the same GPU on other laptops as well.
- Bad: Expensive
- Bad: limits you to laptops that have a Thunderbolt 3 port
If you can afford it I would 100% recommend off-board GPUs.
