i got sick again so the financial update and also this thread are late. i’ll get the financial update up at a later point, or i might just combine it with january since there’s not that much to report as far as i can tell
i got sick again so the financial update and also this thread are late. i’ll get the financial update up at a later point, or i might just combine it with january since there’s not that much to report as far as i can tell
Thank you for your kind words. Every day gets brighter.
For homelab, I’m not 100% sure yet. I’m at least going to be getting a Synology NAS to replace my ancient Lenovo EMC2. I really wanted to get some hardware for running LLama 2 and KobaldCPP, but I’m struggling to find something that’s equal parts not noisy (I live in a studio), affordable(ish), and that has the minimum specs I’d need. I was unironically considering a Mac Mini with a rack converter because of the energy efficiency and powerful iGPU, but sadly they only ship up to 32GB of RAM. Since my reading suggests I’d want at least 64GB of RAM for LLama 2’s 70B version, I’m having to try some other way of doing things. I just wish I didn’t live in a studio so I could grab a secondhand rackmount server without worrying about noise levels. 😭
I have the same problem; my flat is only about 50sqm. Judging by the way things are going, I think there’s a chance Nvidia will release some consumer-grade hardware meant for LLMs in the near-ish future. Until they reveal their next lineup, although it may seem like a poor financial decision, I’m just sticking to using the cloud for running llms.
I’m also hoping to get my hands on some raspberry pis too. I would like to build a toy k3s cluster at some point and maybe run my own mastodon instance. :)