I have been thinking a lot about digital sovereignty lately and how quickly the internet is turning into a weird blend of surreal slop and centralized control. It feels like we are losing the ability to tell what is real because of how easy it is for trillionaire tech companies to flood our feeds with whatever they want.
Specifically I am curious about what I call “kirkification” which is the way these tools make it trivial to warp a person’s digital identity into a caricature. It starts with a joke or a face swap but it ends with people losing control over how they are perceived online.
If we want to protect ourselves and our local communities from being manipulated by these black box models how do we actually do it?
I want to know if anyone here has tried moving away from the cloud toward sovereign compute. Is hosting our own communication and media solutions actually a viable way to starve these massive models of our data? Can a small town actually manage its own digital utility instead of just being a data farm for big tech?
Also how do we even explain this to normal people who are not extremely online? How can we help neighbors or the elderly recognize when they are being nudged by an algorithm or seeing a digital caricature?
It seems like we should be aiming for a world of a million millionaires rather than just a room full of trillionaires but the technical hurdles like isp throttling and protocol issues make that bridge hard to build.
Has anyone here successfully implemented local first solutions that reduced their reliance on big tech ai? I am looking for ways to foster cognitive immunity and keep our data grounded in meatspace.
I’m exactly doing this atm. I’m running a homelab on a $200 USD lenovo p330 tiny with a Tesla P4 GPU, via Proxmox, CasaOS and various containers. I’m about 80% finished with what I want it to do.
Uses 40W at the wall (peak around 100W). IOW about the cost of a light bulb. Here’s what I run -
LXC 1: Media stack
Radarr, Sonarr, Sabnzdb, Jellyfin. Bye bye Netflix, D+ etc
LXC 2: Gaming stack
Emulation and PC gaming I like. Lots of fun indie titles, older games (GameCube, Wii, PS2). Stream from homelab to any TV in house via Sunshine / Moonlight. Bye bye Gforce now.
LXC 3: AI stack
Llama.cpp + llama-swap (AI back ends)
Qdrant server (document server)
Openwebui (front end)
Bespoke MoA system I designed (which I affectionately call my Mixture of Assholes, not agents) using python router and some clever tricks to make a self hosted AI that doesn’t scrape my shit and is fully auditble and non hallucinatory…which would otherwise be impossible with typical cloud “black box” approaches. I don’t want black box; I want glass box.
Bye bye ChatGPT.
LXC 4: Telecom stack
Vocechat (self hosted family chat replacement for WhatsApp / messenger),
Lemmy node (TBC).
Bye bye WhatsApp and Reddit
LXC 5: Security stack
Wireguard (own VPN). NPM (reverse proxy). Fail2Ban. PiHole (block ads).
LXC 6: Document stack
Immich (Google photos replacement), Joplin (Google keep), Snapdrop (Airdrop), Filedrop (Dropbox), SearXNG (Search engine).
Once I have everything tuned perfectly, I’m going to share everything on Github / Codeberg. I think the LLM stack alone is interesting enough to merit attention. Everyone makes big claims but I’ve got the data and method to prove it. I welcome others poking it.
Ultimately, people need to know how to do this, and I’m doing my best to document what I did so that someone could replicate and improve it. Make it easier for the next person. That’s the only way forward - together. Faster alone, further together and all that.
PS: It’s funny how far spite will take someone. I got into media servers after YouTube premium, Netflix etc jacked their prices up and baked in ads.
I got into lowendgaming when some PCMR midwit said “you can’t play that on your p.o.s. rig”. Wrong - I can and I did. It just needed know how, not “throw money at problem till it goes away”.
I got into self hosting LLM when ChatGPT kept being…ChatGPT. Wasting my time and money with its confident, smooth lies. No, unacceptable.
The final straw was when Reddit locked my account and shadow banned me for using different IP addresses while travelling / staying at different AirBNBs during holiday “for my safety”.
I had all the pieces there…but that was the final “fine…I’ll do it myself” Thanos moment.
Wait how did you set it up to avoid haloucinations? Is there a guide you followed that you can point me to?
Possibly a dumb question, so I tentatively pre-apologize: is LXC “Linux Container?”
Is there somewhere I can follow to see this if you end up open sourcing it? Sounds pretty interesting (personally I’m looking into a k3s-based setup but it’s always interesting to see how others do things)
Yep! I will mirror it here -
https://github.com/BobbyLLM
(Its empty rn / place holder only).
I had a bunch of prelim write-ups on r/LocalLLM and r/LocalLlama and r/homelab but they’re in the shadowrealm now due to reddit ban (fuck reddit)
I will also post it on @homelabs and @privacy here; I think my MoA design is worthwhile enough to maybe even merit a post on HackerNews…but I want to cross all T’s and dot all I’s before I get into that bar fight lol.
Take it one step further and host your repo somewhere other than github. Codeberg, perhaps?