Link to Github

In this blog post, I walk through how to take a lone Docker image and use k3s to manage it on a single-node machine (i.e. my desktop). The idea behind Docker is an isolated, self-sufficient container that can run anywhere, k3s and its bigger brother k8s (Kubernetes) is used to manage, in a declarative fashinon, these containers.

The core model of Kubernetes, and the main difference from simply calling docker run is the declarative reconciliation. Kubernetes is a control system built around a single loop:

Serving a small, open-source LLM behind vLLM, containerized on a single desktop GPU.

The repo is here: https://github.com/codecalligrapher/llm-serving-vllm-k8s

The target is to have a single-node Kubernetes, with Grafana to monitor and a throughput benchmark.

Status: containerized OpenAI-compatible endpoint running on an RTX 3060 Ti (8GB). K8s, monitoring, and benchmark are scoped below but not yet built.

Why vLLM

Mainly because of the OpenAI-compatible server, so the endpoint is v1/chat/completions and any OpenAI client works against it.

The primary goal of langugae models is next-word/next-sequence prediction. The transformer architecture is built on the premise of “attention”, this was developed to solve the precursor’s weakness in modeling long-length text sequences (namely recursive neural networks).

In its most basic format, an RNN is a reduce over a sequence with a carried accumulator (in machine-learning terms, the current state).

For context, reduce(function, iterable, initializer) goes through the iterable from left to right, applying the function at each call. It takes the output of function(accumulator, item) at each call, returning the new accumulator and moves to the next item. The initializer is the starting accumulator/state

Job hunting has a side effect nobody warns you about: your inbox turns into a landfill. Every job board you’ve ever touched (CaribbeanJobs, random “noreply@jobs2web.com” aggregators, LinkedIn digests) starts sending a daily dump of postings, 90% of which have nothing to do with what you’re looking for. I didn’t want to unsubscribe, since some of those matches are genuinely useful, and I didn’t want to write fifty brittle Gmail filter rules. So I built a small agent to sort it for me.

1. Install sudo dnf install hyprland

2. Remove autogenerated=1 from ./config/hypr/hyperland.conf to remove warning

3. Chance scale to 1 monitor=,preferred,auto,1.5 The above is resolution, (something), scale

4. Adding dependencies, packages: sudo dnf copr enable solopasha/hyprland

Then sudo dnf install wayland-devel wayland-protocols-devel hyprlang-devel pango-devel cairo-devel file-devel libglvnd-devel libglvnd-core-devel libjpeg-turbo-devel libwebp-devel libjxl-devel gcc-c++ hyprutils-devel hyprwayland-scanner

Installing waybar: sudo dnf install waybar

5. Configuring binds

$mainMod = ALT #changes from the default windows key

For shortcuts

Basic: bind = MODS, key, dispatcher, params