The primary goal of langugae models is next-word/next-sequence prediction. The transformer architecture is built on the premise of “attention”, this was developed to solve the precursor’s weakness in modeling long-length text sequences (namely recursive neural networks).

In its most basic format, an RNN is a reduce over a sequence with a carried accumulator (in machine-learning terms, the current state).

For context, reduce(function, iterable, initializer) goes through the iterable from left to right, applying the function at each call. It takes the output of function(accumulator, item) at each call, returning the new accumulator and moves to the next item. The initializer is the starting accumulator/state

Job hunting has a side effect nobody warns you about: your inbox turns into a landfill. Every job board you’ve ever touched (CaribbeanJobs, random “noreply@jobs2web.com” aggregators, LinkedIn digests) starts sending a daily dump of postings, 90% of which have nothing to do with what you’re looking for. I didn’t want to unsubscribe, since some of those matches are genuinely useful, and I didn’t want to write fifty brittle Gmail filter rules. So I built a small agent to sort it for me.