Building a Local, Tool-Calling Agent to Tame My Job-Alert Inbox

Job hunting has a side effect nobody warns you about: your inbox turns into a landfill. Every job board you’ve ever touched (CaribbeanJobs, random “noreply@jobs2web.com” aggregators, LinkedIn digests) starts sending a daily dump of postings, 90% of which have nothing to do with what you’re looking for. I didn’t want to unsubscribe, since some of those matches are genuinely useful, and I didn’t want to write fifty brittle Gmail filter rules. So I built a small agent to sort it for me.

This post covers email-relabeler: what it does, how it’s put together, and specifically how tool-calling turns a chat model into something that can act on a real inbox instead of just talking about it.

The problem with keyword filters#

Gmail’s native filters are good at syntactic matching: “if sender is X” or “if subject contains Y.” What they can’t do is semantic matching: reading a job posting’s body and deciding whether it’s an AI Engineering / Data Science / MLOps role, or a bakery production manager job that happened to come from the same alert service.

That distinction lives in unstructured text. A keyword filter can’t make that call; an LLM can. So the design splits the problem into two layers:

  1. A cheap, deterministic prefilter: a single Gmail search query that narrows thousands of inbox messages down to “things that are plausibly job alerts,” using known senders and subject strings.
  2. An LLM judgment layer: for messages that survive the prefilter, a model reads the body and decides whether it’s actually relevant, then labels and archives it.

The prefilter handles roughly 95% of the volume reduction with no model call. The model only gets invoked for the ambiguous remainder, which keeps cost down and limits how much surface area the LLM has to mess up.

Why “agent” and not just “a script that calls an LLM once”#

A single LLM call is a function: text in, text out. What I needed was closer to a small worker that could search the inbox with a query it constructs itself, pull the full body of a specific message, reason about relevance, and, only for messages it deems relevant, take a write action (label + archive).

That’s a multi-step process where the next action depends on the result of the previous one. A search result decides which message to fetch; a fetched body decides whether a label gets applied. That’s the textbook shape of an agent: a loop where a model repeatedly decides what to do next until it’s done.

Tool-calling is the mechanism that makes that safe and structured.

Tool-calling, concretely#

LLMs don’t have arms. They can only produce text. Tool-calling (function calling) is the convention that turns that text into action safely: instead of letting the model write Python or shell commands directly, you hand it a menu of typed functions it’s allowed to request, and your own code is the only thing that actually executes them.

Here’s the tool menu from main.py:

tools = [
    {"type": "function", "function": {
        "name": "search_emails",
        "description": "Search Gmail",
        "parameters": {"type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"]},
    }},
    {"type": "function", "function": {
        "name": "get_body",
        "description": "Get body+URLs for a message ID",
        "parameters": {"type": "object",
            "properties": {"message_id": {"type": "string"}},
            "required": ["message_id"]},
    }},
    {"type": "function", "function": {
        "name": "apply_label",
        "description": "Label + archive a message",
        "parameters": {"type": "object",
            "properties": {"message_id": {"type": "string"}, "label": {"type": "string"}},
            "required": ["message_id", "label"]},
    }},
    {"type": "function", "function": {
        "name": "classify_message",
        "description": "Judge if a job relates to AI Engineering/Data Science/MLOps from subject+body",
        "parameters": {"type": "object",
            "properties": {"subject": {"type": "string"}, "body": {"type": "string"}},
            "required": ["subject", "body"]},
    }},
]

Each entry is a JSON Schema description of a function: name, plain-English description, arguments. This schema goes to the model alongside the conversation on every call. The model never executes anything itself; it can only emit a structured request like:

{"name": "search_emails", "arguments": {"query": "newer_than:1d subject:\"Job Alert\""}}

Your code decides whether to actually run that request. Here, that’s execute_tool(), a plain if name == "search_emails": ... dispatcher that calls the real Gmail API and returns a real result.

That’s the whole security model in one sentence: the model can only ever request one of four fixed, narrow, typed operations. It cannot touch Gmail, the filesystem, or anything else directly. No arbitrary code execution, no shell access, no letting the model write a query and run it itself. The attack surface is exactly four functions, and the riskiest one (apply_label, the only one that mutates the mailbox) is scoped to the gmail.modify OAuth permission. It can’t delete anything, read other Google services, or send mail.

The agent loop#

With the tools defined, the loop that drives the agent fits in one breath:

def run_agent(svc, goal: str):
    messages = [{"role": "user", "content": goal}]
    while True:
        resp = ollama.chat(model="llama3.1", messages=messages, tools=tools)
        msg = resp["message"]
        messages.append(msg)
        if not msg.get("tool_calls"):
            return msg["content"]
        for call in msg["tool_calls"]:
            result = execute_tool(svc, call["function"]["name"], call["function"]["arguments"])
            messages.append({"role": "tool", "content": json.dumps(result)})

Walking through it:

  1. Seed the conversation with a goal, e.g. “find today’s job alert emails and label the relevant ones.”
  2. Call the model with the running message history and the tool manifest.
  3. If the reply contains tool_calls, the model isn’t done reasoning yet; it needs more info or wants to act.
  4. For each requested call, execute it against the real Gmail API and append the JSON result back into the conversation as a "role": "tool" message.
  5. Loop. Call the model again with the new information in context.
  6. The loop ends the moment a response comes back with no tool calls. That’s the model saying “I’m done, here’s my answer.”

Same request → execute → observe → repeat pattern that underlies most agent frameworks (LangChain agents, OpenAI’s function-calling loop, Claude’s tool use), just without a framework wrapping it. ollama.chat() already speaks the OpenAI-style tool-calling protocol, so the loop is about 10 lines of plain Python with no dependency beyond the ollama client.

Why local, why Ollama#

The model is Ollama running llama3.1 on the machine itself, not a hosted API. Three reasons:

  • Privacy. Email bodies never leave the machine. For something reading a personal inbox, that’s worth more than a marginal quality bump from a bigger hosted model.
  • Cost. No per-token billing, so the agent can run daily, or more often, without API spend creeping up.
  • Latency doesn’t matter. This isn’t a chat UI with a user waiting on a response; it’s a background job. A locally-served 8B-class model being slower than GPT-4 is irrelevant here.

The tradeoff is classification quality: a local 8B model is less reliable at nuanced judgment calls than a frontier hosted model. That’s why the architecture keeps the model’s job as narrow as possible (decide relevance from a subject and body, request a label) instead of trusting it with anything open-ended.

The Gmail side: OAuth and the narrowest scope that works#

Talking to Gmail goes through google-api-python-client, authenticated via OAuth2 (google-auth-oauthlib). The scope requested is just:

SCOPES = ["https://www.googleapis.com/auth/gmail.modify"]

gmail.modify allows reading and changing labels but not deleting messages or accessing settings, which is the minimum needed for “read a message, add a label, remove it from the inbox.” First run opens a browser for the consent screen and caches the token in token.json so later runs are silent. Both token.json and credentials.json are gitignored, since they’re effectively bearer credentials for the account.

Labeling in Gmail’s API has no first-class “archive” action. Archiving a message just means removing the special INBOX label. So apply_label does exactly that:

def execute_tool(svc, name, inp):
    ...
    if name == "apply_label":
        label_id = get_or_create_label(svc, inp["label"])
        svc.users().messages().modify(
            userId="me",
            id=inp["message_id"],
            body={"addLabelIds": [label_id], "removeLabelIds": ["INBOX"]},
        ).execute()
        return {"status": "ok"}

One label gets added (created on the fly via get_or_create_label if it doesn’t exist), INBOX gets removed, and the message disappears from the inbox view while staying searchable under its new label. No delete, no send, no scope beyond what the task needs.

How the two layers connect in practice#

main() runs the full pipeline end to end. First, the cheap, deterministic Gmail query does its job with no model call involved:

QUERY = 'newer_than:1d (subject:"Job Alert" OR from:jobalerts@caribbeanjobs.com OR from:@noreply.jobs2web.com)'

Every message that survives that query is a candidate, not an automatic label. For each one, main() builds a per-message goal (fetch the body, classify it, label it only if relevant) and hands that goal to run_agent():

goal = (
    f'Message ID {m["id"]} is a candidate job alert email with subject: "{subject}". '
    f"Call get_body to fetch its content, then call classify_message with that subject "
    f"and body to judge whether it relates to AI Engineering, Data Science, or MLOps. "
    f'If it is relevant, call apply_label with message_id="{m["id"]}" and label="{LABEL}". '
    f"If it is not relevant, do not call apply_label — just say so."
)

This is where the agent loop earns its keep. The model decides, in order, to call get_body, then classify_message, and only reaches for apply_label if the classification comes back relevant. classify_message runs as its own JSON-mode call so its output is a clean {"relevant": bool, "reason": str} rather than free text the agent has to parse mid-loop. Nothing gets labeled on sender or subject match alone anymore; the body has to earn it.

Roadmap from here:

  • Generalize past the single “AI Engineering / Data Science / MLOps” category to arbitrary user-defined categories
  • Replace per-message print() output with a short digest of what got labeled and why

Takeaways#

  • Tool-calling is the boundary between “the model thinks” and “the model acts.” Keeping that boundary a short list of narrow, typed, purpose-built functions is what makes it safe to let a model act on a real system at all.
  • Don’t make the model do work a regex can do for free. The deterministic prefilter handles the bulk of the filtering before any model call happens; the LLM is reserved for the one judgment call that’s genuinely ambiguous.
  • Scope credentials to the task. gmail.modify, not full Gmail access. The agent’s blast radius is bounded by what its OAuth grant even allows, independent of what the code does.
  • You don’t need a framework to build an agent. The entire loop (call model, check for tool calls, execute, append result, repeat) is about ten lines of Python once the model API speaks the tool-calling protocol natively.

The code for this lives in main.py; the full tool list, setup steps, and roadmap are in the repo’s README.md.