Build a Slack triage agent, the job OpenAI’s dots post describes, with free software

What the agent is allowed to do

This is the part I decide before writing code, every time. The agent gets a Slack bot token with three scopes: read the history of channels it is in, read channel names, and post messages. It gets an app-level token so it can connect in Socket Mode. It cannot delete, cannot DM people, cannot read channels it was not invited to. The model it calls runs on the same machine and sees only the message text. And the single action that changes anything outside Slack, filing an issue, runs only after a person presses a button.

OpenAI's page, to its credit, describes the same posture for dots: proactive work uses tools restricted to read-only, and custom rules decide what the dot may do alone, what needs approval, and what is blocked. The difference is that here the rules are forty lines of Python you can read.

Why Socket Mode

A Slack app normally receives events at a public HTTPS URL, which means a domain, a certificate, a reverse proxy and an open port on the machine running your agent. For a thing that lives on a Mac mini or a small VPS I think that is the wrong trade. Socket Mode turns it around: the app opens a WebSocket out to Slack, Slack pushes events down it, and nothing on your machine listens for inbound connections. Slack's own documentation recommends Bolt for it, and notes that Socket Mode apps cannot be listed in the public marketplace, which for an internal agent is irrelevant.

That also makes it a daemon rather than a scheduled job: the socket has to stay open. So this agent is a systemd service with Restart=always, not a timer, and the difference matters for the setup pages in this series. If you want the cron-shaped version instead, poll conversations.history every five minutes and skip Socket Mode entirely; the triage step is identical.

The judgement step: a small local model with a fixed shape

The model's whole job is to fill in a form: is this a bug, which area, how severe, a one-line summary, one question for the reporter. I use Ollama's structured output, which takes a JSON schema in the format field and refuses to return anything that does not match it. That one feature is what makes a 1.5-billion-parameter model usable here: it cannot ramble, cannot invent a field, and the area it picks has to be one of the ones I listed, each of which maps to a named owner.

triage.py#!/usr/bin/env python3
"""Triage step: a Slack message in, a proposal out. The model never acts; it fills in a form a human confirms.
Runs against a local Ollama (OLLAMA_URL, default http://localhost:11434) with a small model."""
import json, os, sys, urllib.request

OLLAMA = os.environ.get("OLLAMA_URL", "http://localhost:11434")
MODEL = os.environ.get("TRIAGE_MODEL", "qwen2.5:1.5b")
OWNERS = {"billing": "@dana", "video-pipeline": "@lera", "instagram-connect": "@sam",
          "website": "@lera", "unknown": "@lera"}

SCHEMA = {                      # Ollama's structured output: the reply must be exactly this shape
    "type": "object",
    "properties": {
        "is_bug": {"type": "boolean"},
        "area": {"type": "string", "enum": list(OWNERS)},
        "severity": {"type": "string", "enum": ["low", "medium", "high"]},
        "summary": {"type": "string"},
        "question_for_reporter": {"type": "string"},
    },
    "required": ["is_bug", "area", "severity", "summary", "question_for_reporter"],
}

PROMPT = """You triage messages from a company Slack channel for a small video software company.
Decide whether the message reports a bug. If it does, pick the product area, a severity, write a
one-sentence summary in neutral words, and one question that would help reproduce it. If it is not a
bug, set is_bug false, area unknown, severity low, and summarise what it is instead. Answer only with JSON.

Message from {user} in #{channel}:
{text}"""


def classify(msg):
    body = {"model": MODEL, "stream": False, "format": SCHEMA, "options": {"temperature": 0},
            "messages": [{"role": "user", "content": PROMPT.format(**msg)}]}
    req = urllib.request.Request(OLLAMA + "/api/chat", data=json.dumps(body).encode(),
                                 headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=600) as r:
        out = json.load(r)
    verdict = json.loads(out["message"]["content"])
    verdict["proposed_owner"] = OWNERS.get(verdict["area"], OWNERS["unknown"])
    verdict["source"] = {"user": msg["user"], "channel": msg["channel"], "ts": msg["ts"]}
    verdict["status"] = "PROPOSED, waiting for a human"   # nothing happens until someone clicks Confirm
    return verdict


if __name__ == "__main__":
    for msg in json.load(open(sys.argv[1])):
        print(json.dumps(classify(msg), ensure_ascii=False))

Temperature zero, because I want the same message to get the same verdict tomorrow. The owner map is deliberately dumb: a dictionary, edited by hand. The moment I let the model choose owners from a free-text field I have an agent that can page the wrong person at 3am.

The Slack side

app.py#!/usr/bin/env python3
"""Slack side: listen in Socket Mode (no public URL), send each new message in the watched channels to
triage.classify, post the proposal back with Confirm / Not a bug buttons, and only when a human presses
Confirm file the issue. Needs SLACK_BOT_TOKEN (xoxb-...), SLACK_APP_TOKEN (xapp-...) and the bot in the channel."""
import json, os
from slack_bolt import App
from slack_bolt.adapter.socket_mode import SocketModeHandler
from triage import classify

WATCH = set(os.environ.get("TRIAGE_CHANNELS", "support").split(","))
app = App(token=os.environ["SLACK_BOT_TOKEN"])


@app.event("message")
def on_message(event, say, client):
    if event.get("subtype") or event.get("bot_id"):
        return                                  # edits, joins and bots: ignore
    info = client.conversations_info(channel=event["channel"])["channel"]
    if info["name"] not in WATCH:
        return
    v = classify({"user": event["user"], "channel": info["name"], "ts": event["ts"], "text": event["text"]})
    if not v["is_bug"]:
        return                                  # say nothing; a triage bot that chats is noise
    say(thread_ts=event["ts"], text="Triage proposal", blocks=[
        {"type": "section", "text": {"type": "mrkdwn", "text":
            "*Looks like a bug.* %s\n*Area:* %s  *Severity:* %s  *Owner:* %s\n*To reproduce, ask:* %s"
            % (v["summary"], v["area"], v["severity"], v["proposed_owner"], v["question_for_reporter"])}},
        {"type": "actions", "elements": [
            {"type": "button", "text": {"type": "plain_text", "text": "Confirm and file"}, "style": "primary",
             "action_id": "confirm", "value": json.dumps(v)},
            {"type": "button", "text": {"type": "plain_text", "text": "Not a bug"}, "action_id": "dismiss"}]},
    ])


@app.action("confirm")
def on_confirm(ack, body, client):
    ack()
    v = json.loads(body["actions"][0]["value"])
    url = file_issue(v)                         # the ONLY line that changes anything outside Slack
    client.chat_update(channel=body["channel"]["id"], ts=body["message"]["ts"], blocks=[],
                       text="Filed: %s (confirmed by <@%s>)" % (url, body["user"]["id"]))


@app.action("dismiss")
def on_dismiss(ack, body, client):
    ack()
    client.chat_update(channel=body["channel"]["id"], ts=body["message"]["ts"], blocks=[],
                       text="Dismissed by <@%s>." % body["user"]["id"])


def file_issue(v):
    with open("filed.jsonl", "a") as fh:          # stub: swap for your tracker's API call
        fh.write(json.dumps(v) + "\n")
    return "filed.jsonl#%s" % v["source"]["ts"]


if __name__ == "__main__":
    SocketModeHandler(app, os.environ["SLACK_APP_TOKEN"]).start()

Three design decisions in there. The bot ignores edits, joins and other bots, so it cannot loop on itself. It says nothing at all when the verdict is not a bug, because a triage bot that comments on lunch plans gets muted in a week. And the proposal is posted in the thread of the original message with two buttons, so the person who confirms is on the record, and the message is updated in place so a channel never fills with stale proposals.

What I actually ran

I do not have a throwaway Slack workspace with a bot installed on this machine, and I was not going to point an experimental agent at the real one. So the Slack listener above is written and reviewed but was not connected to Slack today; the Bolt pattern it uses is the documented Socket Mode pattern and nothing in it is exotic. What I did run, end to end, is the judgement step, which is the part that could actually go wrong: five made-up support messages, including a billing complaint, a lunch invitation and a caption-sync bug, through triage.py against a local model. I had meant to do that in a container too, but the Ollama image download kept failing on a busy afternoon, so the model ran under a standalone Ollama binary on my Mac instead; the script and the API call are identical either way.

messages.json[
  {"user": "priya", "channel": "support", "ts": "1760950001.0001",
   "text": "Since this morning the Instagram connect button just spins. Tried two browsers. Our account is @northstar.coffee, on the Growth plan."},
  {"user": "marco", "channel": "support", "ts": "1760950002.0002",
   "text": "Invoice for October shows 3 seats but we only have 2 users now, can someone fix the amount before it charges on Friday?"},
  {"user": "jules", "channel": "general", "ts": "1760950003.0003",
   "text": "Anyone fancy a team lunch on Thursday? Thinking the Lebanese place."},
  {"user": "ana", "channel": "support", "ts": "1760950004.0004",
   "text": "The clip that got scheduled last night has captions that are a few seconds behind the speaker, first time I've seen this. Post is still in drafts so no harm yet."},
  {"user": "tom", "channel": "support", "ts": "1760950005.0005",
   "text": "Pricing page loads blank on Safari on my iPad, works fine on my laptop."}
]

Output, 20 October 2026, qwen2.5:1.5b on CPU, five messages in 4 min 25 s{"is_bug": true, "area": "website", "severity": "medium", "summary": "Instagram connect button not working", "question_for_reporter": "Can you provide steps to reproduce the issue?", "proposed_owner": "@lera", "source": {"user": "priya", "channel": "support", "ts": "1760950001.0001"}, "status": "PROPOSED, waiting for a human"}
{"is_bug": false, "area": "billing", "severity": "low", "summary": "Invoice amount for October is incorrect.", "question_for_reporter": "Please confirm the correct number of seats and users.", "proposed_owner": "@dana", "source": {"user": "marco", "channel": "support", "ts": "1760950002.0002"}, "status": "PROPOSED, waiting for a human"}
{"is_bug": false, "area": "unknown", "severity": "low", "summary": "Request for team lunch on Thursday", "question_for_reporter": "What is the preferred location for the team lunch on Thursday?", "proposed_owner": "@lera", "source": {"user": "jules", "channel": "general", "ts": "1760950003.0003"}, "status": "PROPOSED, waiting for a human"}
{"is_bug": false, "area": "video-pipeline", "severity": "low", "summary": "User reports a video clip with captions that are delayed.", "question_for_reporter": "Please provide the video file and the expected and actual captions to help reproduce the issue.", "proposed_owner": "@lera", "source": {"user": "ana", "channel": "support", "ts": "1760950004.0004"}, "status": "PROPOSED, waiting for a human"}
{"is_bug": true, "area": "website", "severity": "low", "summary": "Pricing page loads blank on Safari on iPad, works fine on laptop.", "question_for_reporter": "Can you please check if the pricing page loads correctly in Safari on iPad?", "proposed_owner": "@lera", "source": {"user": "tom", "channel": "support", "ts": "1760950005.0005"}, "status": "PROPOSED, waiting for a human"}

I am glad I ran it rather than describing it, because the result is more useful than a clean demo would have been. Every reply matched the schema, every area was one from my list, and the lunch invitation was correctly left alone. But a 1.5-billion-parameter model is a sorter, not a judge. It called the spinning Instagram button a bug but filed it under website rather than instagram-connect, so the proposal would have gone to me instead of the person who owns that integration. It called the delayed captions not a bug, with the right area and a sensible reproduction question, which is the most dangerous kind of miss: a real defect that the bot would have stayed silent about. And it took four and a half minutes for five messages on CPU, which is fine for a support channel and hopeless for a busy one.

Two conclusions. The Confirm button is doing real work, which is the whole argument of this series: an unattended judgement needs a human on the far side of it. And the not a bug branch should not be silent at this model size; I would post the proposal in-thread either way and let the Dismiss button carry the no. A 7B model, or a paid API for the judgement step only, would fix most of this at a cost I can measure per message, and that is the trade I would make before letting the bot stay quiet.

Keeping it alive

This is the Linux core page pattern: a dedicated user, a venv, secrets in a root-owned environment file, a service that restarts. Ollama runs as its own service on the same box, listening only on localhost.

/etc/systemd/system/triage.service[Unit]
Description=Slack triage agent
After=network-online.target
Wants=network-online.target

[Service]
User=agent
WorkingDirectory=/opt/triage
EnvironmentFile=/etc/triage/env
ExecStart=/opt/triage/venv/bin/python /opt/triage/app.py
Restart=always
RestartSec=5
NoNewPrivileges=yes
ProtectSystem=strict
ReadWritePaths=/opt/triage

[Install]
WantedBy=multi-user.target

Terminalsudo systemctl daemon-reload
sudo systemctl enable --now triage.service
journalctl -u triage.service -f

On a Mac the same thing is a launchd agent with KeepAlive set, which is how my own jobs run; the Mac guide has the file. Memory is the number to watch: a 1.5B model on CPU wants about two gigabytes while it thinks, which is fine on a Mac mini and tight on the smallest VPS tiers.

Where this sits against dots

What dots has that this does not: a cloud computer with a browser, 4,000 app connections, and a model that can go and reproduce the bug rather than ask a question about it. What this has that dots does not: a bill of zero, a permission list you can read in one screen, and a verdict that is the same every time. For a five-person support channel I would run this. For a company where the agent should actually fix the bug, dots or something like it is the honest answer, and I would still want the Confirm button. The next build in the series moves from proposing to acting, with an inbox and calendar agent that can only ever draft.

Sources

1. OpenAI, Introducing dots (accessed October 20, 2026)
2. CBS News, Sam Altman unveils dots, OpenAI's new AI personal agent (accessed October 20, 2026)
3. Slack, Using Socket Mode (accessed October 20, 2026)
4. Slack, Bolt for Python (accessed October 20, 2026)
5. Ollama, Structured outputs (accessed October 20, 2026)

Written by Lera Leonteva. The classification step was run once on 20 October 2026 with qwen2.5:1.5b under a standalone Ollama 0.40.6 binary on my Mac, against the five messages shown; the Slack listener was written and reviewed but not connected to a workspace that day.