AI Automations for Developer Tools Companies

The candidates look the same as they do anywhere. Intake, extraction, reconciliation, first drafts, watching a queue. What changes in a developer tools company is where the mistakes land.

The queue you want to automate is public

Support here happens in the open. An issue on the repository, a thread in chat, a post on the forum, a discussion nobody closed. An automation that replies in any of those is writing in public, under your name. The page ranks, and gets linked in other threads for years.

That changes the design before it changes any code. Anything the workflow would post in public goes through a person every time, with no confidence threshold that promotes it to unattended later. What runs unattended is the half that touches only your own systems. Labelling, routing, linking duplicates, setting a milestone. Lifting the version and runtime out of the issue template into fields. A wrong label costs a maintainer ten seconds. A wrong answer sitting under a bug report costs more goodwill than the whole workflow saves in a year.

Triage is a version question before it is a bug question

Nobody can separate a real defect from a usage error without three facts: which version, which runtime, and whether there is a reproduction. Finding those in a page of free text is the manual work. It is the part a model is genuinely good at. The information is already in the text, and somebody has to pull it out.

So the first thing we build here is usually an extractor, not a responder. It reads the issue and pulls out five things. The version, the runtime, the operating system, the first line of the error, and the stack frames belonging to your package. Then it checks that version against your release list, and flags whether a reproduction is present. Then it routes and stops. Nothing in that path is visible to the person who filed the issue, which is why it can run on its own from day one.

Every release breaks an example somewhere

The other recurring cost in this industry is drift. The code moves and the documentation stays where it was. An example stops compiling, a parameter is renamed, a response comes back with a different shape. Nobody finds out until somebody opens an issue about the docs.

Half of that wants a deterministic test and no model at all. Where that is true we say so and build it. Extract every code block in the docs, run them in CI against the current release, fail the build when one breaks. That is a better answer than anything with a prompt in it, and it is cheaper to run.

The half that does want a model is the comparison. Take the public API that changed in a release and find the pages and examples that mention it. The output is a short list of what probably needs rewriting, for a human. The list, not the rewrite.

Whose source code is sitting in the ticket

A stack trace is customer code. It carries file paths, internal package names, the shape of somebody’s private module tree and occasionally a token pasted in by accident. Sending it to a third-party model is a disclosure decision. The person who has to defend that decision is rarely the person who built the workflow.

We raise it in scoping because it changes the architecture, not the wording of a policy. Redaction before the call, a model running inside your own infrastructure, or a rule that a named channel never leaves your network at all. Whichever you pick, it has to exist before the first ticket goes through it. It also has to be written somewhere a security reviewer can read without asking you.

The people who inherit this build tools for a living

Your engineers read code all day and they will read this. So the prompt lives in the repository as a file, under review, with the evaluation cases in the directory next to it. The workflow runs on your CI or your own infrastructure. There is no hosted canvas somewhere with one person’s login on it.

That is the standard for all of our automations. In this industry it is also the difference between two outcomes. A handover that survives contact with the team, and a project deleted in the first review. A team that maintains a build system will not adopt a workflow they cannot read end to end, and they are correct to refuse.