Serverless technical documentation automation with Amazon Bedrock

Problem

Techne runs training and knowledge-transfer sessions recorded on video, some over three hours long. Turning each recording into usable documentation (operations manual, troubleshooting, business rules) was days of manual work: someone watched it, transcribed what mattered, and wrote it up. It did not scale, and the bottleneck was not the writing, it was extracting faithfully what was said without inventing what was not.

AI-generated technical documentation has a specific failure mode that makes it useless when it goes wrong: hallucination. A fabricated command, error code, or table name is not a cosmetic detail, it corrupts the material and misleads whoever follows the procedure. The architecture problem, then, was not wiring a model to a queue; it was building the net that separates what the AI may diverge on from what it may not.

Approach

I designed a serverless, event-driven pipeline. Uploading a video to S3 triggers, via EventBridge, a Step Functions state machine that runs the heavy processing in an ECS Fargate container (2 vCPU, 8 GB) and leaves no server standing between executions.

The first part is transcription. Videos longer than 3.5 hours are split into chunks, each chunk goes to AWS Transcribe with diarization (up to 10 speakers, pt-BR), and the partial transcripts are reassembled with corrected time offsets. This step is the most expensive and the slowest, so it is idempotent: before transcribing, the processor looks in S3 for an already-finished transcript (or already-transcribed chunks) for that job and reuses it, so a rerun after a failure does not pay the transcription bill again.

The second part is where fidelity is decided. Instead of asking “generate a document” in a single call, I broke generation into seven chained stages.

Each stage does one thing and passes an explicit artifact forward: cleaning the transcript with timestamps preserved, extracting the technical content as JSON, mapping problem to solution, structuring the skeleton, writing the Markdown, and then the stage that holds up the fidelity promise, the audit. The token budget is allocated per stage as needed (writing uses the model’s full capacity; extraction, a quarter of it), which avoids truncating a section mid-way.

Hallucination is contained in three layers, not one. The first is a strict grounding system prompt that forbids the model from fabricating URLs, commit hashes, error codes, version numbers, commands, or code that do not appear in the transcript; if the instructor clicked a button, the text describes the interface action, it does not invent an equivalent terminal command. The second is stage 6, an auditor: the draft goes back to the model under a skeptical prompt, at temperature zero, with the original transcript alongside, and every technical claim must be backed by a timestamp in the source; whatever is not, gets removed or generalized. The third is deterministic: a regex glossary corrects terms that automatic transcription always mangles (product and module names it turns into near-homophones), without depending on the model’s interpretation.

The official terminology comes from a RAG knowledge base. Reference documents live in a pgvector vector store on RDS PostgreSQL, indexed by the Titan v2 embedding in 300-token chunks with 20% overlap; the writing and audit stages query that base (top-5 retrieval, minimum score 0.5) to spell the technical names the canonical way. Choosing RDS single-AZ as the vector store, rather than OpenSearch Serverless, cut the cost of that layer from hundreds to tens of dollars per month, roughly 85% cheaper, with performance to spare for the vector volume in play.

Technology

AWS Step Functions for orchestration; ECS Fargate (2 vCPU, 8 GB) for processing; AWS Transcribe with diarization; Amazon Bedrock with Claude Sonnet 4.5 for generation; RAG with Amazon Bedrock Knowledge Base over pgvector on RDS PostgreSQL 16 (Titan v2 embeddings). DynamoDB tracks the status of each execution, SNS notifies on completion, and the processor runs in a Python 3.12 container. Resilience through exponential retry (3 attempts), a circuit breaker (opens on 5 failures, resets after 5 minutes), and transcription polling capped at 2 hours. Output in Markdown and DOCX. Estimated cost around one dollar per 3-hour video.

Result

The pipeline turns an hours-long video session into structured technical documentation in minutes, and the same transcript feeds seven document types (troubleshooting, operational guide, operations manual, business rules, configuration and migration, conceptual guide, responsibilities and FAQ) without transcribing again.

What makes it usable is not the speed, it is the fidelity. The three anti-hallucination layers (grounding prompt, auditor with timestamp verification, and deterministic glossary) exist precisely because wrong technical documentation is worse than none. The architecture point was not plugging a model into a queue; it was deciding where the AI may diverge (writing, summarizing) and where it may not (names, commands, codes), and building the net that enforces that boundary.