Case Study · Workflow Scoping

Evaluated an AI authoring tool for a secure enterprise workflow

Environment Oxygen XML Editor, DITA
Role Pilot design, workflow evaluation, and UAT
Focus Workflow fit, operational readiness, and deployment decisions

I designed and ran a pilot of AI-assisted authoring using Oxygen XML Editor and its Positron AI add-on. Security requirements forced the tool through an internal proxy and language model, which reduced its built-in functionality and shifted prompt design, testing, maintenance, and user support onto our team.

The problem

Technical writers were under pressure from frequent release cycles while working in a structured DITA environment with strict content and validation requirements. AI could generate useful text and valid DITA, but writers still had to move between tools, copy and paste output, validate markup, and decide how the generated content fit into existing review and publishing workflows.

The pilot tested whether AI assistance could reduce this friction inside the team’s existing authoring environment without adding excessive review or operational overhead.

What was built

The pilot tested AI-assisted authoring against real documentation work rather than isolated prompt output. I developed prompts for common tasks, defined criteria for reviewing quality and usability, coordinated UAT with writers, analyzed the results, and reported on capability, workflow fit, and operational readiness.

Prompt development

  • Created prompts for summarization, short descriptions, and DITA transformation.
  • Designed prompts for repeatable use across documentation tasks.
  • Tested them against real content using defined quality criteria.

UAT and feedback

  • Coordinated user acceptance testing with pilot participants.
  • Collected feedback across task types.
  • Analyzed results for usability patterns and workflow friction.

Operational assessment

  • Evaluated output against authoring and validation requirements.
  • Assessed licensing, maintenance, governance, and support needs.
  • Identified deployment constraints and reported them to stakeholders.

What the pilot found

Grammar correction, formatting, and short-description prompts performed most reliably. Participants saw value in AI-assisted authoring, but basic prompt support did not remove enough work to create meaningful time savings for experienced writers.

The same features had different value for different users. Basic editorial prompts could support training, consistency, and less experienced writers. Experienced writers needed capabilities that reduced real authoring work rather than adding another editing layer.

For experienced writers, the strongest opportunity was workflow-aware support inside the authoring environment: creating topics, inserting valid structured content, updating maps, and assisting with steps inside the content model.

Tasks such as outlining and question generation scored lower and required more review. The pilot showed that usefulness depended on the task, the writer's experience, and how directly the tool could support the real workflow.

Key finding A successful AI pilot does not prove that the workflow is ready to deploy.

Four operational constraints shaped the deployment decision even though the core capability worked:

License dependency

The AI add-on required Oxygen Enterprise licenses, creating a cost barrier before the workflow could be used broadly.

Model instability

Model updates broke existing prompts without warning, creating silent regressions with no clear owner responsible for detecting or fixing them.

Proxy incompatibility

The internal AI proxy interfered with the AI add-on's built-in features, causing missing functionality and connection errors in participant environments.

Prompt maintenance burden

Every prompt required ongoing design, testing, and maintenance, but no governance model existed for managing a shared prompt library. Supporting the workflow would also have required ongoing ownership of user requests, prompt revisions, regression monitoring, and quality issues.

The decision

The pilot confirmed that AI-assisted authoring could help within a defined scope. Sustainable deployment, however, required workflow integration, prompt governance, tooling maintenance, licensing clarity, and ongoing evaluation.

The team deferred full adoption and proposed a lighter approach: a custom AI interface inside the CMS with lower overhead, clearer ownership, and fewer features to maintain.

The capability was useful, but the original implementation path was too heavy for the value it delivered. The pilot suggested that a smaller, more tightly integrated system that we owned could be a better fit for this context.

Related methods and patterns

Method used Smallest Reliable System Scoping A method for defining the smallest AI-supported workflow that delivers useful value without creating unnecessary maintenance or review work.
Pattern observed Workflow Bloat AI-supported workflows that add more review, correction, or coordination than the original process because the automation boundary was defined too broadly.