About Olivia Kelly

I start by clarifying the purpose: What problem are we trying to solve, who is experiencing it, and what would meaningful improvement look like?

From there, I look at how the work actually happens, where information or workflows break down, and what creates unnecessary effort. The right response might involve content, search, workflow design, measurement, or AI. The technology comes after the problem is understood.

My background spans technical content, product work, knowledge systems, and workflow design. I use that experience to trace problems across the full system and help teams choose changes that are useful, supportable, and proportionate to the need.

This portfolio documents the decisions, evidence, and lessons behind that work.

Tools and environments

Oracle APEX, Oxygen XML Editor, DITA, Markdown, Oracle Cloud Infrastructure AI services, OpenSearch, Adobe Analytics, Google Search Console, Excel, CSV-based evaluation workflows, and prompt-based LLM evaluation.

Current research areas include RAG evaluation, LLM-as-judge methods, RAGAS-style metrics, and observability for AI-enabled content systems.