Case study · Creator membership platform
99.9% tagging accuracy

Scaling manual tagging accuracy across a 10-person research team

10 researchers · 9 creator sites · 3-round rollout

Publishing / Creator economy · Operations, data quality and process design · Project · via Hugo

99.9%
Tagging accuracy
10
Researchers led
9
Creator sites covered
3
Rollout rounds
The business problem

An automated book-tagging pipeline failed, so every book in every creator post across nine sites had to be identified and tagged by hand.

The diagnosis

With 10 people in one shared tracker, the real risks were duplicated work, inconsistent edition choices and small errors adding up unseen.

The intervention

Standardised statuses, built a triage pass for orphaned rows, a closed-loop exception process, weekly QA sampling and written edition-selection rules.

The business impact

99.9% accuracy across thousands of posts, all nine sites kept current, and a QA process that scaled to a later launch.

What I built
Researcher training guideTask trackerEdition-selection rulesQA sampling log

Documents kept private because they contain client details. Happy to walk through them in an interview.

What happened next

The QA and exception process scaled to a later site launch without extra headcount.

Overview. The platform helps book creators share their content and earn from their audiences. When automated tagging could not be trusted, I led a 10-person research team to do the work manually, with direct ownership of accuracy.

My role
  • Owned end-to-end accuracy for a tagging workflow spanning nine creator sites and thousands of posts.
  • Directed 10 researchers identifying the correct book edition from each post.
  • Ran a phased rollout: newest posts across every site first, then 50 posts back per site, then a full year back.
The challenge

Every book had to be matched to a purchasable edition without duplicate entries and without pausing any creator's feed.

What I did

What I built

  • 01A controlled status vocabulary (Started, In progress, Done, Skipped) enforced with data validation, so no two people tagged the same post.
  • 02A recurring triage pass that caught rows with no owner and no status before they became backlog.
  • 03A closed-loop exception process for broken covers, unavailable posts, unreleased editions and import failures.
  • 04A defined QA process: a fixed weekly sample per researcher, an error taxonomy and a feedback loop into training.
  • 05Velocity tracking by round and by site, so lagging sites got more researchers early.
  • 06Written edition rules (paperback first, then hardcover; skip if no eligible format) so 10 people made the same call.
Before and after
  • Tagging accuracy99.9%
  • Sites kept current9 of 9
  • Researchers10
  • Replaced ad-hoc spot checks with an auditable QA trail behind the 99.9% figure.
  • Delivered current feeds and bookshelves for every site without pausing new content.
Next case · 0193% CSAT vs 90% targetBuilding a quality framework for a 500+ agent healthcare operation →

More case studies

Full index