The skill: Writing metric definitions that leave no room for interpretation. Ambiguous metrics create parallel realities where every team has a different number for the same thing.
In a Nutshell
- Every metric needs six things: numerator, denominator, time window, inclusion criteria, exclusion criteria, and data source.
- The grandma test. If someone outside your team can't understand the definition, rewrite it.
- One definition per metric, company-wide. "Active user" means one thing. Not three things depending on which team you ask.
- Exclusions are not optional. Bots, test accounts, internal users, sandbox environments. List them explicitly.
- Document the SQL, not just the English. The English definition says what it means. The SQL shows how it's calculated. You need both.
- Version your definitions. When a definition changes, note the date and explain what changed and why. Historical comparisons across definition changes are meaningless without context.
- Review quarterly. The business evolves. A metric defined 18 months ago might no longer reflect what matters.
Anatomy of a good metric definition
Here's what a complete metric definition looks like in practice:
Name: Weekly Active Users (WAU)
English definition: The count of unique users who performed at least one core action in the last 7 days.
Numerator: COUNT(DISTINCT user_id)
Denominator: N/A (this is a count, not a rate)
Time window: Rolling 7-day window, recalculated daily
Core actions included: page_view, feature_used, content_created, message_sent
Exclusions: Internal users (email domain = company.com), test accounts (user_id in test_accounts table), bot traffic (flagged by bot detection pipeline)
Data source: events.core_events table, joined with users.user_profiles for exclusion flags
Owner: Data team (for definition), Product team (for what counts as a "core action")
Last reviewed: Q1 2026
Change log:
- Q4 2025: Added message_sent to core actions after launching messaging feature
- Q2 2025: Switched from 30-day to 7-day window to better reflect engagement patterns
Notice what's not ambiguous. There's no "active means engaged with the product" hand-waving. You know exactly which events count, which users are excluded, where the data lives, and who owns the definition. Two engineers looking at this will write the same SQL query. That's the goal.
Watch out for stopping at the English definition. "Active user = someone who uses the product regularly." That is a vibe, not something two engineers can independently turn into the same SQL query.
Do's and Don'ts
Written with ❤️ by a human (still)