Operational definitions determine whether two people measuring the same process produce comparable results. A team may agree that customer requests are late while disagreeing about when the clock starts, what counts as a working day and whether waiting for customer information stops the clock. Those disagreements can make an improvement project appear successful or unsuccessful for the wrong reason.
Before collecting more data, define the event, unit, boundary and decision rule. This article uses an illustrative service-desk example to show how a measurement definition is developed, tested and maintained. The numbers are invented for explanation rather than reported organisational results.
Contents Two analysts and one unreliable dashboard • Decision 1 Choose the unit being counted • Decision 2 Name the characteristic precisely • Decision 3 Fix the start and stop events • Decision 4 Set the threshold and exclusions • Decision 5 Define the denominator • Decision 6 Explain who classifies borderline cases • A completed specification for operational definitions • Test the definition before full collection • Explain the measure to the people doing the work • Keep the measure comparable over time
Two analysts and one unreliable dashboard
A service manager asks two analysts to count late responses from the previous week. Analyst A counts 18. Analyst B counts 31. They used the same ticket database.
Analyst A measured time until the first personal reply, excluded weekends and removed tickets awaiting customer information. Analyst B measured time until resolution, included all elapsed hours and retained paused tickets. Neither calculation is necessarily arithmetically wrong. They measured different things under the same label.
The ASQ explanation of quality metrics illustrates why explicit measurement boundaries matter. A useful operational definition translates a broad idea into observable rules that another person can apply. The service desk needs those rules before drawing conclusions from a trend chart.
Decision 1 Choose the unit being counted
Decide whether the unit is a ticket, customer, interaction or service request. One customer may send three messages about a single problem. If every message becomes a separate defect opportunity, the result changes with communication behaviour as well as service performance.
For this example, define the unit as one eligible support ticket representing one customer issue. Duplicate tickets concerning the same issue are linked to the original and excluded from the primary denominator. Keep the duplicates visible in a separate operational measure if they indicate confusion or unnecessary demand.
Explain how mixed requests are handled. A ticket asking about an invoice and a technical failure may need to be split according to the service’s established process. Do not let analysts decide differently each week without documenting the rule.
Decision 2 Name the characteristic precisely
Replace late response with a measurable characteristic. First substantive response time is different from resolution time, acknowledgement time and time until the customer receives a satisfactory answer.
For the illustrative measure, use first substantive response time. An automatic receipt does not qualify. A personalised message asking a necessary clarification question may qualify if it addresses the issue and advances handling. A generic we are looking into it message does not.
This distinction should be tested using real examples. If the boundary between substantive and generic remains subjective, add examples and counterexamples. A definition that sounds precise in a meeting may still be difficult to apply in the database.
Decision 3 Fix the start and stop events
Set the start as the time an eligible ticket arrives in the support system during published service hours. Tickets received outside those hours begin at the next opening time. Set the stop as the recorded time of the first substantive response.
These are illustrative policy choices, not universal standards. A service promising support around the clock needs different rules. An organisation should not exclude out-of-hours waiting if doing so contradicts what customers were promised.
Specify time zones and daylight-saving handling if records cross locations. Define how imported tickets and missing timestamps are treated. Excluding records with missing values may improve the apparent result, so report their number and investigate why they are missing.
Decision 4 Set the threshold and exclusions
Assume the service has an established commitment to respond within four business hours. A ticket breaches the commitment when measured time exceeds four hours. A response at exactly four hours meets this illustrative rule; if the organisation uses another boundary, state it.
List exclusions narrowly: test tickets, spam and verified duplicates. Tickets involving difficult customers or unfamiliar technical issues remain eligible. Removing difficult cases would distort the measure and hide precisely the work the service needs to improve.
Do not apply a customer-wait pause to first response time if the customer has not yet received a substantive reply. Pause rules may belong to a resolution-time measure, but importing them into every metric creates confusion.
For project context, read The Case HQ’s guide to Lean and Six Sigma. Reliable measurement helps a team decide what kind of improvement is needed before selecting tools.
Decision 5 Define the denominator
Suppose 12 of 80 eligible tickets breached the first-response commitment. The observed breach proportion is 12 divided by 80, or 15 per cent. Describe the period and eligible population beside the result.
Do not divide by all 100 records in the export if 20 are excluded under the definition. Conversely, do not remove a ticket simply because its outcome is inconvenient. Keep a reconciliation showing total exported records, valid exclusions, missing classifications and eligible records.
Distinguish defects from defective units. One ticket might contain several errors but still count as one ticket breaching the response commitment. A measure of individual errors would require a different definition and possibly a different denominator. Mixing the two can produce percentages that have no clear interpretation.
The NIST engineering statistics handbook provides background on measurement and statistical analysis. Use appropriate methods only after confirming what the observations actually represent.
Decision 6 Explain who classifies borderline cases
Some cases will remain ambiguous. An analyst may find a response that answers half the question while omitting the urgent issue. Decide who reviews such cases and what evidence they use.
Retain the original response, classification and short reason. If the definition changes after review, identify whether earlier records need consistent reclassification. Otherwise, a chart may show a measurement change as if it were a process improvement.
A supervisor should not quietly change classifications to meet a target. Separate legitimate correction from performance management pressure. Where the measure informs important decisions, arrange a proportionate independent review of disputed cases.
A completed specification for operational definitions
| Field | Illustrative specification |
| Measure | Tickets breaching first substantive response commitment |
| Unit | One eligible ticket representing one issue |
| Start | Receipt during service hours or next opening time |
| Stop | First response that substantively advances handling |
| Threshold | More than four business hours |
| Exclusions | Test records, spam and verified duplicates |
| Denominator | All eligible tickets received in the reporting period |
| Missing data | Report separately and investigate |
Attach example tickets to this specification. The table alone cannot settle every judgement about what substantively advances handling. The examples should include an automatic acknowledgement, a meaningful clarification request and a generic holding message.
The ASQ description of check sheets offers a practical data-collection perspective. A recording form should carry the definition into daily use rather than rely on staff remembering a meeting discussion.
Test the definition before full collection
Select a small pilot set that includes ordinary and borderline tickets. Ask two analysts to classify them independently using the written specification. Compare both their decisions and the reasons for disagreement.
Suppose they agree on 16 of 20 classifications. That 80 per cent agreement is a useful signal to investigate, not a universal acceptance threshold. With a small and deliberately varied set, the main purpose is to locate unclear rules rather than certify the entire measurement system.
Discuss the four disagreements without telling one analyst simply to follow the other. The problem may be ambiguous guidance, inaccessible records or a genuine difference in expert judgement. Revise the specification where appropriate and test again using new examples.
For an assessment requiring more formal agreement analysis, choose a method suited to the data and prevalence of categories. A percentage alone does not account for agreement that could arise through the distribution of classifications.
Explain the measure to the people doing the work
Give frontline staff an opportunity to challenge the definition using examples from their day. They may identify situations that an analyst cannot see in exported records, such as a customer replying through a different channel. Keep the final rule accessible beside the collection form. Training should include classification practice, with reasons for the answer, rather than only a presentation of the target.
Keep the measure comparable over time
Give the specification a version number, effective date and owner. When service hours or commitments change, record the consequences for trend comparisons. Annotate charts if a definition changes and avoid presenting the series as fully comparable without explanation.
Review unintended incentives. Staff might send a superficial message to stop the clock while leaving the issue unresolved. A companion measure of resolution quality, reopened tickets or customer experience can reveal whether speed has displaced useful service.
Do not turn every concern into a dashboard metric. Select a small set that helps the team understand its work. A precise measure remains only one view of performance and should be interpreted alongside relevant context.
If you need a broader improvement pathway, explore The Case HQ’s quality-management guide and the paid Certified Lean Six Sigma Green Belt programme. Begin your next data-collection exercise by asking two people to apply the definition independently. Their disagreements will tell you where the measurement needs work.

Responses