Call operations research
How consistent are call-disposition codes across a small phone team?
A measurement protocol for checking whether a small phone team applies call-disposition codes the same way and where the codes stop matching the notes.

Table of contents
Disposition codes are only useful if the team applies them the same way. In a small office, two agents can describe the same call with different labels. This review offers a way to measure the gap between the code and the note. It also asks whether the code list itself is the problem.
Key takeaways
- Score the disposition against the call note, not the label alone.
- Use a fixed set of boundary examples for every reviewer.
- Separate a wrong code from a code that has no clear definition.
- Report agreement by agent and by code before changing the list.
Research question and scope
This review asks how consistently a small phone team applies its call-disposition codes. The unit is one call with a disposition and a written note. The review measures agreement between the code and the note, and agreement between reviewers. It does not measure whether the caller was satisfied.
The scope covers inbound service and booking calls handled by a receptionist or a small support team. It excludes calls that were never dispositioned and calls handled entirely by an automated system. It also excludes codes that are used only for internal training. The scope should be fixed before the sample is drawn.
The review separates a wrong code from a code that lacks a clear definition. If two experienced agents disagree about the same call, the problem may be the code list rather than the agent. Report the two cases separately so the fix can match the cause. A training session will not fix a vague definition.
Record the number of codes and the number of agents. A long code list in a small team is a common source of inconsistency. A short list with clear examples is easier to apply. The count is context for the result, not a judgment on its own.
Definitions and unit of analysis
Define consistency as the share of calls where the disposition matches the note and the code definition. A code that is technically allowed but does not describe the note is not consistent. The reviewer should use the written definition, not personal habit. The definition should be available during scoring.
Define reviewer agreement as the share of a small overlap sample where two reviewers assign the same code. A high agreement on a wrong definition still produces a consistent but useless result. Report both the note match and the reviewer agreement. The two measures answer different questions.
The unit is the call, not the agent. An agent with a low match rate may be handling a harder call mix. Score by agent as a grouping variable, but do not treat the raw rate as a performance grade. The call type and the complexity should be reported beside the agent.
Decide before collection how to treat a call with more than one issue. Use the primary reason the caller stated, or the code list rule if it provides one. Record the secondary issue in the note. A call with two issues should not be counted twice. A clear rule prevents an inflated sample.
Sampling and data dictionary
Draw the sample from the call records for a fixed period. Include every dispositioned call that meets the scope, or a random sample if volume is high. Record the export date, the queue, and the row count. Keep the export with the report so the sample can be rebuilt.
For each call, capture the agent, the code, the note, the duration, and the call type. Add the customer outcome and any linked follow-up. These fields let the reviewer see whether the code fits the whole call. A short data dictionary keeps the scoring consistent.
Score the call without reading the agent name first. A reviewer who knows the agent may be influenced by a prior impression. Use a randomized order and a fixed form. A second reviewer should score a small overlap sample to check agreement.
Record the code list version that was in use during the sample. A list that changed mid-period will produce a false inconsistency. Split the sample by version if a change occurred. A stable code list is the basis for a fair comparison.
Measuring agreement
Report the share of calls where the disposition matches the note and definition. Also report the share where two reviewers agree. A high note match with low reviewer agreement suggests that the reviewers are using different standards. A low note match with high reviewer agreement suggests that the code list is the problem.
Break the result down by code and by agent. A single code may account for most of the mismatch because its definition is vague. A single agent may need coaching, or may handle a harder mix. The breakdown should lead to a specific fix, not a general instruction to be more careful.
Report the share of calls with more than one plausible code. A high share means the code list overlaps and the agent has to guess. The list should be simplified or the priority rule should be stated. A clear priority rule is often enough to improve consistency.
Compare the result for calls with a short note and calls with a detailed note. A short note may be too thin to support the code, while a detailed note may be easier to match. The note standard is part of the disposition standard. Report both so the team knows what to fix.
Analysis and limitations
The review cannot show whether the caller received the right outcome. A consistent code can still describe a poor call. The disposition measure is a data-quality check, not a service-quality check. Pair it with a short call review if the team wants a fuller picture.
The reviewer may interpret a note differently from the agent who wrote it. A note that is clear to the writer can be ambiguous to a reader. Use the code definition and a short note standard to reduce the gap. Record the ambiguity as a finding rather than a personal error.
The review cannot control for the complexity of the call. A simple booking may be easy to code, while a complaint may fit several categories. Record a simple complexity label and compare within it. A fair comparison is more useful than an overall rate that mixes different calls.
Give the finding to the team with one testable change, such as a shorter code list and a boundary example for the most common ambiguity. Record the change date and repeat the review with the same method. A second sample shows whether the change reached the agents. It also protects the team from judging a new list on its first week.
Repeating the review
Repeat the observation with the same code list version and the same note standard. A change in either can move the result without any change in agent behavior. Report the version beside the number. The reader should be able to see what was tested.
Keep the boundary examples in the appendix so a new reviewer can apply the same standard. A measure that depends on one supervisor is fragile. Write the examples into the operations note. The method should survive a staff change.
Agree on the next review date and the person who owns the call export. Put the date in the quality calendar. A review that is not repeated becomes a one-time story. The next review should ask whether the code list is still in use, not only whether the rate moved.
Use the result to simplify the code list rather than add another training module. A short list with clear examples is easier to apply under pressure. Ask the agents which code they hesitate to use. Remove or rename that code before asking for more consistency.
Sources
- U.S. Bureau of Labor Statistics, customer service representatives
- U.S. Small Business Administration, manage your business
- U.S. General Services Administration
Keep the code list visible during the call rather than only in a training document. An agent who has to search for the right code will choose the closest one under pressure. A short list with plain labels is easier to use than a detailed taxonomy. The list should be reviewed with the agents who use it every day. A code that no one can explain in one sentence is a candidate for removal. A short list that the team trusts is more valuable than a detailed list that invites guessing.