Route support tickets with Python, then evaluate a classifier

A ticket moves quickly from queue to queue and still waits all afternoon for somebody able to help. A router can be fast at every reassignment while making the complete journey slower. Follow it to an accepted investigation, not just a filled-in queue field.

A model can classify the words in a request, but the categories, examples and receiving teams determine whether that label helps. The answer proposes a destination. It neither diagnoses the fault nor proves that the team accepted responsibility.

This guide starts with a small keyword program so the first decision is easy to inspect. Then it follows a ticket past that decision, including rejection and human triage. Those later steps reveal what a more capable model would actually need to improve.

A keyword router with a human-triage result

The complete Python example below uses keywords, not a trained NLP model. It proposes a queue when exactly one category matches; zero matches or overlapping categories produce human triage. Save it as route_ticket.py and run it with Python 3. This gives a later model a transparent baseline to improve on.

import re

KEYWORDS = {
    "Network Team": {"network", "dns", "vpn"},
    "Security Team": {"phishing", "malware"},
    "Software Team": {"application", "software"},
    "Hardware Team": {"hardware", "battery"},
}

def propose_queue(text):
    words = set(re.findall(r"[a-z]+", text.lower()))
    matches = [team for team, keys in KEYWORDS.items() if words & keys]
    return matches[0] if len(matches) == 1 else "Human triage"

if __name__ == "__main__":
    ticket = "I'm having trouble accessing a specific website."
    print(propose_queue(ticket))

The sample prints Human triage because none of the defined keywords appears in the ticket. The same fallback handles a message whose words match more than one category. The rule is easy to inspect, but its category match is still only a proposal: keywords do not understand negation or establish operational ownership.

Try the phrase “This is not a network problem.” The program still sees network and proposes the network queue. It has matched a word, not understood the sentence. That small failure is useful: before adding a learned model, you can already name one behavior the replacement ought to handle better.

Consider a hypothetical report of an intermittent login failure. A classifier sends it to networking; that team's triager rejects it as an identity issue. Predicting again from the same ambiguous text can recreate the same handoff loop. A triage owner needs to obtain the missing operational detail and resolve who will investigate before another reassignment merely restarts the cycle.

The keyword baseline has an ordinary abstention path

Read diagram description

This illustrates the published example, not a production NLP classifier. Exactly one matching category proposes a queue; zero or multiple matches go to human triage. Keywords do not understand negation or establish ownership. Diagram labels: Ticket text: Tokenize and compare category keywords; Exactly one category matches: Propose that queue; Zero or multiple matches: Send to human triage.

For the login ticket, the useful measure runs until a capable team accepts the investigation. Label accuracy alone misses the repeated handoffs after the first prediction. Including that elapsed time and keeping a manual route for unresolved ambiguity shows whether classification actually shortens the customer’s wait.

Human triage needs available people and can make mistakes too. It helps when attention can resolve missing information or a consequential ambiguity. If the same unclear ownership keeps recurring, repair the categories and handoff rather than pile more tickets on the fallback.

Moving from this program to a learned classifier changes how the destination is chosen. Instead of maintaining the keyword sets, you train on ticket text paired with reviewed destination labels. The model learns statistical patterns in those examples and applies them to new text. That can accommodate wording the keyword list misses, but it also makes the quality of the historical labels part of the program’s behavior. The next step is therefore to examine the labels before choosing a training algorithm.

The first queue may differ from the team that resolved the ticket

The first queue records an initial guess; the resolving team may reveal where useful work happened. Choose which history the model should learn. Otherwise a classifier can reproduce yesterday’s detours with excellent agreement against labels that call those detours correct.

Protect customer data before reuse and remove duplicate cases across training and evaluation sets. Hold out a later period to expose changes in vocabulary and ownership. This checks whether the approach still works when the wording or team structure differs from the examples it learned.

Inspect results by queue and consequential ticket type, not only in aggregate. A rare destination that is almost never selected correctly can disappear inside a favorable overall accuracy score. Scikit-learn’s evaluation documentation covers precision, recall and confusion matrices that help make those errors visible.

For an illustrative set of 100 tickets, imagine 95 belong to a common queue and five to a specialist queue. A router that sends everything to the common queue scores 95% overall accuracy while missing every specialist case. Looking at results for each destination exposes what the headline percentage conceals.

Thresholds and correction after a wrong assignment

For a learned classifier, choose routing thresholds using examples outside training and the cost of a wrong destination. A score is not automatically the probability that its answer is correct. Cases below the chosen threshold need a staffed route for resolution so uncertainty does not leave the request ownerless.

Begin with suggestions a triager can review. Include correction effort, handoffs and time to capable ownership in the comparison with the keyword baseline. Severe incidents must retain their established escalation route; text classification should not silently downgrade urgency while trying to select a queue.

When teams or products change, revisit the evaluation and keep the ticket history and reason for each reassignment. That record lets the receiving team continue the investigation rather than rediscover why the earlier route failed. Customer reliability engineering connects that history with service impact.

Try the intermittent-login ticket again. Does the new approach obtain the missing detail and reach a team that accepts it, or merely choose networking faster? The label is part of the journey. The user still needs somebody to pick up the work.