PROJECT / 005
SupportDesk
NLP banking intent classification baseline.
01 / THE PROBLEM
The engineering problem
Customer support systems require automated triage to route queries effectively. While the industry frequently defaults to large language models (LLMs) or heavy transformers, these introduce high inference latency, high compute overhead, and deployment complexity for classification tasks that can be solved with lightweight baselines.
CONSTRAINTS
77 fine-grained banking intents
Low-latency inference requirement
Model evaluation rigor
ENGINEERING FOCUS
High class-overlap text classification
Sub-millisecond TF-IDF vectorization
Macro & weighted F1 evaluation across all classes
02 / ARCHITECTURE
How the system works
SUPPORTDESK / ML PIPELINE & BENCHMARK EVALUATION — OPTIMAL MODEL
Optimal configuration: Extending TF-IDF to bigrams captures two-word financial terminology ('card stolen', 'pin reset'), lifting Macro F1 to 0.8950 with sub-5ms CPU latency.
03 / IMPLEMENTATION
What was built
- 01Trained on the Banking77 benchmark dataset containing 13,082 customer queries across 77 fine-grained intents.
- 02Engineered text feature extraction pipelines testing unigram vs. unigram + bigram representations.
- 03Trained and tuned L2-regularized Logistic Regression classifiers across multiple C values.
- 04Achieved 89.46% validation accuracy and a Macro F1 score of 0.8950, vastly exceeding the 1.3% random baseline.
- 05Conducted error analysis and confusion matrix evaluations across high-overlap intent pairs.
- 06Packaged the pipeline into a reproducible, documented evaluation script with serialized model artifacts.
04 / DECISIONS
Technical decisions
05 / RELIABILITY
Reliability and failure handling
06 / LIMITATIONS
Known limitations and future work
Pragmatic machine learning, text feature engineering, disciplined empirical evaluation, and sensible model selection.