Sentiment Analysis: Rules Engine vs AI-Based Techniques
At my current company, Abound, we have a constant need to explore and develop insights about the user sentiment and decipher (at scale) whether a piece of text reflects positive, negative, or neutral emotions.
From gauging customer satisfaction to doing root-cause-analysis, sentiment analysis is at the heart of providing quality support and developing key focus areas for everything that we do at Abound.
I'm guessing other smaller startups are in a similar boat as they start to receive a large number of customer requests for support
We evaluated 2 main approaches: Rules-Based Engines and AI-Based Techniques
1. Rules-Based Sentiment Analysis
How It Works
A rules-based system uses predefined sets of linguistic rules, dictionaries, and logical conditions to classify sentiments. For example:
Words like "good," "excellent," or "amazing" are flagged as positive.
Words like "bad," "terrible," or "awful" are flagged as negative.
Modifiers like "very" or "not" adjust the sentiment score.
Rules engines rely on human expertise to curate these rules and are typically implemented with conditional logic (e.g., if-else statements).
Interpretability: Every decision is traceable to a specific rule, making it easier to debug and refine.
Quick Setup: Useful for small datasets where building machine learning models isn't feasible.
Low Resource Requirement: No need for extensive computational power or training data.
Domain Specificity: Tailored rules can be extremely effective in narrow or specialized domains.
At Abound, we even ended up providing higher weightage to newer conversations so that if a user was having a poor experience earlier, and now has shifted to a more positive experience lately, that their conversation score is showing an improvement.
Scalability: Expanding rules to cover complex scenarios becomes unwieldy.
Limited Scope: Misses nuances, such as sarcasm, idioms, or contextual sentiment shifts.
Rigidness: Cannot adapt to new language trends or word meanings without manual updates.
2. AI-Based Sentiment Analysis
How It Works
AI-driven sentiment analysis leverages machine learning (ML) or deep learning (DL) models. These models are trained on large datasets, learning patterns in how words and phrases express sentiment. Popular techniques include:
Bag of Words (BoW) or TF-IDF: Basic ML models using word frequency as features.
Recurrent Neural Networks (RNNs) or Transformers: Advanced DL models like BERT, which understand context and semantics.
Adaptability: Learns from data and adapts to different contexts without predefined rules.
Scalability: Handles large, diverse datasets with ease.
Nuanced Understanding: Captures sentiment in complex sentences, including sarcasm or implicit meanings.
Automation: Requires minimal human intervention once deployed.
Data Dependency: Requires vast amounts of labeled data for training.
Computational Cost: Needs powerful hardware for training and inference.
Black Box Nature: Models are harder to interpret and debug compared to rules-based systems.
Bias Risk: Can inadvertently learn biases from the training data.
Rules Engine vs AI Techniques: When to Use What?
Why Rules-Based Systems Excel with Limited Data
Rules engines thrive in environments with:
Limited Data: Small datasets don’t provide enough examples for training AI models.
Specific Domains: Highly specialized vocabulary or contexts where human-defined rules outperform generalized models.
Resource Constraints: Organizations with limited budget or expertise
Transparent Decision-Making: When interpretability and traceability are critical, rules-based systems are superior.
Example Use Case:
A smaller startup(such as ours), that is utilizing 1,000 customer reviews might use a rules engine to classify sentiments, leveraging a simple dictionary and modifiers.
Why AI Techniques Shine with Large Data
Large Data Sets Exist: Abundant labeled data enables training robust, nuanced models.
Varied Contexts: AI handles diverse language patterns and evolves with the data.
Scalability is Required: Processing millions of inputs across multiple languages is a breeze for AI-based solutions.
Subtle Sentiment Insights Are Needed: Advanced models can detect sarcasm, mixed emotions, or sentiment shifts within a text.
Example Use Case:
A multinational corporation analyzing millions of tweets daily to gauge global brand sentiment would benefit from an AI-based approach.
The choice between a rules-based engine and AI-based techniques depends on the problem's scale, data availability, and resource constraints. Rules-based systems are like a scalpel—precise but limited in scope—while AI models are like a Swiss Army knife, capable of handling a wide range of tasks but requiring investment and preparation.
We were already talking to Zendesk and they have an AI based QA tool that can do a great job of providing insights to us, however the price was too high for us and we found that the models were often classifying conversations incorrectly as it lacked the specific domain specific insights.
Each approach has its place in the sentiment analysis toolbox, and the real art lies in knowing which tool to use for the task at hand. In the end, we decided to go with an in-house tool that we developed using Python and applied Simple Rules while parsing the content.
