In my previous post, “Untangling AI Jargon: From Predictive Models to Generative AI,” I introduced the concept of “AI-washing”. It wasn’t me who coined this term; it was first coined by the AINow institute, describing marketing practices that overstate the role of Artificial Intelligence in the product or service they are promoting.
There is no doubt that AI is a transformative power that will change every aspect of our lives, and telco software is no different. However, when an electric toothbrush is being marketed as AI-powered, this is where I get to worry. There’s so much hype around AI, that it’s crucial to separate genuine innovation from exaggerated claims. Luckily, I’m here to help.
In this post, I will provide you with the tools to evaluate vendors’ AI claims. By the end, you’ll know how to differentiate impactful AI solutions from insufficient or misleading #fakeAI, and ensure your investments drive real business value.
What is #FakeAI?
Not all AI are created equal. #FakeAI doesn’t necessarily mean a solution is fraudulent. Instead, it’s a label for AI claims that overstate capabilities, mislead decision-makers, or lack meaningful functionality. It raises concerns regarding transparency, consumer trust in the AI industry, and compliance with security regulations, potentially hampering legitimate advancements in AI.
At its core, artificial intelligence refers to systems capable of simulating human-like intelligence to perform tasks such as learning, reasoning, problem-solving, and adapting. Real AI evolves and improves over time by learning from data and experiences, often using machine learning (ML), deep learning (DL), or generative AI.
#FakeAI, by contrast, may perform valuable functions but fail to fulfill the benefits of true AI. Let’s break down some common AI-washing practices and examples that help illustrate this phenomenon.
Rebranded automation
One of the most common forms of AI-washing involves rebranding automation as artificial intelligence. Automation tools often rely on rule-based systems, where decisions are made based on predefined logic, heuristics, or templates. While these systems can streamline workflows, they lack the hallmark features of AI, such as learning, reasoning, or adaptability.
Rule-based systems were among the earliest forms of AI (often referred to as symbolic AI), but they operate within rigid boundaries defined by human-written rules. Vendors often rebrand these as “AI-powered” solutions, even though they cannot adapt or evolve.
Consider, for example, chatbots. Many of the prevailing solutions are marketed as AI-powered, despite being founded on simple rule-based decision trees. While they can automate simple interactions, they fail to handle nuanced conversations or unexpected inputs, highlighting their static nature.
Predictive models
Predictive Models analyze historical data to forecast future outcomes, often using statistical techniques like regression analysis, clustering and time series modeling. While these models are highly valuable for specific, narrowly defined use cases, they lack the ability to learn, adapt, or improve over time—a hallmark of modern AI systems such as Totogi’s PlanAI. AI builds on the foundations of statistical predictive modeling but goes further by introducing dynamic learning capabilities, enabling it to refine predictions, integrate new data streams, and handle more complex, real-time scenarios.
Separating chaff from wheat : Evaluating AI Claims
Understand the AI spectrum
AI in telecom spans a range of advanced capabilities, from machine learning to generative AI. Each step along this spectrum represents increasing complexity, adaptability, and transformative potential. Understanding these is key to evaluating AI solutions and ensuring they align with your operational needs. For a more detailed explanation of these distinctions, revisit our earlier post, “Untangling AI Jargon: From Predictive Models to Generative AI.”
- Machine Learning (ML): Machine learning is the starting point of modern AI, enabling systems to learn from data and improve over time. Unlike static models, ML algorithms adapt dynamically, making them ideal for handling complex, evolving use cases like customer segmentation or churn reduction.
- Deep Learning (DL): Deep learning builds on machine learning by mimicking human neural networks, allowing systems to analyze vast amounts of unstructured data. DL excels at interpreting nuanced inputs like call transcripts, social media sentiment, and image recognition, making it indispensable for telcos seeking advanced customer insights and operational efficiency.
- Generative AI: Generative AI is the frontier of artificial intelligence, capable of creating entirely new content or executing complex tasks autonomously. GenAI applications in telecom range from co-pilots, which assist decision-making, to fully autonomous AI agents that excel in particular tasks, such as coding, QA, customer service and many others. Generative AI’s adaptability makes it transformative, but its success depends on high-quality training data, effective safeguards against inaccuracies, and seamless integration with existing systems.
By understanding this spectrum—from ML’s ability to adapt and improve, to DL’s capacity for handling unstructured data, to GenAI’s creative and autonomous potential—telco leaders can evaluate AI solutions that truly align with their strategic goals.
Evaluate explainability and transparency
Understanding how an AI system arrives at its conclusions is critical for building trust and ensuring the solution delivers reliable results. A common failing among AI solutions is their “black box” nature, where predictions or outputs are opaque, leaving users uncertain about their validity or risks. Without clear insights into the decision-making process, even high-performing AI can generate skepticism and undermine confidence. Telco leaders must demand transparency from vendors, ensuring their solutions are explainable, auditable, and built on relevant, high-quality data. By probing these areas, evaluators can distinguish robust AI from insufficient or overhyped systems.
Explainability techniques
Explainability refers to an AI’s ability to clarify how it makes decisions, highlighting which factors contribute to specific outcomes. For example, if a churn prediction model identifies a 30% risk for a customer, explainability tools like SHAP (SHapley Additive exPlanations) or LIME (Local Interpretable Model-agnostic Explanations) can show whether the prediction is driven by usage patterns, payment history, or customer support interactions. These tools help users verify that the AI is functioning logically and ethically, without needing to spend days to analyze and understand (or even worse- guess) AI’s decisions.
Questions to ask:
- What explainability tools does your AI use?
- How can users see which factors contributed most to a decision?
- Can your system explain decisions in non-technical terms for business teams?
Auditability
Auditability ensures the AI’s outputs align with real-world expectations by enabling evaluators to test the model’s performance under various conditions. While vendors might not disclose their proprietary algorithms or source code, they should allow telcos to validate predictions using test datasets or sandbox environments. This process helps verify that the AI behaves as claimed and meets telecom-specific requirements.
Questions to ask:
- Can we audit the AI’s performance using our own test datasets?
- How does your model behave in edge cases or scenarios specific to telecom (e.g., identifying fraud or predicting network congestion)?
- What tools or processes do you provide to monitor the model post-deployment?
Training data transparency
The quality and diversity of the training data directly impact an AI’s performance. Vendors should disclose the types of data used to train their models, particularly in telecom where domain-specific knowledge is essential. Models trained on generic datasets may fail to capture the complexities of telecom operations, leading to suboptimal results.
Questions to ask:
- What datasets were used to train the model? Were they domain-specific?
- How does the AI handle telecom-specific variables, such as customer churn patterns or network traffic trends?
- How frequently is the model retrained to stay relevant with new data?
By demanding clarity in these areas, telco leaders can ensure that the AI solutions they adopt are not only effective but also trustworthy and aligned with their unique operational needs. Transparency in explainability, auditability, and training data separates robust AI from the #FakeAI that fails to deliver real value.
Accuracy and relevance
Accuracy means achieving consistent performance across diverse scenarios while minimizing errors and biases. For telecom, accuracy must also align with operational priorities, such as improving customer satisfaction or optimizing network configurations. Relevance ensures the AI’s predictions or outputs are meaningful and actionable within the specific domain.
- Metrics: AI systems should be evaluated using precise metrics such as precision, recall, and F1 scores, which measure the balance between false positives and false negatives. For example, a churn prediction system with high precision but low recall might flag only obvious cases, missing subtler churn risks.
- Robustness: evaluate consistency across varied datasets, including edge cases. For instance, an AI trained only on high-volume urban network data may falter when applied to rural or mixed-use cases. Testing the system under realistic telecom scenarios is critical to identifying potential weaknesses.
- Dynamic relevance: Evaluate how well the AI integrates real-time data to adapt predictions or outputs. For example, a customer segmentation tool should adjust dynamically to behavioral changes like increased usage during promotions or seasonal trends.
Safeguards
AI systems must incorporate rigorous safeguards to minimize the risks of erroneous outputs, particularly in high-stakes scenarios like network optimization or customer communications. These safeguards ensure that errors are detected early, flagged appropriately, and mitigated through additional layers of validation or human intervention.
- Error detection and escalation mechanisms: Advanced AI tools should include automated processes to flag anomalies or low-confidence results. For example, a generative AI writing customer responses should alert human reviewers when uncertain or when generating outputs outside predefined confidence thresholds.
- Hallucination mitigation: Generative AI systems are prone to hallucinations—plausible but incorrect outputs. Mitigation strategies include rigorous pre-training validation, reinforcement learning with human feedback (RLHF), and real-time monitoring of high-risk outputs. Vendors should demonstrate how their systems identify and suppress hallucinations in sensitive applications.
- Fail-Safe mechanisms: For mission-critical operations, AI systems should include rollback or override options to prevent harmful outputs from being implemented automatically. For instance, a network optimization AI might propose changes but require human approval for deployment if confidence levels are below a specific threshold.
Accuracy rooted in metrics and robustness, and safeguards designed for real-world reliability ensure AI solutions are not only fit for purpose but also resilient and trustworthy in dynamic environments.
Delivering on the promise
AI offers immense potential for telecom, but separating genuine innovation from exaggerated claims is vital to ensure investments deliver real-world value. By focusing on the nuances of explainability, accuracy, relevance, and safeguards, telco leaders can navigate the complexities of evaluating AI solutions effectively. Machine learning’s adaptability, deep learning’s ability to process unstructured data, and generative AI’s creative capabilities all require tailored evaluation criteria to uncover their true impact. This approach empowers decision-makers to leverage AI as a transformative force in customer experience, network optimization, and operational efficiency, positioning their organizations for sustained success.
Why Totogi?
At Totogi, we adopt an AI-first ethos that combines transparency, adaptability, and measurable impact. From PlanAI’s hyper-personalized offers to BSS Magic’s agentic AI services, our tools empower telcos to thrive in a competitive landscape.
Curious about how Totogi’s AI-native solutions can transform your operations? Let’s talk!
