Paper deep dive
OTel: Building Domain-Specialized Telecom LLM Foundations for Intelligent Networks
Farbod Tavakkoli, Roderic Paulk, Jorden Terrazas, Kenneth Church, Mark Austin, Louis Powell, Gregory Diamos, Lina Bariah, Syed Ali Raza Zaidi, Maryam Hafeez, Ali Maatouk, Imtiaz Karim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/22/2026, 2:17:17 AM
Summary
The paper introduces Open Telco (OTel), an open-source AI resource designed to address the lack of domain-specific infrastructure for telecommunications AI. OTel provides curated datasets (OTel-Embedding, OTel-Reranker, OTel-LLM, OTel-Safety) derived from 3GPP, GSMA, O-RAN, and other telecom standards, along with 30 full-parameter post-trained baselines for embedding, reranking, and language models. The resource aims to democratize telecom AI research by providing reproducible training data and evaluation protocols, achieving significant performance improvements over base models in retrieval and generation tasks.
Entities (14)
Relation Signals (13)
Open Telco → achievesmetric → MRR@10
confidence 95% · reranking reaches 0.952 MRR@10
Open Telco → achievesmetric → NDCG@10
confidence 95% · embedding retrieval reaches 93.5% NDCG@10
Open Telco → provides → OTel-Safety
confidence 95% · OTel releases four derived datasets: OTel-Embedding, -Reranker, -LLM, and -Safety
Open Telco → provides → OTel-Reranker
confidence 95% · OTel releases four derived datasets: OTel-Embedding, -Reranker, -LLM, and -Safety
Open Telco → provides → OTel-LLM
confidence 95% · OTel releases four derived datasets: OTel-Embedding, -Reranker, -LLM, and -Safety
Open Telco → provides → OTel-Embedding
confidence 95% · OTel releases four derived datasets: OTel-Embedding, -Reranker, -LLM, and -Safety
Open Telco → collaborateswith → Yale University
confidence 90% · Yale University providing roughly 680K question-answer-source triples
Open Telco → collaborateswith → AT&T
confidence 90% · AT&T Chief Data Office
Open Telco → usesdatafrom →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Frontier AI models have advanced rapidly, but they still struggle with telecom-specific tasks. We present Open Telco (OTel), an open telecom AI resource with derived datasets for retrieval, reranking, instruction tuning, and safety/abstention, plus 30 full-parameter post-trained baselines across embedding, reranking, and language models. The community has already engaged substantially with the resource: as of May 3, 2026, the released models have been downloaded over 16 million times, and the project has received 157+ pieces of media coverage worldwide. Building on prior open telecom datasets and benchmarks, OTel provides documented telecom data sources, held-out evaluation partitions, trained embedding models, rerankers, context-grounded LLMs, and safety/abstention data in one unified resource. OTel post-training improves performance across all three model families: embedding retrieval reaches 93.5% NDCG@10, reranking reaches 0.952 MRR@10, and language-model correctness reaches 88.2%. We release OTel as a reproducible starting point and invite the community to expand the data, improve embedding and reranking models, and build stronger context-grounded telecom LLMs.
Tags
Links
- Source: https://arxiv.org/abs/2608.15436v1
- Canonical: https://arxiv.org/abs/2608.15436v1
Trouble viewing inline? Open PDF directly →
Full Text
21,598 characters extracted from source content.
Expand or collapse full text
OTel: Building Domain-Specialized Telecom LLM Foundations for Intelligent Networks Farbod Tavakkoli ‡ , Roderic Paulk ‡ , Jorden Terrazas ‡ , Kenneth Church ‡ , Mark Austin ‡ , Louis Powell Φ , Gregory Diamos § , Lina Bariah Υ , Syed Ali Raza Zaidi Ψ , Maryam Hafeez Ψ , Ali Maatouk Ω , Imtiaz Karim ¶ ‡ AT&T Chief Data Office, Φ GSMA, § RelationalAI, Υ Khalifa University, Ψ University of Leeds, Ω Yale University, ¶ The University of Texas at Dallas Abstract Frontier AI models have advanced rapidly, but they still struggle with telecom-specific tasks. We present Open Telco (OTel), an open telecom AI resource with derived datasets for retrieval, reranking, instruction tuning, and safety/abstention, plus 30 full-parameter post-trained baselines across embed- ding, reranking, and language models. The community has already engaged substantially with the resource: as of May 3, 2026, the released models have been downloaded over 16 million times, and the project has received 157+ pieces of media coverage worldwide. Building on prior open tele- com datasets and benchmarks,OTelprovides documented telecom data sources, held-out evaluation partitions, trained embedding models, rerankers, context-grounded LLMs, and safety/abstention data in one unified resource.OTelpost- training improves performance across all three model fami- lies: embedding retrieval reaches 93.5% NDCG@10, rerank- ing reaches 0.952 MRR@10, and language-model correctness reaches 88.2%. We releaseOTelas a reproducible starting point and invite the community to expand the data, improve embedding and reranking models, and build stronger context- grounded telecom LLMs. 1 Introduction Telecommunications infrastructure underpins nearly every aspect of modern digital life, yet it remains one of the most technically demanding domains for AI systems to navigate. The standards that govern it, 3GPP specifications running to hundreds of thousands of pages, O-RAN alliance documents, GSMA permanent reference documents, and IETF RFCs are dense, highly interdependent, and continuously revised. A single specification can span dozens of releases; a single ques- tion about network behavior may require synthesizing con- tent across multiple working groups and document versions. For AI systems to be genuinely useful in this environment, general-purpose language modeling is not sufficient. What is required is domain-specific training data at scale, retrieval pipelines tuned to the structure and vocabulary of telecom corpora, and evaluation benchmarks that reflect the kinds of questions practitioners actually ask. ∗ Corresponding authors: farbod.tavakkoli@att.com, lpowell@gsma.com, imtiaz.karim@utdallas.edu The academic and industry research community has rec- ognized this need and responded with a growing ecosystem of telecom-specific benchmarks. This research has focused on curating public 5G datasets for LLMs to utilize [6], evalu- ating broad telecom knowledge and standards understanding across thousands of questions drawn from 3GPP and related sources [7] and domain-specific mathematical reasoning [1], focused specifically on O-RAN [2], targeting 5G root cause analysis in operational log data [8]. The GSMA Open Telco AI Leaderboard [3] brings these fragmented efforts together into a shared evaluation interface, making it possible to com- pare models across multiple telecom tasks in a single view. These benchmarks represent a significant collective achieve- ment: they have established, with increasing precision, what telecom AI systems should be able to do. What remains missing, however, is equally important: a unified open resource for building the systems that can meet those benchmarks. Evaluation without training infras- tructure creates an asymmetry. Practitioners who want to develop telecom-capable retrieval, reranking, or generation models must assemble their own training data from hetero- geneous sources, design their own cleaning pipelines, im- plement their own evaluation setups, and release their own baselines, all independently, without a shared foundation. This makes progress slow, comparisons unreliable, and repro- ducibility difficult. Domain AI ecosystems in medicine, law, and finance have shown that this asymmetry can be resolved by releasing not just benchmarks but also aligned training resources and reproducible baselines that the community can build on directly. We address this gap with Open Telco (OTel) ∗ , an open telecom AI resource that covers the full Retrieval-Augmented Generation (RAG) pipeline from data collection through model release.OTelis the product of a large-scale collab- orative effort involving more than 100 domain experts from industry and academia. It releases four derived datasets: OTel-Embedding,-Reranker,-LLM, and-Safetybuilt from roughly 1.1 million raw training points spanning 3GPP speci- fications, GSMA documents, O-RAN alliance materials, RFCs, industry whitepapers, and telecom academic papers, and ∗ https://huggingface.co/farbodtavakkoli Accepted at the ACM AI Leadership Summit, Breakthrough Impact Highlights Track, 2026. arXiv:2608.15436v1 [cs.AI] 15 Aug 2026 OTel Team cleaned to 326,767 high-confidence examples through a rig- orous multi-stage quality pipeline. Alongside the data,OTel releases 30 full-parameter post-trained baselines covering embedding models, rerankers, and language models of vary- ing sizes, each trained on the matchingOTeldataset under a shared reproducible recipe and evaluated on held-outOTel partitions. The community response has been immediate and sub- stantial. As of May 2026, the releasedOTelmodels have been downloaded over 16 million times and the project has received 157+ pieces of media coverage worldwide across telecom trade press, general technology outlets, and regional business media. This level of engagement, within months of release, reflects the scale of unmet demand for open telecom AI infrastructure. Baseline results further vali- date the approach: OTel fine-tuning improves performance consistently across all three model families, with embedding retrieval reaching 93.5% NDCG@10, reranking reaching 0.952 MRR@10, and language-model correctness reaching 88.2%. We releaseOTelnot as a finished product but as a reproducible starting point, a foundation the community can expand with new data, stronger models, multilingual coverage, and richer evaluation. 2 Technical Contribution OTel’s technical contribution spans three interconnected lay- ers: a curated dataset family derived from heterogeneous tele- com sources, a suite of full-parameter post-trained baselines covering the RAG pipeline, and a reproducible evaluation protocol that ties the two together. Each layer is designed to be independently useful and collectively coherent. Dataset collection and curation. TheOTelsource corpus encompasses six categories of publicly available telecom documents: GSMA permanent reference documents, 3GPP specifications, O-RAN alliance specifications across multi- ple working groups, the IETF RFC series, topical telecom material including eSIM and roaming documentation, and industry whitepapers and telecom academic papers. Over 100 domain experts contributed approximately 1.1 million raw training points, with Yale University providing roughly 680K question-answer-source triples derived from arXiv tele- com papers, 3GPP standards, telecom Wikipedia articles, and Common Crawl telecom pages; GSMA contributing 158K examples from PRDs and the Discover portal; NetoAI pro- viding 100K RFC-derived examples; and Khalifa University, the University of Leeds, and UT Dallas together contributing over 160K O-RAN and whitepaper examples. The incoming data presented two distinct structural chal- lenges. The Yale contribution consisted of question-answer- source triples without explicit retrieval passages, where source documents can span hundreds of thousands of tokens. Con- verting these into retrieval-ready supervision required a dedi- cated six-stage enrichment pipeline: (i) grouping QA pairs by source document and joining them to the full document text; (i) creating candidate passages via sliding window and se- mantic chunking; (i) mining hard negatives both within and across source documents, followed by reranker rescoring to improve negative quality; (iv) applying fact-grounded claim verification by decomposing reference answers into atomic claims and checking each against the candidate context; (v) selecting the minimal sufficient context through greedy min- imization; lastly (vi) formatting the verified examples into contrastive training structures suitable for Multiple Nega- tives Ranking Loss and related retrieval objectives. The need for aggressive cleaning was confirmed empiri- cally. We trained separate full-parameter embedding models on independent data shards and evaluated each on a common held-out set. Performance varied dramatically, from 71.1% to 3.9% Acc@1 across shards, revealing substantial noise in the raw corpus. The final cleaning pipeline applied four sequential filters heuristic filtering to remove malformed and low-information examples, reranker-based semantic fil- tering to remove query-passage pairs with low semantic alignment, embedding-based semantic filtering to remove near-duplicates and low-coherence examples, and global deduplication. For the Yale subset, this reduced 680K exam- ples to 220,334. For the remaining contributors, 420K ex- amples were reduced to 106,433. The final retained corpus contains 326,767 high-confidence training points. These are released in four task-specific formats ordered by the RAG workflow.OTel-Embeddingprovides anchor, positive, and up to five hard-negative passages per exam- ple for contrastive retrieval training.OTel-Rerankerpro- vides query-passage pairs with binary relevance labels for cross-encoder training.OTel-LLMprovides context grounded instruction-tuning prompts with reference completions and abstention flags indicating whether the retrieved context is sufficient to answer.OTel-SafetyextendsOTel-LLMwith examples specifically designed for abstention tuning, where retrieved context is off-topic or insufficient and the correct model behavior is to decline rather than generate. All four datasets are released under Apache-2.0 as derived QA pairs; raw source documents are not redistributed, and release doc- umentation describes provenance, intended use, and source- specific constraints at the asset level. Post-trained baselines.OTelreleases 30 full-parameter post-trained baselines: 10 embedding models ranging from 22M to 8B parameters, 3 rerankers ranging from 0.6B to 8B parameters, and 17 language models ranging from 270M to 32B parameters. Each baseline starts from a publicly avail- able open-weight checkpoint including models from the Gemma 3, Qwen 3, OLMo 3, Mistral 3, LFM2, and RNJ-1 families and is full-parameter post-trained on the matching OTeldataset. Training was distributed across AMD MI300X, MI325X, and MI355X GPUs alongside NVIDIA A100 and H100 nodes, with large runs reaching 94.2% GPU utilization across 256 AMD MI325X GPUs. OTel OTelfine-tuning improves performance consistently across all three model families and all parameter scales. For em- bedding models, NDCG@10 improves by +9.6 to +60.2 per- centage points over the respective base models (shown in Figure 1). For rerankers, MRR@10 improves by +0.535 to +0.598 absolute across all three sizes, with all models exceed- ing 0.944 MRR@10 after fine-tuning, compared to 0.35–0.42 for the base models. For language models, LLM-as-judge cor- rectness improves by +3.7 to +10.0 percentage points (shown in Figure 2). The largest model,OTel-LLM-27B-IT, reaches 88.2% correctness;OTel -LLM-8.3B-ITis the strongest mid-size baseline at 79.6%, outperforming other models in its weight class by a substantial margin;OTel-LLM-1.2B-IT provides a competitive low-latency option at 74.4%. While larger models generally raise the upper envelope, the results also show that architecture and training family matter inde- pendently of scale:OTel-LLM-8.3B-IToutperforms several models with two to four times as many parameters. Evaluation protocol. Held-out evaluation partitions follow a 90/10 train/eval split for LLMs and embeddings and a 95/5 split for rerankers, all with seed 42. LLMs are evaluated by GPT-4o mini LLM-as-judge correctness on context-grounded answer generation: models receive retrieved telecom context and are judged on whether their answer is correct relative to that context and a reference answer. This scoping is in- tentional theOTelLLMs are designed for RAG, not unre- stricted open-ended QA, and results should be interpreted accordingly. Embeddings are evaluated by NDCG@10, which captures ranking quality across the top retrieved passages. Rerankers are evaluated by MRR@10, which captures how quickly the first truly relevant passage is promoted. All re- sults include standard errors computed via bootstrap resam- pling (푛=10). Collaborator-led retrieval stress tests from UT Dallas and the University of Leeds provide additional diagnostic evi- dence beyond the aggregate held-out splits. These tests eval- uated O-RAN retrieval on fixed-chunk question pools and examined retrieval behavior across 3GPP, GSMA PRD, O- RAN, RFC, whitepaper, and academic paper subdomains. The results are not reported as formal benchmark tables because protocols were not yet standardized, but they sur- face a consistent pattern: O-RAN retrieval is comparatively strong across models, while academic paper and GSMA PRD examples remain weaker areas, suggesting that targeted cu- ration and harder negatives in these subdomains should be a priority for future OTel releases. 3 Broader Implications The significance ofOTelextends well beyond its benchmark numbers. Telecom is among the most consequential and least AI-accessible technical domains: its standards are vast, its terminology is specialized, its documents are interdependent, and the cost of errors in deployment, such as misconfigured Figure 1. Fine-tuning improvement of embedding retrieval Figure 2. Fine-tuning improvement of language model networks, incorrect protocol interpretation, and unsafe au- tomation, can be severe.OTelrepresents a systematic effort to make this domain accessible to the broader AI research community, and its design choices reflect lessons from anal- ogous efforts in other high-stakes fields. Democratizing telecom AI research. By releasing aligned training data, post-trained baselines, and a reproducible eval- uation protocol in a single unified resource,OTelremoves the infrastructure barrier that has historically limited telecom AI research to organizations with the resources to assemble all three independently. A research group that previously needed to negotiate data access, design a cleaning pipeline, train models from scratch, and build an evaluation harness OTel Team can now start fromOTel’s baselines and focus their effort on the specific improvement they want to make. This mirrors the role that shared resources have played in accelerating progress in medical AI following MedQA [5], legal AI follow- ing LegalBench [4], and financial AI following FinBen [9]. In each of those domains, the availability of shared training data and reproducible baselines compressed years of scattered effort into a coherent research agenda.OTelis designed to play the same role for telecom. Industry uptake and real-world validation. The scale of community engagement 16 million model downloads and 157+ media mentions within months of release provides early evidence thatOTelis addressing a genuine and widespread need. This is not merely academic interest: the coverage spans telecom trade press, general technology outlets, and regional business media, reflecting demand from practition- ers across the industry who need domain-specific AI tools and have not previously had access to open, reproducible starting points. The multi-organizational authorship ofOTel spanning a major US carrier, a global industry consortium, academic institutions on four continents, and hardware and software infrastructure partners also reflects a level of cross- sector coordination that is rare in AI research and that posi- tionsOTelas a genuinely community-owned resource rather than a single organization’s release. Responsible deployment and safety. Responsible deploy- ment is central toOTel’s design rather than an afterthought. The context-grounded RAG framing is itself a safety mecha- nism: by training models to answer from retrieved context rather than from parametric memory alone,OTelreduces the risk of fluent but incorrect outputs on technical tele- com questions. TheOTel-Safetydataset and its associated abstention-tuned model variants extend this further, training models to recognize when retrieved context is insufficient or off-topic and to decline rather than confabulate. Model cards, dataset cards, Croissant 1.1 metadata with full Responsible AI field coverage, and asset-level provenance documentation are included for all 30 baselines and four datasets. Limitations and future directions.OTelis an open start- ing point, not a finished product, and its current limitations define a concrete research agenda. The resource is English- only and text-centric, which excludes significant portions of the global telecom community and omits the diagrams, tables, and structured data that are central to many telecom docu- ments. The main-paper evaluation relies on held-out splits from theOTelcorpus itself rather than fully independent external benchmarks, which limits the ability to assess cross- domain generalization. Collaborator stress tests reveal mean- ingful subdomain variation that aggregate scores obscure. Finally, the current release does not include per-subdomain reporting as a first-class evaluation output, making it difficult to track improvements in specific areas over time. FutureOTelreleases should address these gaps systemat- ically: expanding to multilingual and multimodal data, im- proving academic-paper and GSMA PRD subsets with bet- ter negatives and larger evaluation pools, establishing per- subdomain metrics as a standard part of the evaluation pro- tocol, and validating on external benchmarks beyondOTel’s own held-out partitions. The resource is architected to sup- port this kind of incremental community-driven growth new data can be contributed through the same enrichment and cleaning pipeline, new models can follow the same training recipe and naming convention, and new evaluation subsets can be added to the held-out partitions without breaking existing comparisons. We invite the community to build on OTel, improve what it provides, and push telecom AI toward the reliability and coverage that the domain demands. Acknowledgments Apart from the coreOTelteam, we also would like to ack- nwoledge the support of Kostikey Mustakas from AT&T Chief Data Office; Antti-Ville Suni, Andrey Ivannikov, Andy Allred, Mark van Heeswijk, Alexander Finn, Kumaran Siva from AMD; Enrique Molero from GSMA; Molham Aref, Niko- laos Vasiloglou from RelationalAI; David Kanter from ML- Commons; Rick Lievano, Ven Kumar, Inayat Wali, Thomas Steagall from Microsoft; Mashroor Hasan Bhuiyan from Uni- versity of Texas at Dallas; Mirza Masfiqur Rahman and Faik Kerem Ors from Purdue University Merouane Debbah, Es- raa Fahmy, Bohao Wang from Khalifa University; Zeinab Nezami, Shehr Bano from University of Leeds; Leandros Tas- siulas and Rex Ying from Yale University, Matt Upson and Nick Sorros from Mantis NLP; Vignesh Ethiraj, Ashwath David from NetoAI. References [1]Vincenzo Colle, Mohamed Sana, Nicola Piovesan, Antonio De Domenico, Fadhel Ayed, and Merouane Debbah. 2026. Telemath: A benchmark for large language models in telecom mathematical problem solving. IEEE Network (2026). [2]Pranshav Gajjar and Vijay K Shah. 2025. Oran-bench-13k: An open source benchmark for assessing llms in open radio access networks. In 2025 IEEE 22nd Consumer Communications & Networking Conference (CCNC). IEEE, 1–4. [3] GSMA. 2025. Open Telco AI Leaderboard. https://huggingface.co/ spaces/GSMA/open-telco-leaderboard. [4]N. Guha et al.2024. LegalBench: A collaboratively built benchmark for measuring legal reasoning in large language models. In Advances in Neural Information Processing Systems, Vol. 36. [5]D. Jin et al.2021. What disease does this patient have? A large-scale open domain question answering dataset from medical exams. Applied Sciences 11, 14 (2021), 6421. [6]Imtiaz Karim, Kazi Samin Mubasshir, Mirza Masfiqur Rahman, and Elisa Bertino. 2023. SPEC5G: A Dataset for 5G Cellular Network Protocol Analysis. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings), Jong C. Park, Yuki Arase, Baotian Hu, Wei Lu, Derry Wijaya, Ayu Purwarianti, and Adila Alfa Krisnadhi (Eds.). Association for Computational Linguistics, Nusa Dua, Bali, 20– 38. doi:10.18653/v1/2023.findings-ijcnlp.3 OTel [7]Ali Maatouk, Fadhel Ayed, Nicola Piovesan, Antonio De Domenico, Merouane Debbah, and Zhi-Quan Luo. 2025. Teleqna: A benchmark dataset to assess large language models telecommunications knowledge. IEEE Network (2025). [8]Mohamed Sana, Nicola Piovesan, Antonio De Domenico, Yibin Kang, Haozhe Zhang, Merouane Debbah, and Fadhel Ayed. 2025. Reasoning language models for root cause analysis in 5G wireless networks. arXiv preprint arXiv:2507.21974 (2025). [9] Q. Xie et al.2024. FinBen: A holistic financial benchmark for large language models. In Advances in Neural Information Processing Systems, Vol. 37.