Small Language Models and Edge Deployment for Low-Connectivity Environments: A Conceptual Framework for AI Under Infrastructure Constraints
DOI:
https://doi.org/10.68854/7cfzk484Keywords:
small language models, edge AI, on-device inference, model compression, quantization, intermittent connectivity, energy, TinyML, low-resource settings, AfricaAbstract
The dominant way of delivering language-model capability, a large model hosted in a remote data centre and reached over a reliable network, embeds assumptions that do not hold for much of the world: continuous connectivity, dependable power, billing in a currency the user earns, and a data pathway that may cross borders. This paper develops a conceptual framework for deploying language models where those assumptions fail. It draws on the rapidly maturing literature on edge and on-device language models, model compression, efficient inference, edge–cloud collaboration, tiny machine learning and inference energy, and on recent reviews of AI deployment in low-resource African settings. It first organises the technical options into a deployment stack, from compression of the model to orchestration between device, local server and cloud. It then characterises the constraint space of low-connectivity environments along five dimensions, connectivity, power, compute, cost and data governance, and argues that the right question is not whether a small model is good enough but whether a deployment pattern fits the constraint profile of its setting. Four patterns are distinguished: cloud-only, edge-first with opportunistic cloud, edge-only, and tiered inference across device, campus or clinic server and cloud. The framework maps constraints to patterns, states six propositions about fit, and derives design principles, offline-first operation, graceful degradation, local-task and local-language evaluation, local corpora, energy budgeting and sovereign hosting, that apply across patterns. The paper closes with illustrative deployment cases drawn from the African literature and a research agenda. It concludes that, in low-connectivity settings, the fit between a model, the task it serves and the infrastructure available determines whether a deployment succeeds, so that the appropriate choice is often a distributed pattern of modest capability rather than the most powerful model reachable only over an unreliable network.
Downloads
References
Argerich, M. F., & Patiño-Martínez, M. (2024). Measuring and improving the energy efficiency of large language models inference. IEEE Access, 12, 80194–80207. https://doi.org/10.1109/access.2024.3409745
Bai, G., Chai, Z., Ling, C., Wang, S., Lu, J., Zhang, N., Shi, T., Yu, Z., Zhu, M., Zhang, Y., Song, X., Yang, C., Cheng, Y., & Zhao, L. (2026). Beyond efficiency: A systematic survey of resource-efficient large language models. ACM Computing Surveys, 58(16), 1–45. https://doi.org/10.1145/3845797
Cai, G., Tian, R., Yang, L., Jia, Y., Li, L., & Wang, J. (2026). Efficient inference for edge large language models: A survey. Tsinghua Science and Technology, 31(3), 1365–1380. https://doi.org/10.26599/tst.2025.9010166
Chen, W., Wang, J., Zhang, Y., Shi, Y., Jiang, T., Zhou, S., Wu, C., Zhang, A., Zhou, C., Wang, M., Liu, X., Hao, X., Wu, Y., Li, Y., Hu, Y., Cao, Z., Lu, Y., Li, M., Ma, Y., … Guo, Y. (2026). On-device large language models: A survey of model compression and system optimization. Artificial Intelligence Review, 59(9), 191. https://doi.org/10.1007/s10462-026-11538-1
Cheng, J., Kang, H., Shao, Y., Li, N., Chen, P., Wang, R., Long, S., Yang, X., & Ma, L. (2026). Survey on efficient large language models: Principles, algorithms, applications, and open issues. IEEE Transactions on Neural Networks and Learning Systems, 37(5), 2025–2045. https://doi.org/10.1109/tnnls.2025.3628671 Dinçer, E., & Kilimci, Z. H. (2026). Real-time and offline large language models on edge devices: A systematic review. PeerJ Computer Science, 12, e3769. https://doi.org/10.7717/peerj-cs.3769
Dritsas, E., & Trigka, M. (2026). Deployment-aware compression of large language models for edge intelligence: A system-level survey. IEEE Transactions on Artificial Intelligence, 7(7), 3703– 3718. https://doi.org/10.1109/tai.2026.3656155
Emmanuel, I. D., & Esenogho, E. (2026). Artificial intelligence for sustainable development in low-resource African contexts: A structured review and adaptive deployment framework. Sustainability, 18(15), 7835. https://doi.org/10.3390/su18157835
Gao, L. (2026). Colonialism in code: Linguistic hegemony and epistemic exclusion in large language models. AI & Society. Advance online publication. https://doi.org/10.1007/s00146-026-03184-6
Heydari, S., & Mahmoud, Q. H. (2025). Tiny machine learning and on-device inference: A survey of applications, challenges, and future directions. Sensors, 25(10), 3191. https://doi.org/10.3390/s25103191
Jiang, S., Zhou, X., Zhang, M., Xu, C., Liao, G., Chen, J., & Cao, J. (2026). Edge large language models: A comprehensive survey. CCF Transactions on Pervasive Computing and Interaction, 8(2), 181– 210. https://doi.org/10.1007/s42486-025-00227-7
Liang, Y., Xu, C., Mei, Y., Zou, H., Guo, J., Fan, X., Wang, T., & Huang, H. (2026). A comprehensive survey on large language model compression for artificial intelligence applications in edge systems. IEEE Internet of Things Journal, 13(10), 20583– 20599. https://doi.org/10.1109/jiot.2026.3675866
Lin, Z., Qu, G., Chen, Q., Chen, X., Chen, Z., & Huang, K. (2025). Pushing large language models to the 6G edge: Vision, challenges, and opportunities. IEEE Communications Magazine, 63(9), 52–59. https://doi.org/10.1109/mcom.001.2400764
Murei, A., & Kamika, I. (2026). Artificial intelligence and edge computing for sustainable smart water-safety monitoring in low-resource communities: A critical review. Limnological Review, 26(3), 51. https://doi.org/10.3390/limnolrev26030051
Oto, P. O. (2026). Autonomous edge artificial intelligence for remote cancer diagnosis in low-resource settings: A structured narrative review and deployment roadmap. Tumor Discovery, 5(3), 025350087. https://doi.org/10.36922/td025350087
Qu, G., Chen, Q., Wei, W., Lin, Z., Chen, X., & Huang, K. (2025). Mobile edge intelligence for large language models: A contemporary survey. IEEE Communications Surveys & Tutorials, 27(6), 3820– 3860. https://doi.org/10.1109/comst.2025.3527641
Revesai, Z., & Chipfumbu, C. T. (2026). Data protection in the era of large language models: Challenges and opportunities. AI and Ethics, 6(4), 356. https://doi.org/10.1007/s43681-026-01223-y
Shonubi, O. A. (2026). The open-source paradox: Africa's digital sovereignty and the structural limits of artificial intelligence autonomy. AI & Innovation, 1(1), e70004. https://doi.org/10.1002/aiv2.70004
Somvanshi, S., Islam, M. M., Chhetri, G., Chakraborty, R., Mimi, M. S., Shuvo, S. A., Islam, K. S., Javed, S., Rafat, S. A., Dutta, A., & Das, S. (2025). From tiny machine learning to tiny deep learning: A survey. ACM Computing Surveys, 58(7), 1–33. https://doi.org/10.1145/3776588
Su, X., Liu, T., Pang, P., Luo, Y. T., & Wong, D. (2026). How can large language models drive environmental sustainability? A systematic scoping review. Sustainability, 18(9), 4327. https://doi.org/10.3390/su18094327
Zheng, Y., Chen, Y., Qian, B., Shi, X., Shu, Y., & Chen, J. (2025). A review on edge large language models: Design, execution, and applications. ACM Computing Surveys, 57(8), 1–35. https://doi.org/10.1145/3719664
Downloads
Published
Issue
Section
License
Copyright (c) 2026 IIAR Journal of Artificial Intelligence and Emerging Technologies
This work is licensed under a Creative Commons Attribution 4.0 International License.
