Artificial Intelligence or AI technology has found its way into smartphones. Modern smartphones are now powered not only with advanced central processors and integrated graphics processors but also with dedicated AI coprocessors or accelerators and engines that are usually designed and built as part of the entire system-on-a-chip package.
Examples include the Neural Engine found in the A series of chips of Apple used in the iPhone and the M series of chips used in high-end iPads, the Ascend found in HiSilicon Kirin chips used in Huawei devices, and the Hexagon Vector Processor and Tensor Processing Unit found in Qualcomm Snapdragon chips used in several Android smartphones and tablets.
Considering the aforesaid applications of Artificial Intelligence in the design and production of smartphones, what exactly is the purpose of using AI technology in consumer electronic devices? How does AI work in these devices? What is the use of AI in smartphones? This article discusses the general purpose, specific functions, and benefits or advantages of AI in smartphones.
Understanding the Purpose of AI in Smartphones
Among the consumer electronic products that are benefitting from artificial intelligence are desktop and laptop computers, mobile devices, virtual personal assistants, automotive electronics, smart home appliances, wearable devices, and distributed sensors, among others. Specific AI applications such as machine learning and deep learning equip these devices with the capabilities to become smarter and maximize their functionalities.
For example, in smart television sets, AI hardware can help in automating specific settings such as volume and picture quality using data obtained from attached sensors, usage patterns of consumers, current trends in the market, and default manufacturer settings. Artificial Intelligence can make the entire consumer user experience more personalized.
The purpose of AI in smartphones center on coprocessing. Specific AI accelerators or engines work alongside a central processor, graphics process, and other processing units such as an image signal processor to improve the overall functionalities of a particular device while adding additional features and capabilities. Note that a coprocessor supplements the functions of a primary processor to accelerate overall system performance.
One of the main benefits of a coprocessor is that it offloads tasks or processors from the main processor such as a CPU to improve efficiency. Specialization is a more specific benefit that involves designating specific processes to a particular hardware component. AI in smartphones fundamentally involves the use of a specialized and dedicated hardware component designed to accelerate Artificial Intelligence and more specific machine learning applications.
Four Specific Functions of AI in Smartphones
To understand better the purpose of AI in smartphones, it is also important to understand the known specific functions of AI accelerators or engines, especially how these coprocessors are used today in these consumer electronic devices.
1. Virtual Assistants for Accurate Voice Recognition and Personalization
Most smartphones nowadays have virtual assistants either built as part of the operating system or as a standalone app. Examples include Siri in iOS and iPadOS of Apple, Bixby of Samsung Electronics, Google Assistant of Google, and Celia of the Android-based EMUI of Huawei. These apps depend primarily on voice inputs but can also utilize hardware inputs to interact with users, perform commands, and answer inquiries.
An AI coprocessor built into a particular smartphone improves the functionalities of these apps. To be specific, the dedicated AI accelerator or engine processes voice inputs through voice recognition or natural language processing. The AI learns to familiarize itself with the voice of a particular user with repeated use to improve voice recognition accuracy and the overall interaction with the virtual assistant application.
Of course, these apps also personalize the user experience of their particular users. AI helps in processing data obtained from usage patterns such as browsing history and usage of specific apps to allow these virtual assistants to provide personalized recommendations.
2. Improving Photo and Video Quality and Expanding Camera Functionalities
The cameras found in smartphones have considerably improved over the years in terms of the quality of the images they take and videos they record, as well as in terms of functionalities. Part of these come from the use of more advanced camera components such as better image sensors and next-generation image signal processors. Part of these also come from the heavy-lifting transpiring in a particular AI accelerator or engine.
A particular AI coprocessor helps in improving the image and video quality by automatically adjusting camera settings such as the amount of light or the level of exposure, the amount of contrast and brightness, and the degree of refinement. AI fundamentally edits the images or videos the minute a user snaps the photo or hit the record button.
Furthermore, artificial intelligence has also enabled newer camera features and functionalities. These include applying bokeh effect or automatic subject separation, removing unwanted elements in the frame, and improving night mode photography.
The cameras in several mid-range and flagship-level smartphones are also capable of providing augmented reality features. These features are used in specialized mobile games such as Pokémon Go, home design apps and virtual online-enabled stores, and social media apps such as the filters and photo enhancements used in TikTok and Instagram.
Some smartphones have features that automatically add metadata to images and videos taken using the cameras, as well as automatically categorizing them in smart folders. There are also devices that automatically assemble, arrange, cut, edit, and render video clips taken to produce a single compilation of videos for uploading on social media.
3. Object and Image Recognition, Voice-to-Text, and Context Awareness
Another benefit of AI in smartphones is that it makes these devices smarter by specifically making them aware of their surroundings. Some devices have specialized digital signal processors and sensors that continuously monitor the surroundings. Data obtained from this monitoring are fed into the chip and processed by digital signal processors and an AI accelerator or engine to enable functionalities and provide a more personalized user experience.
Some specific use cases include health monitoring applications that track and record the steps made by an individual user or the amount of his or her nighttime sleep and provide necessary health-related recommendations. Another use case is in location tracking to provide recommendations of nearby establishments or in enhancing navigation apps.
AI is also responsible for advanced smartphone features such as object and image recognition. These features also take advantage of the existing camera systems in a particular device to recognize real-world objects or images. Examples of use cases include converting print into editable text, recognizing a particular object for online identification, and recognizing faces for added camera functionality or biometrics security.
Remember that AI-equipped devices can process natural languages with better precision. This is helpful in applications that take advantage of converting voice to text or speech to text. Specific uses include hands-free messaging and notetaking, voice commands through voice inputs, language translation, and automatic transcription of voice recordings.
4. Improving the Functionalities and Efficiency of the Chip and Entire System
Optimization is another benefit and function of AI in smartphones. A particular AI coprocessor does not only improve the functionalities of a device but also makes its entire system more efficient. The native machine learning capabilities of the coprocessor learn from data obtained from the day-to-day interaction of a particular user with his or her device to allow specific hardware optimizations tailored to specific use cases.
The optimizations center primarily on optimizing the performance of an entire chip or its specific professors such as the CPU and GPU. The particular AI accelerator or engine is responsible for actively monitoring a particular use case scenario and communicating with the entire chip to determine the best combination of hardware settings.
Heavy tasks such as running mobile games or other resource-intensive apps would tell the CPU to utilize its more powerful cores to maximize the processing capabilities of the device. On the other hand, during moderate to light tasks, the CPU would use the more energy-efficient cores to conserve power and battery life. AI automates and optimizes these processes.
An AI coprocessor also ensures hardware-software optimization. It pairs apps with specific hardware settings or a combination of chip settings to ensure that the entire system is working as efficiently as possible. The result of all these optimizations transpiring under the hood is a smoother and streamlined user experience.
The Evolving Roles of AI in Smartphones
Artificial Intelligence or AI in smartphones takes several forms and has specific applications. These include smart photography, speech recognition, object and image recognition, personalization of user experience, and automation and optimization of system processes, among others. Remember that the primary purpose of an AI accelerator or engine is to act as a coprocessor that helps other processors and components become more efficient.
Fundamentally, Artificial Intelligence has made smartphones smarter through active processing of data, native machine learning, and context awareness. It has expanded and improved the performances and functionalities of smartphones and other consumer electronic devices such as tablets and wearables, thus making AI coprocessors a staple part of systems and AI capabilities a standard feature of modern devices.
FURTHER READINGS AND REFERENCES
- Morra, L., Mohanty, S. P., and Lamberti, F. 2020. “Artificial Intelligence in Consumer Electronics.” IEEE Consumer Electronics Magazine. 9(3): 46-47. DOI: 1109/mce.2019.2962163
- Sarker, I. H., Hoque, M. M., Uddin, M. K. et al. 2021. “Mobile Data Science and Intelligent Apps: Concepts, AI-Based Modeling and Research Directions.” Mobile Networks and Applications. 26: 285-303. DOI: 10.1007/s11036-020-01650-z
