Apple Unveils AI-Driven Desktop Robot, Smart Speaker for 2024

Apple is revamping its strategy in the artificial intelligence sector with a series of ambitious new products, including a desktop robot, a more human-like version of Siri, a smart speaker with a display, and a home security camera. The desktop robot, slated for release in 2027, is a central component of Apple's AI strategy. Meanwhile, the smart speaker with a display is planned for launch next year, marking Apple's entry into the entry-level smart home device market.

Another significant growth opportunity lies in home security systems. The new camera will be a core component of Apple's security system, enabling the automation of home functions. This approach is designed to enhance the appeal of Apple's product ecosystem, making its products more attractive to consumers.

These initiatives are part of Apple's broader effort to revitalize the company. Its latest major project, the Vision Pro headset, has yet to achieve sales success, and its most popular products have seen little design change in years. Additionally, Apple has faced criticism for not keeping pace with the generative AI revolution. OpenAI, for instance, could potentially threaten Apple's core business areas by collaborating with former Apple design chief Jony Ive to develop AI-based new devices.

While Apple is still in the early stages of turning around its AI software, executives believe that the development of hardware products is key to its revival. This will help Apple compete in new areas with rivals like Samsung Electronics and Meta. Apple's spokesperson declined to comment, and given that these products have not been officially announced, the company's plans remain subject to change or cancellation. Many of these initiatives and their timelines depend on Apple's continued progress in AI-driven software.

Apple's CEO recently addressed employees, emphasizing the importance of success in the AI field and hinting at upcoming new products. Beyond home devices, Apple is preparing to launch a thinner, redesigned iPhone this year. In the longer term, the company plans to introduce smart glasses, a foldable phone, a 20th-anniversary iPhone, and an improved version of earphones called the N100. Additionally, Apple is planning a large foldable device that combines features of the MacBook and iPad.

Apple aims to boost sales of its flagship products, which have seen a slowdown in growth for several years. The company has also canceled expansion plans in certain new areas, such as autonomous vehicles, further increasing the pressure to find new revenue streams. These new initiatives will also help counter claims that the company is no longer as innovative as it once was.

Last year, reports indicated that Apple was advancing a desktop robot project codenamed "J595" and developing a new smart home strategy. More details about its entry into this market have since emerged, which is significant for its AI goals. The desktop robot resembles an iPad mounted on a movable arm, which can rotate and reposition to follow users around a room. Like a human head, it can turn towards the person speaking or calling it and even attempt to attract the attention of those not facing it.

Apple envisions users placing the robot on a desk or kitchen counter to assist with tasks, watch media, and manage daily schedules. FaceTime calls will be a key feature, with the screen automatically adjusting to focus on people in the room. Apple is testing a feature that turns the iPhone screen into a joystick, allowing users to rotate and move the robot to showcase different people or objects during video calls.

The robot's standout feature is its new Siri voice assistant, designed to integrate into conversations among multiple people. It can interact with users throughout the day and retrieve information more easily. The idea is for the device to act as a person in the room, such as interrupting a conversation about dinner plans to suggest nearby restaurants or relevant recipes. It is also designed for two-way discussions, similar to OpenAI's voice mode.

Apple plans to place Siri at the core of the device's operating system and give it a visual personality to make it more lifelike. This method, called "Bubbles," is somewhat similar to "Clippy," the animated paperclip that served as a virtual assistant in Microsoft Office in the 1990s. Apple has experimented with making Siri look like an animated version of the "Finder" icon, which represents the smiley face of the Apple computer file management system. However, the final design is still undecided, with designers considering options closer to the "Memoji" style, which are the fun characters representing Apple user accounts.

The prototype features a horizontal display of about 7 inches, similar in size to the iPad mini. An electric extendable arm can push the display out from the base by about half a foot in any direction. Some familiar with the product refer to it as the "Pixar Lamp," named after the famous logo of the animation film company. Apple had previously revealed some of its research in this area: in January, the company published a paper detailing a lamp that uses robotics to move.

Apple has multiple teams across its AI, hardware, software, and interface design departments working on this project, led by Kevin Lynch, who previously drove the development of smartwatch software and automotive-related work. The tech giant is also developing other robots, including a wheeled mobile robot similar to Amazon's "Astro" and a humanoid robot model. Additionally, Apple has a team actively developing a large mechanical arm for use in manufacturing or retail back-end tasks, codenamed T1333, which is expected to take several years to materialize.

The new operating system for the desktop robot, codenamed "Charismatic," is designed for multi-user environments. Its interface primarily features a clock and small widgets, which are typically used for specific tasks. "Charismatic," previously known as "Pebble" and "Rock" during development, combines design elements from the Apple TV and Apple Watch operating systems. It includes features like multi-user mode and dashboard themes, such as one based on the comic "Snoopy."

These devices are designed for sharing, with a front-facing camera that scans the user's face as they approach, automatically adjusting the layout, functions, and content based on the user's preferences. Some versions of the software use circular app icons with a hexagonal app grid layout. Apple plans to include many of its core apps, such as Calendar, Camera, Music, Reminders, and Notes. However, the interface will heavily rely on voice interaction and widgets rather than frequent switching between different apps.

While the device will have a touch screen, the primary input method will be Siri and an upcoming feature called "App Intents," which allows users to control the interface and applications precisely through voice commands. The hardware itself resembles Google's Nest Hub but has a square shape with thin black or white borders and rounded corners. The non-robotic 7-inch display sits on a hemispherical base that houses some electronic components and has holes at the bottom edge for speakers and microphones. It can also be wall-mounted.

This launch will mark Apple's official entry into the smart home market, a decade after Amazon and Google began selling screen-equipped smart speakers. For Apple, the home environment is a crucial target market, especially as more users watch content and automate home devices through their living rooms. Apple has a strong presence in the mobile device market and has quickly expanded into the automotive sector through CarPlay. However, this success has not translated into the smart home market. Despite launching HomeKit in 2014 to control third-party devices, Apple's HomePod speaker has not achieved significant success.

At the core of the new home devices, as well as existing products like the iPhone and iPad, is a comprehensive overhaul of Siri's underlying system. Engineers are developing a version codenamed "Linwood," which features a new "brain" based on large language models—the foundation of generative AI. This version aims to use personal data to answer questions, a feature that has been delayed due to issues with the current version. The new software, internally codenamed "LLM Siri," is expected to launch as early as next spring. However, development is ongoing: Apple is preparing a redesigned smart assistant for the iPhone and iPad, which will debut early next year.

During a recent internal employee meeting, the senior vice president of software engineering revealed that the company is undertaking an unprecedented comprehensive overhaul. He stated, "The end-to-end improvements we've made to Siri have given us the results we need, allowing us to achieve not only our initial plans but also a much larger upgrade than we originally envisioned." He also noted that no other project has received as much attention as this one. "Linwood" is built on technology developed by Apple's Foundation Models team, but the company also has a competing project called "Glenwood," which aims to support Siri using external technology. The final decision on which models to use has not been made, but Apple is testing with Anthropic PBC's "Claude" model. Earlier this year, the former head of the Vision Pro project, who is now overseeing the Siri project, was involved in supervising both initiatives.

During the development of the desktop robot, Apple's engineers extensively used ChatGPT and Google Gemini to build and test features. Across Apple's AI and Siri teams, software developers are increasingly integrating third-party systems into their development processes. Apple is also developing a home security camera codenamed J450, designed for home security with the ability to recognize individuals and perform tasks automatically. The device will be battery-powered, with a lifespan of several months to a year, comparable to similar products. It features face recognition and infrared sensors to identify people in the room. Apple envisions users installing cameras throughout their homes to assist with automation, such as turning off lights when someone leaves a room or playing a family member's favorite music. Apple plans to develop various types of cameras and home security products as part of a new hardware and software series, competing with Amazon, Google, and Roku. The company is also testing a doorbell that uses face recognition technology to unlock doors. Apple has already launched iCloud+ services with online storage for monitoring footage, but these services are designed for third-party cameras.