New Apple Acquisition Reinforces On-Device Generative AI Focus
LLMs at the edge and maybe some image generation
Apple quietly acquired a French startup focused on image analysis in December 2023, according to France-based business publisher Challenges. The acquisition price was not disclosed, but Apple informed the European Commission about the transaction.
The company’s original focus was data compression for images. Fast Company reported that Datakalab worked with the French government during the COVID-19 pandemic to identify whether citizens were wearing face masks on public transportation. However, in 2023, the company posted about developing compression for large language models (LLM) and quantizing them to a scale suitable for on-device deployment.
This acquisition appears unrelated to the Darwin AI takeover earlier this year. Darwin’s machine vision technology is expected to be employed in Apple manufacturing facilities to improve efficiency and quality. This may be critical in helping Apple diversify beyond its traditional Chinese manufacturing base.
Datakalab is all about on-device generative AI software capabilities. In particular, Apple would like to run native LLMs on iPhones that don’t require continuous call-backs to cloud infrastructure.
Quantization
Quantization is a technique used to reduce the memory footprint of an LLM. The “large” in large language models can be incompatible with portable devices. Many LLMs can only run on large servers or server clusters. This has led to recent investments in developing more capable small-large language models (SLM) or baby LLMs. However, there is generally a trade-off in quality after undergoing compression. In addition, many of the SLMs are still too large for a typical handheld device that must also run other applications.
This is what would make a company like Datakalab attractive to Apple. In a May 2023 research paper, Datakalab researchers commented:
Deep neural networks (DNNs) are ubiquitous in computer vision and natural language processing, but suffer from high inference cost. This problem can be addressed by quantization, which consists in converting floating point operations into a lower bit-width format. With the growing concerns on privacy rights, we focus our efforts on data-free methods. However, such techniques suffer from their lack of adaptability to the target devices, as a hardware typically only supports specific bit widths. Thus, to adapt to a variety of devices, a quantization method shall be flexible enough to find good accuracy v.s. speed trade-offs for every bit width and target device. To achieve this, we propose REx, a quantization method that leverages residual error expansion, along with group sparsity.
The company also posted AI-generated images created by a quantized version of Stable Diffusion in the middle of 2023, “4x smaller” and also faster than the unmodified model. Apple would be attracted to the company’s combination of expertise in image and language model compression. Datakalab also boasted on its LinkedIn profile that it had three patents pending and had published nine research papers.
Generative AI on the Edge
Apple is all about the iPhone. It is also about controlling its own fate. While Apple was negotiating with Google and OpenAI about embedding their LLMs into iPhone services in 2024, the company would like to minimize these dependencies. The negotiations are smart because they suggest Apple could introduct meaningful generative AI features for the iPhone before year-end. There is little chance of that occurring if based solely on in-house development in 2024. Of course, the negations also indicate that Apple is not close to delivering against its own limited vision generative AI experiences for smartphone users.
For several reasons, Apple would like to run LLMs and AI image generation models on devices.
It would like to control the experience locally. That will enable lower latency and tighter integration into iOS.
Apple wants to claim unmatched privacy protection for users. An LLM that processes fully or mostly on-device has fewer points of potential failure when it comes to privacy data leakage.
Apple would like to keep its costs low. iPhones are already expensive. They will become more expensive if Apple is sending large payments to LLM providers.
Apple would like to create more personalized user experiences without providing too much insight into iPhone user behavior to other tech companies.
Siri could use a significant upgrade. A quantized LLM with zero latency or privacy data leakage could be an ideal path to fulfilling Siri’s promise, which was never quite fulfilled and seemed to regress recently.
Datakalab’s research team presents an opportunity to achieve these goals by developing quantization technology in-house. However, don’t expect to see anything soon on this front. Apple has historically taken at least two years before technology from acquired companies arrives in core products. You may very well be waiting until June 2026 to see some fruit from the acquisition.
One additional point is worth noting. Datakalab previously had experience with image recognition. Darwin AI is also a vision system. These technologies could be applied to both the iPhone and Vision Pro for image recognition. That might be a bonus.
What I like about Apple is that they don't rush. Not publicly at least. I think they're doing themselves a favor by only deploying this in the iPhone on their own conditions. On-device LLMs makes the most sense, even if it is still a year away.