Lumai promises energy-efficient AI inference through optics

Lumai announces that its upcoming Iris Aura server will perform pre-filling with about one-tenth of the energy of a GPU. Its Iris Nova server has been available for evaluation since April 2026. This promise pertains to a specific phase of inference, as the costs of data conversion and operations assigned to electronics limit potential gains.
Lumai targets initial query processing
Pre-filling involves more calculations than decoding: it handles the received request and then circulates it through the model. Essentially, it repeats multiplications between an unchanged matrix and vectors. Decoding, on the other hand, gradually constructs the response, word by word or token by token. It also performs multiplications, but the time required mainly depends on retrieving matrices from memory and transferring them.
This distinction is essential for comparing the energy expenditures of a GPU to those of an optical system. Based in Oxford, England, Lumai is among the companies focused on pre-filling. It describes Iris Nova as a feasibility demonstrator, rather than a server that has already fulfilled the energy promise announced for its successor.
Phillip Burr, product manager at Lumai, argues that light propagation in air, rather than in integrated chip waveguides, will allow surpassing GPUs. The device employs a programmable spatial light modulator: this membrane made up of pixels, akin to an advanced liquid crystal display, acts on both the intensity and the light phase. It replaces the fixed attenuator of Joseph W. Goodman's historical device.
A vector is represented by light, then reproduced N times using a lens before passing through the modulator. Another lens collects the result, which is converted back into digital electronic data; nonlinear functions remain executed by electronics. According to Burr, the entire multiplication between vector and matrix occurs within an optical cycle.
Gains also depend on conversions
Companies in the sector aim for energy efficiency that is 10 to 80 times greater than that of current GPUs. These targets are not guaranteed results at scale, and achieving them before further advancements in electronics remains uncertain. Peter McMahon, associate professor of applied physics and engineering at Cornell in Ithaca, New York, has been working on optical computing for over a decade. He notably warns against comparisons that overlook the conversion of data into light signals.
In 2022, his team demonstrated an optical neural network using less than one photon per multiplication, with one photon corresponding to approximately 2.5 x 10⁻¹⁹ joules. A multiplication on a classical computer can require between 10⁻¹⁴ and 10⁻¹² joules. This comparison suggests a potential saving factor of 100,000, but McMahon reminds us that optical encoding was not included in the experiment's accounting.
His example estimates electronic multiplication at 1 picojoule, compared to 1,000 picojoules for encoding an element of the vector in light form. With these assumptions, electronics maintain the advantage as long as the vector contains fewer than 1,000 elements. McMahon considers this example to be fairly close to real-world conditions. Based on calculations conducted on various neural networks, he roughly places the favorable threshold for optics between N equal to 1,000 and N equal to 10,000.
Another limitation relates to Amdahl's law: accelerating a portion of the work does not eliminate the cost of the rest. McMahon takes the example of 90% of a program being able to be accelerated, while the remaining 10% limits the speed gain to 10x. The energy objective of inference differs from a speed objective, but the principle remains applicable. Depending on the chosen technique, the portion suitable for optics varies without reaching 100%, particularly because nonlinear operations are added to matrix multiplications.
Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisissez votre rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
Light performs multiplications without switching
Multiplications between vectors and matrices, or between matrices, represent the most computationally demanding component of current AI. Inference executes thousands of them every second. As early as 1978, pioneer Joseph W. Goodman and his colleagues proposed performing these with optical components, whose quality has since significantly improved.
In their device, a row of N diodes represents the elements of a vector of length N: a brighter diode corresponds to a higher value. A cylindrical lens vertically spreads the beams to reproduce this vector N times. The light then passes through an attenuator containing N by N pixels, each of which reduces intensity according to the corresponding value in the matrix. This passage performs the multiplications; a cylindrical lens positioned perpendicularly then sums the contributions from each row before measuring them with detectors.
For an N by N matrix and a vector of length N, the calculation requires N² multiplications. The example where N equals 3 thus includes 9 multiplications and three accumulations, each grouping additions. In electronics, each multiplication requires energy: the expenditure thus follows N².
With optical attenuation, multiplication is passive as long as the matrix remains unchanged. The passage of light through the attenuator then requires almost no energy. In this theoretical reasoning, the expenditure pertains to encoding the vector and reading the result, following N rather than N². However, this growth advantage does not exempt one from considering the concrete cost of these operations.
Companies are also exploring networks
Optalysys, based in England, is introducing elements of optical computing into the communication circuits of data centers, which already use light signals. Its elementary optical component on chip performs a Fourier transform. In Austin, Texas, Neurophos is developing another variant of Goodman’s device, relying on metamaterials to enhance the power of a compact device. These complex structures can manipulate light at a scale smaller than its wavelength, even though those designed by Neurophos are larger than a wavelength.
These projects accompany the growth of a field that has long been primarily confined to research, despite an idea that is over 70 years old. The market for the companies involved is valued at $2.22 billion, compared to $0.4 billion in 2022; projections foresee exponential growth in the coming years. Startups are emerging in the United States, China, Germany, and Israel, with ambitions to compete with GPU manufacturers like Nvidia. Theoretically, optics could perform their calculations with significantly less power.
The recent surge is driven by AI, particularly by inference, which is the use of models to respond to user requests. This activity represents the majority of AI's energy consumption and has led to the construction of vast energy-hungry data centers. McMahon attributes much of the interest in photonics to this energy constraint, for which optics is considered a possible response. The industry is already investing billions of dollars in advanced GPUs and other chips to make inference more energy-efficient.
Funding has also enabled the development of other types of AI algorithms, scientific computing solutions, and technologies for network processing. Logan Wright, who serves as an assistant professor of applied physics at Yale (New Haven, Connecticut) and was previously a postdoctoral researcher in McMahon's lab, hopes to see strong engagement from many companies in this direction. According to him, the advancements made could benefit photonic systems whose applications extend beyond artificial intelligence.
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.