ST Sees AR Glasses Replacing Smartphones
August 5, 2021 Anne-Françoise Pelé
Could a pair of glasses be the next big computing platform?
Marco Angelici, MEMS Micro Actuators Business Unit director at STMicroelectronics, thinks the 2020s will be the decade when augmented-reality glasses replace smartphones. The key technology enabler for the transition will be laser-beam scanning (LBS), according to Angelici and ST.
At the end of 2020, 5.27 billion people — 67% of the global population — subscribed to mobile services, according to the GSM Association. There will be nearly half a billion new subscribers by 2025, taking the total number of subscribers to 5.7 billion, or 70% of the world’s people. Providing all those customers with anytime, anywhere access to information is essential, but the smartphone user experience has its limitations.
“We see a decline in phone innovation,” said Angelici. “We are at the dawn of another revolution in the mobile user interface,” characterized by the shift from mobile phones to smart glasses. “Instead of watching your phone while you are walking, moving your head up and down, you would be able to watch the world around you, interact with people, get your information directly at eye level.”
Perceiving an inflection point, Angelici said, “Augmented reality can be the next computing platform, and in the long term, it can replace mobile phones.”
The first AR smart glasses were a bold attempt to bring people more seamlessly into the information age. The Google Glass concept was brilliant, but the execution was lacking. Priced at $1,500, the strange-looking glasses didn’t deliver the experience or the value that users expected.
Lessons were learned from those missteps, Angelici said. All-day wearable smart glasses must be comfortable and fashionable, support the user’s prescription lenses on the glasses, deliver critical application-specific information, and be reasonably priced.
ST is convinced that LBS will be a key enabling technology for small-form–factor AR smart glasses that can check all of those boxes. Toward that end, the company has built an ecosystem of technology developers, suppliers, and manufacturers to develop a communication device tailored to today’s customers’ needs.
Building an ecosystem
A deciding factor for any alliance is the complementary expertise of its members. In October 2020, ST, Applied Materials, Dispelix, Mega1, and Osram established the LaSAR (Laser Scanning for Augmented Reality) Alliance to create an ecosystem that will enable and accelerate the design and manufacture of AR wearable devices such as smart glasses and head-mounted displays.
“Today, we are working together, learning from each other, finding tradeoffs in order to find the best specification for the full system,” said Angelici. “The ultimate scope is standardization.”
The alliance brings together a MEMS micromirror platform from ST, compact illumination sources from Osram, and waveguide elements from Dispelix and Applied Materials. Mega1 is responsible for integrating those devices into a small optical light engine.
After IEEE Industry Standards and Technology Organization (ISTO) announced the formation of the LaSAR Alliance in March, it received membership requests from 10 to 15 additional companies, said Angelici.
The alliance’s first milestone is to develop a lightweight, compact, all-day wearable and fashionable pair of glasses for mass production in 2022. Expectations are high, as partners target weight lower than 60 g, power consumption under 500 mW, and brightness beyond 1,000 nits. There is, however, a debate over the field of view, acknowledged Angelici. “We believe that a 30° field of view is more than enough for all-day wearable glasses. We don’t need to cover the sides, as we won’t be playing games with these wearable glasses. We are talking about receiving messages, infographics, into our eyes.”
With LBS, he claimed, it is possible to achieve up to a 100° field of view, but there is always a tradeoff: “The more I open, the more power it consumes.”
ST and Quanta Computer are developing the optical, electronic, and photonics design to enable volume manufacturing of AR smart glasses in line with the LaSAR Alliance. “They [Quanta] have put a lot of effort in R&D, and they will have a full pair of glasses based on our laser-beam scanning technology, embedding the application processor, the connectivity, the sensors, the glasses, and the lenses,” said Angelici.
Optimizing MEMS micromirrors
ST has extensive experience in MEMS design, development, and manufacturing, with competencies in electrostatic, electromagnetic, and piezoelectric technologies. The group started investing in MEMS micromirrors in 2009 and boosted its activities by acquiring bTendo, an Israeli startup specializing in LBS solutions, in 2012. Since then, it has sealed partnerships with Intel, MicroVision, LeddarTech, and North (now Google).
MEMS micromirrors are used in LBS systems to project visible images or infrared patterns. They work by deflecting laser beams emitted from laser diodes to project images onto the required field of view. The beam deflection is generally performed using a combination of two mirrors rotating on perpendicular axes. (Source: STMicroelectronics)(Source: STMicroelectronics)
Detractors note that LBS is not a mature technology, but Angelici is convinced of its “high value” and outlined its brightness and power-efficiency advantages. “We have a flying spot, meaning we are not illuminating the full frame but [instead are] illuminating pixel by pixel where the content is needed,” he said. “If I am projecting the time on my screen, I have 95% of the screen black and 5% white, and I am illuminating only this path. We don’t need to buffer the full frame; we can just buffer the line where the pixel is going to land.”
By working with application developers, ST can visualize where the content will be projected and where the white spots should be. It can then design components that are faster in resuming operation from power down and “switch off the lasers to consume zero” when the pixel is not painted, Angelici said.
Over the past few months, ST has confirmed its commitment to LBS through strategic initiatives and partnerships.
Besides the LaSAR Alliance to develop AR eyewear applications, ST and Intel have worked on a MEMS micromirror for integration in Intel’s RealSense LiDAR depth camera L515. RealSense is a 10-meter distance depth camera with a very large field of view, and ST said its MEMS micromirror enables continuous laser scanning across the entire field of view.
In the meantime, to advance the adoption of piezoelectric MEMS, ST has established the Lab-in-Fab R&D line in collaboration with Singapore’s A*Star Institute of Microelectronics and Japanese vacuum-device manufacturer ULVAC. “For AR, we have defined that piezoelectric is the best compromise in terms of dimension, power consumption, and performance,” said Angelici. “Now we have a fast-prototyping line in Singapore for piezo actuation, so there will be a seamless transfer from prototype to production.”
ST is also collaborating with OQmented, a 2019 spinoff from Germany’s Fraunhofer Institute for Silicon Technology, on the development of MEMS mirror-based LBS solutions. OQmented claims its Lissajous scan pattern and vacuum encapsulation Bubble MEMS technology improve resolution, energy consumption, and chip size while ensuring long-term reliability for the hermetically sealed micromirrors. Angelici commented, “It’s really cutting by a factor of 10 the power consumption of a MEMS mirror if you put it under vacuum.”
OQmented joined the LaSAR Alliance in May.
Initiating and planning
ST has set up what it calls a one-stop shop for LBS. Dubbed MEMS ScanAR, it comprises MEMS mirrors, MEMS mirror drivers, laser diode drivers, mirror control loops, and relay optics.
To accelerate the integration cycle, ST developed Star0, a first reference design for AR smart glasses based on the MEMS ScanAR components, Dispelix’s lens, and Osram’s three-color RGB module. It then developed a second generation, Star1, that reflects insights gained from the LaSAR Alliance, Angelici said. Based on thin-film piezoelectric technology, Star1 features a 65° field of view, 1,280 × 720-pixel resolution, and 50% less power consumption at a similar size to Star0 (0.7 cc). ST is currently sampling Star1 to key partners and expects to have it “assembled in engineering samples for customers” in the third quarter of this year, said Angelici.
Star0 has power consumption of 1,220 mW for a full white display and of 742 mW for a 10% white spot. In contrast, Star1 has a power consumption of 781 mW for a full white display and of 312 mW in a 10% white spot, said Angelici.
ST has defined a piezoelectric process roadmap and expects to improve the actuation efficiency by 50% within the next couple of years. For a design of the same performance, it anticipates a 50% power consumption reduction and 30% die size reduction; for a design of the same size and power consumption, it anticipates 1,080p resolution or a 90° FoV.
“Without changing anything — just by working on the material and process, which is enabled by this big investment in Singapore — we can provide scalability to this technology,” said Angelici.
Asked when the 50% actuation efficiency target will be achieved, Angelici said that “2023–2024 is when we expect the augmented reality market for all wearables to start flying, and it will be available by then.”
Of course, his group is not just developing components. “Our vocation is billions of units,” said Angelici, specifying that the group can mass-produce its MEMS mirrors at its 8-inch fabs in Milan and Singapore. “We are also using two new fabs for drivers, so we are ready to go with volumes when augmented reality takes off.”