Bernard C. Kress is Partner Optical Architect in Microsoft´s HoloLens Project—and a mastermind of the mixed reality platform. He is also member of the SPIE´s board of Directors and Chairman of the Digital Optical Technologies Conference at the World of Photonics Congress in Munich (June 23-27, 2019). In our interview, Kress explains his visions and current trends in the field of Mixed Reality—and he spans the bow to the SPIE conference in Munich.
Bernard C. Kress: When I started my career as an optics engineer in the telecom sector 20 years ago, everybody wanted to increase the bandwidth of optical fibers due to the rapid development of the internet. After the dotcom bubble burst, I started to work in the field of microoptics, holography and diffractive optics. At that time no one would have imagined, that companies like Microsoft would be even remotely interested in optical engineers. But today at Microsoft we have one of the largest optical engineering groups in the world. And also Apple has a group evitable with the scale of Zeiss. Microsoft, Apple, Google, Facebook and also Amazon and Intel are actively recruiting at the major optics universities in the US and worldwide. They just can´t get enough optics experts. We are in the midst of the boom of augmented and virtual reality (AR/VR). At Microsoft we are convinced that Mixed Reality is more than smart glasses—it is the new computing platform.
Kress: One part of my answer is absolutely clear: HoloLens is aimed at enterprises, not at consumers. Because there is no consumer market. In the beginning, we didn´t really know, how HoloLens would be used. The first version was a shot in the dark. But now we have tons of user-feedback, which allowed us to specifically improve the hardware. The major return on investment for companies is cost avoidance. The applications range from remote service to training and qualification or assistance with complex repairs that can be done much faster with support of augmented reality. HoloLens is embedded in a whole ecosystem. Companies that use our Mixed-Reality Device, are developing their own contend. They create highly specialized killer apps that produce their return on investment.
Kress: HoloLens is part of the intelligent edge which gives enterprises access to our cloud services. That is our core business. Microsoft will not be a hardware company. But in the beginning of the market nobody else makes that level of quality in hardware. We are forced to heavily invest and make it on our own. We hope that contract manufacturers and other companies will get started and will produce excellent hardware themselves. I think it will be necessary for several big players to join forces—as they did in the CD/DVD market—to bring down the hardware-costs. In AR/VR and Mixed Reality most of the revenues will be accounted for by software and content. The AR/MR market is expected to grow over $ 100 Billion after 2020. Our goal is, that Microsoft´s Mixed Reality Cloud Services can be on every single device on the planet. Market analysis expect that the main part of the AR/MR market will happen in Asia.
Kress: We need further improvement of all building blocks to optimize the optics, the size and weight, thermal management and so on. There are challenges concerning the sources that can be LEDs, Lasers, so called vertical-cavity surface-emitting lasers (VCSELs) or also inorganic LED-arrays and OLEDs. Lots of companies and research institutes in the photonics community are working on it. Also the display-technology is not determined yet: will they be non-emissive like LCOS in Magic Leap One, DLP in DigiLens smart glasses or HTPS LCF in Google Glass, or even laser lit phase panels for holographic display, or will they be emissive, like OLEDs in ODG smart Glasses or the new high brightness iLEDs microdisplay panels; or even scanning devices such as in the new HoloLens 2 and North Smart Glasses with laser MEMS scanner display? Also we need better combiner optics, like waveguides and improved eye-tracking and 3D-Sensors.
Kress: In HoloLens2 we make use of time-of-flight (TOF) sensors and artificial intelligence to perform real time semantic spatial mapping, to provide accurate hologram/reality interaction, a key to the mixed reality experience. Four environmental understanding black&white cameras offer accurate six Degrees of Freedom, HD-Video-Cameras and also gesture sensing devices as well as a spatial sound systems with several microphones. I think the main interest is in waveguides and in the eye tracking technologies. They track the gaze, the pupil locations and the eye vergence, part of the oculomotor 3D cues. In addition they can scan the Iris for secure user authentication. All that is important to change the focus, but also to identify different users and automatically adapt HoloLens to their given optical properties. That’s essential for the immersive effect, for individual comfort but also for typical workflows in enterprises, where more than one person will use the glasses. Mixed reality creates new working environments. So when users return to their current project, HoloLens automatically leads them to the right mixed reality space, where colleagues in a very different part of the world may already have continued the work.
Kress: No. The only way to learn is to observe the market development. There are several technologies that already failed. Like several other companies we believe in the waveguide technology. Others swear upon free space optics. A third group favors freeform prisms. The race is not decided yet. But regarding the recent failures of very promising and well funded competitors, we feel encouraged in our technology road map, in which curved waveguides and also green and blue VCSEL´s could play an important role—and I think they will be available soon. Success will rest on two pillars: the wearable and visual comfort and the immersion.
Kress: We are directly asking for ideas of the photonics community in SPIEs optical design challenge. Companies come up with their current challenges and present it to the academic community. It is all about joining forces. And our Digital Optical Technologies Conference in Munich goes in the same direction. Half of the papers are based on AR/VR or Mixed Reality. But that might change to sensors for the internet of things (IoT) or LiDAR-Technologies for autonomous driving. Digital Optics have a very wide range of applications. Currently it is about the optimization of displays for three-dimensional mixed reality experiences. But we will also have a session about switchable, tunable and digitally reconfigurable optics, which is very exiting! Their capacity to change their functionalities is very important for mixed reality. The optics that we are talking about are digital in two different definitions: on the one hand, they are fabricated with digital masks in lithography. On the other hand their functionality is digital. Most of them are micro-optics manufactured in wafer scale, which is also a digital technology. And often, the drivers for their development come from Digital Industries that is the case in the field of waveguide optics. On top of that, the design of such optics is also a digital process that requires outstanding software. The fabrication and mass replication of such digital waveguides is often done via direct write electron-beam lithography and ion beam etching followed by nano-imprint lithography. You will learn lots of this on our Conference in Munich.