volvo on road_展示世界上第一个由Varjo和Volvo制作的逼真的混合现实演示

本文主要是介绍volvo on road_展示世界上第一个由Varjo和Volvo制作的逼真的混合现实演示，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

volvo on road

Get a behind-the-scenes look at an ambitious automotive project made with Unity from Varjo, the maker of industrial VR/XR headsets known for their superior visual quality in VR.

在幕后观察一下由Varjo联合开发的一个雄心勃勃的汽车项目，该公司是工业VR / XR头戴设备制造商，以其出色的VR视觉质量而闻名。

The team at Varjo is behind some of the most innovative projects in the world of mixed reality. They previously shared a photogrammetry-based environment they created in VR with us, and today they will share how they, together with Unity and Volvo, broke new ground on a demo that brings the real and virtual worlds together like never before.

Varjo 的团队支持混合现实世界中一些最具创新性的项目。他们以前与我们共享了在VR中创建的基于摄影测量的环境，今天，他们将与Unity和Volvo一起分享如何将演示将真实世界和虚拟世界融合到一起的演示的新突破。

Get a first-hand look at this project in-person at Unite Copenhagen. Volvo and Varjo will be on-site showcasing this experience, as well as speaking in multiple sessions:

在 Unite Copenhagen 亲身体验此项目。沃尔沃和瓦尔霍将在现场展示这种体验，并分多次演讲：

How Volvo embraced real-time 3D and shook up the auto industry
沃尔沃如何拥抱实时3D并撼动了汽车行业
Creating next-gen VR and MR experiences using Varjo VR-1 and XR-1
使用Varjo VR-1和XR-1创建下一代VR和MR体验
Future mobility, smart cars, and autonomous driving: Preparing for a new era at Volvo
未来的出行，智能汽车和自动驾驶：为沃尔沃的新时代做准备

Get tickets now

立即获取门票

使混合现实具有真实感

(Making mixed reality photorealistic)

The content below is courtesy of Varjo.

以下内容由Varjo提供。

Mixed reality means blending virtual content with the real world. So far mixed reality has been accomplished with optical see-through, where the user sees digital objects augmented on top of reality through a pair of glasses. This is fine for portraying infographics or playing games, but for realistic scenes, it offers little value. Optical see-through devices can’t display black or opaque content on top of the real world. Everything appears hazy and holographic.

混合现实意味着将虚拟内容与现实世界融合在一起。到目前为止，已经通过光学透视实现了混合现实，用户可以通过一副眼镜看到在现实之上增强的数字对象。这对于描绘图表或玩游戏是很好的，但对于现实场景而言，它提供的价值很小。光学透明设备无法在现实世界的顶部显示黑色或不透明的内容。一切都显得朦胧而全息。

We at Varjo wanted to get rid of this limitation and be able to render photorealistic, opaque content – where it is impossible to distinguish between what is real and what is virtual. Our mission was to make photorealistic mixed reality possible with video pass-through. Video pass-through means using cameras to digitize the world in real-time, and then showing the combined result of real mixed with virtual to the user.

Varjo的我们希望摆脱这种限制，并能够渲染逼真的，不透明的内容-在这里无法区分真实和虚拟。我们的任务是通过视频传递使逼真的混合现实成为可能。视频直通是指使用摄像头将世界实时数字化，然后向用户显示真实与虚拟混合的组合结果。

Before we could achieve this, we first needed a VR headset capable of displaying the real world in human-eye resolution. That is why we released our first human-eye resolution product VR-1, targeted for professional users, to the market in February 2019.

在实现这一目标之前，我们首先需要一款能够以人眼分辨率显示现实世界的VR头戴式耳机。因此，我们于 2019年2月向市场发布了第一款针对专业用户的人眼分辨率产品 VR-1 。

And at Augmented World Expo 2019 in Santa Clara, we showed a glimpse of the magic that can be accomplished with video pass-through. We publicly demonstrated our new headset XR-1 Developer Edition for the first time with a joint demo with Volvo, made with Unity. With XR-1, you can blend virtual content seamlessly with reality with extremely low latency and integrated eye-tracking, in superior resolution.

在圣克拉拉举行的2019年增强世界博览会上，我们展示了可以通过视频传递实现的魔力一瞥。我们通过与Unity共同制作的与Volvo的联合演示，首次公开展示了我们的新耳机 XR-1 Developer Edition 。借助XR-1，您可以以极低的延迟和集成的眼动跟踪将虚拟内容与现实无缝地融合在一起，并具有出色的分辨率。

Here’s how the world’s first photorealistic mixed reality demo was created.

这是世界上第一个真实感混合现实演示的创建方式。

This video is unmodified material shot through the Varjo XR-1 Developer Edition. With XR-1, you can see photorealistic virtual content blending with reality in a full field of view. You can also switch seamlessly from XR to full VR.

该视频是通过Varjo XR-1开发人员版拍摄的未经修改的资料。 使用XR-1，您可以在真实的视野中看到逼真的虚拟内容与现实的融合。 您还可以从XR无缝切换到完整VR。

使用沃尔沃和Unity开发XR-1演示 (Developing the XR-1 demo with Volvo and Unity)

Varjo began working on a video-pass through mixed reality headset in early 2018. The collaboration between Varjo and Volvo also started in spring 2018, as Volvo outlined the need for an XR headset that would allow them to test various elements of future cars – such as heads up displays, new materials, and UI for infotainment systems – inside a real car while driving on a real test track. The high requirements on readability and low-latency needed to drive a car on a test-track pushed Varjo to succeed in product development.

Varjo从2018年初开始研究通过混合现实耳机进行视频传递。Varjo与沃尔沃之间的合作也于2018年Spring开始，因为沃尔沃概述了需要XR耳机以使他们能够测试未来汽车的各种元素–在真实的汽车中，同时在真实的测试轨道上行驶时，作为抬头显示器，新材料和信息娱乐系统的UI。在测试轨道上驾驶汽车对可读性和低延迟的高要求促使Varjo在产品开发方面取得成功。

Given how well Unity already worked for the VR-1, it was a natural choice to try out how the virtual objects would appear in mixed reality. The fact that Unity is easy to integrate and extend with C++ libraries, such as our own Varjo plug-in, made it possible for us to extend our plug-in to support mixed reality. By simply defining the empty background in a VR scene to be replaced by the video-pass-through signal, we were quickly able to see virtual objects in a real environment.

鉴于Unity已经为VR-1做好了工作，因此自然地尝试虚拟对象如何在混合现实中出现是很自然的选择。 Unity易于与C ++库(例如我们自己的Varjo插件)集成和扩展，这一事实使我们能够扩展插件以支持混合现实。通过简单地定义VR场景中的空白背景以替换为视频直通信号，我们可以快速看到真实环境中的虚拟对象。

The close collaboration and fast iterations were made possible by Unity’s ease of use, as our team was developing and improving pass-through simultaneously while working hand in hand with our customer. A year later, the first public demonstration of XR-1 brought to life the capabilities of our technology, combined with Volvo’s superior models and photorealistic Unity graphics.

Unity的易用性使紧密协作和快速迭代成为可能，因为我们的团队正在与客户携手并进的同时开发和改进传递功能。一年后，XR-1的首次公开演示将我们的技术功能与沃尔沃的高级模型和逼真的Unity图形结合起来，栩栩如生。

The demo illustrates the power of video pass-through mixed reality as opposed to optical see-through. In this demo, you have the following steps:

该演示说明了视频直通混合现实与光学直通相反的功能。在此演示中，您需要执行以下步骤：

1. Experience real reality

1.体验现实

You see the real world around you through the XR-1 headset. The real world is streamed with <10ms latency via the high-res cameras in the front plate. You see the world in a full field of view and at a high resolution with a 90Hz framerate, which gives a sensation of not wearing any headset at all (i.e., seeing the real world with your own eyes). You can walk around and explore the real world freely.

您可以通过XR-1耳机看到周围的真实世界。通过前板上的高分辨率摄像头，真实世界的延迟小于10ms。您可以在90Hz帧频下以全视角和高分辨率看到世界，这给人的感觉是根本没有戴任何耳机(即，用自己的眼睛看到现实世界)。您可以四处走走，自由探索现实世界。

2. Enter photorealistic mixed reality

2.输入逼真的混合现实

A beautiful Volvo XC60 builds up in front of you. It first appears as a stylized transparent blue wireframe. The virtual car is anchored to the real floor in the room around you and oriented so that the chair in the booth is aligned with the driver’s seat of the virtual car. The viewer can take a seat in the real chair and is still able to see the real surroundings through the wireframe.

美丽的沃尔沃XC60会在您眼前建立。它首先显示为程式化的透明蓝色线框。虚拟汽车将固定在您周围房间的真实地板上，并定向，以使展位中的椅子与虚拟汽车的驾驶员座椅对齐。观看者可以坐在真正的椅子上，仍然能够通过线框看到真实的环境。

The car now turns into a solid model, and the surfaces goes from transparent to opaque. The virtual car casts shadows on the floor of the real world, and looking on the car’s surface, it is possible to see that the real world is reflected in the car’s surfaces. The reflections come from an HDR cube map that was taken during setup on the exact spot of the car. The same cube map is also used for ambient lighting.

现在，汽车变成了实体模型，并且表面从透明变为不透明。虚拟汽车在真实世界的地板上投下阴影，然后在汽车表面上看，可以看到真实世界在汽车表面上得到了反射。反射来自在安装过程中在汽车确切位置拍摄的HDR立方体贴图。相同的立方体贴图也用于环境照明。

This is the first time the viewer sees opaque mixed reality, and the effect is stunning. You can still see the real world and your colleagues through windscreens.

这是观众第一次看到不透明的混合现实，并且效果惊人。您仍然可以通过挡风玻璃看到现实世界和您的同事。

How it was accomplished: The car model was provided to Varjo by Volvo. Because the resolution of the headset and the car was so high, we needed to do as much pre-processing as possible. The lighting was baked in the DCC to textures and multiplied in custom shaders. The baked textures only dealt with occlusion and the shading is still affected by the skybox.

完成方式： 沃尔沃(Volvo)将汽车模型提供给了Varjo。由于头戴式耳机和汽车的分辨率很高，因此我们需要进行尽可能多的预处理。灯光在DCC中烘焙为纹理，并在自定义着色器中成倍增加。烘焙的纹理仅处理遮挡，并且阴影仍受天空盒的影响。

Mattias Wilkenmalm from Volvo handled asset creation and wrote custom car paint shaders that delivered superior results. We simply modified them to get the look and transitions we needed. The final model is around 7 million polygons and has around 150 4K textures.3. Switch seamlessly into virtual reality – and back.

沃尔沃(Volvo)的Mattias Wilkenmalm负责资产创建，并编写了定制的汽车油漆着色器，从而获得了出色的效果。我们只是对其进行了修改，以获得所需的外观和过渡。最终模型大约有700万个多边形，并具有大约150个4K纹理。 3.无缝切换到虚拟现实–然后再切换。

The viewer is then asked to step outside the car, and we transition to Venice. The last pieces of the real world around the viewer are now disappearing in a unique transition as the reality transforms into a virtual scene of Venice, where the car is parked in one of the alleys. The reflections in the car are now those of Venice, and the shadows of the car are now landing on the streets of Venice.

然后要求观众走出车外，然后我们转到威尼斯。随着现实转变为威尼斯的虚拟场景，观看者周围的现实世界的最后一部分现在正以独特的过渡消失，威尼斯被停在了一条小巷中。汽车中的反射现在是威尼斯的反射，汽车的阴影现在降落在威尼斯的街道上。

After a while, we transition back from virtual to the real world. The user can now go around the virtual car and see all the details and reflections. This shows that XR-1 offers the ability to still interact with others and select only the parts you want to virtualize.How it was accomplished: To make the transitions visually pleasing, Volvo’s Timotei Ghiurau, Lead, Virtual Experiences, and XR Research, used world space 3D noise with alpha cutouts to bring in the car and the environment. This is fast to do in the fragment and it looks very cool. It was a perfect combination when dealing with tight deadlines. Noise functions can be fetched from Unity’s Keijiro’s repository.

一段时间后，我们从虚拟世界过渡到现实世界。用户现在可以在虚拟汽车周围转转并查看所有细节和反射。这表明XR-1提供了仍然与其他人交互并仅选择要虚拟化的部分的功能。 如何实现： 为了使过渡看起来令人愉悦，沃尔沃的Timotei Ghiurau，Virtual Experiences和XR Research负责人将世界空间3D噪声与Alpha切口一起用于汽车和环境。在片段中这样做很快，而且看起来很酷。在处理紧迫的最后期限时，这是一个完美的组合。噪声函数可以从Unity的Keijiro的存储库中获取。

To get the smooth transition for the car reflections, the Venice environment was added to a separate layer so that the real-time reflection probe was only rendering the minimal amount of geometry. The reflection probe was rendered at 30 frames per second while the scene renders at a much higher frame rate. Having the transition visible in the reflection probe added much more immersion to the scene.

为了获得汽车反射的平滑过渡，将威尼斯环境添加到单独的图层中，以便实时反射探针仅渲染最少数量的几何图形。反射探针以每秒30帧的速度进行渲染，而场景以更高的帧速率进行渲染。在反射探针中看到过渡效果可以使场景更具沉浸感。

The fact that XR-1 is a first-of-a-kind headset to offer the ability to seamlessly switch from real reality into mixed reality onwards into full VR and back to reality again makes for a very impressive demo. It is a Matrix-like moment to see the surrounding reality disappear and replaced by a virtual scene – and then traveling back.

XR-1是首款头戴式耳机，能够无缝地从真实现实转换为混合现实，再转换为完整VR并再次回到现实，这一事实令人印象深刻。这是一个类似于矩阵的时刻，可以看到周围的现实消失并被虚拟场景取代，然后返回。

—

Our thanks to Varjo for sharing this post with us. Join us at Unite Copenhagen to see Varjo VR-1 and XR-1 Developer Edition first hand.

感谢 Varjo 与我们分享这篇文章。加入我们的团结哥本哈根送 ee值Varjo VR-1和XR-1开发版的第一手资料。