VR Headset See-through Tech to Help in MR Development

  • VR headsets’ see-through technology can enable an early MR experience and also help develop the ecosystem necessary for MR applications.
  • Depth perception is essential in MR applications to demonstrate the occlusion phenomena. The current quality of the VR VST is barely adequate as it is achieved through cameras. However, this can be improved by incorporating an additional ToF sensor.
  • There will be significantly more VR headsets with VST going forward, promoting a rise in the number of MR applications.


Virtual reality (VR) headsets use real-time six degrees of freedom (DOF) and visual simultaneous localization and mapping (SLAM) to receive real-world information, such as 3D coordinates and object recognition. Initially, the technology was employed for the hand-tracking and Guardian system. Nonetheless, some VR headsets incorporate images from monochrome cameras and, unexpectedly, extend the capabilities of video see-through (VST) to mixed reality (MR). Meta has launched the Quest Pro VR headset, while ByteDance has the PICO4 with full-color cameras to provide MR applications that are more realistic.

The see-through technology used in current VR headsets is called VST, while the see-through technology used in augmented reality (AR) devices is called the optical see-through (OST). The VST technology blends the digitized three-dimensional view of the real world with a computer-generated simulation of reality, which is then shown on the opaque display. By using VST along with the appropriate rendering technique and algorithm, it is possible to enable interaction and occlusion between virtual and actual objects, as well as reconstruct a digital three-dimension model of the physical world.

Unlike VR headsets, which use cameras to obtain the see-through feature, AR devices are manufactured with the see-through capability. The OST function of AR devices allows virtual images to be superimposed directly over real-world vision. The resolution of real-world images in AR devices is significantly higher than in VR headsets. However, current AR devices allow rather limited interaction with the real environment as occlusion cannot be achieved in the absence of depth sensors.


Advantages of VR VST

The VR VST technology allows users to interact with their environment without taking off the VR headset by means of a physical keyboard, mouse, touchpad or controller. It is even capable of creating expensive gadgets in the virtual world, such as multiple displays. Going forward, more experimental AR and MR features will likely be integrated into VR headsets, considerably enriching the ecosystem and usefulness of such devices.

Current challenges faced by VR VST

  • Both stereo vision and depth perception are crucial for MR. In the future, VR headsets equipped with a time-of-flight (ToF) sensor will be able to capture more accurate distance information than the camera-equipped ones currently available.
  • The VST of a VR headset barely achieves binocular vision. One of the reasons is that there is only one camera for VR VST, which makes it difficult to measure the distance of a close object. Additionally, there will still be warping and distortion of field-of-view objects. Future algorithms must improve this, or designers need to avoid letting users perceive items located closer. Additionally, more color and white balance sensors will be required for making more realistic approximations of the actual color.
  • In a novel application of VR, VST technology can be used to bring several displays in the virtual world by creating a few virtual displays. However, the current resolution of virtual displays in VR headsets is less than that in real-world displays. Nevertheless, this issue can be resolved by increasing the resolution of the display on the VR headset, though it would cost more.
  • Latency is a crucial characteristic of the VST VR headset because processors must encode the image captured by the camera. Therefore, the speed and bandwidth of processors will affect the performance of VR systems as future mid-range and high-end VR headsets in general will support 4K video streams. Therefore, the VR headset SoC’s requirement should be higher than that of smartphones. 


The VST technology in VR headsets can blur the lines between VR and AR, making it possible for VR to provide MR features. MR can be used for numerous purposes apart from just gaming. It can be used in learning platforms, for athletic purposes, webinars and digital twins in the manufacturing industry. Although VR VST still has a few issues that need to be fixed, it can lower the entry barrier for MR applications. Increasing adoption by users and developers can help in the improvement of the VR headset VST technology, which will lead to the creation of more MR apps. In turn, these MR apps will drive the development of AR hardware and ecosystems. After all, AR devices have certain advantages that cannot be replicated by VR devices.


Related Posts


Brady Wang

Brady Wang has more than 20 years working experience in high-technology companies from semiconductor manufacturing to market intelligence, and strategy advisory. Brady’s major coverage in Counterpoint is semiconductors. Prior to joining Counterpoint, Brady Wang worked for Gartner for 11 years. He started his career at TSMC as an engineer for 6 years.

Term of Use and Privacy Policy

Counterpoint Technology Market Research Limited


In order to access Counterpoint Technology Market Research Limited (Company or We hereafter) Web sites, you may be asked to complete a registration form. You are required to provide contact information which is used to enhance the user experience and determine whether you are a paid subscriber or not.
Personal Information When you register on we ask you for personal information. We use this information to provide you with the best advice and highest-quality service as well as with offers that we think are relevant to you. We may also contact you regarding a Web site problem or other customer service-related issues. We do not sell, share or rent personal information about you collected on Company Web sites.

How to unsubscribe and Termination

You may request to terminate your account or unsubscribe to any email subscriptions or mailing lists at any time. In accessing and using this Website, User agrees to comply with all applicable laws and agrees not to take any action that would compromise the security or viability of this Website. The Company may terminate User’s access to this Website at any time for any reason. The terms hereunder regarding Accuracy of Information and Third Party Rights shall survive termination.

Website Content and Copyright

This Website is the property of Counterpoint and is protected by international copyright law and conventions. We grant users the right to access and use the Website, so long as such use is for internal information purposes, and User does not alter, copy, disseminate, redistribute or republish any content or feature of this Website. User acknowledges that access to and use of this Website is subject to these TERMS OF USE and any expanded access or use must be approved in writing by the Company.
– Passwords are for user’s individual use
– Passwords may not be shared with others
– Users may not store documents in shared folders.
– Users may not redistribute documents to non-users unless otherwise stated in their contract terms.

Changes or Updates to the Website

The Company reserves the right to change, update or discontinue any aspect of this Website at any time without notice. Your continued use of the Website after any such change constitutes your agreement to these TERMS OF USE, as modified.
Accuracy of Information: While the information contained on this Website has been obtained from sources believed to be reliable, We disclaims all warranties as to the accuracy, completeness or adequacy of such information. User assumes sole responsibility for the use it makes of this Website to achieve his/her intended results.

Third Party Links: This Website may contain links to other third party websites, which are provided as additional resources for the convenience of Users. We do not endorse, sponsor or accept any responsibility for these third party websites, User agrees to direct any concerns relating to these third party websites to the relevant website administrator.

Cookies and Tracking

We may monitor how you use our Web sites. It is used solely for purposes of enabling us to provide you with a personalized Web site experience.
This data may also be used in the aggregate, to identify appropriate product offerings and subscription plans.
Cookies may be set in order to identify you and determine your access privileges. Cookies are simply identifiers. You have the ability to delete cookie files from your hard disk drive.