White Paper: Benefits of 10, 25, 50 and 100GigE Cameras for VR, AR and MR Applications

The line between the real and virtual worlds continues to blur thus creating awe-inspiring experiences that not too long ago were only found in works of fiction. Whether through Virtual Reality, Augmented Reality, or Mixed Reality, individuals can break down the wall between reality and the imagination.

For most people, these technologies may still seem like they are too far off into the future to even think about but they have been around for quite a few years already with a deep history and rapid evolution.

Recognized as the most well-known of the three technologies, virtual reality (VR) immerses users in a completely virtual environment that is generated by a computer thus completely shutting out the real-world. Using a head-mounted display (HMD) or headset, users can experience a world of images and sounds which tricks the individual into thinking they are part of that world. As VR continues to become more advanced, it even offers the chance to manipulate and move objects with special controllers but all content remains computer generated.

Although VR has exploded in the past five years, it has been a concept for much longer. As early as the 1930s, inventors and writers envisioned simulated environments where one could escape the real world through machines and art.

In the 1950s, multi-sensory simulation was created for users to feel like they were “in” a movie. By the 1960s, head-mounted display devices were created to serve as windows into the virtual world. Some might remember the View-Master which was introduced as early as 1939 which allowed 3-D viewing using “reels” containing seven stereoscopic 3-D pairs of small transparent color film photographs.

Although slightly less familiar than virtual reality, augmented reality is something that affects individuals almost daily without them even realizing it. In augmented reality (AR), users see and interact with the real world while digital content is added to it. One of the most recent examples of AR is the game Pokémon GO, where users search for virtual creatures in their real-world environments. This game is continually evolving and is now taking on Mixed Reality characteristics.

From the first virtual yellow line marker in live NFL games, puck tracers in NHL hockey, and golf ball tracers on the PGA tour, AR has had a progressive impact over the years. Beginning in the 1960’s and 1970’s, inventors developed systems that would display computer-generated graphics overlaid onto the real world, surrounding users in an interactive environment. While this technology has drastically evolved in the last decade alone, its growth is only expected to continue.

Mixed reality is truly that: the merging of real and virtual worlds to produce new environments and visualizations where real and digital objects co-exist and interact in real time. Using an MR headset along with next-generation imaging and sensing technologies, MR provides the ability to interact with a virtual environment while remaining in the real world thus merging the real and the imaginary which offers an experience that can change the way people work and play today. Originally introduced in 1994, MR has moved far beyond just displays, and now includes environmental input, spatial sound, and location. While this technology is still rapidly evolving, it is said that it will revolutionize how people view and interact with the world.

VR, AR, and MR, all have similar challenges. Largely, these challenges come down to the generation of 3D content. They can require tight timing synchronization between image captures from multiple camera sources for an optimal experience. They can require excellent image quality under a variety of lighting conditions. Some applications must have images which are high resolution while still maintaining an appropriate frame rate. They can have camera setups which span large areas including stadiums so the simplest options are preferred for system installation and maintenance. They can require many more cameras to complete a system for an optimal experience. Lastly, as in most applications, cost can be an important factor.

Emergent’s 10, 25, 50 and 100GigE cameras provide a few options to tightly control the timing synchronization between cameras. Emergent has extensive experience in synchronizing cameras using a variety of methods including PTP, hardware triggering, etc, and have been able to demonstrate microsecond accuracies with minimal effort.

Emergent cameras only use the best sensors from top vendors such as Sony, Gpixel, and AMS and we work hard to squeeze every ounce of performance out of these sensors which leads to improvements in dynamic range which is important to VR/AR/ MR applications in that systems often need to deal with varying lighting conditions.

Sony sensors currently range from 0.5MP @ 1595 fps to as high as 31MP @ 35 fps. Sony’s 4th generation of sensors, also known as Sony Pregius S range from 5MP @ 269 fps to 25MP @ 98 fps. These newer sensors combine high resolution and frame rates and are a great fit for VR, AR, and MR applications. Gpixel sensors currently range from 5MP @ 290 fps to 65MP @ 35 fps and are on the rise. AMS has the 50MP @ 30 fps.

The Sony IMX253 (12MP @ 80fps), IMX535 (12MP @ 192 fps), IMX532 (16MP @145 fps), IMX531 (20MP @ 100 fps), and IMX530 (24MP @ 98 fps) are currently the most popular for VR/AR/MR applications.

Emergent’s 10, 25, 50 and 100GigE cameras, coupled with Myricom’s MVA or Mellanox’ VMA, equals or out-performs other interfaces in every technical category and its price performance is the best. With VMA or MVA, all the shortcomings associated with 1GigE w.r.t. CPU utilization, latency and jitter are obliterated. Industry acceptance is excellent as well.

Here, the closest competitor is CXP. Low cost commodity SFP+ SFP28, ans QSFP28 transceivers create a virtually limitless link length. In comparison, CXP6 and CXP12 will achieve 72m and 30m respectively but can go further with expensive fiber adapters.

It is important to distinguish between SFP+/SFP28/ QSFP28 transceivers and CXP or CameraLink fiber adapters. Transceivers are low cost commodity product which slots directly in the camera and NIC while the fiber adapters are expensive bulky external pieces of hardware (often the same size as the camera itself or even bigger) produced by very few suppliers and these sit somewhere in-between the camera CXP or CameraLink cable(s) and the fiber link which, interestingly enough, employs the same SFP+ transceivers that Emergent cameras employ directly in the camera. And, naturally, you need one fiber adapter at each end of the link since the grabber will typically have the same interface as the camera.