Late 2018, Jaunt China had moved its strategy to LBE, which was promising at that time in China. That most of the use cases are in shopping malls or VR stores. Where the cost is very expensive in my opinion. (Around 20 USD per 10 minutes).
From Jaunt's aspect, they had great advantage as they had lots of VR videos, and had hosted several events before.
So the remaining part is the VR game. I lead the team for protyping LBE VR cases.
Tracking:
At first, we cooperated with a VR tracking company in Beijing. They used Outside-in strategy and put several sensors gridly arranged on top of the room.
Motion Capture:
They also wrote a calibration software to adjust, and sent a motion capture package to us, which looked like this. The core tool kit in Unity is also steam VR.
However, putting that many sensors is quite costy, while at that time, htc Vive Pro had just been released, their acuracy of position tracking is comparable.
Using steam VR to design LBE prototype is quite simple, the hardest part is modeling, at that time we don't have any 3D artists.
MR Video capture:
I used Intel Real Sense RGBD camera, which by then was already stopped producing.
The recording software is Mixcast by blueprint reality.
Here is the link: https://mixcast.me/
The depth filter at that time was still under development. It can filter your shape, but the hand or arm by my best practise of setting threshold were still like frog's flippers :)
The other way is to filter by specific range of colors. White is by default. While in testing, how much degree of grey is pretty tricky. You can see the video below to check how do we optimize the threshold and how do we set up the room environment.
You can see transparent would be specified, while in this room there are lots of iron shelf, and trackers from other side. In real time the room could be modified.
Lidar interaction:
This idea is originated from team Lab. In the video below, you can see when the character walks on the hill, the waterfall effect would drop alongside the body.
The technology is usually not that hard. They used lidar to detect the distance for specific angle has been changed.
In this project, I used Hokuyo sensor:
Here is the behavior of its setup software.
After checked the manual of its API, I also developed the prototype in Unity.
The next video is a example, that when distance is changed and the sensor get each frame, it can easily calculate the distance.
Wechat miniapp appointment:
After the VR LBE game eventually setup, I also wrote a WeChat miniapp for the appointment system, which is my first time of creating WeChat miniapp. Though nowadays, many cases that might be useful in WeChat miniapp can be modualized and have lots of templates, at that time we have to use the development tools to create wechat miniapp. The layout is designed by wxml (it's own format similar to xml, and wxss, it's similar css layout file).
The background I used Tencent cloud function, as WeChat is presented from Tencent, their connection is much easier. Cloud function is a no-sgl db system, that send and receive requests via json.
