VR simulates human action

Posted by benjy on Fri, 07 Jun 2019 03:16:44 +0200

First of all, Thanksgiving God's sharing, there is no way ahead, the latter want to enjoy the cool is difficult.
Drawing on the address of the Great God: http://www.gad.qq.com/article/detail/27127

I. Preparation

    1. Equipment
    HTC Vive tracker (Tracker X3, Controller X2): A set of HTC Vive devices has two handles and a tracker, then order two tracers.

    2. Plug-ins
    Unity's Assets Store plug-ins can be found by searching for "Final IK" (it's too expensive to buy directly, but God is selfless).
    (2) Of course, you also need to download Steam VR Plugin as a plug-in in the Assets Store (as anyone who has touched VR should know)

II. Thoughts

1.IK, also known as inverse kinematics, is a plug-in used to maintain correct joint bending.

2. How to distinguish trackers?
(1) HTC Vive can identify 15 devices (locator + tracker), judge the device type according to the example of Valve.VR.ETrackedDeviceClass, and then use Steam VR_TrackedObject. isValid to judge whether the device is available. If available, get its index.
(2) First judge foot 1, foot 2, waist, hand 1, hand 2 according to the value of Y-axis direction; then take helmet as reference, if the vector direction of foot 1-foot 2 is the right side of helmet, foot 1 is the right foot, hand is the same.

3. How to synchronize model actions?
It is not possible to directly set the equipment as a moving target of four joints. Because the foot and waist trackers cannot coincide with the model joint points, and there is also a difference in the rotation angle.
Five new objects are created, and the rotation of the five objects is the rotation of the joints when they are recognized, and then they are set as the sub-nodes of the tracker.

III. Realization

Post the code identifying the tracker:

 * CheckTrackers/InitPositions The sub-objects are wrist, ankle and waist, which are dragged to the corresponding position of the model.
 * Start detecting trackers using the coprocess CheckTrackers(), and the orderedList stores the resulting location index
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class GetTrackers : MonoBehaviour
    public Dictionary<int, Transform> trackerTrans = new Dictionary<int, Transform>();//Storage tracker index and transform
    public List<int> orderedList = new List<int>();//index of Storage Tracker
    public Transform[] originBodyTrans = new Transform[5];//Limbs + Waist (Following Target)
    public Transform bodyRightTrans;//Get the right-hand coordinates of the helmet to distinguish the left and right trackers

    [SerializeField] private GameObject trackers;//Parent node, retrieve tracker

    //The order is: right foot, left foot, waist, right hand, left hand.
    private SteamVR_TrackedObject[] trackerArray;
    private Dictionary<int, Transform> devicesIndexArray = new Dictionary<int, Transform>();//A Tracker for Storage Recognition

    public static GetTrackers instance;
    // Use this for initialization
    void Awake()
        instance = this;

        trackerArray = trackers.GetComponentsInChildren<SteamVR_TrackedObject>();

        for (int i = 0; i < trackerArray.Length; i++)
            trackerTrans[(int)(trackerArray[i].index)] = trackerArray[i].transform;//Get the attributes of 15 subnodes

    /// <summary>
    /// Start Detecting Tracker
    /// </summary>
    /// <returns></returns>
    public IEnumerator CheckTrackers()
        while (true)
            yield return new WaitForFixedUpdate();
            for (int i = (int)SteamVR_TrackedObject.EIndex.Device1; i < (int)SteamVR_TrackedObject.EIndex.Device15; i++)
                Valve.VR.ETrackedDeviceClass deviceClass = Valve.VR.OpenVR.System.GetTrackedDeviceClass((uint)i);

                if (deviceClass == Valve.VR.ETrackedDeviceClass.Controller ||
                    deviceClass == Valve.VR.ETrackedDeviceClass.GenericTracker)//Handle or tracker
                    for (int m = 0; m < trackerArray.Length; m++)
                        if ((int)trackerArray[m].index == i && trackerArray[m].isValid)
                            if (!devicesIndexArray.ContainsKey(i))
                                devicesIndexArray.Add(i, trackerTrans[i]);//Add to Dictionary
                                orderedList.Add(i);//Add the order of key s to the array
                                Debug.Log(deviceClass.ToString() + i + ": " + trackerTrans[i].position.y);
                            if (devicesIndexArray.Count == 5)//If there are five trackers
                                yield return StartCoroutine(BindingTrickersToModel());//Distinguishing Trackers by Location
                                yield break;//End the cycle
    /// <summary>
    /// Determine the joint corresponding to the tracker
    /// </summary>
    /// <param name="devicesIndexArray">Storage tracker index</param>
    /// <returns></returns>
    IEnumerator BindingTrickersToModel()
        yield return null;
        //According to Y, distinguish hands, feet and waists first
        //1, 2 feet, 3 waists, 4, 5 hands
        for (int i = 0; i < orderedList.Count - 1; i++)
            for (int j = orderedList.Count - 1; j > i; j--)
                float y1 = devicesIndexArray[orderedList[i]].position.y;
                float y2 = devicesIndexArray[orderedList[j]].position.y;
                if (y1 > y2)
                    var temp = orderedList[i];
                    orderedList[i] = orderedList[j];
                    orderedList[j] = temp;

        //Horizontal direction vector
        Vector3 footVector = devicesIndexArray[orderedList[0]].position - devicesIndexArray[orderedList[1]].position;
        footVector.y = 0;
        Vector3 handVector = devicesIndexArray[orderedList[3]].position - devicesIndexArray[orderedList[4]].position;
        handVector.y = 0;

        if (Vector3.Angle(new Vector3(bodyRightTrans.right.x, 0, bodyRightTrans.right.z), footVector) > 90)
            //0 right 1 left
            var temp = orderedList[0];
            orderedList[0] = orderedList[1];
            orderedList[1] = temp;
        if (Vector3.Angle(new Vector3(bodyRightTrans.right.x, 0, bodyRightTrans.right.z), handVector) > 90)
            //3 right 4 left
            var temp = orderedList[3];
            orderedList[3] = orderedList[4];
            orderedList[4] = temp;

            orderedList[0], orderedList[1], orderedList[2], orderedList[3], orderedList[4]);

        devicesIndexArray.Clear();//Empty the dictionary

Topics: Steam Unity