Leap Enabling the Unity 3D Bootcamp Demo – Step by Step + Leap HUD

In this Blog, I take the unity 3D standard demo (The Bootcamp) and I modify it to be Leap Enabled and supply below the step by step instructions for the reader to be able to reproduce this. The motivation behind Leap Enabling Boot Camp was to answer some key questions:

  • Can it be done?
    Answer: YES
  • How hard is it to Leap Enable a FPS or a 3rd Person Shooter?
    Answer: Relative Simple
  • How easy is it to navigate?
    Answer: Depends on your perspective.  The modifications I suggest below will allow you to experiment as well as simply use what I came up with.  I tried some different combinations such as using a single hand to control movement, mouse look as well as shooting.   This worked well in open space, but was not as usable in close quarter. Issues were mainly with unexpected “strafing” when you are trying to change the mouse look only. Ended up fixing this by adding a new function to return leap input as stepped input
  • Single Hand, or Hand and Keyboard for Navigation?
    Answer: Again, the answer is most likely affected by desired gameplay and preference.  I ended up having 4 variables, one for strafing (leap or not) one for forward and backward (leap or not), another for shooting and yet another for mouse look. Can you navigate and shoot with one hand? Sure… The demo shows how that is done.
  • Easiest way to “shoot” while navigating?
    Answer: Tried initially a rotation of the hand from horizontal to vertical to indicate shooting, but that affected the accuracy of the positioning, an undesirable side effect.  I then simply using the act of splaying your fingers slightly – shooting if 4 or more fingers are detected, and that worked quite well.
    Key thing (included in steps below) is to allow shooting via traditional controls as well as Leap, so you can easily switch between themselves
  • Finally, I found it very useful to create a HUD to visualize what the Leap is doing Navigation wise, so I could see if I needed to tweak anything.  See image below with the HUD on the right.

LeapEnabledBootCampImg1

If you are interested in the final result. Check out this Video of what it looks like.

Right… On with the Tutorial then…

Step 1 Create a new project.

I called mine LeapEnabledBootCampMake sure you check the following packages to import “Bootcamp.unitypackage”.
If you do not have this package on your machine, go ahead and download it from the Unity Asset store (it’s free)

LeapEnabledBootCampImg2

Step 2 Make sure that the Bootcamp Scene is working properly

Open the Bootcamp unity scene and run it. Make sure the soldier comes up and you can go forward and backwards and strafe left and right with the arrows, and that you can also change the camera looking angle using the mouse.

Step 3 (Optional) Rename the Bootcamp scene so you can experiment

I renamed mine to LeapEnabledBootcamp

Step 4 Create a Plugins folder under the asset directory

Step 5 Copy the Leap Files from UnityAssets in the SDK to your Plugins Directory and root directory

If you have unity Pro, you can copy the 3 files (Windows) from the Leap UnitySandbox\Assets\Plugins directory to your newly created Plugins director
If you have Unity Free only, then you need to copy Leap.dll and LeapCSharp.dll to the root directory of your game, and LeapCSharp.NET3.5.dll to your Plugins directory.
For more information on getting Unity Standard (free) working with Leap, read my Blog (http://pierresemaan.com/getting-the-leap-to-work-with-unity-free-version-not-pro/ )

Step 6 copy LeapUnityExtensions.cs to the Plugins folder

LeapUnityExtensions.cs is in the Assets\Scripts directory from the examples in the SDK under UnitySandbox.

Step 7 copy pxsLeapInput.cs to the Plugins folder

Download pxsLeapInput.cs and copy the script to the Plugins folder.  The Script has to be in Plugins as we need to reference it from a javascript script later and it needs to be compiled first (so the javascript engine can see it)

Step 7 a (optional) examine the pxsLeapInput Script

Note this is a refinement of my previous pxsLeapInput.  It includes some key enhancements such as:
1      Added support for Mouse x and Mosue y axis
2      If Hand is null, pxsLeapInput will get the default input axis (so, you can move your hand away from the Leap and work with the game as per usual).
3      Added a function GetHandGesture for firing.  Splayed fingers = fire, rotate hand clockwise = aim.
4      Added a function GetHandAxisStep which returns a “clipped” or “stepped”

private static float GetHandAxisPrivate(string axisName, bool scaled)
{
​// Call Update so you can get the latest frame and hand
​Update();
​float ret = 0.0F;
​if (m_Hand != null)
​{
​​// SNIP
​​case "Mouse X":
​​​// rotation is preferred (more usable).  
​​​// if using rotation for something else,
​​​// can use Palmdirection by uncommenting below.
​​​ret = ret = -2 * PalmNormal.x ;
​​​// ret =  4 * (PalmDirection.x ) + 1.00F ;
​​​m_Errors = "ret Mouse.x = " + ret.ToString();
​​​break;
​​case "Mouse Y":
​​​// use z axis as this is the most pronounced
​​​// change when TITLTing a hand
​​​​ret =  2 * PalmNormal.z ;
​​​​// m_Errors = "ret Mouse.Y = " + ret.ToString();
​​​​break;
​​// SNIP
​}
​else
​{
​​// Hand is Null, so return the standard axis
​​switch (axisName)
​​{
​​case "Depth":
​​​// depth for leap = vertical axis
​​​ret = Input.GetAxis("Vertical") ;
​​​break;
​​default:
​​​// return the axis name from input if hand is null
​​​// and the special cases above do not apply
​​​ret = Input.GetAxis(axisName) ;
​​​break;
​}
​}
​return ret;
}

And

public static float GetHandAxisStep(string axisName)
{
​// Assume dead zone of -0.5 to 0.5
​// always return -1, 0 or +1
​float ret = GetHandAxisPrivate(axisName, true);
​if (ret < -0.5F) ​{ ​​ret = -1.0F; ​} ​else if (ret > 0.5F)
​{
​​ret = 1.0F;
​}
​else
​{
​​ret = 0;
​}
​return ret;
}

Note that it is a Classic class (not Monobehavior).  It’s a singleton and includes properties to easily get Frame and Hand from the latest Update.

Step 8 Modify SoldierController.js script to allow for Leap Input

In the Bootcamp demo, we have the soldier movement (forward, backward, sideways) in SoldierController.js.
As usual, we setup some variables to indicate whether we use LeapEnabled input
Note that with my new script, where it falls through to normal Axis behaviour if the Leap hand is null, this may seem redundant, but the main motivation is to be able to easily enable and disable individual axis from Leap Control to “play” with best setting for the Leap.  After all, this is a tutorial.

public var leapEnabledVerticalAxis : boolean = false;
public var leapEnabledHorizontalAxis : boolean = false;
public var leapEnabledFire : boolean = false;
public var hideMouse : boolean = false;

Next, we find where we use Input.GetAxis and modify these to use Leap if Leap is enabled.

// moveDir = new Vector3(Input.GetAxis("Horizontal"), 0, Input.GetAxis("Vertical"));
// pxsLeap
var x : float = 0;
var z : float = 0;

if (leapEnabledHorizontalAxis == true)
{
	x = pxsLeapInput.GetHandAxisStep("Horizontal");
}
else
{
	x = Input.GetAxis("Horizontal");
}

if (leapEnabledVerticalAxis == true)
{
	z = pxsLeapInput.GetHandAxisStep("Depth");
}
else
{
	z = Input.GetAxis("Vertical");
}

Note that when we modify the fire behavior, we make sure you can still fire using the normal inputs

if (leapEnabledFire == true)
{
​fireLeap = pxsLeapInput.GetHandGesture("Fire1") && weaponSystem.currentGun.freeToShoot && !dead && !inAir;
}
//Check if the user if firing the weapon
fire = fireLeap || (Input.GetButton("Fire1") && weaponSystem.currentGun.freeToShoot && !dead && !inAir);

Step 9 Modify SoldierCamera.js script to allow for Leap Input

Now we modify the SoldierCamera script to accept Leap input.
We setup a variabl at the top of the script

public var leapEnabled : boolean = false;

We then modify the input.  Note that there are two places to modify.  One which controls the camera and another area where there is logic to re-set the idle timer when there is no movement.  Don’t forget to do both.  I ignored the Orbit logic and found the camera orbiting as it thinks the soldier is idle.

if (leapEnabled == true)
{
​// Make sure we reset the idle timer if we are moving via Leap, otherwise, we enter into Orbit Mode with funny results
​if(orbit && (Input.GetKeyDown(KeyCode.O) || pxsLeapInput.GetHandAxisStep("Horizontal") != 0.0 || pxsLeapInput.GetHandAxisStep("Depth") != 0.0 || soldierController.aim || soldierController.fire))
​{
​​GoToOrbitMode(false);
​}
}
else
{
​if(orbit && (Input.GetKeyDown(KeyCode.O) || Input.GetAxis("Horizontal") != 0.0 || Input.GetAxis("Vertical") != 0.0 || soldierController.aim || soldierController.fire))
​{
​​GoToOrbitMode(false);
​}
}

Also

if (leapEnabled == true)
{
​x += Mathf.Clamp(pxsLeapInput.GetHandAxis("Mouse X") * a.x, -maxSpeed.x, maxSpeed.x) * deltaTime;
​y -= Mathf.Clamp(pxsLeapInput.GetHandAxis("Mouse Y") * a.y, -maxSpeed.y, maxSpeed.y) * deltaTime;
​// print (pxsLeapInput.Errors);
}
else
{
​x += Mathf.Clamp(Input.GetAxis("Mouse X") * a.x, -maxSpeed.x, maxSpeed.x) * deltaTime;
​y -= Mathf.Clamp(Input.GetAxis("Mouse Y") * a.y, -maxSpeed.y, maxSpeed.y) * deltaTime;
}

Step 10 (Optional) Create a Leap Navigation HUD to visualize Leap Input

As you will see from the video recording, I found it useful to create a HUD to visualize Leap Input.
The steps to do so are listed here briefly, as I will create a separate blog tutorial to detail the exact steps to keep everything separate, also because the HUD is not 100% coupled with the Leap (conceptually anyway). Steps were:
. Create an empty GameObject
. Attach a script HudLeap to it
. The script has some texture variables, so we can then drag bitmaps we create (I used PowerPoint to create mine) onto these textures
. The HudLeap has an OnGUI
. OnGUI uses the pxsLeapInput to get the leap inputs we are using to navigate
. We then draw the representation on screen
Check out the HudLeap.js script for the details.

Step 11 Enable Leap Input

Select the Soldier in the Scene and in the SoldierController.js script component in the inspector, select the checkbox to make it Leap Enabled.
Select the “Soldier Camera” in the Scene and in the SoldierCamera.js script component in the inspector, select the checkbox to make it Leap Enabled.

Step 12 Play

If you play the scene now, you will note that you can control the car with your hand.
Note that you can see the effect live with the Leap HUD.
LeapEnabledBootCampImg4