Leap Enabling the Unity 3D Bootcamp Demo – Step by Step + Leap HUD

In this Blog, I take the unity 3D standard demo (The Bootcamp) and I modify it to be Leap Enabled and supply below the step by step instructions for the reader to be able to reproduce this. The motivation behind Leap Enabling Boot Camp was to answer some key questions:

  • Can it be done?
    Answer: YES
  • How hard is it to Leap Enable a FPS or a 3rd Person Shooter?
    Answer: Relative Simple
  • How easy is it to navigate?
    Answer: Depends on your perspective.  The modifications I suggest below will allow you to experiment as well as simply use what I came up with.  I tried some different combinations such as using a single hand to control movement, mouse look as well as shooting.   This worked well in open space, but was not as usable in close quarter. Issues were mainly with unexpected “strafing” when you are trying to change the mouse look only. Ended up fixing this by adding a new function to return leap input as stepped input
  • Single Hand, or Hand and Keyboard for Navigation?
    Answer: Again, the answer is most likely affected by desired gameplay and preference.  I ended up having 4 variables, one for strafing (leap or not) one for forward and backward (leap or not), another for shooting and yet another for mouse look. Can you navigate and shoot with one hand? Sure… The demo shows how that is done.
  • Easiest way to “shoot” while navigating?
    Answer: Tried initially a rotation of the hand from horizontal to vertical to indicate shooting, but that affected the accuracy of the positioning, an undesirable side effect.  I then simply using the act of splaying your fingers slightly – shooting if 4 or more fingers are detected, and that worked quite well.
    Key thing (included in steps below) is to allow shooting via traditional controls as well as Leap, so you can easily switch between themselves
  • Finally, I found it very useful to create a HUD to visualize what the Leap is doing Navigation wise, so I could see if I needed to tweak anything.  See image below with the HUD on the right.

LeapEnabledBootCampImg1

If you are interested in the final result. Check out this Video of what it looks like.

Right… On with the Tutorial then…

Step 1 Create a new project.

I called mine LeapEnabledBootCampMake sure you check the following packages to import “Bootcamp.unitypackage”.
If you do not have this package on your machine, go ahead and download it from the Unity Asset store (it’s free)

LeapEnabledBootCampImg2

Step 2 Make sure that the Bootcamp Scene is working properly

Open the Bootcamp unity scene and run it. Make sure the soldier comes up and you can go forward and backwards and strafe left and right with the arrows, and that you can also change the camera looking angle using the mouse.

Step 3 (Optional) Rename the Bootcamp scene so you can experiment

I renamed mine to LeapEnabledBootcamp

Step 4 Create a Plugins folder under the asset directory

Step 5 Copy the Leap Files from UnityAssets in the SDK to your Plugins Directory and root directory

If you have unity Pro, you can copy the 3 files (Windows) from the Leap UnitySandbox\Assets\Plugins directory to your newly created Plugins director
If you have Unity Free only, then you need to copy Leap.dll and LeapCSharp.dll to the root directory of your game, and LeapCSharp.NET3.5.dll to your Plugins directory.
For more information on getting Unity Standard (free) working with Leap, read my Blog (http://pierresemaan.com/getting-the-leap-to-work-with-unity-free-version-not-pro/ )

Step 6 copy LeapUnityExtensions.cs to the Plugins folder

LeapUnityExtensions.cs is in the Assets\Scripts directory from the examples in the SDK under UnitySandbox.

Step 7 copy pxsLeapInput.cs to the Plugins folder

Download pxsLeapInput.cs and copy the script to the Plugins folder.  The Script has to be in Plugins as we need to reference it from a javascript script later and it needs to be compiled first (so the javascript engine can see it)

Step 7 a (optional) examine the pxsLeapInput Script

Note this is a refinement of my previous pxsLeapInput.  It includes some key enhancements such as:
1      Added support for Mouse x and Mosue y axis
2      If Hand is null, pxsLeapInput will get the default input axis (so, you can move your hand away from the Leap and work with the game as per usual).
3      Added a function GetHandGesture for firing.  Splayed fingers = fire, rotate hand clockwise = aim.
4      Added a function GetHandAxisStep which returns a “clipped” or “stepped”

private static float GetHandAxisPrivate(string axisName, bool scaled)
{
​// Call Update so you can get the latest frame and hand
​Update();
​float ret = 0.0F;
​if (m_Hand != null)
​{
​​// SNIP
​​case "Mouse X":
​​​// rotation is preferred (more usable).  
​​​// if using rotation for something else,
​​​// can use Palmdirection by uncommenting below.
​​​ret = ret = -2 * PalmNormal.x ;
​​​// ret =  4 * (PalmDirection.x ) + 1.00F ;
​​​m_Errors = "ret Mouse.x = " + ret.ToString();
​​​break;
​​case "Mouse Y":
​​​// use z axis as this is the most pronounced
​​​// change when TITLTing a hand
​​​​ret =  2 * PalmNormal.z ;
​​​​// m_Errors = "ret Mouse.Y = " + ret.ToString();
​​​​break;
​​// SNIP
​}
​else
​{
​​// Hand is Null, so return the standard axis
​​switch (axisName)
​​{
​​case "Depth":
​​​// depth for leap = vertical axis
​​​ret = Input.GetAxis("Vertical") ;
​​​break;
​​default:
​​​// return the axis name from input if hand is null
​​​// and the special cases above do not apply
​​​ret = Input.GetAxis(axisName) ;
​​​break;
​}
​}
​return ret;
}

And

public static float GetHandAxisStep(string axisName)
{
​// Assume dead zone of -0.5 to 0.5
​// always return -1, 0 or +1
​float ret = GetHandAxisPrivate(axisName, true);
​if (ret < -0.5F) ​{ ​​ret = -1.0F; ​} ​else if (ret > 0.5F)
​{
​​ret = 1.0F;
​}
​else
​{
​​ret = 0;
​}
​return ret;
}

Note that it is a Classic class (not Monobehavior).  It’s a singleton and includes properties to easily get Frame and Hand from the latest Update.

Step 8 Modify SoldierController.js script to allow for Leap Input

In the Bootcamp demo, we have the soldier movement (forward, backward, sideways) in SoldierController.js.
As usual, we setup some variables to indicate whether we use LeapEnabled input
Note that with my new script, where it falls through to normal Axis behaviour if the Leap hand is null, this may seem redundant, but the main motivation is to be able to easily enable and disable individual axis from Leap Control to “play” with best setting for the Leap.  After all, this is a tutorial.

public var leapEnabledVerticalAxis : boolean = false;
public var leapEnabledHorizontalAxis : boolean = false;
public var leapEnabledFire : boolean = false;
public var hideMouse : boolean = false;

Next, we find where we use Input.GetAxis and modify these to use Leap if Leap is enabled.

// moveDir = new Vector3(Input.GetAxis("Horizontal"), 0, Input.GetAxis("Vertical"));
// pxsLeap
var x : float = 0;
var z : float = 0;

if (leapEnabledHorizontalAxis == true)
{
	x = pxsLeapInput.GetHandAxisStep("Horizontal");
}
else
{
	x = Input.GetAxis("Horizontal");
}

if (leapEnabledVerticalAxis == true)
{
	z = pxsLeapInput.GetHandAxisStep("Depth");
}
else
{
	z = Input.GetAxis("Vertical");
}

Note that when we modify the fire behavior, we make sure you can still fire using the normal inputs

if (leapEnabledFire == true)
{
​fireLeap = pxsLeapInput.GetHandGesture("Fire1") && weaponSystem.currentGun.freeToShoot && !dead && !inAir;
}
//Check if the user if firing the weapon
fire = fireLeap || (Input.GetButton("Fire1") && weaponSystem.currentGun.freeToShoot && !dead && !inAir);

Step 9 Modify SoldierCamera.js script to allow for Leap Input

Now we modify the SoldierCamera script to accept Leap input.
We setup a variabl at the top of the script

public var leapEnabled : boolean = false;

We then modify the input.  Note that there are two places to modify.  One which controls the camera and another area where there is logic to re-set the idle timer when there is no movement.  Don’t forget to do both.  I ignored the Orbit logic and found the camera orbiting as it thinks the soldier is idle.

if (leapEnabled == true)
{
​// Make sure we reset the idle timer if we are moving via Leap, otherwise, we enter into Orbit Mode with funny results
​if(orbit && (Input.GetKeyDown(KeyCode.O) || pxsLeapInput.GetHandAxisStep("Horizontal") != 0.0 || pxsLeapInput.GetHandAxisStep("Depth") != 0.0 || soldierController.aim || soldierController.fire))
​{
​​GoToOrbitMode(false);
​}
}
else
{
​if(orbit && (Input.GetKeyDown(KeyCode.O) || Input.GetAxis("Horizontal") != 0.0 || Input.GetAxis("Vertical") != 0.0 || soldierController.aim || soldierController.fire))
​{
​​GoToOrbitMode(false);
​}
}

Also

if (leapEnabled == true)
{
​x += Mathf.Clamp(pxsLeapInput.GetHandAxis("Mouse X") * a.x, -maxSpeed.x, maxSpeed.x) * deltaTime;
​y -= Mathf.Clamp(pxsLeapInput.GetHandAxis("Mouse Y") * a.y, -maxSpeed.y, maxSpeed.y) * deltaTime;
​// print (pxsLeapInput.Errors);
}
else
{
​x += Mathf.Clamp(Input.GetAxis("Mouse X") * a.x, -maxSpeed.x, maxSpeed.x) * deltaTime;
​y -= Mathf.Clamp(Input.GetAxis("Mouse Y") * a.y, -maxSpeed.y, maxSpeed.y) * deltaTime;
}

Step 10 (Optional) Create a Leap Navigation HUD to visualize Leap Input

As you will see from the video recording, I found it useful to create a HUD to visualize Leap Input.
The steps to do so are listed here briefly, as I will create a separate blog tutorial to detail the exact steps to keep everything separate, also because the HUD is not 100% coupled with the Leap (conceptually anyway). Steps were:
. Create an empty GameObject
. Attach a script HudLeap to it
. The script has some texture variables, so we can then drag bitmaps we create (I used PowerPoint to create mine) onto these textures
. The HudLeap has an OnGUI
. OnGUI uses the pxsLeapInput to get the leap inputs we are using to navigate
. We then draw the representation on screen
Check out the HudLeap.js script for the details.

Step 11 Enable Leap Input

Select the Soldier in the Scene and in the SoldierController.js script component in the inspector, select the checkbox to make it Leap Enabled.
Select the “Soldier Camera” in the Scene and in the SoldierCamera.js script component in the inspector, select the checkbox to make it Leap Enabled.

Step 12 Play

If you play the scene now, you will note that you can control the car with your hand.
Note that you can see the effect live with the Leap HUD.
LeapEnabledBootCampImg4

17 thoughts on “Leap Enabling the Unity 3D Bootcamp Demo – Step by Step + Leap HUD

  1. I think that this is awesome, and my friend ordered me a leap motion for my birthday, and i am a unity programmer—nerdgasm

    • Charlie, make sure you block out some decent hour to play with teh leap once it arrives. It is very addictive.
      Glad you enjoyed the tutorial. Working on my next one, which is modifying my pxsLeapInput script so it is aware when the App is deployed as a WebPlayer and communicates theough hosted template via leap JavaScript api in that case.

  2. Pingback: Leap Motion et Unity 3D | Unity 3D – France

  3. Pingback: Menu GUI options for Leap Motion and Unity3D | Pierre's Blog

  4. Hello Pierre,

    I downloaded the full project. I can’t seem to operate the soldier with the leap motion device. I also tried to do it from the very beginning following your instructions and the one by Joe Ward. Still can’t make the soldier move at all but I can move it with the mouse and keyboard. I checked the inspector to make sure all leap-related checkboxes are checked (both Soldier Controller and Soldier Camera – both under Soldier_Locomotion).
    Also, is there a difference if I put the LeapCSharp.NET3.5.dll directly under Assets (as joe instructed) or on the Plugins folder (you instructed)?

    I have been using the 0.8 SDK and I use a Mac

      • maybe you share how you solved your problem, i have not get my device yet, but i think as soon as they are delivering the devices, people will come to this site for different questions.

      • Hi, I’d appreciate if you could share with us how to make it work. Everything seems to be set-up but the character doesn’t respond to input from Leap. Thanks.

        • Mike. Did you download the full source code? If it is not working, the most common error is to forget to put the leap dlls in the correct place. Check my blog on making leap work with Unity Free or the Leap Motion knowledge base that was written subsequently. Even if you have Pro, you will need to copy 2 dlls to the root directory to have the leap working while in the editor.
          Where did you get up to in the step by step?

          • Hi Pierre, Got it working. The confusing part was copying the dlls to the Assets/Plugins directory. Note to anyone who has the same problem: In Unity Pro, you need to replace the files in Assets/Plugins in the Bootcamp project with files from LeapSDK/Examples/Unity SandBox/Assets/Plugins. Thanks.

  5. Hi Pierre, thanks so much for all the work you’ve put into these Unity/Leap blog posts. I am new to both Unity and Leap Motion but your articles have helped me feel confident in achieving useable results in a reasonable timeframe.

    I’m trying to add Leap functionality to a basic Unity project (rotate a Plane around based on PalmNormal coordinates) and have set up my document just like in this article (without importing the bootcamp package). Here is the code I have placed on my plane object:

    using UnityEngine;
    using System.Collections;
    using Leap;

    public class LeapMove : MonoBehaviour {
    public float speed = 3.0f;

    void Start ()
    {
    }

    void Update ()
    {
    float x = pxsLeapInput.GetHandAxisStep(“Tilt”) * Time.deltaTime * speed;
    float z = pxsLeapInput.GetHandAxisStep(“Rotation”) * Time.deltaTime * speed;
    transform.Translate(x, 0, z);
    }
    }

    The error Unity returns is: UnityException: Input Axis Tilt is not setup.

    Obviously changing the names of different Axes in the InputManager fixes the error but still won’t control the tilting of the plane with the Leap.

    Thanks again for these great articles, keep up the good work!

    • Hi Ryan,

      if you look at the code for GetHandAxisStep, it basically implements its own logic and if the “axis” requested does not have a special implementation, it falls through to the native Input.GetAxis.
      The error you are getting is because the axis “Tilt” is not implemented, although it clearly is in the function… so, perhaps put some Debug.LogError statements in the function to show the axis name that is being received into the function to make sure there are no weird errors…
      BUT… My guess looking at the code is that we are falling to the part of the if statement where Hand==Null… which means that there be another err such as the Leap is disconnected, it is etc, and hence Hand = null and hence the code is falling through to the default handler…

      Have you followed the setup under my post on Getting leap working with Unity (such as copying the appropriate DLLs to the root folder etc?)

      Pierre

      • You were right. I did not have all my Leap files in the correct place. Thanks for verifying, I thought my code was the culprit 🙂

        Thanks again Pierre,

        Cheers!

  6. Pingback: Tutorial: Leap Enabling the Unity 3D Bootcamp Step by Step | Unofficial LEAP Motion blog | Techdemos & Experiments

  7. Hey. Thanx for your great work on this. It’s very useful.
    I just noticed an error in pxsLeapInput.cs at line 187.
    Is now: PalmNormal = m_Hand.PalmPosition.ToUnity();
    Should be: PalmNormal = m_Hand.PalmNormal.ToUnity();

Leave a Reply

Your email address will not be published. Required fields are marked *