Leap Enabling the Unity3D Car Tutorial

In this Blog, I take a Tutorial from unity 3D (The Car Racing Tutorial) and I modify it to be LeapEnabled and supply below the step by step instructions for the reader to be able to reproduce this.

Files and Resources Needed.

If you are interested in the final result. Check out this Video of what it looks like.

Step 1 Create a new project. 

I called mine LeapEnabledCarTutorial
Make sure you check the flowing packages to import “Car Tutorial.unitypackage”.  If you do not have this package on your machine, go ahead and download it from the Unity Asset store (it’s free) 

step1

Step 2 Make sure that the Complete Scene is working properly

Open the CompleteScene unity scene and run it. Make sure the car comes up and you can go forward with the up arrow and steer with the left and right arrow

 step2

Step 3 Duplicate the Complete Scene so you can experiment

Choose edit.. Duplicate Scene

This will create a duplicate of the CompleteScnee called CompleteScene 1

I renamed mine to LeapCompleteScene

step3

Step 4 Create a Plugins folder under the asset directory

Step 5 Copy the Leap Files from UnityAssets in the SDK to your Plugins Directory and root directory

If you have unity Pro, you can copy the 3 files (Windows) from the Leap UnitySandbox\Assets\Plugins directory to your newly created Plugins director
If you have Unity Free only, then you need to copy Leap.dll and LeapCSharp.dll to the root directory of your game, and LeapCSharp.NET3.5.dll to your Plugins directory.
For more information on getting Unity Standard (free) working with Leap, read my Blog (http://pierresemaan.com/getting-the-leap-to-work-with-unity-free-version-not-pro/ )

step5astep5b

Step 6 copy LeapUnityExtensions.cs to the Plugins folder

LeapUnityExtensions.cs is in the Assets\Scripts directory from the examples in the SDK under UnitySandbox.

Step 7 copy pxsLeapInput.cs to the Plugins folder

Download pxsLeapInput.cs (see the top of the Tutorial for all downloads) and copy the script to the Plugins folder.  The Script has to be in Plugins as we need to reference it from a javascript script later and it needs to be compiled first (so the javascript engine can see it)

Step 7 a (optional) examine the pxsLeapInput Script

Note that it is a Classic class (not Monobehavior).  It’s a singleton and includes properties to easily get Frame and Hand from the latest Update.

The key function is GetHandAxisPrivate

private static float GetHandAxisPrivate(string axisName, bool scaled)
{
// Call Update so you can get the latest frame and hand
Update();
float ret = 0.0F;
if (m_Hand != null)
{
  Vector3 PalmPosition = new Vector3(0,0,0);
  Vector3 PalmNormal = new Vector3(0,0,0);
  Vector3 PalmDirection = new Vector3(0,0,0);
  if (scaled == true)
  {
    PalmPosition = m_Hand.PalmPosition.ToUnityTranslated();
    PalmNormal = m_Hand.PalmNormal.ToUnity();
    PalmDirection = m_Hand.Direction.ToUnity();
  }
  else
  {
    PalmPosition = m_Hand.PalmPosition.ToUnity();
    PalmNormal = m_Hand.PalmPosition.ToUnity();
    PalmDirection = m_Hand.Direction.ToUnity();
  }
  switch (axisName)
  {
  case "Horizontal":
    ret = PalmPosition.x ;
    break;
  case "Vertical":
    ret = PalmPosition.y;
    break;
  case "Depth":
    ret = PalmPosition.z ;
    break;
  case "Rotation":
    ret = -2 * PalmNormal.x ;
    break;
  case "Tilt":
    ret = PalmNormal.z ;
    break;
  case "HorizontalDirection":
    ret = PalmDirection.x ;
    break;
  case "VericalDirection":
    ret = PalmDirection.y ;
    break;
  default:
    break;
  }
}
if (scaled == true)
{
  if (ret > 1) {ret = 1;}
  if (ret < -1) {ret = -1;}
}
return ret;
}

I tried to make the interaction with the Leap as similar as possible as interactions that Unity has with other inputs.  So, in Unity, you would normally call Input.getAxis (axisName) to get the mouse, or arrow key buttons etc… So, I created similar

Step 8 Modify car.js script to allow or Leap Input

Ok, now for the final step.  We need to modiy the car.js script from the Racing Ttorial to allow for Leap Input.  This is where our design for GetHandAxis function will come in handy as it wil be super easy to modify car.js

1st, add a variable at the top of the script somewhere to hold a value as to whether the script is LeapEnabled (note the default s false)

// pxs modify for Leap
// store a variable to indicate whether we are using Leap as the input here
public var LeapEnabled : boolean = false;

Next find the function GetInput()

Note the 1st 2 lines in the function

throttle = Input.GetAxis("Vertical");
steer = Input.GetAxis("Horizontal");

Replace these with the following

// pxs Modify for Leap
if (LeapEnabled == true)
{
  throttle = pxsLeapInput.GetHandAxis("Depth");
  print("Throttle: " + throttle.ToString());
  steer = pxsLeapInput.GetHandAxis("Rotation");
  print("Steer: " + steer.ToString());
}
else
{
  throttle = Input.GetAxis("Vertical");
  steer = Input.GetAxis("Horizontal");
}

Run the LeapCompleteScene and note that it still works (with arrow keys etc)

Step 9 Final Step – Enable Leap Input

Select the Car in the Scene and in the car.js script component in the inspector, select the checkbox to make it Leap Enabled.

step9 

Step 10 Play

If you play the scene now, you will note that you can control the car with your hand.

Forwards and backwards is from eth hand position on top of the Leap (so towards the screen is forward for the car, away from the screen is backwards)
And the hand rotation is used to steer

step10

Enjoy

39 thoughts on “Leap Enabling the Unity3D Car Tutorial

  1. thank you for an amazing tutorial !! i am using mac and i was trying to build this scene, but after compilation is done, the resulting application file was not launching, any ideas ?

    • Hi Khaled
      On the Mac, there are differences to which assets to use and where to place the files for Leap.
      Check out this knowledge base article that Joe Ward put together on the subject – https://developer.leapmotion.com/articles/creating-leap-apps-with-unity-standard-license
      In all cases, I would try a step by step approach.
      for example, if you disable LeapEnabled in the car.js script, then it will not call pxsLeapInput and you can check that it works ok with the keybaord… if it does, the issue is definitely in your leap enablement… you will notice that I store an error message in the line Errors = Errors + ‘\r’ + ‘\n’ + “Controller could not be created”; Make that variable public so you can print it out on the console in car.js using print(pxsLeapInput.Errors) and you can get at least visibility of what is going wrong, but I suspect Joe’s article may solve it for you.
      Hope that helps
      Pierre

  2. Thank you for the tutorial. I`m making a scene where both hands should appear. Is there any way I can make both hands appear in the scene?

    • I wonder if it is possible to get input data for a second hand using the code above. I tried adding a second hand ID, which is hand[1] instead of hand[0], to the script. But it doesn`t seem to work properly. As a beginner in this field, I`m still a little hazy on a few techinical details regarding Leap technology. Any help would be greatly appreciated. Thank you.

      • Hi Jungsu Lee. It certainly is possible… A couple of different appraoched come to mind.
        Current code takes the 1st hand all the time.
        You could modify to have a getlefthandaxis and get a righthandaxis.
        As well as checking for the existence of a hand in the frame, you would need to add some logic as to when is it the left or the ight, such as with the position of the x coordinate.
        1. check the palm position and if it is negative, assume it is the left hand and if it is positive, it is the right.
        2. You have to check that there are indeed 2 hands in the scene. (If you are for example always requiring 2 hands). Check that handlist.count > 1

        It depends on what you are trying to achieve really.

  3. Hey, Thats a Great tutorial up there.. thanks
    The Unity3D Car Tutorial also includes C# script for controlling the car I was wondering if i could directly integrate the leap code into C# script which controls the car instead of Js ???

    • Hi Vivek,
      You can absolutely use the C# scripts. I wrote all the leap enablement side in C#, I just chose to integrate with the javascript version as this was the default in the CompletedScene that I downloaded.
      Integrating with teh C# should be just as simple or easy.
      have you tried it yet?

      Pierre

      • Thank you for your reply, i did try to modify c# script to receive leap inputs, the problem the car goes out of control the car . Since c# script uses Input.GetKey instead of Input.GetAxis as in JS, i replaced it with roll value of leap (ie if i tilt my hand to the left the roll angle is negative and if i tilt to the right the roll angle will be positive) so if the roll value is negative the car should move the left and the other way round for positive roll values.
        please help me out here ..

        //THE ORIGINAL SCRIPT
        if (Input.GetKey (KeyCode.LeftArrow))
        steerInput = -1;
        if (Input.GetKey (KeyCode.RightArrow))
        steerInput = 1;

        //LEAP ENABLED SCRIPT

        if(pxsLeapInput.GetRoll()10f)
        steerInput = 1;

        • Hi Vivek

          Not sure what GetRoll does..
          from my original code posted,
          you should be able to simply use the below
          If the steering is the wrong direction, simply multiply by -1
          so change

          steerInput = pxsLeapInput.GetHandAxis(“Rotation”);
          to
          steerInput = -1 * pxsLeapInput.GetHandAxis(“Rotation”);

          code which should work
          // pxs Modify for Leap
          if (LeapEnabled == true)
          {
          steerInput = pxsLeapInput.GetHandAxis(“Rotation”);
          }
          else
          {
          if (Input.GetKey (KeyCode.LeftArrow))
          steerInput = -1;
          if (Input.GetKey (KeyCode.RightArrow))
          steerInput = 1;
          }

  4. hi Pierre,

    I follow your post but did not succeed. I am using a Unity3D 4.1 Pro. Do you think whether this will be a problem? Should it be 3.5? What is the version you are using?

    Terry

    • Hi Terry
      I used 4.0 when I wrote the tutorial, so should not be a problem.
      Did you try downloading the full source code from GitHub and running it?
      What issues are ou finding?

  5. Hi Pierre,
    That was an awesome tutorial. Even as a newbie, I’m able to get it work. I wonder if there is any way to stop the car when the hand is removed? currently the car goes backwards. Thanks in advance.

    • Thanks for the kind words Gnana.
      I have implemented a “HandLost” function in my projects after this post… basically, I have a timer (usually 1/2 sec) variable and I increment this with each frame. I then check the frame for frame.hands().count and if the count s >0, I reset the timer. if the timer is > 1/2 second, my HandsLost() returns true
      I usually use HandsLost to initiate a PAUSE function in the game.

      • Thanks for the reply Pierre, as told in the post, I’m completely newbie to this stuff, so I’d be enlightened if you could give more information about where to include this functions. Thanks.

        • I’ve implemented, thanks a lot for your advice. I’ve done as follows on pxsLeapInput.cs.
          I’ve added a stopwatch and added new function to check if more than 1 second hands are lost.
          // Adding the following function to check if hands are lost – GSi – 29072013
          public static bool HandLost()
          {
          if(stopWatch.ElapsedMilliseconds > 1000)
          {
          handLost = true;
          }
          else
          {
          handLost = false;
          }
          return handLost;
          }

          Then updated the update() function as follows.
          public static void Update()
          {
          if( m_controller != null )
          {

          Frame lastFrame = m_Frame == null ? Frame.Invalid : m_Frame;
          m_Frame = m_controller.Frame();
          if (m_Frame != null)
          {
          if (m_Frame.Hands.Count > 0)
          {
          // If pause, resume the game – GSi – 29072013
          if(UnityEngine.Time.timeScale == 0)
          {
          UnityEngine.Time.timeScale = 1;
          }
          m_Hand = m_Frame.Hands[0];
          stopWatch.Stop();
          stopWatch.Start();
          }
          // Adding the following code to pause the game if hands are lost – GSi – 29072013
          else
          {
          if(HandLost())
          {
          UnityEngine.Time.timeScale = 0;
          stopWatch.Stop();
          }
          }
          }
          }
          }

          Is there any better way to do this? Thanks.

    • Skyler,
      sounds like you cannot compile your project, what steps have you tried already (such as deleting folders except the key ones etc)… Looks like you are getting an access permission issue?
      Does it compile without the leap part?
      Pierre

  6. Hi Pierre,

    When I reach the end of step 8 I encounter an error which tells me:
    Assets/Plugins/pxsLeapInput.cs(92,68): error CS1061: Type `Leap.Vector’ does not contain a definition for `ToUnityTranslated’ and no extension method `ToUnityTranslated’ of type `Leap.Vector’ could be found (are you missing a using directive or an assembly reference?)

    I’m using Unity 4.1.2f1. I’m rather new to scripting so any help would be appreciated. Thanks!

    Edwin

  7. I am so glad I found this tutorial. But I got stuck on step 8. I kept getting compilation error said ” Unknown identifier: pxsLeapInput”, I am using unity3d free version, and I wasn’t able to figure out how to compile script in Plugin folder. Can you help me?

    Thanks

  8. Hi Pierre its amazing tutorial, but i have problem when i add two hand, i follow your reply comment to Jungsu Lee, its your reply:

    “Hi Jungsu Lee. It certainly is possible… A couple of different appraoched come to mind.
    Current code takes the 1st hand all the time.
    You could modify to have a getlefthandaxis and get a righthandaxis.
    As well as checking for the existence of a hand in the frame, you would need to add some logic as to when is it the left or the ight, such as with the position of the x coordinate.
    1. check the palm position and if it is negative, assume it is the left hand and if it is positive, it is the right.
    2. You have to check that there are indeed 2 hands in the scene. (If you are for example always requiring 2 hands). Check that handlist.count > 1

    It depends on what you are trying to achieve really.”

    my problem :
    – i get dtetc two hand, but the control for car its just 1 hand and its hand first detect, so if i use two hand the car its not work preffectly

    can u help me for this problem, please… 🙂

    • Hi, So you can get 2 hands detected, but now you want to control the car using 2 hands.
      Are you looking at steering the car like a normal car (pretending you are holding a steering wheel?) if so, you need to look at the function pxsLeapInput.GetHandAxis(“Rotation”)
      Note that the current implementation uses

      case “Rotation”:
      ret = -2 * PalmNormal.x ;
      break;

      PalmNormal is basically the vector from the middle of your palm perpendicular to the your palm. so, by looking at the X value, I was getting a rotation of the palm value (to steer). BTW, since I wrote the code, Leap Motion introduced Yaw as a property of the palm, so doing the same ting today, I would use that.
      BUT… Back to you… if you want to use it as a steering wheel, I would compare the Y value of the left palm versus the Y value of the right Palm and return a rotation accordingly. so if the right palm is lower that he left, you are steering right and vice versa.

      Be careful, though… As I use the palm Z axis position to go forward or backwards. If you use both hands to steer, how can you control forward and backwards motion?

      My last comment on this… We have seen implementations such as Google Earth Leap Motion implementation, and they have used a similar scheme to my original one (one hand control) versus games such as the Disney Sugar Rush car racing, which I personally found very hard to control as it keeps loosing one hand or another…. (Subtly saying 2 hands to control a car is very old fashioned thinking, and with the Leap you can afford to break physical mapping paradigms :-))

  9. Hi, in step 9 i can’t see Enable Leap Input , so when i run the game, it didn’t link leap motion. But I set step 1 to step 8 is OK!!

    How can I link Leap ??

    • Hi Sing Fei. The Leap Enabled checkbox comes from the fact that you created a variable in Step8’called LeapEnabled that is public.
      // pxs modify for Leap
      // store a variable to indicate whether we are using Leap as the input here
      public var LeapEnabled : boolean = false;

  10. Hi, Pierre. I followed your tutorial and your code works well but it is hard for me to control the car to behave well enough.
    First , it is hard to turn directions when the car at a high speed. Second, the leap motion detect the hand position too accurate, it keeps back and forth when I need to pull the car back and turn its direction, how can I make a better user experience?

    • Hi Michael,

      You can add a “dead zone” in the throttle, to make sure you only act on significant Z position of the hand.
      Something like below (doing it from memory, so hope i get syntax correct)
      throttle = pxsLeapInput.GetHandAxis(“Depth”);
      with
      throttle = pxsLeapInput.GetHandAxis(“Depth”);
      if (Mathf.Abs(throttle) < 0.25) { throttle = 0; }

Leave a Reply

Your email address will not be published. Required fields are marked *