About Pierre Semaan

Experienced hands-on Product Management and Marketing executive with a successful track record in product strategy, execution and operations in a global context spanning Americas, Europe, APAC, Middle East and Africa. Executive experience across all stages of the product lifecycle, from analyzing and formulating a market approach, to defining, developing, and delivering great products and on to product positioning, marketing, go to market strategy and finally including field operations and sales. Proven results in Enterprise, SMB and Service Provider markets and solutions, encompassing verticals such as Banking, Financial Services, Telecommunications, Retail, Government and Defence. Strategic change agent with a track record of building successful, high energy and effective teams. Experience in managing cross functional teams that impact the entire organization, including senior leadership experience in Product Management, Product Marketing, Marketing Communications, and R&D.

Aspect ratio switching, Full Screen and Unity

As part of developing my game http://games4leap.com/rock-paper-scissors/, I had to deal with Aspect Ratio switching and going back and forth from full screen to other aspect ratios.

I found this wiki article on how to do this in Unity.
http://wiki.unity3d.com/index.php?title=AspectRatioEnforcer

This worked well for me for switching between aspect ratios in the editor and also at run-time, but I found that switching back and forth from FullScreen was an issue (it seemed that the aspect ratio does not get updated after the 1st switch?).  Anyhow, I modified the code so that it does 2 things I needed.

  1. Switches automatically if the aspect ratio changes
  2. Handles full screen switching well.

The other thing I made is change the backgroundCam.clearFlags to CameraClearFlags.Skybox instead of CameraClearFlags.SolidColor.  This way, you do not get the black rectangles at different aspect rations, but the skybox from the main skybox.  So, if you set the skybox to something matching your background, all is good.

Code Below – Just copy this into an AspectUtility.cs file and attach to your main camera (credit to initial AspectUtility as per link above)

 

using UnityEngine;
public class AspectUtility : MonoBehaviour {
    int lastScreenHeight  = 0;
    int lastScreenWidth  = 0;
    bool lastfullscreen = false;
    public float _wantedAspectRatio = 1.777778f; // (16:9)
    // public float _wantedAspectRatio = 1.3333333f; // 4:3
    public bool landscapeModeOnly = true;
    static public bool _landscapeModeOnly = true;
    static float wantedAspectRatio;
    static Camera cam;
    static Camera backgroundCam;

	void Awake () {
		Screen.fullScreen = true;
		_landscapeModeOnly = landscapeModeOnly;
        cam = camera;
        if (!cam) {
            cam = Camera.main;
			//Debug.Log ("Setting the main camera " + cam.name);
        }
		else {
			//Debug.Log ("Setting the main camera " + cam.name);
		}

        if (!cam) {
            //Debug.LogError ("No camera available");
            return;
        }
        wantedAspectRatio = _wantedAspectRatio;
        SetCamera();
	}

	void Update()
	{
		// check the screen height and witdh
		if ((Screen.height != lastScreenHeight) || (Screen.width != lastScreenWidth) || (Screen.fullScreen != lastfullscreen)) 
		{
			lastScreenWidth = Screen.width;
			lastScreenHeight = Screen.height;
			lastfullscreen = Screen.fullScreen;
			SetCamera();
		}
	}

    public static void SetCamera () {
		float currentAspectRatio = 0.0f;
		if(Screen.orientation == ScreenOrientation.LandscapeRight ||
			Screen.orientation == ScreenOrientation.LandscapeLeft) {
			//Debug.Log ("Landscape detected...");
        	currentAspectRatio = (float)Screen.width / Screen.height;
		}
		else {
			//Debug.Log ("Portrait detected...?");
			if(Screen.height  > Screen.width && _landscapeModeOnly) {
				currentAspectRatio = (float)Screen.height / Screen.width;
			}
			else {
				currentAspectRatio = (float)Screen.width / Screen.height;
			}
		}
        // If the current aspect ratio is already approximately equal to the desired aspect ratio,
        // use a full-screen Rect (in case it was set to something else previously)

		// Debug.Log ("currentAspectRatio = " + currentAspectRatio + ", wantedAspectRatio = " + wantedAspectRatio);

        if ((!Screen.fullScreen) && ((int)(currentAspectRatio * 100) / 100.0f == (int)(wantedAspectRatio * 100) / 100.0f)) 
		{
            cam.rect = new Rect(0.0f, 0.0f, 1.0f, 1.0f);
            if (backgroundCam) {
                Destroy(backgroundCam.gameObject);
            }
            return;
        }

        // Pillarbox
        if (currentAspectRatio > wantedAspectRatio) {
            float inset = 1.0f - wantedAspectRatio/currentAspectRatio;
            cam.rect = new Rect(inset/2, 0.0f, 1.0f-inset, 1.0f);
        }
        // Letterbox
        else {
            float inset = 1.0f - currentAspectRatio/wantedAspectRatio;
            cam.rect = new Rect(0.0f, inset/2, 1.0f, 1.0f-inset);
        }
        if (!backgroundCam) {
            // Make a new camera behind the normal camera which displays black; otherwise the unused space is undefined
            backgroundCam = new GameObject("BackgroundCam", typeof(Camera)).camera;
            backgroundCam.depth = int.MinValue;
            // backgroundCam.clearFlags = CameraClearFlags.SolidColor;
            backgroundCam.clearFlags = CameraClearFlags.Skybox;
            backgroundCam.backgroundColor = Color.black;
            backgroundCam.cullingMask = 0;
        }
    }

	public static int screenHeight {
		get {
			return (int)(Screen.height * cam.rect.height);
		}
	}

	public static int screenWidth {
		get {
			return (int)(Screen.width * cam.rect.width);
		}
	}

	public static int xOffset {
		get {
			return (int)(Screen.width * cam.rect.x);
		}
	}

	public static int yOffset {
		get {
			return (int)(Screen.height * cam.rect.y);
		}
	}

	public static Rect screenRect {
		get {
			return new Rect(cam.rect.x * Screen.width, cam.rect.y * Screen.height, cam.rect.width * Screen.width, cam.rect.height * Screen.height);
		}
	}

	public static Vector3 mousePosition {
		get {
			Vector3 mousePos = Input.mousePosition;
			mousePos.y -= (int)(cam.rect.y * Screen.height);
			mousePos.x -= (int)(cam.rect.x * Screen.width);
			return mousePos;
		}
	}

	public static Vector2 guiMousePosition {
		get {
			Vector2 mousePos = Event.current.mousePosition;
			mousePos.y = Mathf.Clamp(mousePos.y, cam.rect.y * Screen.height, cam.rect.y * Screen.height + cam.rect.height * Screen.height);
			mousePos.x = Mathf.Clamp(mousePos.x, cam.rect.x * Screen.width, cam.rect.x * Screen.width + cam.rect.width * Screen.width);
			return mousePos;
		}
	}
}

Menu GUI options for Leap Motion and Unity3D

I have been playing around with the Leap Motion and Unity3D mainly, and received some positive encouragement from early tutorials I published in this area around getting the Leap to work with Unity standard as well as the Leap enablement of the BootCamp tutorial and the Leap enablement of the Car tutorial.
For the past few months, I have been working on my first commercial application (Paper Scissors Rock) specifically targeted at the Leap Motion controller.
As part of that effort, I have experimented with various types of interaction with Unity. From a gameplay POV, it all depends on your game, and the key thing (for me anyway) is to use the uniqueness of the Leap, either as far as using the 3rd dimension (z axis) or the gestures support.
From a “Menu” support, we enter the realm of 2D, where the interaction (often) is more akin to a touch screen.
Imagine a simple 2D game / screen and you want to push some buttons on the screen. Experiments included:
Experiment 1: Initially, I used the concept of the Z Axis to “touch” button. So you move you hand / finger around and change the mouse cursor as a result, and then to “click”, you move your finger/hand close to the screen.
Pros: very easy to implement. Simply monitor the z axis and if it crosses a “threshold”, you implement the “click” behaviour.
Cons: Usability is not great. I tested this and while I could use it, every other person that tried to use it was confused (because of the lack of tactile feedback, it is tough to know when you are “clicking”.
Mitigation: I initially implemented a “cursor glow” which basically overlays a glow on the cursor when it is in “clicking” range, and also had to implement a “lock-out” function to ensure only 1 click get registered.. But frankly, the usability was just not there.
Experiment 2: Gestures. You can use gestures (either the built in ones or your own) to signify a click.
Pros: Again, easy to implement, you just watch for a gesture and implement the “click” functionality
Cons: 1st, I had problem with the “gesture” impacting the cursor position (as you hand / finger) tends to change ion a gesture. I settled (as an example) for shooting wit played fingers etc. (see the Bootcamp tutorial) which made that quite usable. Same as above, you would need to implement a lock-out (lucky for shooting, it had its own timeout between bullets)
Experiment 3: “Hover Clicks”. This is where you hover over a button, receive feedback (usually by a clockwise animation) and then “click”. This has been discussed very extensively at the Leap Motion forums and theLIFT even provided some free UI elements to help (see here)
Pros: Very usable, I found that the testers loved this interaction
Cons: Can be hard to retrofit existing code / games.
With that last point in mind, I ended up creating a very simple hover click concept, without having to retrofit all my UI elements. To do that, I basically did the following.
I had an existing game with menus that were implemented in Unity3D via using a Raycast.

feedbackhover

Retrofitting existing code in Unity 3D

To “retrofit” hover buttons, I had to essentially add 3 key things:

  1. First I had to add a test raycast to check whether I am on top of something “click-able”
  2. Then I implemented a generic function which keeps track of how long I am hovering on top of a clickable and sets the “hoverClick” variable accordingly
  3. Lastly I implemented a visual feedback in “onGUI.”

Step 1

I modified the Update function of my main script to include the test raycast to check whether I am on top of a “clickable”
Check out the final results if you are interested: http://games4leap.com/rock-paper-scissors/
And some sample code below

// do a test ray to see if we are on top of a clickable
Ray testRay = Camera.mainCamera.ScreenPointToRay(CurrMousePos);
if (Physics.Raycast(testRay, out hit))
{
if (hit.collider.gameObject.GetComponent())
	{
		// The below is specific to my game.
// But basically need to get a unique string for what you are hovering on top of
		buttonHover = hit.collider.gameObject.GetComponent();
		string clickablebutton = buttonHover.id + buttonHover.who + buttonHover.idLevel.ToString();
		// MouseOverClickable keeps track of whether we are on top of the same button
		// as well as starting the timer to see if a hoverclick is initiated.
		MouseOverClickable(clickablebutton);
	}
	else
	{
		// we are not on top off a button, so still call MouseOverClickable, but with an ever changing ID
		MouseOverClickable("");
		clickableElapsedTime = 0F;
	}
}
else
{
	MouseOverClickable("");
	clickableElapsedTime = 0F;
}

 

This is the function that basically keeps track of whether we are hovering on top of

We basically keep track of how long we are hovering the same button and if we are hovering greater than 1 second, then we set a variable (hoverClick to true),.  In the update function of the code, we check for hoverClick and implement that functionality

public  void MouseOverClickable(string ClickableID)
{
	if (ClickableID == "")
	{
		lastClickableID = ClickableID;
		hoverClick = false;
		clickableElapsedTime = 0F;
	}
	else if (ClickableID == lastClickableID)
	{
		clickableElapsedTime = clickableElapsedTime + Time.deltaTime;
		if (clickableElapsedTime > 1)
		{
			// elapsed time is greater than 1 second, so trigger a hoverClick
			hoverClick = true;
			clickableElapsedTime = 0F;
		}
	}
	else
	{
		lastClickableID = ClickableID;
		hoverClick = false;
		clickableElapsedTime = 0F;
	}
}

The code in OnGUI that actually draws the timer functionality

void OnGUI () 
{
	// so we only need to show the "timer glow" if we have the conditions of:
	if (clickableElapsedTime > 0) 
	{
		// Work out which if 12 bitmaps to use (showing progression clockwise)
		int glowTimerIndex = Mathf.CeilToInt(11 * clickableElapsedTime);
		if (glowTimerIndex > 11)
		{
			glowTimerIndex = 11;
		}
		Texture2D cursorGlowImg = CursorGlowArray[glowTimerIndex];
		graphicRect = new Rect(x - (cursorGlowImg.width/2), y - (cursorGlowImg.height/2) , cursorGlowImg.width, cursorGlowImg.height);;
		GUI.DrawTexture(graphicRect, cursorGlowImg);
	}
}

Leap Enabling the Unity 3D Bootcamp Demo – Step by Step + Leap HUD

In this Blog, I take the unity 3D standard demo (The Bootcamp) and I modify it to be Leap Enabled and supply below the step by step instructions for the reader to be able to reproduce this. The motivation behind Leap Enabling Boot Camp was to answer some key questions:

  • Can it be done?
    Answer: YES
  • How hard is it to Leap Enable a FPS or a 3rd Person Shooter?
    Answer: Relative Simple
  • How easy is it to navigate?
    Answer: Depends on your perspective.  The modifications I suggest below will allow you to experiment as well as simply use what I came up with.  I tried some different combinations such as using a single hand to control movement, mouse look as well as shooting.   This worked well in open space, but was not as usable in close quarter. Issues were mainly with unexpected “strafing” when you are trying to change the mouse look only. Ended up fixing this by adding a new function to return leap input as stepped input
  • Single Hand, or Hand and Keyboard for Navigation?
    Answer: Again, the answer is most likely affected by desired gameplay and preference.  I ended up having 4 variables, one for strafing (leap or not) one for forward and backward (leap or not), another for shooting and yet another for mouse look. Can you navigate and shoot with one hand? Sure… The demo shows how that is done.
  • Easiest way to “shoot” while navigating?
    Answer: Tried initially a rotation of the hand from horizontal to vertical to indicate shooting, but that affected the accuracy of the positioning, an undesirable side effect.  I then simply using the act of splaying your fingers slightly – shooting if 4 or more fingers are detected, and that worked quite well.
    Key thing (included in steps below) is to allow shooting via traditional controls as well as Leap, so you can easily switch between themselves
  • Finally, I found it very useful to create a HUD to visualize what the Leap is doing Navigation wise, so I could see if I needed to tweak anything.  See image below with the HUD on the right.

LeapEnabledBootCampImg1

If you are interested in the final result. Check out this Video of what it looks like.

Right… On with the Tutorial then…

Step 1 Create a new project.

I called mine LeapEnabledBootCampMake sure you check the following packages to import “Bootcamp.unitypackage”.
If you do not have this package on your machine, go ahead and download it from the Unity Asset store (it’s free)

LeapEnabledBootCampImg2

Step 2 Make sure that the Bootcamp Scene is working properly

Open the Bootcamp unity scene and run it. Make sure the soldier comes up and you can go forward and backwards and strafe left and right with the arrows, and that you can also change the camera looking angle using the mouse.

Step 3 (Optional) Rename the Bootcamp scene so you can experiment

I renamed mine to LeapEnabledBootcamp

Step 4 Create a Plugins folder under the asset directory

Step 5 Copy the Leap Files from UnityAssets in the SDK to your Plugins Directory and root directory

If you have unity Pro, you can copy the 3 files (Windows) from the Leap UnitySandbox\Assets\Plugins directory to your newly created Plugins director
If you have Unity Free only, then you need to copy Leap.dll and LeapCSharp.dll to the root directory of your game, and LeapCSharp.NET3.5.dll to your Plugins directory.
For more information on getting Unity Standard (free) working with Leap, read my Blog (http://pierresemaan.com/getting-the-leap-to-work-with-unity-free-version-not-pro/ )

Step 6 copy LeapUnityExtensions.cs to the Plugins folder

LeapUnityExtensions.cs is in the Assets\Scripts directory from the examples in the SDK under UnitySandbox.

Step 7 copy pxsLeapInput.cs to the Plugins folder

Download pxsLeapInput.cs and copy the script to the Plugins folder.  The Script has to be in Plugins as we need to reference it from a javascript script later and it needs to be compiled first (so the javascript engine can see it)

Step 7 a (optional) examine the pxsLeapInput Script

Note this is a refinement of my previous pxsLeapInput.  It includes some key enhancements such as:
1      Added support for Mouse x and Mosue y axis
2      If Hand is null, pxsLeapInput will get the default input axis (so, you can move your hand away from the Leap and work with the game as per usual).
3      Added a function GetHandGesture for firing.  Splayed fingers = fire, rotate hand clockwise = aim.
4      Added a function GetHandAxisStep which returns a “clipped” or “stepped”

private static float GetHandAxisPrivate(string axisName, bool scaled)
{
​// Call Update so you can get the latest frame and hand
​Update();
​float ret = 0.0F;
​if (m_Hand != null)
​{
​​// SNIP
​​case "Mouse X":
​​​// rotation is preferred (more usable).  
​​​// if using rotation for something else,
​​​// can use Palmdirection by uncommenting below.
​​​ret = ret = -2 * PalmNormal.x ;
​​​// ret =  4 * (PalmDirection.x ) + 1.00F ;
​​​m_Errors = "ret Mouse.x = " + ret.ToString();
​​​break;
​​case "Mouse Y":
​​​// use z axis as this is the most pronounced
​​​// change when TITLTing a hand
​​​​ret =  2 * PalmNormal.z ;
​​​​// m_Errors = "ret Mouse.Y = " + ret.ToString();
​​​​break;
​​// SNIP
​}
​else
​{
​​// Hand is Null, so return the standard axis
​​switch (axisName)
​​{
​​case "Depth":
​​​// depth for leap = vertical axis
​​​ret = Input.GetAxis("Vertical") ;
​​​break;
​​default:
​​​// return the axis name from input if hand is null
​​​// and the special cases above do not apply
​​​ret = Input.GetAxis(axisName) ;
​​​break;
​}
​}
​return ret;
}

And

public static float GetHandAxisStep(string axisName)
{
​// Assume dead zone of -0.5 to 0.5
​// always return -1, 0 or +1
​float ret = GetHandAxisPrivate(axisName, true);
​if (ret < -0.5F) ​{ ​​ret = -1.0F; ​} ​else if (ret > 0.5F)
​{
​​ret = 1.0F;
​}
​else
​{
​​ret = 0;
​}
​return ret;
}

Note that it is a Classic class (not Monobehavior).  It’s a singleton and includes properties to easily get Frame and Hand from the latest Update.

Step 8 Modify SoldierController.js script to allow for Leap Input

In the Bootcamp demo, we have the soldier movement (forward, backward, sideways) in SoldierController.js.
As usual, we setup some variables to indicate whether we use LeapEnabled input
Note that with my new script, where it falls through to normal Axis behaviour if the Leap hand is null, this may seem redundant, but the main motivation is to be able to easily enable and disable individual axis from Leap Control to “play” with best setting for the Leap.  After all, this is a tutorial.

public var leapEnabledVerticalAxis : boolean = false;
public var leapEnabledHorizontalAxis : boolean = false;
public var leapEnabledFire : boolean = false;
public var hideMouse : boolean = false;

Next, we find where we use Input.GetAxis and modify these to use Leap if Leap is enabled.

// moveDir = new Vector3(Input.GetAxis("Horizontal"), 0, Input.GetAxis("Vertical"));
// pxsLeap
var x : float = 0;
var z : float = 0;

if (leapEnabledHorizontalAxis == true)
{
	x = pxsLeapInput.GetHandAxisStep("Horizontal");
}
else
{
	x = Input.GetAxis("Horizontal");
}

if (leapEnabledVerticalAxis == true)
{
	z = pxsLeapInput.GetHandAxisStep("Depth");
}
else
{
	z = Input.GetAxis("Vertical");
}

Note that when we modify the fire behavior, we make sure you can still fire using the normal inputs

if (leapEnabledFire == true)
{
​fireLeap = pxsLeapInput.GetHandGesture("Fire1") && weaponSystem.currentGun.freeToShoot && !dead && !inAir;
}
//Check if the user if firing the weapon
fire = fireLeap || (Input.GetButton("Fire1") && weaponSystem.currentGun.freeToShoot && !dead && !inAir);

Step 9 Modify SoldierCamera.js script to allow for Leap Input

Now we modify the SoldierCamera script to accept Leap input.
We setup a variabl at the top of the script

public var leapEnabled : boolean = false;

We then modify the input.  Note that there are two places to modify.  One which controls the camera and another area where there is logic to re-set the idle timer when there is no movement.  Don’t forget to do both.  I ignored the Orbit logic and found the camera orbiting as it thinks the soldier is idle.

if (leapEnabled == true)
{
​// Make sure we reset the idle timer if we are moving via Leap, otherwise, we enter into Orbit Mode with funny results
​if(orbit && (Input.GetKeyDown(KeyCode.O) || pxsLeapInput.GetHandAxisStep("Horizontal") != 0.0 || pxsLeapInput.GetHandAxisStep("Depth") != 0.0 || soldierController.aim || soldierController.fire))
​{
​​GoToOrbitMode(false);
​}
}
else
{
​if(orbit && (Input.GetKeyDown(KeyCode.O) || Input.GetAxis("Horizontal") != 0.0 || Input.GetAxis("Vertical") != 0.0 || soldierController.aim || soldierController.fire))
​{
​​GoToOrbitMode(false);
​}
}

Also

if (leapEnabled == true)
{
​x += Mathf.Clamp(pxsLeapInput.GetHandAxis("Mouse X") * a.x, -maxSpeed.x, maxSpeed.x) * deltaTime;
​y -= Mathf.Clamp(pxsLeapInput.GetHandAxis("Mouse Y") * a.y, -maxSpeed.y, maxSpeed.y) * deltaTime;
​// print (pxsLeapInput.Errors);
}
else
{
​x += Mathf.Clamp(Input.GetAxis("Mouse X") * a.x, -maxSpeed.x, maxSpeed.x) * deltaTime;
​y -= Mathf.Clamp(Input.GetAxis("Mouse Y") * a.y, -maxSpeed.y, maxSpeed.y) * deltaTime;
}

Step 10 (Optional) Create a Leap Navigation HUD to visualize Leap Input

As you will see from the video recording, I found it useful to create a HUD to visualize Leap Input.
The steps to do so are listed here briefly, as I will create a separate blog tutorial to detail the exact steps to keep everything separate, also because the HUD is not 100% coupled with the Leap (conceptually anyway). Steps were:
. Create an empty GameObject
. Attach a script HudLeap to it
. The script has some texture variables, so we can then drag bitmaps we create (I used PowerPoint to create mine) onto these textures
. The HudLeap has an OnGUI
. OnGUI uses the pxsLeapInput to get the leap inputs we are using to navigate
. We then draw the representation on screen
Check out the HudLeap.js script for the details.

Step 11 Enable Leap Input

Select the Soldier in the Scene and in the SoldierController.js script component in the inspector, select the checkbox to make it Leap Enabled.
Select the “Soldier Camera” in the Scene and in the SoldierCamera.js script component in the inspector, select the checkbox to make it Leap Enabled.

Step 12 Play

If you play the scene now, you will note that you can control the car with your hand.
Note that you can see the effect live with the Leap HUD.
LeapEnabledBootCampImg4

Leap Enabling the Unity3D Car Tutorial

In this Blog, I take a Tutorial from unity 3D (The Car Racing Tutorial) and I modify it to be LeapEnabled and supply below the step by step instructions for the reader to be able to reproduce this.

Files and Resources Needed.

If you are interested in the final result. Check out this Video of what it looks like.

Step 1 Create a new project. 

I called mine LeapEnabledCarTutorial
Make sure you check the flowing packages to import “Car Tutorial.unitypackage”.  If you do not have this package on your machine, go ahead and download it from the Unity Asset store (it’s free) 

step1

Step 2 Make sure that the Complete Scene is working properly

Open the CompleteScene unity scene and run it. Make sure the car comes up and you can go forward with the up arrow and steer with the left and right arrow

 step2

Step 3 Duplicate the Complete Scene so you can experiment

Choose edit.. Duplicate Scene

This will create a duplicate of the CompleteScnee called CompleteScene 1

I renamed mine to LeapCompleteScene

step3

Step 4 Create a Plugins folder under the asset directory

Step 5 Copy the Leap Files from UnityAssets in the SDK to your Plugins Directory and root directory

If you have unity Pro, you can copy the 3 files (Windows) from the Leap UnitySandbox\Assets\Plugins directory to your newly created Plugins director
If you have Unity Free only, then you need to copy Leap.dll and LeapCSharp.dll to the root directory of your game, and LeapCSharp.NET3.5.dll to your Plugins directory.
For more information on getting Unity Standard (free) working with Leap, read my Blog (http://pierresemaan.com/getting-the-leap-to-work-with-unity-free-version-not-pro/ )

step5astep5b

Step 6 copy LeapUnityExtensions.cs to the Plugins folder

LeapUnityExtensions.cs is in the Assets\Scripts directory from the examples in the SDK under UnitySandbox.

Step 7 copy pxsLeapInput.cs to the Plugins folder

Download pxsLeapInput.cs (see the top of the Tutorial for all downloads) and copy the script to the Plugins folder.  The Script has to be in Plugins as we need to reference it from a javascript script later and it needs to be compiled first (so the javascript engine can see it)

Step 7 a (optional) examine the pxsLeapInput Script

Note that it is a Classic class (not Monobehavior).  It’s a singleton and includes properties to easily get Frame and Hand from the latest Update.

The key function is GetHandAxisPrivate

private static float GetHandAxisPrivate(string axisName, bool scaled)
{
// Call Update so you can get the latest frame and hand
Update();
float ret = 0.0F;
if (m_Hand != null)
{
  Vector3 PalmPosition = new Vector3(0,0,0);
  Vector3 PalmNormal = new Vector3(0,0,0);
  Vector3 PalmDirection = new Vector3(0,0,0);
  if (scaled == true)
  {
    PalmPosition = m_Hand.PalmPosition.ToUnityTranslated();
    PalmNormal = m_Hand.PalmNormal.ToUnity();
    PalmDirection = m_Hand.Direction.ToUnity();
  }
  else
  {
    PalmPosition = m_Hand.PalmPosition.ToUnity();
    PalmNormal = m_Hand.PalmPosition.ToUnity();
    PalmDirection = m_Hand.Direction.ToUnity();
  }
  switch (axisName)
  {
  case "Horizontal":
    ret = PalmPosition.x ;
    break;
  case "Vertical":
    ret = PalmPosition.y;
    break;
  case "Depth":
    ret = PalmPosition.z ;
    break;
  case "Rotation":
    ret = -2 * PalmNormal.x ;
    break;
  case "Tilt":
    ret = PalmNormal.z ;
    break;
  case "HorizontalDirection":
    ret = PalmDirection.x ;
    break;
  case "VericalDirection":
    ret = PalmDirection.y ;
    break;
  default:
    break;
  }
}
if (scaled == true)
{
  if (ret > 1) {ret = 1;}
  if (ret < -1) {ret = -1;}
}
return ret;
}

I tried to make the interaction with the Leap as similar as possible as interactions that Unity has with other inputs.  So, in Unity, you would normally call Input.getAxis (axisName) to get the mouse, or arrow key buttons etc… So, I created similar

Step 8 Modify car.js script to allow or Leap Input

Ok, now for the final step.  We need to modiy the car.js script from the Racing Ttorial to allow for Leap Input.  This is where our design for GetHandAxis function will come in handy as it wil be super easy to modify car.js

1st, add a variable at the top of the script somewhere to hold a value as to whether the script is LeapEnabled (note the default s false)

// pxs modify for Leap
// store a variable to indicate whether we are using Leap as the input here
public var LeapEnabled : boolean = false;

Next find the function GetInput()

Note the 1st 2 lines in the function

throttle = Input.GetAxis("Vertical");
steer = Input.GetAxis("Horizontal");

Replace these with the following

// pxs Modify for Leap
if (LeapEnabled == true)
{
  throttle = pxsLeapInput.GetHandAxis("Depth");
  print("Throttle: " + throttle.ToString());
  steer = pxsLeapInput.GetHandAxis("Rotation");
  print("Steer: " + steer.ToString());
}
else
{
  throttle = Input.GetAxis("Vertical");
  steer = Input.GetAxis("Horizontal");
}

Run the LeapCompleteScene and note that it still works (with arrow keys etc)

Step 9 Final Step – Enable Leap Input

Select the Car in the Scene and in the car.js script component in the inspector, select the checkbox to make it Leap Enabled.

step9 

Step 10 Play

If you play the scene now, you will note that you can control the car with your hand.

Forwards and backwards is from eth hand position on top of the Leap (so towards the screen is forward for the car, away from the screen is backwards)
And the hand rotation is used to steer

step10

Enjoy

Getting the Leap to work with Unity (Free Version not Pro)

I am lucky enough to have been accepted into the Leap Developer Program. If you do no know what the Leap is, then check out www.leapmotion.com

I have some ideas as to how to use this interface (definitely not as replacement to a touch screen, but something that can use the 3rd dimension since it has “depth”)  Anyway, in order to experiment,  I decided to experiment using a gaming engine and settled on unity3d (www.unity3d.com) as it seems to be quite well supported.

Here started my dilemma  Unity3D has 2 versions, a free one (for companies making less than 100K a year and with some limitations) and a Pro one (quite reasonable price, 1.5K). But, to just experiment, I wanted to use the Free one and not have to shell out 1.5K.

The problem is that in the Free version of Unity, you are not allowed to use a native DLL (you can however use .net managed DLLs).

So, with Leap supporting C# and .net, I thought it would be quite simple to get the free version working… and indeed it was… But, noone seems to have blogged the exact steps, and all posts I have seen, just hint that this is possible without the actual step by steps to how to do it.

Hence… (Drum Roll please)… below are the step by step introductions on how I got it working (really simple after all).

1st, I am running Windows 7, 64Bit and version 0.74 of the SDK (downloaded this morning)

Step 1 – Make sure you have a Leap, and have downloaded the SDK and your leap is working – Run the Leap Visualizer to make sure that the leap is working well.

Step 2 – Locate the Unity Assets folder.  This is located under [SDK Path]\LeapSDK\lib\UnityAssets\Plugins (there are 3 files there – Leap.dll, LeapCShare.dll and LeapCSharp.NET3.5.dll

Step 3 – Locate and make a copy of the Unity Sandbox folder.  [SDK Path]\LeapSDK\Examples\UnitySandbox

Step 4 – Copy the Unity Sandbox folder somewhere else, where we will experiment. I called mine UnityFreeSandbox

Step 5 – Run the UnitySandbox.exe already built under the UnitySandbox folder.  It should run and you show be able to use your leap to interact with the Cube (note the picture below showing that it can detect by finger).

workingUnitySandbox

Step 6 – Now, open the UnitySandbox project within Unity itself. (in UniySandbox folder, Go to subfolder Assets\Scenes folder and open UnitySandbox scene).

Step 7 – Now inside Unity Editor, run the program.  You will note that it will work, but you will get errors logged in the console “License error – plugins are only supported in Unity Pro” (see screen shot below)

WorkingwithinUnitybutwitherrors

Step 8 – Now, within the Unity Editor… Go to Assets\Plugins and you should see 3 files (Leap.dll, LeapCShare.dll and LeapCSharp.NET3.5.dll).  DELETE the 1st 2, so you are only left with LeapCSharp.NET3.5.dll there).

Step 9 – Run the program within Unity and you will note that it will NOT work and the console shows an error (see below)
DllNotFoundException: LeapCSharp
Leap.LeapPINVOKE+SWIGExceptionHelper..cctor ()

NotWorkingwithinUnityFree

Step 10 – Now copy 2 of the 3 files from UnityAssets folder (see step 2) [SDK Path]\LeapSDK\lib\UnityAssets\Plugins , namely, Leap.dll and LeapCShare.dll to the root folder where you copied UnitySandbox – See Step 4 (I called mine UnityFreeSandbox)

Step 11 – Run your project and you should see that it runs, and works with no errors in the console.

Step 12 – Build your project and create an EXE, if you place it in the root folder UnityFreeSandbox where your dlls are, it will work with no problems.  If you want to deploy it, you will need to ensure that the 2 Dlls copied in Step 10 are deployed with your EXE (in the same folder would be my choice)

Step 13 – Enjoy and let me know how you get on with Leap and Unity.