Hand Tracking

Inputs

occludeHandsboolean

Whether hands occlude 3d objects or let everything render over them.

leftHandDefaultJointstring

The default joint that objects are attached to, with the options being: palm,wrist,thumb-0,thumb-1,thumb-2,thumb-3,index-0,index-1,index-2,index-3,mid-0,mid-1,mid-2,mid-3,ring-0,ring-1,ring-2,ring-3,pinky-0,pinky-1,pinky-2,pinky-3,wrist_to_thumb,wrist_to_index,wrist_to_mid,wrist_to_ring,wrist_to_pinky. Note for numbered fingers: 0 = closer to palm and 3 = finger tip.

rightHandDefaultJointstring

The default joint that objects are attached to, with the options being: palm,wrist,thumb-0,thumb-1,thumb-2,thumb-3,index-0,index-1,index-2,index-3,mid-0,mid-1,mid-2,mid-3,ring-0,ring-1,ring-2,ring-3,pinky-0,pinky-1,pinky-2,pinky-3,wrist_to_thumb,wrist_to_index,wrist_to_mid,wrist_to_ring,wrist_to_pinky. Note for numbered fingers: 0 = closer to palm and 3 = finger tip.

Events

onLeftTracking

Called each frame when leftHand is tracking, providing all the hand tracking data in a data object. Example usage: handTrackingComponent.onLeftTracking.add((data)=>{ print(data.joints.thumb1.position2D); // gets 2d screenspace position of thumb1 joint. print(data.position3D); // gets 3d position of the general hand. print(data.rotation3D); // gets z rotation of the general hand, useful for 3d outputs. print(data.rotation2D); // gets z rotation of the general hand, useful for 2d outputs. });

onRightTracking

Called each frame when rightHand is tracking, providing all the hand tracking data in a data object. Example usage: handTrackingComponent.onRightTracking.add((data)=>{ print(data.joints.thumb1.position2D); // gets 2d screenspace position of thumb1 joint. print(data.position3D); // gets 3d position of the general hand. print(data.rotation3D); // gets z rotation of the general hand, useful for 3d outputs. print(data.rotation2D); // gets z rotation of the general hand, useful for 2d outputs. });

onLeftHandShown

triggered when left hand shown

onRightHandShown

triggered when right hand is visible.

onLeftHandHidden

Triggered when left hand is hidden

onRightHandHidden

triggered when right hand is hidden.

onLeftChefsKiss

called if left hand does chefs kiss gesture.

onRightChefsKiss

called if right hand does chefs kiss gesture.

onLeftThumbUp

called if left hand does thumb up gesture.

onRightThumbUp

called if right hand does thumb up gesture.

onLeftPoint

called if left hand does point gesture.

onRightPoint

called if right hand does point gesture.

onLeftPeace

called if left hand does peace sign

onRightPeace

called if right hand does peace sign

onLeftCount1

Called when left hand counts 1

onRightCount1

Called when right hand counts 1

onLeftCount2

Called when left hand counts 2

onRightCount2

Called when right hand counts 2

onLeftCount3

Called when left hand counts 3

onRightCount3

Called when right hand counts 3

onLeftCount4

Called when left hand counts 4

onRightCount4

Called when right hand counts 4

onLeftCount5

Called when left hand counts 5

onRightCount5

Called when right hand counts 5

onLeftPalmOpen

Called when left hand is opened fully.

onRightPalmOpen

Called when right hand is opened fully.

onLeftFistClosed

Called when left hand is closed fully. Useful for grabbing or holding objects with handles. E.g. use when holding a bottle.

onRightFistClosed

Called when right hand is closed fully. Useful for grabbing or holding objects with handles. E.g. use when holding a bottle.

onLeftPinchDown

Called when left hand index and thumb pinch down to pinch.

onRightPinchDown

Called when right hand index and thumb pinch down to pinch.

onLeftPinchUp

Called when left hand index and thumb release from pinching.

onRightPinchUp

Called when right hand index and thumb release from pinching.

onLeftRock

When left hand makes a clench fist.

onRightRock

When right hand makes a clench fist.

onLeftScissors

When left hand makes a scissors gesture.

onRightScissors

When right hand makes a scissors gesture.

onLeftPaper

When left hand makes a paper gesture.

onRightPaper

When right hand makes a paper gesture.

onLeftSwipeLeft

called when left hand swipes to the left.

onLeftSwipeRight

called when the left hand swipes to the right.

onLeftSwipeUp

called when left hand swipes upwards.

onLeftSwipeDown

called when left hand swipes downwards.

onRightSwipeLeft

Called when the right hand swipes left.

onRightSwipeRight

Called when the right hand swipes right.

onRightSwipeUp

Called when the right hand swipes upwards.

onRightSwipeDown

Called when the right hand swipes downwards.

More in Camera, Audio & Sensors

• Device CameraGives your Lens the camera picture, live or frozen as a still. Your Lens can show that picture on a surface, or keep the still to use later.
• Face EventsMakes your Lens react when you open your mouth, smile, kiss, raise your eyebrows or tilt your head. It also tells the Lens where your eyes, nose and mouth are on screen.
• Face MLChanges your face with one ready-made effect, for example an anime look, a meme grin or a skull. Say the look you want, and your Lens keeps that one effect.
• FaceLandmarksFollows where your eyes, nose, mouth and forehead are, so a sticker or text can sit on them and move with you. Ask for hearts on your eyes.
• GyroscopeTurns tilting and shaking your phone into a control for your Lens. Ask for a tilt game, for example a maze where you roll a ball to the exit.
• HapticsMakes your phone vibrate at the moments you choose. Ask for a buzz when you tap the screen or score a point.
• Picture in PicturePuts a frame with the live camera on the screen, with or without a border. Tap the frame to swap in a photo from your device.
• Selfie WidgetShows a small live camera view on the screen, as a circle or a rectangle. Choose where it sits and how big it is, for example in the corner of a game.
• SoundPlays a sound in the Lens. Add the sound you want, and the Lens can start it, pause it and resume it.
• Touch EventsAdds tap, drag and swipe controls, so you touch the screen to control the Lens. Say what each touch does, for example tap to drop a coin.
• Voice ChangerChanges your voice while you speak. Ask for the effect you want, for example a deep voice, a robot, an alien or an echo.
• ZoomZooms the view in on any point of the screen, smoothly or held in place the whole time. Say where to zoom, how close, and whether it repeats back and forth.