A Prime Example of HCI Application
Unity3D
[Link]
Yingcai Xiao
Unity3D
Engine
IDE
Visual Programming
Assets
Tutorial
Examples
Installation
Download at [Link]
Free for students and company < $100,000/y
Supporting multi-platforms.
Create your first Unity3D project
Getting started with a simple Unity3D project
Start Unity3D
New Project (3D or 2D)
File->Save Scene
Font size too small?
1. Right-click on the desktop
2. Select “Display settings”
3. “Change the size of text”
4. Default to 125%, change to 175%
for Unity IDE presentations
Getting started with a simple Unity3D
project
Assets->Import Packages
- Characters
- Particle Systems
- Vehicles
Select components before importing each package.
Object Creation
GameObject->3D object->sphere
Edit->Frame Select (to show the created object)
Hold onto the arrows to move the sphere or change
the position in the Inspector.
Note: Y is up.
Inspector and transform panel
Moving, translation, rotation and scaling tools
Unity 3D: Component
Select Created Game Object->Component->Physics-
>Rigidbody
• Rigid Objects: non-deformable with physical properties
(gravity, inertial).
• Non-rigid Objects:
• Deformable: changeable geometry
• Breakable: changeable topology.
• Intangible Objects: No predefined shape.
fire, clouds, …
Animation
Window-
>Animation
Click on the object
to be animated.
Component-
>Miscellaneous-
>Animation
Animation
Create New Animation Clip
Click on the record button (red) at the top-left.
Save the new animation clip file.
Click on Curves.
Select Add Property->transform position, rotation
or scaling
Animation
Add key frame on the timeline as many as you want.
Click on the red button again to finish making the
animation clip.
Animation (Unity5)
Add key frame on the timeline as many as you want.
Click on the red button again to finish making the
animation clip.
Animation
Go back to Unity window.
Under Inspector Animation , change the
name of animation clip from none to the
one that you have made.
Animation Event
Allows you to call functions in the object’s
script at specified points in the timeline.
Add a new Animation Event by double-
clicking the Event Line or by using the
Event button.
Animation Event
When you add an event, a dialog box will
appear to prompt you for the name of the
function and the value of the parameter you
want to pass to it.
Unity 3D: Light
• Game Object ->Light -> Directional Light (Default)
• Move and rotate just like any other object.
Terrain
• GameObject 3D Object Terrain
• In the hierarchy panel, select Terrain.
• In the Inspector: set x = -5, y = 0, z = -5.
• Click on one of the action icons in Terrain (Script) to
raise/lower terrain, paint height, smooth height, paint
texture, place trees, paint details, terrain setting. Adjust
brush size to 1 before performing the operations.
• Paint Texture: Edit Texture -> Select
Terrain
To place trees, you have to build tree first (Game Object-
>3D object ->Tree), then choose“Edit Trees -> Add
Trees” first to add different types of trees.
In the Add Trees popup window, you need click on the
little circle at the right-most of the “Tree” row.
Select, say, Palm and then click on “Add” in the “Add
Tree” window.
Go back to the Inspector, select “Mass Place Trees” from
available “Trees” to add.
Physics
Unity3D provide Physics library.
- Rigidbody, collider, joint, force and etc.
Rigidbody component : gravity automatically added
Collision detection : box, sphere, capsule , mesh, whelle
and terrain.
Collision call back function : OnCollisionEnter,
OnCollision and OnCollisionExit.
Player (for a third-person game)
• Make sure to save the scene “File->Save Scene”
(Ctrl S”) and save the project
“File->Save Project”
• Next, we need add the player.
• In the Project window, drag“Standard Assets-
>Character Controllers->3rd Person Controller”
to the Hierarchy window.
• In the Hierarchy window, double-click on the 3 rd
Person Controller.
• Click on the “Move selected object” icon. Then
move the controller to the top of the terrain. You may
have to adjust your view angle by clicking on the
‘xyz” icon to see the position. Click the middle of
the icon to get the perspective view.
Unity 3D: Play
• Now click on the “Play” icon and use
the arrow keys to controll the player.
• You should see the player running
around and make sure he does not run
off the edge.
Under the Hood
HCI Programming Essentials
EDP: Event Driving Programming
[Link] generators (input devices)
[Link]
[Link] loop
[Link] mapping (compile time)
[Link] dispatching (run time)
[Link] handlers
[Link]
[Link]
HCI Programming in Unity
1. Event generators (input devices)
Unity takes input from almost all input devices:
CLI: keyboard
GUI: mouse, touch screen
NUI: Kinect, LeapMotion, Smart Phones
HCI Programming in Unity
2. Events
2.A. Virtual Events: Horizontal/Vertical Movements
2.B. Software Events: Collision
2.C. Idle Event: for background activities
2.D. User defined events: at any key points of
computation.
HCI Programming in Unity
3. Event loop
Unity takes care of it.
3.A. At coding time: visual programming
3.B. At application run time: API libraries built into
the application.
HCI Programming in Unity
4. Event mapping (compile time)
4.A. At coding time: visual programming
4.B. At application run time: can be done, but no API
yet.
HCI Programming in Unity
5. Event dispatching (run time)
At application run time: done by unity built-in
runtime libraries.
Visual Programming:
Events distributed by the Unity IDE to
[Link] when coding
[Link] when simulating (play)
[Link] when in the debugging mode.
HCI Programming in Unity
6. Event handlers
6.A. Written by the application programmer.
6.B. Built-in by the Game Engine (Physics Engine, AI
Engine, …)
6.C. Imported as part of the prefabricates.
HCI Programming in Unity
7. Computation
7.A. Written by the application programmer.
7.B. Built-in by the Game Engine (Physics Engine, AI
Engine, …)
7.C. Imported as part of prefabricates.
HCI Programming in Unity
8. Output
8.A. Audio: sound tracks
8.B. Video: animated graphics
8.C. Haptic: force feedback
HCI Programming in Unity
8.B. Graphics Output
8.B.1. Local: OpenGL
8.B.2. Remote: WebGL
8.B.3. Others: XR: VR/AR/MR
HCI Programming Essentials
EDP: Event Driving Programming
[Link] generators (input devices)
[Link]
[Link] loop
[Link] mapping (compile time)
[Link] dispatching (run time)
[Link] handlers
[Link]
[Link]
HCI Programming Essentials
EDP: Event Driving Programming
Project: directory and files for a specific
game project.
C:\Users\xiao\Documents\New Unity
Project 1
\Assets (anything you can reuse)
\Library (binary files)
Unity 3D IDE
IDE: Integrated Development Environment
Project: directory and files for a specific
game project.
C:\Users\xiao\Documents\New Unity
Project 1
\Assets (anything you can reuse)
\Library (binary files)
Unity 3D: Assets
C:\Users\xiao\Documents\New Unity
Project 1\Assets (anything you can reuse)
\Standard Assets
\OpenNI
\Scripts
\_Scenes
\Materials
\Artwork
Unity 3D: Standard Assets
C:\Users\xiao\Documents\New Unity Project 1\
Assets\Standard Assets
Objects: (Look)
\Tree
\Terrain
\Charater
Lights: (Look)
\Light Flares
\Light Cookies
Code: (Feel: control, interaction, animation, …)
\Scripts
Unity 3D: Objects
C:\Users\xiao\Documents\New Unity Project 1\
Assets\Standard Assets\Charater:
Prefab: (Predefined Objects)
First Person, 3rd Person
\Source:
\Prototype (Look)
[Link]
\Materials (properties)
\Textures (images)
\Scripts (Feel: actions)
Java Scripts: [Link]
C#: [Link]
Unity 3D: Scripts
Languages:
Interpreted : Java Script
Compiled: C#
Usages:
General: under Project\Scripts
[Link]
Objects: attached to objects
[Link]
Unity 3D: Library
cashe:
for speeding up processing
metadata:
data that describes data
previews:
for previewing scenes
ScriptAssemblies:
compiled object assemblies for scripts
Resources for 3D model
[Link]
Autodesk Maya
[Link]
k-maya/overview
Blender
[Link]