COMP5517 Human Computer Interaction

09. Interactive Computer Graphics

Overview

2
What is Computer
Graphics?

M.C. Escher – Smaller and Smaller (1956)

3
What is Computer
Graphics?
 Creation, Manipulation, and Storage of
geometric objects (modeling) and their
images (rendering)
 Display those images on screens or
hardcopy devices
 Image processing (2D graphics)
 3D computer graphics
 Others: GUI, Haptics, Displays (VR)...

4
Why computer graphics?

 Fun! Visible!
 Everywhere
 Visual system offers:
 Parallel input
 Parallel processing
 Computer graphics: ideal for human-computer
communication

5
Applications
 Graphs and Charts  Computer Art
 Computer-Aided Design  Movies
 Virtual Reality  Games
 Data Visualization  Graphical User Interfaces
 Education and Training (GUI)

6
What drives computer
graphics?
 Movie Industry
 Leaders in quality and artistry
 Not slaves to conceptual purity

 Big budgets and tight schedules

 Reminder that there is more to

CG than technology
 Hey, How’d they do that?

 Defines our expectations

7
Movies
expression depth of field

fracture

motion
water

reflection hair
8
What drives computer
graphics?
 Game Industry
 The newest driving force in CG
 Why? Volume and Profit
 This is why we have commodity GPUs
 Focus on interactivity
 Cost effective solutions
 Avoiding computing and other tricks
 Games drive the baseline

9
Computer Graphics for
Gaming

Super Mario Run

Super Mario Bros.

10
What drives computer
 Medical Imaging and Scientific Visualization
graphics?
Tools for teaching and diagnosis
 No cheating or tricks allowed
 New data representations and modalities
 Drive issues of precision and correctness
 Focus on presentation and interpretation of data
 Construction of models from acquired data

Nanomanipulator, UNC Joe Kniss, Utah Gordon Kindelman, Utah

11
Medical Imaging & Scientific
Visualization

Bruckner and Groeller,

Holten, TU/e, 2007
TU Vienna, 2007

12
What drives computer
graphics?
 Computer Aided Design (CAD)
 Mechanical, Electronic, Architecture,...
 Drives the high end of the hardware market
 Integration of computing and display resources
 Reduced design cycles = faster systems, sooner

ProEngineer, [Link] 13
Computer Aided Design
(CAD)

14
Computer Aided Design
(CAD)

 AutoDesk
 IAME 2-stroke race kart engine
15
What drives computer
graphics?
 Graphic User Interfaces (GUI)
 [Link]

16
What is Computer
Graphics?
 Look at 5 areas
 Hardware
 Rendering
 Interaction
 Modeling
 Scientific Visualization

17
What is Computer
Graphics?
 Look at 5 areas
 Hardware
 Rendering
 Interaction
 Modeling
 Scientific Visualization

18
Hardware: Amazing
Changes
 Fundamental architecture shift
 Dual computing engines:
 CPU and GPU
 More in GPU than CPU

19
Hardware: Amazing
Changes
 Fast, cheap GPUs
 ~HK$5000
 Cheap memory
 Displays at low cost (LCD screen)
 How many monitors do you have/use?

20
LCD Display (Special LCD)

21
LCD Display (Build Your Own Special
LCD)

22
Beyond the laptop screen
 24 screen configuration, Virginia Tech

• Gigapixel display
• 50 LCD touchscreens

23
Beyond the laptop screen
 Microsoft Surface

• Apple iPad

24
Beyond the laptop screen

Head mounted displays

Parachute trainer
US Navy

25
Beyond the laptop screen

 Roll-up screen, Philips

26
Relation of Graphics Model
& Image

27
Schematic

Model Display Image

interaction

User

28
Also…

Computer
Graphics

Model Image

Pattern
Recognition

Image
Processing

29
From model to image

Model World View NDC Display

Graphics pipeline

Coordinates and transformations

30
From model to image

Model World View NDC Display

Cylinder:
x 2  y 2 r 2
0  z h
Local or modeling coordinates
Geometric modeling
31
From model to image

Model World View NDC Display

Position cylinders in scene:

World coordinates

32
From model to image

Model World View NDC Display

Look at cylinders:

Viewing coordinates

Visible surfaces, shading

33
From model to image

Model World View NDC Display

Display:
1
Normalized Device Coordinates

0 1
34
 From model to image

Model World View NDC Display

Display on screen:
0
1024 Device Coordinates

768
Interaction
35
Generating graphics

 Special-purpose programs
 Photoshop, Powerpoint, AutoCAD, StudioMax, Maya,
Blender, PovRay, …

 General graphics libraries and standards

 Windows API, OpenGL, Direct3D,…

36
What is Computer
Graphics?
 Look at 5 areas
 Hardware
 Rendering
 Interaction
 Modeling
 Scientific Visualization

37
Rendering

 Many think/thought graphics synonymous

with rendering
 Well researched
 Working on second and third order effects
 Fundamentals largely in place

38
Rendering

 Major areas:
 Ealiest: PhotoRealism
 Recent: Non-Photorealistic Rendering (NPR)
 Recent: Image-Based Rendering (IBR)

39
Rendering

 Ray Tracing has become practical

 Extremely high quality images
 Photorealism, animation, special effects
 Accurate rendering, not just pretty

40
Ray Tracing

41
Ray Tracing

42
Ray Tracing

43
Ray Tracing

44
Rendering Realism

Morning Evening
45
Rendering Realism

Cornel Measurement Lab

46
Specular Reflection vs. Diffuse
Reflection

47
Rendering Realism

Real Synthetic

Shirley, et. al., cornell

48
Is this real?

m fajaro, usc
49
Terrain Modeling:
Snow and Trees Added

s premoze, [Link]., utah

50
Rendering/Modeling Hair

51
Humans

Final Fantasy (Sony) Jensen et al. 52

Is Photorealism
Everything?

53
Is Photorealism
Everything?

54
Non-Photorealistic
Rendering (NPR)

b gooch, [Link]., utah

55
Tone Shading

56
Non-Photorealistic Rendering
(NPR)

57
Image-Based Rendering
(IBR)
 Model light field
 Do not have to model geometry
 Good for complex 3D scenes
 May leave holes where no data is available

58
3D Scene Capture

Fuchs [Link]., UNC

UNC and UVA 59
3D Scene Recreation

60
360o Scan

p willemsen, et. al., utah

61
What is Computer
Graphics?
 Look at 5 areas
 Hardware
 Rendering
 Interaction
 Modeling
 Scientific Visualization

62
Interaction

 Way behind rest of graphics spectacular

advances
 Still doing WIMP:
 Windows, icons, menus, pull-downs/pointing
 Once viewed as “soft” research
 Turns out to be one of hardest problems

63
Interaction still needs...
 Better input devices
 Better output devices
 Better interaction paradigms
 Better understanding of HCI
 Bring in psychologists

Human-Computer Interaction

64
What is Computer
Graphics?
 Look at 5 areas
 Hardware
 Rendering
 Interaction
 Modeling
 Scientific Visualization

65
Modeling
 Many model reps
 Bezier, B-spline, box splines, simplex splines,
polyhedral splines, quadrics, super-quadrics,
implicit, parametric, subdivision, fractal, level
sets, etc (not to mention polygonal)

66
Modeling

 Physically-Based Modeling
 Newton
 Behavior as well as geometry
 Materials Modeling
 Metal, cloth, organic forms, fluids, etc.
 Procedural (growth) modeling

67
Modeling... is hard

 Complexity
 Shape
 Specifying
 Realistic constraints
 Detail vs concept
 Tedious, slow

s drake, et. al., utah

68
Modeling is hard

 Mathematical challenge
 Computational challenge
 Interaction challenge
 Display challenge (want 3D)
 Domain knowledge, constraints

69
Growth (Procedural)
Modeling

70
Growth (Procedural)
Modeling

71
Model Capture

72
Models

D Johnson and
J D St Germain, Utah

Russ Fish et al., Utah

73
What is Computer
Graphics?
 Look at 5 areas
 Hardware
 Rendering
 Interaction
 Modeling
 Scientific Visualization

74
Scientific Visualization

National Library of Medicine

Visual Human

75
Others
 Rasterization, modeling, viewing, lighting,
texture mapping and ray tracing
 GUI and Interaction in three-dimensions

76
Thank You for Listening!