VRM AVATAR SYSTEM

Line-by-Line Code Explanation

How a 3D Talking Avatar Works in the Browser

[Link] • @pixiv/three-vrm • WebGL

1. What This File Does
This JavaScript module loads a VRM (Virtual Ready-Made) 3D avatar, renders it inside a browser
<canvas> using WebGL via [Link], and animates it with three states: idle breathing/blinking, bot
speaking with lip sync, and user listening pose. Three global window functions let [Link] control the
avatar.

Component What It Does

Scene The 3D world — an empty container every object lives in

Camera The viewer's eye — defines angle, zoom and clipping range

Renderer GPU-powered painter that draws every frame onto the canvas

VRM Model The 3D avatar with a full skeleton and facial expressions

Animation Loop Runs ~60× per second, updates poses, expressions, redraws

Analogy: A film studio. Scene = the studio room. Camera = film camera on a tripod. Renderer =
projector. VRM = the actor. Animation loop = the film strip running at 60 fps.

2. Imports
import * as THREE from 'three';
Imports the entire [Link] library as the namespace THREE. [Link] wraps raw WebGL (the browser
GPU API) into easy-to-use objects like Scene, Camera, Mesh, Light.
Analogy: WebGL is raw lumber and nails. [Link] is the power tools that let you build without knowing
every nail.

import { GLTFLoader } from 'three/addons/loaders/[Link]';

Imports a file-loader that understands GLTF/GLB/VRM 3D model files — the most popular web 3D
format, used by Blender, Unity, Sketchfab exports, and more.
Analogy: Like importing a PDF reader, but for 3D model files.

import { VRMLoaderPlugin, VRMUtils } from '@pixiv/three-vrm';

VRMLoaderPlugin is a GLTF parser extension that extracts VRM-specific data (humanoid bones, facial
expressions, spring bones). VRMUtils provides helper functions for fixing common VRM quirks.
Analogy: VRM is a dialect of GLTF. VRMLoaderPlugin is the dialect translator. VRMUtils is a spell-
checker for the translated result.

Other 3D Format Options

Format Notes
 GLTF / GLB Best for web — used here. Blender, Unity, Sketchfab all export
it

OBJ + MTL Old, no animation support

FBX Common in games — needs FBXLoader from [Link] extras

COLLADA (.dae) XML-based, mostly legacy now

VRM Anime humanoid avatars by pixiv — extends GLTF

Resources
• [Link] docs: [Link]
• three-vrm GitHub: [Link]
• VRM specification: [Link]

3. Canvas & Scene Setup

const canvas = [Link]('vrm-canvas');
Grabs the HTML <canvas> element — the actual pixel grid where [Link] paints. Without targeting a
canvas, the renderer has nowhere to draw.

const shadowEl = [Link]('model-shadow');

Gets a CSS shadow div placed under the avatar. This is a simple design element (a blurred circle), not
a real 3D shadow — shown and hidden alongside the avatar.

const scene = new [Link]();

Creates the 3D world — an empty container. Every object (lights, model, helpers) must be added to the
scene before it can be rendered.
Analogy: Building an empty theatre stage before putting actors or lights on it.

Camera
const camera = new [Link](44, [Link] /
[Link], 0.1, 20);
Creates a perspective camera — the type that mimics the human eye (far objects look smaller than
near ones). The four arguments are:
Argument Meaning

44 Field of view in degrees — like a 44mm camera lens

innerWidth / innerHeight Aspect ratio — stops image being squished or stretched

0.1 Near clipping plane — objects closer than 0.1 m are invisible

20 Far clipping plane — objects beyond 20 m are invisible

 Analogy: A real camera. FOV = zoom ring. Aspect = portrait vs landscape. Near/far clipping = minimum
and maximum focus distance. Anything outside that range is cut out.

[Link](0, 1.1, 3.8);

Places the camera at X=0 (centred), Y=1.1 m (chest height), Z=3.8 m (in front of the avatar). In
[Link], the Z axis points toward the viewer.

[Link](0, 0.8, 0);

Points the camera lens at coordinate (0, 0.8, 0) — roughly the avatar's upper chest / lower face. This is
where the camera focuses.
Analogy: [Link] = where you put the tripod. lookAt = which direction you aim the lens.

4. Renderer
const renderer = new [Link]({ canvas, alpha: true, antialias: true });
Creates the WebGL renderer — the GPU-powered engine that composites the entire 3D scene into 2D
pixels on the canvas every frame.
Option Effect

canvas Which HTML canvas to paint on

alpha: true Transparent background so CSS / HTML shows behind the

avatar

antialias: true Smooths jagged diagonal edges (costs a small GPU budget)

[Link]([Link]([Link], 2));
Uses the screen's native pixel density for sharp rendering but caps at 2×. Retina displays are 3× but
rendering at 3× triples GPU workload — 2× is the sweet spot.
Analogy: Print resolution. 2× is sharp enough for any screen. 3× is overkill for display use and wastes
GPU.

[Link] = [Link];
Sets colour output to sRGB — the standard colour space monitors expect. Without this, colours appear
washed out and incorrect because the GPU outputs linear light values.

[Link]('resize', () => { ... });

On window resize: updates the camera's aspect ratio and the renderer's pixel dimensions so the 3D
content fills the new size correctly without distortion.
Analogy: A smart TV that automatically readjusts picture ratio when you rotate it.
5. Lighting
[Link] has no default lighting — a scene with no lights is completely black. You manually compose a
lighting rig. This file uses a classic 2-light setup.

[Link](new [Link](0xffffff, 1.6));

Ambient light fills the whole scene equally from every direction with no shadows. 0xffffff = white. 1.6 =
intensity (fairly bright).
Analogy: Overhead office fluorescent lighting — illuminates everything uniformly, creates no shadows.

const key = new [Link](0xffffff, 1.2); [Link](0.5, 2, 2);

The main key light, shining from upper-front-right. Creates highlights that make the face look 3D and
vivid. 'Key' is from professional 3-point lighting terminology.
Analogy: The main spotlight aimed at an actor on a theatre stage. It creates shadows and makes the
face pop.

const fill = new [Link](0xffffff, 0.4); [Link](-1, 0.5,

1);
A weaker fill light from the left softens the harsh shadows the key light creates. Intensity 0.4 = one-third
of the key light.
Analogy: The reflector board a photographer uses during a portrait shoot to bounce light into shadowy
areas.

[Link] Light Types

Type Behaviour

AmbientLight Equal light from all directions — no shadows

DirectionalLight Parallel rays from a direction — like sunlight

PointLight Light from a point in all directions — like a light bulb

SpotLight Cone-shaped beam — like a flashlight or stage spotlight

HemisphereLight Sky colour from above, ground colour from below

6. State Variables
Variable Purpose

vrm = null Holds the loaded VRM model object after loading completes

vrmVisible = false Tracks whether the avatar is currently shown on screen

pendingShow = false vrmShow() was called before model loaded — show it when
ready
 animState = 'idle' Current animation: 'idle' | 'bot' (speaking) | 'user' (listening)

Analogy: A stage manager's checklist — which actor is on stage, are the lights on, what scene are we
in.

7. Loading the VRM Model

const loader = new GLTFLoader(); [Link](parser => new
VRMLoaderPlugin(parser));
Creates the file loader and teaches it VRM by registering the plugin. Without registration, the loader
reads the raw 3D mesh but ignores all humanoid bone assignments and expression data.

[Link]('/static/[Link]', (gltf) => {...}, (p) => {...}, (e) => {...});
Asynchronously downloads and parses the VRM file. Three callback functions handle: success (model
loaded), progress (percentage update), and error (file missing/corrupt).

vrm = [Link];
Extracts the VRM object from the loaded GLTF. The VRMLoaderPlugin stores it at [Link]
after parsing is complete.

VRMUtils.rotateVRM0(vrm);
VRM version 0 models face backwards by default (a legacy quirk from early VRM spec). This utility
rotates the model 180° to face forward. Not needed for VRM 1.0 files.
Analogy: Correcting a photo that was scanned facing the wrong direction.

[Link]([Link]);
Removes skeleton bones that have no influence on any mesh vertex. Fewer active bones = fewer
matrix calculations per frame = better performance.

[Link]((child) => { if ([Link]) [Link] =

false; });
Disables frustum culling on all skinned meshes. Frustum culling hides objects outside the camera view
to save GPU, but avatar limbs can legally move off-screen — disabling prevents them popping out.
Analogy: Normally [Link] hides things outside the camera frame. But a waving arm might go off-
screen briefly — we disable this optimisation so body parts don't vanish mid-gesture.

[Link] = false; [Link]([Link]);

Adds the model to the scene but keeps it invisible until vrmShow() is called.

if (pendingShow) { pendingShow = false; _show(); }

If vrmShow() was called before loading finished, show the model now. pendingShow acts as a 'show
me when ready' deferred flag.
8. Helper Functions
const getBone = name => vrm?.humanoid?.getNormalizedBoneNode(name) ?? null;
Returns the Transform node (position/rotation/scale object) for a named skeleton bone. Optional
chaining (?.) safely returns null if VRM is not loaded yet. You then rotate the returned node to move that
body part.
Analogy: Like asking a puppet master for the string that controls the left arm. He hands you the string —
you pull it to move the arm.

const setExpr = (name, v) => vrm?.expressionManager?.setValue(name, v);

Sets a named facial expression to a value between 0.0 (fully off) and 1.0 (fully on). The
expressionManager blends multiple expressions simultaneously.
Analogy: A slider panel on a puppet's face. Pull the 'aa' slider to 0.8 and the mouth opens 80%. Pull the
'blink' slider to 1.0 and the eyes close fully.

VRM Standard Facial Expressions

Expression Name Effect

aa Open mouth — vowel A shape. Used for lip sync

ih / ou / ee / oh Other vowel mouth shapes for detailed lip sync

blink Both eyes closed simultaneously

blinkLeft / blinkRight Individual eye blink

happy Happy expression (eyebrows raised, slight smile)

sad Sad expression

angry Angry expression

surprised Wide eyes, open mouth

relaxed Soft, calm expression

9. setIdlePose — Default Resting Position

Called once after model load. Rotates specific bones into a natural resting position so the avatar does
not stand in a stiff T-pose.

Bone Rotation Applied

leftUpperArm z = +[Link]/3 (60°) — arm raised out to left side

rightUpperArm z = -[Link]/3 (60°) — arm raised out to right side

leftLowerArm z = +0.2 rad — slight forearm bend

 rightLowerArm z = -0.2 rad — slight forearm bend

leftHand z = +0.1 — natural wrist angle

rightHand z = -0.1 — natural wrist angle

spine x = +0.05 — very slight forward lean

chest x = -0.03 — subtle counter-rotation for realism

head x = +0.05 — subtle downward head tilt

Analogy: A choreographer positioning a dancer before the show. Arms slightly out at 60 degrees, natural
body lean — not a robot T-pose.
[Link] uses radians. [Link] = 180°. [Link]/3 = 60°. [Link]/2 = 90°. Convert: degrees ×
([Link]/180).

10. Animation Functions

doIdle — Breathing & Blinking
function doIdle(dt, t) { ... }
dt = delta time (seconds since last frame, ~0.016 at 60 fps). t = total elapsed seconds since page load.
Called every frame in all three states.

[Link].z = [Link](t * 1.1) * 0.007;

Sways the spine left-right using a sine wave. t*1.1 = speed (1.1 full oscillations per second). 0.007 =
amplitude (barely noticeable, just like a real person standing still).
Key insight: [Link]() produces a smooth -1 to +1 wave, infinitely. Multiplying by small numbers (0.007)
makes motion subtle. Multiplying t by a number controls the speed.

[Link].y = [Link](t * 0.38) * 0.022;

Very slow neck rotation (0.38 cycles/second) creates a subtle natural micro-sway, like a person thinking
or waiting.

blinkTimer += dt; const c = blinkTimer % 3.5;

Accumulates time, uses modulo (%) to repeat a 3.5-second blink cycle. The avatar blinks once every
3.5 seconds.

if (c < 0.06) setExpr('blink', c / 0.06); else if (c < 0.13)

setExpr('blink', (0.13-c) / 0.07); else setExpr('blink', 0);
The blink animation: 0–0.06 s eyes closing (expression ramps 0→1). 0.06–0.13 s eyes opening (ramps
1→0). Remaining 3.37 s: eyes fully open. Total blink = 0.13 s — same as a real human blink.
Analogy: A timer that resets every 3.5 seconds. For the first 0.13 seconds it blinks, then holds open for
the rest. Just like a real person.
doBot — Speaking Animation
function doBot(dt, t) { doIdle(dt, t); lipPhase += dt * 11;
setExpr('aa', [Link](0, [Link](lipPhase) * 0.5 + 0.3) * 0.85);
[Link].x = 0.05 + [Link](t * 4.8) * 0.03; }
Runs while the bot is speaking. Inherits all idle motion and adds rapid lip movement and a head nod.

lipPhase += dt * 11: Advances lip phase at 11 rad/s — rapid oscillation creating a talking pace
[Link](lipPhase) * 0.5 + 0.3: Sine centred at 0.3 with ±0.5 amplitude — mouth bounces between -
0.2 and 0.8
[Link](0, ...): Clamps to zero — mouth cannot go negative (cannot over-close)
* 0.85: Scales peak to 85% open — slightly under fully open for realism
[Link].x = 0.05 + [Link](t * 4.8) * 0.03: Subtle head bob at 4.8 Hz — natural speaking
motion
Analogy: A marionette whose mouth opens and closes while the operator speaks. 11 rad/s creates a
natural-paced talking rate — not too slow, not cartoon-fast.

doUser — Listening Animation

function doUser(dt, t) { doIdle(dt, t); setExpr('aa', 0);
[Link].x = 0.01 + [Link](t * 0.85) * 0.005; }
Called when the user is speaking (bot is listening). Mouth fully closed (aa=0). Spine has a slightly more
forward lean at a slower breathing rate — classic attentive-listening posture.
Analogy: A receptionist leaning slightly forward, mouth closed, attentively listening to a patient describe
their symptoms.

11. The Animation Loop

(function animate() { requestAnimationFrame(animate); ...
[Link](scene, camera); })();
An Immediately Invoked Function Expression (IIFE) that recurses via requestAnimationFrame. Runs
approximately 60 times per second, matching the monitor refresh rate.
Analogy: A film projector — advance one frame, project it, advance the next, project it. 60 times a
second. Each call to animate() is one frame.

const dt = [Link]((now - prevTime) / 1000, 0.05);

Delta time = seconds since last frame (~0.0167 at 60 fps). Capped at 0.05 s (20 fps equivalent) to
prevent huge animation jumps if the tab was backgrounded or the page froze.
Why delta time? Without it, animations run twice as fast on 120 Hz monitors vs 60 Hz. Multiplying
movements by dt makes them frame-rate independent — same speed on all screens.

if (animState === 'bot') doBot(dt, t); else if (animState === 'user') doUser(dt,
t); else doIdle(dt, t);
Routes to the correct animation function based on the current state. State is changed externally by
[Link]().
 [Link]?.update(); [Link](dt);
[Link]() applies all queued expression value changes to the 3D mesh vertices.
[Link](dt) runs VRM physics — spring bones (hair, loose clothing) and other VRM internal
systems.
Analogy: Setting expression values is like queuing puppet string movements on a notepad. .update() is
when the puppet master actually pulls the strings.

[Link](scene, camera);
The final step every frame — instructs the GPU to rasterise the entire 3D scene from the camera's
point of view and paint the result onto the canvas pixels.

12. Public API — Global Functions

Three functions are attached to window so [Link] (in a different module) can control the avatar without
importing this file.

[Link]()
[Link] = () => { if (!vrm) { pendingShow = true; return; } _show();
};
Makes the avatar visible. If VRM has not finished loading yet, sets pendingShow=true so it auto-shows
when loading completes. Call when the user starts a call.

[Link]()
[Link] = () => { pendingShow = false; if (!vrm) return;
vrmVisible = false; [Link] = false;
shadowEl?.[Link]('hidden'); };
Hides the avatar and its CSS shadow. Clears any pending show. Call when the call ends.

[Link](state)
[Link] = (state) => { animState = state; if (state !== 'bot')
setExpr('aa', 0); };
Changes the avatar animation state. When leaving 'bot' state, immediately zeroes the mouth
expression so it does not freeze mid-open.
State When to Use

'idle' Call has just started — bot not speaking or listening yet

'bot' Bot is generating and playing its audio response

'user' User is speaking — bot is actively listening

Integration tip: In [Link], call vrmSetState('bot') inside setBotSpeaking(), vrmSetState('user') inside

setUserSpeaking(), and vrmSetState('idle') inside setActive().
13. Ways to Improve This
Real Audio-Driven Lip Sync
Currently lip sync is a synthetic sine wave. For real audio-driven lip sync, use the Web Audio API
AnalyserNode to read actual audio amplitude and drive the 'aa' expression:
[Link](dataArray); const amp =
[Link]((a,b)=>a+b) / [Link] / 255; setExpr('aa', [Link](1,
amp * 1.5));
This makes lip movement react to actual voice volume in real time.

Emotion Expressions
Add happy/concerned/surprised expressions based on conversation tone. Subscribe to the transcript
SSE stream, detect emotion keywords ('great', 'sorry', 'urgent') and call setExpr('happy', 0.5) etc.

Smooth State Transitions

Instead of instant expression cuts when switching states, lerp (linearly interpolate) between values for
smooth transitions:
currentAA = [Link](currentAA, targetAA, dt * 8); setExpr('aa',
currentAA);

Clinic Background
Add a reception desk and clinic background using a PlaneGeometry with a photo texture, or a 360°
HDRI environment map with [Link].

Resources
Resource Link / Description

[Link] Docs [Link] — complete API reference

[Link] Examples [Link] — 500+ interactive live demos

three-vrm [Link] — VRM library with

examples

VRM Models [Link] — free VRM avatar downloads

VRoid Studio Free Windows/Mac app to create custom VRM avatars

Mixamo [Link] — free character animations (GLTF

export)
[Link] Journey [Link] — the best [Link] learning course