PLANNING PHASE OUTPUT

SYSTEM DESCRIPTION & BACKGROUND

There currently exists no publicly available, industry standard solution for realistic lighting in 2d
animation as well as 2d motion graphics. The current pipelines and processes that do exist to
apply realistic lighting effects in 2d animation tend to require a heavy amount of planning in
preproduction and be incredibly labor intensive. This holds especially true in post-production,
particularly for compositing artists. This system: RALS (Robust animation and lighting system)
aims to be an open-source solution to the problem of creating volumetric, realistic and detailed
lighting in 2d animation. RALS offers a variety of customizable and custom-tailored solutions to
create realistic lighting effects, for a wide range of surfaces, lighting situations.
Problem/Opportunity:

Currently existing solutions for lighting in 2d animation are essentially nonexistent with all
currently existing solutions and pipelines consisting of makeshift manual lighting simulation
manually done by artists via heavy compositing in post-production and planning in pre-
production. Many productions as a result don’t even consider light and shadow in their pipeline
with characters and objects being depicted with flat colors with no use of light to indicate form.

The ones that do consider light and shadow often resort to crude and simplistic depictions of
lighting, with lights and shadows often being depicted with discrete singular colors without the
blending or soft gradation that often occurs in real life lighting situations. Some productions may
go further with basic specular highlights, blurred shadows and procedurally generated rim
lighting via compositing but that tends to be about it with any further lighting being too labor
intensive to be realistic for most conventional animated productions.

Scope:
RALS is a lighting system solely intended for lighting 2d animated scenes in post-production.
RALS does not cover audio, 3D or live action post processing.

Objectives:

1) RALS provides artists with an intuitive, user-friendly GUI for lighting/relighting and 2d
scenes.
2) RALS is made to be easily integrated into currently existing postproduction pipelines as
well as smooth and efficient use with currently existing compositing suites.
3) RALS is expected to dramatically cut down on time required for lighting in 2d animation
through automation of tedious tasks required for high fidelity lighting.
4) RALS shall support a hybrid workflow with integration of both procedural and manual
lighting setups.
Expected value:

Through the creation of this system, I expect to see the creation of more specialized roles for
lighting in postproduction for animation as well as a more efficient distribution of labor
resources during postproduction in animation studios. We expect the system to facilitate a more
streamlined efficient postproduction pipeline for artists such as compositors, effects artists,
lighting artists with regards to the creation of high-fidelity lighting in 2d animation.

We expect the system to radically cut down on the time it takes to light and relight scenes in 2d
animation as well as deliver higher quality results than would be possible with conventional
lighting techniques traditionally seen in 2d animation. Another benefit of the system is that it
allows postproduction artists to work more focused jobs with deeper specializations rather than
the jack of all trades style work patterns generally seen in a lot of studios. This mode of work
tends to be particularly prevalent in smaller studios and studios working with harsh time
constraints. RALS being open source, also enables studios to create their own versions to better
fit into their specialized production pipelines.

Feasibility summary:

1) Technical feasibility: I plan to build RALS by utilizing already existing systems and
graphics API’s such as (C++/Python core/Vulkan/OpenGL) and leverage open-source
graphics libraries such as ffmpeg, for video IO and encoding and decoding. Libraries
such as ffmpeg also enables me to work with a variety of different video codecs and
formats when rendering out the final frames.
I also plan to leverage research on monocular depth estimation for the development of the
automated depth map generation feature. I also plan on the integration of DPT models in
RALS for better depth estimation Based on all components of relying on existing mature
technologies with extensive support ecosystems I judge RALS to be technically feasible.
2) Economic feasibility: RALS will be funded primarily by the RALS foundation. An
independent nonprofit organization that aims to provide funding and support for RALS
development costs. The RALS foundation will be funded primarily by donations from
corporate sponsorships from GPU vendors, graphics tablet manufacturers, tech and media
giants, companies such as adobe, Nvidia, Ubisoft, Epic games, Activision, Ubisoft, Intel
etc. The RALS foundation will also be funded by other complementary revenue streams
such as RALS studio subscriptions.
RALS studio is a learning platform that aims to offer in-depth courses, project files, live
workshops and Resources for RALS users. The platform will charge both annual and
monthly subscription fees. RALS also aims to make profits from merchandise sales, i.e.
RALS Lamps, RALS pens, RALS mugs etc. Both the RALS platform and merchandise
sales begin immediately upon launch with corporate sponsorships secured before release.
The development of RALS will be handled by a small team of developers/designers paid
 through funding secured by the RALS foundation deposited. Analyzing the revenue
streams and general cost structure of RALS. I judge RALS to be economically feasible.
3) Operational feasibility: RALS is designed to be seamlessly integrated into most
animation postproduction pipelines and compositing suites i.e. DaVinci resolve, adobe
after effects, nuke compositing, fusion etc. It is also made with an incredibly intuitive
user interface to facilitate effective navigation. RALS autosaves project files in the .rals
format as a measure against system crashes. RALS provides intuitive easy to follow
courses and programs via the RALS studio online platform. These courses can be used to
train staff and they come with project files to practice using RALS. Using the provided
materials on the RALS studio platform, users can expect to gain intermediate to advanced
mastery of the software within a week.

Based on this, I believe RALS to be operationally feasible, economically feasible and technically
feasible.

ANALYSIS PHASE OUTPUT

Fact finding and requirements gathering

In order to make sure RALS was perfectly catered to the needs of animation studio’s I sent out
multiple surveys and questionnaires on forums centered around art and animation from multiple
online communities. This included polycount, blender forums, Blackmagic forums, artstation,
adobe community. I observed currently existing postproduction pipelines and walkthroughs to
deduce common pain points and design around them. I also had several interviews with multiple
different professional composting artists to get a general idea of the post-production compositing
workflow.

Based on my findings, I was able to outline a system proposal bringing together all the essential
system requirements necessary for RALS and presenting a clear idea of how RALS ought to
function in a real operational environment.
Functional requirements

During my finding facts and requirement gathering, I gathered some essential requirements that
RALS needs to be able to perform:

1) Enable users to create, layer, group, name and organize different lighting setups atop each
other
2) Allow artists to manually create rim lighting, core shadows, ambient lighting, specular
highlights, via Bezier curves with multiple controls and settings.
3) Allow artists to animate Lighting shapes and movement on objects via keyframed Bezier
curves that track object motion both in real time and batch time.
4) Allow artists to track and adjust keyframes via timeline
 5) Allow artists to automatically generate lighting via proxy model generation, using
automatic depth map generation/painting on imported image sequences and videos.
6) Allow artists to import and export a variety of file formats into the software for relighting
including MP4, MKV, AVI, PNG, JPG, JPEG, GIF, TIF, TIFF, WEBM, WEBP, NOL,
APNG, MOV, SPRITESHEET, TGA, EXR, SGI
7) Enable artists to combine manual lighting generated via depth map or proxy model with
manually keyframed animated lighting.
8) Allow artists to control the behavior of different lighting systems based on the material
the light is interacting with. i.e. matte surfaces, rough surfaces, translucent surfaces,
transparent surfaces, iridescent surfaces, skin, foliage, general environment.
9) Allow artists to isolate individual light groups to avoid dealing with management of the
complex multipoint diffuse lighting that often happens in real life since lights in real life
often interact with other lights. Spotlights interact with ambient lights, Specular
highlights interact with subsurface scattering and so on.
Non-functional requirements

Supplementing the core functionality of RALS, I also considered the non-functional

requirements to ensure it’s a viable part of any post-production pipeline.

1) The system’s interface shall be easy to navigate and intuitive

2) The system shall autosave every 15 minutes to assist in backing up data after crashes.
3) The system shall support rendering on mid-range gpu’s and integrated GPU’s
4) The system should be able to run on mid-to-high-range hardware
5) The system should be compatible with windows, macOS and Linux
6) The system should be able to work with a multitude of different video codecs for
rendering
7) The system size shall not take more than 4gb of space
8) The system shall support OpenColorIO for consistent color space management.
These nonfunctional requirements make RALS not only useful but enjoyable for whoever is
using it.

User and system interactions

RALS is generally designed with one user in mind: The lighting artist. The job of the lighting
artist is to light the entirety of the shot. The lighting artist should be able to import scenes in the
necessary file format (primarily image sequences or video), draw light spots and shadow spots
onto characters with Bezier curves as well as animate said Bezier curves with tracked keyframes.
The lighting artist should also be able to paint or automatically generate depth maps for character
lighting.
The lighting artist should also be able to integrate the characters/objects with the environment or
the rest of the shot after lighting has been applied to the focus of the shot by the lighting artist.
When done with the lighting process, the compositing artist should also be able to render
completed scenes as image sequences or videos to be exported to other compositing software for
color correction, video effects etc.

USE CASE DIAGRAM

This diagram is a very high-level overview of the core functionality of RALS. It illustrates the
relationship between RALS and the lighting artist as well as the various ways in which the
lighting artist may interact with the system. The lighting artist imports the image sequence from
a location on their computer. Once the image sequence is imported, the lighting artist inserts a
light setup layer. When they try and create the light setup layer, the program displays a popup
box with an entry field for the layer name. Once inputted, the lighting artist selects the type of
material being lit. They can choose either between either a standard matte material preset or
custom materials and presets e.g. iridescent surfaces, translucent surfaces, skin etc. When the
artist chooses the material, they choose between either a Bezier keyframe based lighting setup
with manually selected keyframes or a procedurally generated depth map for every frame on the
timeline. If the layer stack is too cluttered, the lighting artist can select and group as well as
isolate light groups. When done lighting the scene. The artist may export the finished project file.
If they so choose, they can manually modify export settings or select custom output settings.

USE CASE DESCRIPTIONS

The following are textual descriptions of the many USE cases found in the RALS use case
diagram
UC-1 Import image sequence
Item Description
Primary actor Lighting artist
Import image sequence into project for
Goal lighting/relighting.
Project is opened and valid file path is chosen for image
Pre-conditions sequence to be imported from
This use case occurs when the lighting artist navigates to
the file path to import image sequence/video or uses
Trigger special key binds to import files.
1) Lighting artist selects import file from file tab
2) System opens file explorer.
3) Artist navigates to folder where image sequence
or video is stored
4) Artist selects media directory
5) They click import
6) Dialog boxes pop up showing successful import.
Main flow 7) Image sequence is now visible on timeline
Image sequence is cached in the .rals project files and
Post-conditions then made available for lighting operations

UC-2 Create light setup layer

Item Description
Primary
actor Lighting artist
Goal Creation of a layer for lighting
Pre-
conditions Image sequence/ video has been imported onto timeline
Trigger This use case is initiated when the lighting artist selects “create new light setup” in the layer view
1) Artist navigates to layer stack and clicks on new light setup.
2) System prompts artist for layer name as well as the type of lighting system and type of material
being lit.
3) Artist enters name and selects type of lighting system, as well as the type of material being lit.
Main flow 4) The light setup is created on the timeline
Post- Layer stack updated with new light setup layer
conditions

UC-3 Insert standard matte setup

Item Description
Primary actor Lighting artist
Insert default material behavior when creating light
Goal setup
This use case is initiated when the lighting artist
navigates to the settings in the light setup window to
Trigger select matte material
1) Artist navigates to layer stack.
2) Artist clicks new light setup.
3) System prompts artist for layer name as well as
the lighting mode and material.
4) Artist selects matte material
5) Artist confirms their choice and closes setup
Main flow window.
Matte lighting setup layer is displayed on layer stack and
Post-conditions timeline

UC-4 Insert custom material setup

Item Description
Primary actor Lighting artist
Insert default material behavior when creating light
Goal setup
This use case is initiated when the lighting artist
navigates to the settings in the light setup window to
Trigger select matte material
1) Artist navigates to layer stack.
2) Artist clicks new light setup.
3) System prompts artist for layer name as well as
the lighting mode and material.
4) Artist selects custom material
5) Artist confirms their choice and closes setup
Main flow window.
Custom lighting setup layer is displayed on layer stack
Post-conditions and timeline
UC-5 Create depth map
Item Description
Primary actor Lighting artist
Create depth map to inform light behavior on
Goal objects and characters.
Artist chose depth based lighting when creating
Pre-conditions light setup
This use case occurs when the lighting artist selects
depth based light generation when creating their
Trigger light setup.
1) Artist navigates to layer stack.
2) Artist clicks new light setup.
3) System prompts artist for layer name as
well as the type of lighting mode and
material.
4) Artist selects auto depth or paint depth.
Main flow 5) Artist confirms their choice.
Post-conditions Lighting mode is confirmed

UC-6 Batch render depth map <<include>>

Item Description
Primary actor Lighting artist
Goal Create depth map for the entire shot
Artist chose depth based lighting setup when
Pre- conditions creating light setup layer
This use case occurs when the artist inserts a depth
Trigger based lighting setup layer on the timeline.
1) Artist navigates to layer stack.
2) Artist clicks new light setup.
3) System prompts artist for layer name as
well as the lighting mode and material
4) Artist selects auto depth or paint depth.
5) Artist chooses either and paints or the
system generates depth map procedurally
6) System displays progress window showing
render progress
7) System displays prompt when done
Main flow rendering depth map
The depth-based light setup layer is now visible on
Post-conditions the timeline and layer stack.
UC-7 Insert manual light keyframes
Item Description
Primary actor Lighting artist
Creating keyframed lighting with Bezier curves to be
Goal tracked over the entire shot.
Artist chose keyframe based lighting in light setup
Pre- conditions window.
This use case occurs when the artist confirms
Trigger keyframe-based lighting in light setup window.
1) Artist navigates to layer stack.
2) Artist clicks new light setup.
3) System prompts artist for layer name as
well as the lighting mode and material
4) Artist selects keyframe based lighting.
5) Keyframe based light setup layer is
created.
6) Artist selects two points on the timeline
and draws lights on individual frames with
Bezier curves with Bezier curves
Main flow 7) System automatically interpolates motion
Keyframes are now visible on the timeline and in
the main viewing panel. Artist can scrub through to
Post-conditions view and tweak light tracking

UC-8 Realtime preview Bezier keyframes

Item Description
Primary actor Lighting artist
Having a Realtime view on lighting changes made
Goal with Bezier curves
Artist chose keyframe based lighting in light setup
Pre- conditions window.
This use case occurs automatically when the artist
inserts a keyframe based lighting setup onto the
Trigger timeline.
1) Artist navigates to layer stack.
2) Artist clicks new light setup.
3) System prompts artist for layer name as
well as the lighting mode and material
4) Artist selects keyframe based lighting.
5) Keyframe based light setup layer is
created.
6) Artist inserts keyframes
7) Artist scrubs timeline for Realtime
Main flow playback
Post-conditions Interpolated layers now visible in Realtime

UC-9 Batch render Bezier keyframes

Item Description
Primary actor Lighting artist
Generating a tracked path of interpolated Bezier
Goal curve based light keyframes for the whole shot.
Artist inserted keyframe based lighting in light setup
Pre- conditions window.
This use case occurs when the artist toggles batch
rendering instead of Realtime on the main view
Trigger panel.
1) Artist navigates to layer stack.
2) Artist clicks new light setup.
3) System prompts artist for layer name as
well as the lighting mode and material
4) Artist selects keyframe based lighting.
5) Keyframe based light setup layer is
created.
6) Artist toggles batch rendering on the main
view panel
7) The artist selects two points on the timeline
to draw Bezier fill curves over the focus of
the shot
8) Artist selects “Render>>Bezier pass”.
9) The system automatically iterates through
Main flow frames and applies interpolation
Post-conditions Rendered sequence viewable on timeline

UC-10 Group light setup layers

Item Description
Primary actor Lighting artist
Grouping together multiple light setup layers into a
Goal singular collapsible group
Pre- conditions Artist has selected layers for grouping
Artist selects multiple layers from layer stack or
Trigger timeline and groups them
1) Artist navigates to layer stack.
2) Artist selects n number of layers
3) Artist clicks ctrl + g
4) Layers get grouped together.
Main flow 5) Artist names layer group.
Multiple different layers now viewable in a
Post-conditions collapsible group
UC-11 Isolate light setup layers
Item Description
Primary actor Lighting artist
Isolating selected light setups and making
Goal everything else invisible
Pre- conditions Artist has selected layers for isolation
Artist selects multiple layers from layer stack or
Trigger timeline and groups them
1) Artist navigates to layer stack.
2) Artist selects n number of layers
3) Artist clicks ctrl + h
4) Layers are automatically made the only
Main flow ones in view.
Post-conditions Only selected layers now visible

UC-12 Select light setup layers

Item Description
Primary actor Lighting artist
Only make selected layers available for arbitrary
Goal operations
Artist has selected layers for arbitrary uniform
Pre- conditions operations
Artists select multiple layers from layer stack or
Trigger timeline.
1) Artist navigates to layer stack.
2) Artist double clicks n number of layers
while holding shift
Main flow 3) All n layers are selected now
Only selected layers are now available for some
Post-conditions arbitrary uniform operation

UC-13 Exporting project file

Item Description
Primary actor Lighting artist
To export final render for finishing touches in
Goal compositing.
Pre- conditions Artist has selected valid directory for final export
Trigger Artist navigates to file tab and clicks export
1) Artist navigates to file tab.
2) Artist clicks export
Main flow 3) System opens export window
 4) Artist clicks export
5) File is exported by default as a 4K AVI file
Exported render file now available in selected
Post-conditions directory

UC-14 Modifying Output settings <<extend>>

Item Description
Primary actor Lighting artist
To offer more control over format and file type of
Goal final render
There is an existing image sequence or video on the
Pre- conditions layer stack
Trigger Artist navigates to file tab and clicks export
1) Artist navigates to file tab.
2) Artist clicks export
3) System opens export window
4) Artist navigates to export settings
5) Artist modifies preset render options
including format, resolution, bit depth,
video codec, default directory etc.
6) Artist clicks on export
Main flow 7) File is exported to chosen directory
Post-conditions File exported with new settings

UC-15 Using preset output settings <<extend>>

Item Description
Primary actor Lighting artist
To enable artists to use premade as well as custom
made presets without constantly manually
Goal changing export settings
There is an existing image sequence or video on the
Pre- conditions layer stack
Trigger Artist navigates to file tab and clicks export
1) Artist navigates to file tab.
2) Artist clicks export
3) System opens export window
4) Artist navigates to presets
5) Artist clicks on chosen preset
Main flow 6) Artist clicks export
Exported render file now available in selected
Post-conditions directory with chosen settings

ACTIVITY DIAGRAMS
This activity diagram gives a lower-level overview of UC-1 from the RALS use case diagram. It
illustrates/models the set of activities and the control flow required for the “Import image
sequence” use case to be take place. The lighting artist clicks on the file tab, clicks on import,
The program displays the file explorer, the artist navigates to a valid media directory and selects
which media file to import. The program then notifies the user that the import was successful
This activity diagram gives a lower-level overview of UC-2 from the RALS use case diagram. It
illustrates/models the set of activities and the control flow required for the “Create light setup
layer” use case to be take place. The lighting artist navigates to the layer stack, clicks on new
light setup, the system prompts the artist to input the layer name as well as the lighting mode and
material, then the lighting setup is created on the timeline.
This activity diagram gives a lower-level overview of UC-3 from the RALS use case diagram. It
illustrates/models the set of activities and the control flow required for the “Insert standard matte
setup” use case to be take place. The lighting artist navigates to the layer stack, clicks on new
light setup, the system prompts the artist to input the layer name as well as the lighting mode and
material, the artist selects matte material and fills out the other fields, the artist confirms their
choice and a standard matte light setup is inserted on the timeline.
This activity diagram gives a lower-level overview of UC-4 from the RALS use case diagram. It
illustrates/models the set of activities and the control flow required for the “Insert custom
material setup” use case to be take place. The lighting artist navigates to the layer stack, clicks
on new light setup, the system prompts the artist to input the layer name as well as the lighting
mode and material, the artist selects custom material and fills out the other fields, the artist
confirms their choice and a custom material setup is inserted on the timeline.
This activity diagram gives a lower-level overview of UC-5 & UC-6 from the RALS use case
diagram. It illustrates/models the set of activities and the control flow required for the “Create
depth map” & “Batch render depth map” use case to be take place. The lighting artist navigates
to the layer stack, clicks on new light setup, the system prompts the artist to input the layer name
as well as the lighting mode and material, the artist selects auto depth or paint depth, the artist
confirms their choice and a depth based light setup is inserted on the layer stack.
This activity diagram gives a lower-level overview of UC-7 from the RALS use case diagram. It
illustrates/models the set of activities and the control flow required for the “Insert manual light
keyframes” use case to be take place. The lighting artist navigates to the layer stack, clicks on
new light setup, the system prompts the artist to input the layer name as well as the lighting
mode and material, the artist selects Keyframe based lighting for the lighting mode, the artist
confirms their choice and a keyframe based light setup is inserted on the layer stack. The artist
then selects two distinct points on the timeline and makes keyframes with Bezier curves. The
system automatically interpolates the keys in Realtime.
This activity diagram gives a lower-level overview of UC-8 from the RALS use case diagram. It
illustrates/models the set of activities and the control flow required for the “Insert manual light
keyframes” use case to be take place. The lighting artist navigates to the layer stack, clicks on
new light setup, the system prompts the artist to input the layer name as well as the lighting
mode and material, the artist selects Keyframe based lighting for the lighting mode, the artist
confirms their choice and a keyframe based light setup is inserted on the layer stack. The artist
then selects two distinct points on the timeline and makes keyframes with Bezier curves. The
system automatically interpolates the keys in Realtime on playback.
This activity diagram gives a lower-level overview of UC-9 from the RALS case diagram. It
illustrates/models the set of activities and the control flow required for the “Real” use case to be
take place. The lighting artist navigates to the layer stack, clicks on new light setup, the system
prompts the artist to input the layer name as well as the lighting mode and material, the artist
selects Keyframe based lighting for the lighting mode, the artist confirms their choice and a
keyframe based light setup is inserted on the layer stack. The artist then toggles batch rendering
on the view panel; the artist then selects two distinct points on the timeline and makes keyframes
with Bezier curves. The artist then selects “Render>>Bezier pass” and the system applies
interpolation for each key.

This activity diagram gives a lower-level overview of UC-10 from the RALS use case diagram.
It illustrates/models the set of activities and the control flow required for the “Group light layers”
use case to be take place. The lighting artist navigates to the layer stack, selects n number of
layers, clicks ctrl + g, The layers get grouped together, artist names the layer group.
This activity diagram gives a lower-level overview of UC-11 from the RALS use case diagram.
It illustrates/models the set of activities and the control flow required for the “Isolate light
layers” use case to be take place. The lighting artist navigates to the layer stack, selects n
number of layers, clicks ctrl + h, The unselected layers get hidden
This activity diagram gives a lower-level overview of UC-12 from the RALS use case diagram.
It illustrates/models the set of activities and the control flow required for the “Select light setup”
use case to be take place. The lighting artist navigates to the layer stack, double clicks n number
of layers while holding shift, All n layers are selected now.
This activity diagram gives a lower-level overview of UC-13 from the RALS use case diagram.
It illustrates/models the set of activities and the control flow required for the “Export project file”
use case to be take place. The lighting artist navigates to the file tab, clicks export, the system
displays the export window, the artist clicks export, the file is then exported with the default
settings.
This activity diagram gives a lower-level overview of UC-14 from the RALS use case diagram.
It illustrates/models the set of activities and the control flow required for the “Export project file”
use case to be take place. The lighting artist navigates to the file tab, clicks export, the system
displays the export window, the artist navigates to presets and modifies preset render options, the
artist clicks export, the file is then exported with the now modified settings.
This activity diagram gives a lower-level overview of UC-15 from the RALS use case diagram.
It illustrates/models the set of activities and the control flow required for the “Export project file”
use case to be take place. The lighting artist navigates to the file tab, clicks export, the system
displays the export window, the artist navigates to presets and selects a preset, the artist clicks
export, the file is then exported with the chosen settings.
REFERENCES:

1) I used visual paradigm as well as starUML for designing UML diagrams. I used visual
paradigm because it was the software I could access the easiest and starUML because It’s
incredibly simple and intuitive and I’ve worked with it before.
2) I based much of the idea for the software off of the software KLAS(Klaus light and
shadow). A proprietary software made in collaboration between SPA studios and Les
films du Poisson rouge for lighting 2d animation. It’s not accessible to the public and
appears to be a fork of Les film du Poisson rouge’s other software MOE, which is also
not freely accessible to the public
3) I based the funding Model for RALS off of the funding model for the blender foundation