2020 IEEE International Conference on Robotics and Automation (ICRA)

31 May - 31 August, 2020. Paris, France

Extracting Legged Locomotion Heuristics with Regularized Predictive Control

Gerardo Bledt1 and Sangbae Kim1

Abstract— Optimization based predictive control is a pow- Offline RPC

erful tool that has improved the ability of legged robots to
execute dynamic maneuvers and traverse increasingly difficult
terrains. However, it is often challenging and unintuitive to
design meaningful cost functions and build high-fidelity models
while adhering to timing restrictions. A novel framework to ex-
Data Driven Analysis
tract and design principled regularization heuristics for legged
locomotion optimization control is presented. By allowing a
simulation to fully explore the cost space offline, certain states
and actions can be constrained or isolated. Data is fit with
simple models relating the desired commands, optimal control
actions, and robot states to identify new heuristic candidates.
Basic parameter learning and adaptation laws are then applied
to the models online. This method extracts simple, but pow-
erful heuristics that can approximate complex dynamics and
account for errors stemming from model simplifications and
parameter uncertainty without the loss of physical intuition
while generalizing the parameter tuning process. Results on
the Mini Cheetah robot verify the increased capabilities due to Fig. 1: Data Driven Heuristic Extraction. Methods pre-
the newly extracted heuristics without any modification to the sented in this paper improve the robot’s capabilities by
controller structure or gains. extracting heuristic models from simulated data.
I. I NTRODUCTION
Legged robots have the potential to be highly dynamic
MIT Cheetah 2 showed the benefits of both heuristics
machines capable of outperforming humans and animals in
and optimization by injecting heuristic regularization into
locomotion tasks over irregular and dangerous environments.
predictive control. This increased the robot’s ability to reject
Unfortunately, they have not yet delivered on that promise as
disturbances and decreased computation time as well as
they still lack the agility and robustness needed to traverse
frequency of undesirable local minima [11]. The robot was
arbitrary terrains with the same grace as animals. However,
able to stabilize multiple gaits at a variety of different
recent advances in the field are showing increasingly impres-
velocities and environmental uncertainty using a single cost
sive behaviors and robustness to both external disturbances
function and set of gains across all situations. Further work
and unexpected obstacles.
proved this type of controller to be tractable on the MIT
Heuristic-based legged locomotion control has shown suc-
Cheetah 3 quadruped hardware platform [12].
cess in traversing rough terrains, particularly with degraded
Parameter adaptation and automatic tuning is notably dif-
perception. Since experts already have knowledge about
ficult in legged systems because of the rapid, harsh discrete
locomotion, it is often a better choice to simply encode intel-
mode changes as feet enter and exit stance and swing
ligent heuristics rather than leave it to an optimization solver.
phases. Early work on automatic tuning for robotic systems
Focchi at al. demonstrate a heuristic planning framework on
concluded that because of the discontinuous gradients it
the HyQ robot to overcome uncertain conditions [1].
was best to develop a controller able to move stably before
Whole-body optimization-based controls using a full dy-
employing the parameter tuning [13]. Other work success-
namics model have been successful in both humanoids
fully used evolutionary optimization that did not attempt to
[2], [3] and quadrupeds [4], [5]. Model Predictive Control
approximate gradients to learn to walk [14], while another
(MPC) approaches have been gaining popularity for their
showed an automatic gain tuning method vastly improved
ability use simple control models for prediction, while still
the robot’s performance without tedious hand tuning [15].
achieving good performance [6], [7], [8]. Results on the
Recently, neural networks have shown promising results
MIT Cheetah 3 robot platform using a linear convex MPC
for learning control policies through simulation. Hwangbo et
framework demonstrated highly dynamic, robust locomotion
al. show a simulation-based learned policy ported over for
with various gaits [9], [10].
real hardware locomotion [16]. Work from DeepMind has
Regularized Predictive Control (RPC) allows known dy- shown the ability to learn to move in complex environments
namics and locomotion principles to be exploited through on various different dynamic systems [17]. However, as is
heuristic regularization. Previous work on a model of the often a drawback of neural networks, it is not always clear
1 Department of Mechanical Engineering, MIT, Cambridge, MA how to modify specific policy parameters in order to tune
02139, USA: gbledt@[Link], sangbae@[Link] the behaviors without the need to retrain the entire network.

978-1-7281-7395-5/20/$31.00 ©2020 IEEE 406

Authorized licensed use limited to: MIT-World Peace University. Downloaded on August 17,2026 at 10:10:47 UTC from IEEE Xplore. Restrictions apply.
A. Contributions 0.3

0.9 0.9
0.25
0.8 0.8

The main contribution of the paper is a framework that 0.2

0.7

0.6
0.7

0.6

provides methods for the extraction of simple legged loco- 0.15

0.5

0.4
0.5

0.4
0.1

motion heuristics from robot experience data. The result is 0.05

0.3 0.3

0.2 0.2

autonomous discovery of simple, but powerful regularization 0

0.1 0.1

0 0

heuristics that improve the performance of the RPC without -0.05

-1 -0.5 0 0.5 1
-0.1
-1 -0.5 0 0.5 1
-0.1
-1 -0.5 0 0.5 1

any explicit changes to control gains or controller structure. (a) JD (χ) (b) JR (χ) (c) J(χ)
Each simple heuristic is generalizable, easily tunable, and
Fig. 2: Regularizing Nonlinear Cost Functions. A generic
carries a clear physical meaning. With newly extracted
nonlinear cost function (2a) with multiple local minima is
heuristics from using this framework, the robot is able to spin
additively regularized with a simple quadratic cost function
in place rapidly at 4 rad rad
s and turn at 2 s while moving at a
m (2b). The resulting cost function (2c) captures a smoothened
high speed of 1.5 s . Previously, these maneuvers were could
nonlinear function with only one minimum.
not be executed stably with the exact same controller. The
heuristics found through the framework are generalizable
and adequate for locomotion under unexpected disturbances.
in [11]. Generally it can be written
−1
NP
II. R EGULARIZED P REDICTIVE C ONTROL min J(χ) = χ̃Tk Wk χ̃k
χ k=0
As opposed to traditional MPC techniques, RPC directly
exploits simple heuristics to simplify complex cost spaces xk+1 = Ak xk + Bk h(χk , Φk ) + dk
s.t.
and find feasible solutions quickly. Rather than treating the ζk (χk , Φk ) ≤ 0
problem of robot locomotion as a black box optimization, ζk0 (χk , χk+1 , Φk , Φk+1 ) ≤ 0
simple physics-based heuristics are encoded into the cost T
where χk = xTk uTk , ∀k ∈ {0, ..., N − 1} are decision

function and constraints to bias the optimization towards
variables and the number of timesteps, N , is user selected.
a sensible solution while remaining free to explore the
Physical feasibility constraints are ζ and ζ 0 as described in
surrounding cost space for the possibility of a better result.
[12], and the gait phase is Φ as introduced in [18]. The
Heuristics for the decision variables, χ, are embedded simplified discrete dynamics matrices are written
directly into the cost function through error terms " 2 # " 2 #
dtk −1 dtk
 
I6 dtk I6

Ak = Bk = 2 I −1 dk = 2 ag
χ̃ = Hχ χ, Φ, xd − χ (1) 06 I6 dtk I dtk ag

where the heuristics Hχ are functions of the robot states, where I is the inertia tensor and ag is the gravity vector. The
 T nonlinear inputs make up the combined forces and torques
x = pT ΘT ṗT Θ̇T , desired states, xd , and the
on the CoM as
scheduled gait phase, Φ, as defined in [18]. The robot model  
is chosen to be a simple lumped mass model with a massless f
h(χk , Φk ) = B
leg assumption making the states the position of the CoM, τ
p, the roll-pitch-yaw Euler Angle CoM orientation, Θ, the X4
I3 

velocity in the world frame, ṗ, and the Euler angle rates, = s f
R(Θk )T ri,k × Φ,i,k i,k

Θ̇. Inputs are chosen to be the foot location vectors for i=1
each foot i from the CoM, ri , and the corresponding ground with R(Θk ) being the rotation matrix of the CoM, and
 force, fi , which are ordered
reaction
T
in the input vector sΦ,i,k ∈ {0 : swing, 1 : contact} being the boolean contact
u = r1T f1T . . . r4T f4T . The combination of states state dependent on the scheduled gait phase of each foot.
and inputs for all predicted timesteps are chosen to be the To demonstrate the effects of regularization, a simple ex-
T
decision variables, χk = xTk uTk . The power of the ample is depicted in Figure 2. The underlying cost function
RPC framework is that it allows extremely rich information (2a) is chosen to be nonlinear with multiple local minima.
about the robot to be embedded in the optimization with no A quadratic regularization function (2b) is designed to be
modification to the controller structure. additive and smooth out the dynamics cost where the result
The inputs are solved for through a nonlinear optimization is a well-conditioned combined cost function (2c) with no
using a direct transcription method [19]. The objective is to extraneous local minima. Through regularization, the shape
minimize the quadratic cost function on the deviation from of the cost function is changed without canceling out the
the state and input heuristics. We pose the optimization- original nonlinear function.
based control essentially as a nonlinear MPC that relies Although these heuristics benefit the RPC, designing them
heavily on regularization heuristics rather than an accurate in practice is challenging and can be unintuitive. Simply
model or realistic dynamics. In fact, the model has massless adding a regularization function does not improve the op-
legs and almost completely linearized dynamics as described timization problem. In fact, it can often yield detrimental

407
Authorized licensed use limited to: MIT-World Peace University. Downloaded on August 17,2026 at 10:10:47 UTC from IEEE Xplore. Restrictions apply.
 We propose a more autonomous method to quickly dis-
0.9 0.9
cover these relationships. This paper will focus on using the
0.8 0.8

0.7 0.7
data gathered from simulations running the RPC offline in
0.6 0.6 different scenarios. Since the solution times do not need to
0.5 0.5 adhere to any real-time constraints offline, the cost space can
0.4 0.4

0.3 0.3
be fully explored for nonintuitive local minima and tested
0.2 0.2
for generality. Running the controller in a simulated environ-
0.1 0.1 ment also allows restriction of certain states to investigate the
0 0
performance without some of the nonlinear coupling effects
-0.1 -0.1
-1 -0.5 0 0.5 1 -1 -0.5 0 0.5 1
that would be present in the real system.
(a) Bad Regularization (b) Good Regularization The states, inputs, commands, gait schedule, and combi-
Fig. 3: Designing Regularization Functions. Regularization nations of each from the simulations make up the set
functions should smooth the dynamics function while not  T
V = xT uT xTd ΦT (ṗ × Θ̇)T t . . . (3)
having a significant effect on the optimal solution.
to be tested as potential functions of each other. Various
simple model fits are applied to the data sets and highly
results. Figure 3 shows how a poorly designed regularization correlated models are identified as candidates for a new
function (3a) can pull the combined minimum away from the heuristic policy. These candidates are further analyzed for
underlying optimum, whereas a well designed regularization repeatability and compared with the results of the predictive
function (3b) simplifies the surrounding cost space, but does optimization to determine if the proposed model is a valid
not affect the optimal solution. The key is to design good extracted heuristic. The framework computes model fits on
heuristics that meaningfully shape the cost space. the robot data during simulation where forward, lateral, and
Qualitatively, a heuristic is good if it causes the system turning rate commands are provided.
to behave as desired when executed. Quantitatively, a set The fits tested for statistical correlation are polynomial
of heuristics is considered good if the optimal solution of and sum of sines fits where the model is described as
the underlying function, the heuristic regularization portion,
D∈[0,9]
and combined cost function are all approximately equal to X
Polynomial: Hvi (vj ) = avni vjn (4)
each other for all states within the set of desired reasonable
n=0
locomotion states written as D∈[1,8]
X
χ∗R ≈ χ∗D ≈ χ∗ , ∀{x|x ∈ X} (2) Sum of Sines: Hvi (vj ) = bvni sin (cvni vj + dvni ) (5)
n=1
This viable operating space, X, is intentionally defined
vaguely allowing for different desired performance metrics. with D being the degree of polynomial and the number
For example, the set of heuristics for quasi-static locomotion of summed sine terms respectively for the heuristic model
may not be adequate for more dynamic movement. What Hvi (vj ) where the dependent variable, vi , is written as a
may be considered failure for one robot, may be perfectly function of the independent variable, vj and both are taken
within reasonable locomotion for another robot. While the from V . These models were deliberately chosen over more
individual parts of the cost function may look very different complicated highly nonlinear functions or neural networks
from each other, the optimal solutions for each part should because they are easily tunable and can provide intuition for
be near each other in most cases. the underlying physical relationships. It is straightforward
to determine the effects of varying the parameters of a
III. E XTRACTING H EURISTICS polynomial or sine wave model, while approximating highly
To design these heuristics, we propose using the frame- nonlinear behaviors.
work described in Figure 4. The framework pairs expert Figure 5 shows an example result for the R2 values for
design with data-driven analysis. Humans provide natural a first order polynomial model fit on data gathered from
intuition that algorithms may not find as easily. Meanwhile, varying the commanded forward speed of the robot. Similar
computers can analyze large amounts of data and recognize plots were made for the various model fits in (4) and (5). In
patterns automatically that may not be obvious to an expert. the past, some of these relationships were discovered through
The framework seeks to exploit the advantages of each experimental observation and parameters were hand tuned,
method to build the richness of the regularization heuristics but with this framework, the model types and associated
that will be embedded in the robot for better online control. parameters automatically discovered. This simplifies the
Expert design encompasses observing nature and analyzing process, while retaining intuition over the robot control.
simple physics. For example, the notion of vertical impulse
scaling [20], or the widely used capture point [21]. These IV. E VALUATING H EURISTICS
required experts in the field to make clever assumptions and Extracted heuristic models are dependent on the explo-
simplifications for force generation and footstep placement ration done in simulation, as well as the choice of dependent
in specific cases. and independent variables to test for correlation. This means

408
Authorized licensed use limited to: MIT-World Peace University. Downloaded on August 17,2026 at 10:10:47 UTC from IEEE Xplore. Restrictions apply.
 Offline Heuristic Design

Expert Design Online RPC

Observing nature Analyzing physics

Data Driven Analysis

Offline RPC
Parameter Learning

Fig. 4: Heuristic Extraction Framework. Expert design and data analysis from running the RPC offline both yield heuristics
that can be used to inform the controller online. Basic Reinforcement Learning adapts parameters to account for model
inaccuracies and timing discretization delays that prove to be difficult to model.

0.8
0.15

0.6 0.1

0.4 0.05

0.2 0

0 -0.05

-0.2 -0.1

-0.4 -0.15

-0.6 -0.2

-0.8 -0.25
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2

(a) Hθ (ṗy ) (b) Hφ (ṗx )

(a) Variable Relationships (b) Heuristic Candidates R2
Fig. 6: Linear Velocity Induced Orientation. The frame-
Fig. 5: Identifying Heuristic Model Candidates. Model work identified the tendency towards linear orientation offset
fits are computed for each of the variable relationships in 6a in pitch and roll with faster forward and lateral trotting.
to identify new heuristic candidates for a subset of V . The
R2 values for a 1st order polynomial fit are seen in 6b.
function of the gait phase with R2 = 0.835 described by the
new heuristic
that while the framework can identify potential candidate
models between two states, they do not necessarily represent Hφ (Φ) = bφ1 sin(cφ1 Φ + dφ1 )
meaningful heuristics. It is still necessary to have a knowl- (8)
= 0.023 sin(12.553Φ + 2.999).
edgeable expert design exploration methods and differentiate
between models with overfitting or false dependencies. The results of which are depicted in Figure 7. Interestingly,
Two of the stronger correlations found were linear depen- this matches the intuition for a periodic pitch limit cycle
dencies of roll, θ, on the robot’s lateral velocity, ṗy , and during a trot gait where c1 = 2πf and the two diagonal
pitch, φ, based on the forward velocity, ṗx . The fits had pairs of legs perform complementary actions over a single
an R2 value of 0.9827 and 0.9533 respectively. These were gait cycle, f = 2, giving cφ1 ≈ 12.566.
found to be By combining the two extracted heuristics for pitch in
equations (7) and (8), we get
Hθ (ṗy ) = aθ1 ṗy + aθ0 = −0.339ṗy + 0.00027 (6)
Hφ (ṗx ) = aφ1 ṗx + aφ0 = 0.073ṗx − 0.0475 (7) Hφ (ṗx , Φ) = Hφ (ṗx ) + Hφ (Φ)
(9)
= a1 ṗx + a0 + b1 sin(c1 Φ + d1 ).
and shown in Figure 6 for both cases. The dominant linear
nature of the relationships is clear, although this does not Figure 8 shows that although the commanded robot speed is
fully fit the data. 1.5 m
s with no pitch, the robot’s dynamics vary in practice.
The same simulations were run again and analyzed, However using heuristic (9) we can approximate the complex
but this time new heuristics were discovered. Namely, a dynamics that take place naturally using only the simple
sinusoidal relationship for the periodic pitch behavior as a intuitive models that are easy to modify.

409
Authorized licensed use limited to: MIT-World Peace University. Downloaded on August 17,2026 at 10:10:47 UTC from IEEE Xplore. Restrictions apply.
 0.03

0.02

0.03 0.01

0
0.02
-0.01

0.01 -0.02

-0.03
0
-0.04

-0.05
-0.01
-0.06
0 2 4 6 8 10 12 14 16
-0.02

-0.03
(a) Pitch Compensation (b) Learned Parameter Adaptation
-0.04

-0.05
Fig. 9: Learned Pitch Compensation Parameters. Both
0 0.2 0.4 0.6 0.8 1
parameters aφ0 and aφ1 are learned online to satisfy the reward
function and averaged to be constants in the robot’s memory.
Fig. 7: Periodic Pitch Behavior. The robot exhibits natural
pitch dynamics that are be extracted from simulation data. disappeared with apitch error of 0.16 ± 0.52◦ for forward
velocities of ṗx ∈ −2, 2 m s . Regardless of the underlying
0.06
0.06
0.05
cause of the errors, we are able to find a simple, easily
0.05
0.04

0.03
tunable approximate model to compensate for them.
0.04
0.02 The same method was applied to iteratively to build the set
0.03
0.01

0
of heuristics, H. In addition to the previously discussed pitch
0

0.2
and roll compensation and the periodic orientation behavior,
0.02

0.4
1.35
heuristics were extracted for foot placement during straight
0.01 0.6

1.5
1.45
1.4
walking, in-place turning, and high speed turning, as well as
0.8

0
1.35 1.4 1.45 1.5 1.55 1.6 1.65 1 1.65
1.6
1.55
a curious error where the robot began to dip in height as it
moved faster. The extracted models are listed in Table I. Note
Fig. 8: Intuitive Approximation. The dark red shows the that the simplest model was chosen to prevent overfitting
adjusted reference pitch during a steady state trot from the despite many datasets also being slightly better approximated
simple heuristics and matches closely to the actual pitch of by higher degree polynomials and larger sum of sines.
the robot in orange from a commanded 1.5 m Newly extracted heuristics were added to the RPC in sim-
s forward speed.
ulation to verify that each one would generalize to situations
other than the exploration in which they were extracted. As
Since the data was gathered in simulation without real- new heuristics are injected into the RPC optimization, the
time computation restrictions, the parameters used for the capabilities of the robot increase. In Figure 10 the robot was
heuristics may often differ on the actual system or even commanded  forward and lateral velocities from (ṗx , ṗy ) ∈
−3.5, 3.5 m 4.0 rad

on the same simulated model with real-time restrictions in s and turning rates of ψ̇ ∈ −4.0, s .
place. Therefore, a very simple parameter learning law was Maximum forward and lateral velocities, as well as turning
developed to adjust the values during operation. A reward rate were all doubled when all models were added.
function is defined as RPC was implemented on the MIT Cheetah 3 and Mini
P Cheetah robots using the IPOPT solver [22] in C++ with the
RHvi = − (vd − vi ) (10)
Φ
optimization running at a situationally-variable frequency of
100-200Hz as described in [12]. New behaviors such as the
where the robot is rewarded for
Ptracking the desired behavior rapid in-place turning and high speed turning which were
summed over a gait period, . The parameter update law
Φ not previously possible when using a naı̈ve optimization are
for the linear model case is written to be now able to be executed as seen in Figure 11. When turning
in place, the feet are placed along the direction of the turn as
( v
a0,k
i
= av0,k−1
i
− ∇RHvi , |vj | < vj,min
∇RHv (11) the robot is given a command of ψ̇d = 6.5 m s . for high speed
av1,k
i
= av1,k−1
i
− i
, |vj | ≥ vj,min .
vj turning with commands of {ṗx,d , ψ̇d } = {1.5 m rad
s , 2 s }, the
This adaptation law was able to successfully learn param- foot placement is dominated by the need to apply centrifugal
eters that reduced errors in the robot at runtime as seen in forces by placing them roughly out along the turning radius.
Figure 9. The controller generalizes to handle unexpected situations
as the heuristics bias the optimization towards a solution
V. R ESULTS without constraining it, meaning that the optimization can
The exact reason for the velocity induced orientation er- still search for better footstep locations and forces. The
rors is not immediately clear, but can likely be a combination controller was tested for robustness by giving the robot an
of various factors including unmodeled dynamics, incor- impulsive push as seen in Figure 12a, as well as commanded
rect friction parameters, timestep discretization, optimization to trot forward at ṗd = 1.5 m s over random debris scattered
solve time, or something we have not thought of. With the over the ground as seen in Figure 12b. In both cases, the
heuristic implemented on the robot, pitch and roll errors robot is able to reject disturbances. More experiment videos

410
Authorized licensed use limited to: MIT-World Peace University. Downloaded on August 17,2026 at 10:10:47 UTC from IEEE Xplore. Restrictions apply.
 3.5 3.5 3.5 3.5 3.5

0.0 0.0 0.0 0.0 0.0

-3.5 -3.5 -3.5 -3.5 -3.5

-3.5 0.0 3.5 -3.5 0.0 3.5 -3.5 0.0 3.5 -3.5 0.0 3.5 -3.5 0.0 3.5

3.5 3.5 3.5 3.5 3.5

0.0 0.0 0.0 0.0 0.0

-3.5 -3.5 -3.5 -3.5 -3.5

-4.5 0.0 4.5 -4.5 0.0 4.5 -4.5 0.0 4.5 -4.5 0.0 4.5 -4.5 0.0 4.5

(a) No Heuristics (b) Hr (ṗ) (c) Hr (ψ̇) (d) Hr (ṗ × Θ̇) (e) HΘ (ṗ, Φ)

Fig. 10: Viable Operating Regions. Each plot from 10a to 10e adds one of the heuristics discussed in the paper. As new
heuristics are discovered and subsequently injected into the optimization, the set of viable operating regions where the robot
is in stable locomotion increases. The dark areas signify where a fall did not occur, but does not qualify the movements.

TABLE I: Extracted Heuristics

Heuristic Extracted Heuristic Model
Forward Stepping Hr (ṗ) = ar1 ṗ + ar0
In-Place Turning Hr (ψ̇) = ar1 ψ̇ + ar0
High Speed Turning Hr (ṗ × Θ̇) = ar1 (ṗ × Θ̇) + ar0
Orientation Compensation HΘ (ṗ) = aΘ 1 ṗ + a0
Θ

Periodic Orientation HΘ (Φ) = bΘ sin(c Θ Φ + dΘ ) (a) Push Rejection (b) Rough Terrain
1 1 1
Height Compensation Hz (ṗx ) = az2 ṗ2x + az1 ṗx + az.
0 Fig. 12: Robust Locomotion. Heuristics generalize to be
robust to external disturbances and unstructured terrain.

the heuristic solution. Finding H∗ is likely unfeasible, but

the proposed method can be used to build the set H, and
better approximate H∗ for a large number of cases. This
will take the burden off of the optimization as the initial
guess and regularized solution will be closer to the actual
(a) In-Place Turning (b) High Speed Turning dynamics-based optimal solution.
Recent work has shown great results in the area of robotic
Fig. 11: Improved Performance. Extracted heuristics allow learning and is being considered as a possible enhancement
the robot to spin in place at 2π rad
s (11a) and make high to the framework. However, in this work we argue that it
speed turns of 2 rad
s while moving forward at 1.5 m
s (11b). is important to have a good grasp of the physical intuition
behind the controller. By choosing the set of heuristics, H,
to be built up from simple functions as described in the paper
are shown in the attached video [23]. Results support the you retain control over important state relationships and
goal of a framework to systematically improves robot perfor- have an awareness for the controller’s possible strengths and
mance by extracting heuristics that approximate unmodeled weaknesses, while allowing a learning or parameter adapta-
dynamics and compensate errors with simple models. tion law pick the values of the gains through the robot’s
experience. A policy network may be able to adaptively
VI. C ONCLUSION
change which heuristics are active in certain situations and
The results have shown controller performance improved would be worth exploring. The generality of the controller
without any explicit changes to the control gains or the makes it possible to explore a variety of techniques without
controller structure. Theoretically, if a set of heuristics, H, any major changes to the core robot control architecture.
is found where χ∗R = χ∗D = χ∗ , ∀{x|x ∈ X} rather
than the approximate equation (2), then these heuristics ACKNOWLEDGMENTS
are the solution to the optimization, H = H∗ , for all This work was supported by the Toyota Research Insti-
of the viable operating conditions, X. This then implies tute, Centers for ME Research and Education at MIT and
that there is no more need for heavy optimization as the SUSTech, Naver Labs, and the Air Force Office of Scientific
optimal dynamics-based solution would always be equal to Research. Video link: [Link] c82NYmk4Q.

411
Authorized licensed use limited to: MIT-World Peace University. Downloaded on August 17,2026 at 10:10:47 UTC from IEEE Xplore. Restrictions apply.
 R EFERENCES vol. 340 of Lecture Notes in Control and Information Sciences, pp. 65–
93, Springer Berlin Heidelberg, 2006.
[1] M. Focchi, R. Orsolino, M. Camurri, V. Barasuol, C. Mastalli, D. G. [20] H. W. Park, S. Park, and S. Kim, “Variable-speed quadrupedal bound-
Caldwell, and C. Semini, “Heuristic planning for rough terrain loco- ing using impulse planning: Untethered high-speed 3d running of MIT
motion in presence of external disturbances and variable perception cheetah 2,” in 2015 IEEE International Conference on Robotics and
quality,” CoRR, vol. abs/1805.10238, 2018. Automation (ICRA), pp. 5163–5170, May 2015.
[2] S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. Dai, F. Per- [21] J. Pratt, J. Carff, S. Drakunov, and A. Goswami, “Capture point: A
menter, T. Koolen, P. Marion, and R. Tedrake, “Optimization-based step toward humanoid push recovery,” in IEEE-RAS Int. Conf. on
locomotion planning, estimation, and control design for the atlas Humanoid Robots, (Genova, Italy), pp. 200–207, Dec. 2006.
humanoid robot,” Autonomous Robots, vol. 40, pp. 429–455, Mar [22] A. Wächter and L. T. Biegler, “On the implementation of an interior-
2016. point filter line-search algorithm for large-scale nonlinear program-
[3] F. Farshidian, M. Neunert, A. W. Winkler, G. Rey, and J. Buchli, ming,” Math. Program., vol. 106, pp. 25–57, Mar. 2006.
“An efficient optimal planning and control framework for quadrupedal [23] “Extracting legged locomotion heuristics with regular-
locomotion,” in ICRA, 2017. ized predictive control,” [Link] c82NYmk4Q,
[4] A. W. Winkler, F. Farshidian, M. Neunert, D. Pardo, and J. Buchli, [Link]
“Online walking motion and foothold optimization for quadruped
locomotion,” 2017 IEEE International Conference on Robotics and
Automation (ICRA), pp. 5308–5313, 2017.
[5] C. D. Bellicoso, F. Jenelten, C. Gehring, and M. Hutter, “Dy-
namic locomotion through online nonlinear motion optimization for
quadrupedal robots,” no. 99, 2018-01-17.
[6] M. J. Powell, E. A. Cousineau, and A. D. Ames, “Model predictive
control of underactuated bipedal robotic walking,” in Proc. of the
IEEE Int. Conf. on Robotics and Automation, pp. 5121–5126, May
2015.
[7] Y. Tassa, T. Erez, and E. Todorov, “Synthesis and stabilization of
complex behaviors through online trajectory optimization,” in Proc. of
the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 4906–
4913, Oct 2012.
[8] M. Naveau, M. Kudruss, O. Stasse, C. Kirches, K. Mombaur, and
P. Soures, “A reactive walking pattern generator based on nonlinear
model predictive control,” IEEE Robotics and Automation Letters,
vol. 2, pp. 10–17, Jan 2017.
[9] G. Bledt, M. J. Powell, B. Katz, J. D. Carlo, P. M. Wensing, and
S. Kim, “MIT cheetah 3: Design and control of a robust, dynamic
quadruped robot,” in Proceedings of the IEEE/RSJ International
Conference on Intelligent Robots and Systems, (Madrid, Spain), Oct.
2018.
[10] J. D. Carlo, P. M. Wensing, B. Katz, G. Bledt, and S. Kim, “Dynamic
locomotion in the MIT cheetah 3 through convex model-predictive
control,” in Proceedings of the IEEE/RSJ International Conference
on Intelligent Robots and Systems, (Madrid, Spain), Oct. 2018.
[11] G. Bledt, P. M. Wensing, and S. Kim, “Policy-regularized model
predictive control to stabilize diverse quadrupedal gaits for the MIT
cheetah,” in Proceedings of the IEEE/RSJ International Conference
on Intelligent Robots and Systems, (Vancouver, Canada), Sept. 2017.
[12] G. Bledt and S. Kim, “Implementing regularized predictive control
for simultaneous real-time footstep and ground reaction force opti-
mization,” in Proceedings of the IEEE/RSJ International Conference
on Intelligent Robots and Systems, (Macau, China), Nov. 2019.
[13] R. Ringrose, “Automatically tuning control systems for simulated
legged robots,” in AAAI 1994, 1994.
[14] S. Chernova and M. Veloso, “An evolutionary approach to gait
learning for four-legged robots,” in 2004 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS) (IEEE Cat.
No.04CH37566), vol. 3, pp. 2562–2567 vol.3, Sep. 2004.
[15] J. D. Weingarten, G. A. D. Lopes, M. Buehler, R. E. Groff, and
D. E. Koditschek, “Automated gait adaptation for legged robots,” in
IEEE International Conference on Robotics and Automation, 2004.
Proceedings. ICRA ’04. 2004, vol. 3, pp. 2153–2158 Vol.3, April
2004.
[16] J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis,
V. Koltun, and M. Hutter, “Learning agile and dynamic motor skills
for legged robots,” Science Robotics, vol. 4, no. 26, 2019.
[17] N. Heess, D. TB, S. Sriram, J. Lemmon, J. Merel, G. Wayne, Y. Tassa,
T. Erez, Z. Wang, S. M. A. Eslami, M. A. Riedmiller, and D. Silver,
“Emergence of locomotion behaviours in rich environments,” CoRR,
vol. abs/1707.02286, 2017.
[18] G. Bledt, P. M. Wensing, S. Ingersoll, and S. Kim, “Contact model
fusion for event-based locomotion in unstructured terrains,” in 2018
IEEE International Conference on Robotics and Automation (ICRA),
(Brisbane, Australia), Accepted 2018.
[19] M. Diehl, H. Bock, H. Diedam, and P.-B. Wieber, “Fast direct multiple
shooting algorithms for optimal robot control,” in Fast Motions
in Biomechanics and Robotics (M. Diehl and K. Mombaur, eds.),

412
Authorized licensed use limited to: MIT-World Peace University. Downloaded on August 17,2026 at 10:10:47 UTC from IEEE Xplore. Restrictions apply.