Software Language Engineering

10. Language Composition

see also MC Ref. Manual, Ch. 7

Prof. Dr. Bernhard Rumpe

Software Engineering
RWTH Aachen

[Link]
Farbe!
 Rep.
Software Language

A software language is a human readable and computer processable language addressing a particular
problem.

• Examples:
 UML: a general-purpose modeling language
 Java: a general-purpose programming language
 XML: a format for structured data

A modeling language is a software language used for modeling software or systems.

2 Software Engineering | RWTH Aachen

Developing Modeling Languages

• Often: • Problems Developing Modeling Languages

 define meta model
 design concrete syntax  Ad-hoc development of languages
 write transformation into a programming language
 No (systematic) reuse of
• In addition?  language syntax
 comfortable editor  context conditions
 context checks in the model  analysis and synthesis techniques
 context checks between models  semantics, pragmatics
 complex analyses / syntheses
 No language-integrating tools
• Which other tools?
 testing?  Result:
 simulation?  too much efforts necessary for DSL & tool development
 security / safety analysis, … ??  suboptimal quality
 no libraries for models
 no libraries for language components

3 Software Engineering | RWTH Aachen

 Rep.
Software Language Engineering

• Software Language Engineering (SLE)

 is the discipline to design useful software languages and
their tool infrastructure in an efficient, systematic way.

• SLE has a specific focus on scalability for

 heterogeneous languages and
 large complex systems

according to [CFJ+16] :

4 Software Engineering | RWTH Aachen

But:

How to engineer a software language?

(and especially a complex modeling language)

5 Software Engineering | RWTH Aachen

Language Composition

Language composition
• is used to build new languages from language components

• Language composition deals with all concepts of a language:

 concrete syntax
 abstract syntax (AST)
 well-formedness rules (“context conditions”)
 semantics (“meaning”)
 pragmatics (“use”)

6 Software Engineering | RWTH Aachen

 Software Language Engineering
10. Language Composition
10.1. Model Composition

Prof. Dr. Bernhard Rumpe

Software Engineering
RWTH Aachen

[Link]
Farbe!
Model Composition

• Projects developing large complex systems use

 many models with complex dependencies 
• Modularity and composition are essential for:
 distributed development
 reuse from libraries
 efficient tools (generation, analysis)

• Composition:
C= AB

• Connects A with B and encapsulates internals

• The principle is well known 

 e.g. modules, classes, abstract data types

8 Software Engineering | RWTH Aachen

Kinds of Model Composition C = A  B

• Automata
 cross product or 
 parallel composition

• Class diagrams
 merge of classes with same name

• Logics 
 conjunction

• Programming languages
 classes (components): import: call and extension

• These are so far all homogeneous (within their language) 

9 Software Engineering | RWTH Aachen

Dimensions of Model Composition

• Syntactic: How does A  B look like?


• Semantic: What does A  B mean?

• Method: How to decompose C into

rather self-contained A and B?

• Organization: Can we develop A and B by


separate parallel groups?

• Technical: Can we compile incrementally & individually?

That means: is there a binding technique for

Code(A)  Code(B)

allowing to avoid Code(A  B)?

10 Software Engineering | RWTH Aachen

How to Compose Heterogeneous Models?

• Heterogeneity of models is typical in development, because

 models describe only aspects of the target system

• How to compose models written in different sublanguages,

like in the UML?

• How to compose models of a DSL with code written in a GPL?

• Heterogeneous model composition needs language composition.

class object
deployment diagrams diagrams
statecharts C++, sequence
diagram Java … __: __: diagrams
__:
OCL

11 Software Engineering | RWTH Aachen

 Software Language Engineering
10. Language Composition
10.2. Forms of Language Operators

Prof. Dr. Bernhard Rumpe

Software Engineering
RWTH Aachen

[Link]
Farbe!
Techniques Operating on Languages

These techniques are available and can be used even together to

achieve language composition:

• Language aggregation, e.g., CD + Statechart


• Language embedding, e.g., Expressions as Guards

• Language inheritance, e.g., FSM become TimedFSM

• Language extension
 conservative extension
 AST-conservative extension

• Language restriction

13 Software Engineering | RWTH Aachen

Language Extension - Starter

• Lets start with one language L1 L1 • Automaton language L1:

• The automaton has SC

 2 states and behavior automaton PingPong {
 2 transitions
 describing a ping pong game state Ping, Pong;

Ping -> Pong

Ping -> Pong

}

14 Software Engineering | RWTH Aachen

Language Extension

• L2 extends L1 L1 • Automaton language L1 is extended by actions in L2:

 by new language concepts  Actions are embedded at multiple places:
extends
SC
L2 behavior automaton PingPong {

• One model contains language concepts of both state Ping, Pong;

languages
Ping -> Pong [ strokes++ ]
• Either L1 or L2 becomes the master language and
the other the multiply embedded sub-language Ping -> Pong [ strokes++ ]
}
• Semantics, code generation is often defined together,
but ideally reuse L1-semantics, generators, etc.
should be possible

15 Software Engineering | RWTH Aachen

Conservative Language Extension

• L2 extends L1 L1 • Automaton language L1 is extended by actions in L2:

 by new language concepts  Actions are embedded at multiple places:
extends
SC
L2 behavior automaton PingPong {

L2 is a conservative extension of L1, if state Ping, Pong;

Models(L2) are a superset of Models(L1)
Ping -> Pong [ strokes++ ]
• … enables reuse of models
Ping -> Pong [ strokes++ ]
L2 is a AST (metamodel) conservative extension of }
L1, if internal representation remains valid
• … enables reuse of L1-functionality also operating on
L2-models

16 Software Engineering | RWTH Aachen

Language Embedding

• A new language L3 embeds L1 L2 • Automaton language L1 and action language L2 are

model concepts from L2 in combined to a language embedding the actions into
the language L2 extends embeds the automaton:
SC
L3 behavior automaton PingPong {

• Models have parts conforming to sublanguages state Ping, Pong;

• Languages L1 and L2 were independently developed Ping -> Pong [ strokes++ ]

• Enables reuse and extension of languages Ping -> Pong [ strokes++ ]

}
• Allows to define language components
 E.g. expressions, literals, type definitions.
• “Glue” can be added, e.g. the square brackets

17 Software Engineering | RWTH Aachen

Language Aggregation

• An aggregated language L1 L2 • Two models:

L3 combines L1, L2, and more …  An automaton and a java class sharing symbols
aggregates (e.g. strokes)
SC
L3 behavior automaton PingPong {
• Models are independent artefacts
 they can be edited, reused, etc. individually state Ping, Pong;

• Models are only semantically composed Ping -> Pong [ strokes++ ];

 there is no model belonging “only” to L3 }

• Models syntactically refer to each other Java

 “Symbols” are imported / exported class Game {
Player a, b;
int strokes = 0;
}

18 Software Engineering | RWTH Aachen

Language Restriction

• L2 extends L1 L1 • Automaton language L2 does not allow the Java

 … by eliminating syntactic concepts actions of L2 anymore:
or additional CoCos extends
SC
L2 behavior automaton PingPong {

L2 is a conservative restriction of L1, if state Ping, Pong;

Models(L2) are a subset of Models(L1)
Ping -> Pong [ strokes++ ]

Ping -> Pong [ strokes++ ]

• Enables:
 Reuse of tooling and some of the models }

 Simpler L2-functions possible Examples:

(e.g. generator may now be computable) • Forbid actions, forbid hierarchy of states
• Only basic types (integer …) used
• Disallow certain names,

19 Software Engineering | RWTH Aachen

 Software Language Engineering
10. Language Composition
10.3. Language Composition Operators

Prof. Dr. Bernhard Rumpe

Software Engineering
RWTH Aachen

[Link]
Farbe!
Component Grammars

• A component grammar defines a sub-language

 MontiCore produces the AST classes but no parser
• Used typically as
 a library of nonterminals, or
 an extensible (incomplete) language with extension points
• Component grammar may have
 interfaces and abstract nonterminals without body, and
 external nonterminals
marks a grammar
as component
1 component grammar InvAutomaton { MG
2 external Invariant;
3 State = "state" Name
4 Invariant
5 ("<<" ["initial"] ">>" | "<<" ["final"] ">>" )* ";" ;
6 }

21 Software Engineering | RWTH Aachen

External Nonterminals

• … define extension points

• AST: generates a (Java) interface ExtInvariant
• … are “filled” using productions when composing grammars
1 component grammar InvAutomaton { MG
2 external Invariant;
can be used like any
3 State = "state" Name other nonterminal
4 Invariant
5 ("<<" ["initial"] ">>" | "<<" ["final"] ">>" )* ";" ;
6 }

1 grammar Automaton3 extends InvAutomaton, ExpressionBasis { MG

2 Invariant = ("[" Expression "]")? ;
3 } nonterminal is “filled” using a production

22 Software Engineering | RWTH Aachen

Language Embedding

• Embedding of independently developed languages into each other is possible using

 language extension and
 overwriting of nonterminals

1 component grammar InvAutomaton { MG

2 external Invariant;
3 State = "state" Name
4 Invariant
5 ("<<" ["initial"] ">>" | "<<" ["final"] ">>" )* ";" ;
6 }
grammar with external
nonterminals is extended

1 grammar Automaton3 extends InvAutomaton, ExpressionBasis { MG

2 Invariant = ("[" Expression "]")? ;
3 } and their nonterminals now use each other

23 Software Engineering | RWTH Aachen

Language Inheritance

• Allows a modular definition of languages by reusing language components

• Existing languages can be extended and modified
 all productions from the super grammar are inherited
 inherited productions can be redefined

1 grammar HierarchicalAutomaton extends Automaton { MG

2 State = "state" Name
3 ("<<" ["initial"] ">>" | "<<" ["final"] ">>" )*
4 (";" | "{" State* "}" ) ;
5 }
HierarchicalAutomaton AST-CD
redefinition of nonterminal String name
State to allow substates:
conservative extension * *
State Transition
boolean initial String from
boolean final String input
* String name String to

24 Software Engineering | RWTH Aachen

Example: Language Inheritance in Expressions

• Adding new infix expressions with certain priority

• Priorities are numbers with a larger step size (to leave open gaps)

1 grammar Expression1 { MG
2 interface Exp;
3 AndExp implements Exp <90> = left:Exp "&&" right:Exp;
4 OrExp implements Exp <80> = left:Exp "||" right:Exp;
5 }

1 grammar Expression2 extends Expression1 { MG

2 XorExp implements Exp <85> = left:Exp "xor" right:Exp;
3 }
priority 85 is between “&&” and “||”
Interface is extended, no overriding needed

25 Software Engineering | RWTH Aachen

Language Restriction by Production Redefinition

• Overriding a production allows to

 restrict variants
 Here: no final states anymore
 But: care for unused attributes in the AST

1 grammar InfiniteAutomaton extends Automaton { MG

2 State = "state" Name
3 ("<<" ["initial"] ">>")?
4 ("; " | "{" State* "}" ) ;
5 }

redefining nonterminal State to AST-CD

forbid a part of the production «gen»
by omitting it in the new body
State
Please note:
boolean initial
Unused attribute (but in the AST *
boolean final
inherited from super-grammar) String name

26 Software Engineering | RWTH Aachen

Language Composition in the Large

For grammars G1, G2, …:

G1
• “G2 extends G1” can be used in the small:
extends
 Override a few/single nonterminals
 Add new nonterminal variants using extends or implements G2
 Here: G2 is an adaptation of G1

• “G2 extends G1” can be used to include libraries:

G1 G2
 G2 nonterminals use G1, but do usually not adapt them
 Here: G2 reuses some library components extends includes

• “G extends G1, G2, G3, …” can be used for component composition: G

 External/interface nonterminals are glued together, resp. filled

 Here: Language G composes languages G1, …

27 Software Engineering | RWTH Aachen

 Software Language Engineering
10. Language Composition
10.4. Conservative Extension

Prof. Dr. Bernhard Rumpe

Software Engineering
RWTH Aachen

[Link]
Farbe!
Overriding Nonterminals

• … is rather flexible, but some restrictions help preserve properties

• (similar to overriding methods in Java)

1) Free modification

2) Conservative extension of the AST

 the AST-functions can be reused (visitors, mills, hc-functions)

3) Conservative extension of the CS

 the models can be reused

4) CS and AS compliance
 the models can be reused

29 Software Engineering | RWTH Aachen

Conservative Extension of the AST

• Conservative extension of the AST • Old:

 preserves all nonterminals in their cardinalities MG
 and all semantically relevant terminals.
S = "state" Name
• Consequence:
 the AST-functions can be reused • New, conservative extension:
(visitors, mills, handcoded-functions) MG
S = "deep-state" Name "{" Body "}"
• Conservative extension allows to
 extend AST (namely the productions)
 rearrange order
 relax cardinalities, e.g. from + to *

30 Software Engineering | RWTH Aachen

Conservative Extension of the Concrete Syntax (CS)

• Conservative extension of the CS • Old:

 preserves and only extends the concrete syntax. MG
S = "state" Name
• Consequence:
 the models can be reused,
because all models remain valid • New variants, CS-conservative:
MG
• It is allowed to S = "state" Name ("{" Body "}")? "endstate"?
 add optionals (?),
 add additional alternatives ( .. | ..)
S = "state" Name+ Body
 and add lists (*)
 relax cardinalities
 from 1 to ?, + or *, and S = (["state"] | ["blue-state"]) Name
 from ?, + to *

31 Software Engineering | RWTH Aachen

Pathological Case: Compliance Problem

• Old: MG
A = s:Name t:Name

• Overridden by: MG

A = t:Name s:Name

• is both CS-conservative and AS-conservative,

• but models are interpreted differently now.

• We note: Rearranging order can be harmful.

32 Software Engineering | RWTH Aachen

 Software Language Engineering
10. Language Composition
10.5. Example: UML as Composed Language

Prof. Dr. Bernhard Rumpe

Software Engineering
RWTH Aachen

[Link]
Farbe!
Example: UML as Language Composition

• Grammars for each language component

• UML diagrams also share language components:
 Basics (Names)
 Types
 Stereotypes, Tags
 Completeness-Info
 OCL-Expressions
 Actions (Statements)

class object
architecture diagrams diagrams
statecharts Fun, C++, sequence
diagram Java, Pyt. … __: __: diagrams
__:
OCL

34 Software Engineering | RWTH Aachen

UML Modularity and Configuration also in Code Generation and Testing Needed

class
architecture object
diagrams statecharts sequence
diagram PLs: Fun., C++, diagrams
__: __: diagrams
Java, Python, …
__:
OCL

parameterized
consistency test code
code
analyzer generator
generator

“smells” &
errors system tests

35 Software Engineering | RWTH Aachen

UML possible Symbol Exchange between Sublanguages

• Numerous kinds of symbols are introduced

• and should be exported to other models

class statecharts object sequence

architecture diagrams
diagram diagrams diagrams
__: __:
C++, OCL __:
Java …

Possible flow of symbols:

attribute, class, method, constant, state, port, object

36 Software Engineering | RWTH Aachen

UML Language Component Embeddings

• Language constructs in UML are not disjoint

• Certain nonterminals define language elements and are reused in other languages
class
diagrams object
architecture statecharts sequence
C++, diagrams diagrams
diagram Java … __: __:

OCL __:

method statement object

expression Reuse of
nonterminals
expression (language
components)

type

type

37 Software Engineering | RWTH Aachen

 UML as Composed Language

• The Language UML deconstructed into a hierarchy of sublanguages (layer 2)

and individual language concepts (base layer)
class
Layer 3: architecture
diagrams
statecharts object sequence
Fun, diagrams
“Multi-Viewpoint” diagram Java … __: __:
diagrams

Language UML __:

OCL

diagrams, logic
PL constructs

method
type
statement
expression

38 Software Engineering | RWTH Aachen

 UML as Variably Configurable Language

• Some of the sublanguages are highly configurable

 E.g. which type system to include, which statements, expressions, …

class
Layer 3: architecture
diagrams
statecharts object sequence
Fun, diagrams
“Multi-Viewpoint” diagram Java … __: __:
diagrams

Language UML __:

OCL

1) choose
between
alternatives

2) extend with
new grammars

3) choose symbol mappings and import flows for diagram composition

39 Software Engineering | RWTH Aachen

Extensible UML

• UML is extensible
 Certain properties are currently not part of the UML 2 standard
 But extension should be easier and more reliable that defining stereotypes only.
class
diagrams object
architecture statecharts sequence
Fun, diagrams diagrams
diagram Java … __: __:

OCL __:

Uncertainty /
SI Units Probability Tabular
Variability Notations
Time

Continuous Geometry /
Behavior Space

In essence: compositional approach to language development is key

40 Software Engineering | RWTH Aachen

Current Research Topics

• Concrete compositions of Software Engineering languages?

 and their benefits in development

• How to combine languages from:

 mechanical engineering (CAD, CAM, CAS…)
 and software modelling?

• How to use multi-modelling for effective simulation?

• How to substantially improve a model-based development process for Cyber-Physical Systems?

• How to manage variability of model languages?

41 Software Engineering | RWTH Aachen

 Software Language Engineering
10. Language Composition
10.6. Generated Compositional Infrastructure

Prof. Dr. Bernhard Rumpe

Software Engineering
RWTH Aachen

[Link]
Farbe!
Composed Parser

• Parsing methods of G2 contains also methods to

parse G1-nonterminals (e.g. D)

• G1-parser still can be used, but delegates to G2-

parser,
 i.e. it looks like the G1-parser but now delivers G2-objects
 it delivers AST nodes from G2 (e.g. subclass g2.C)
with signature of G1 (e.g. superclass g1.C)

• Consequence: G1-functionality can fully be reused

grammar G1 { C = ... ; D = ... ; } MG

grammar G2 extends G1 { C = ... ; E = ... ; }

43 Software Engineering | RWTH Aachen

Composed Mill to Retrieve Builders

• In composed languages the mills are composed as well:

 composed mill delivers the proper builders
 old mills also deliver the proper builders of the G2-language

• Advantage: grammar G1 { C = ... ; } MG

 Reuse of G1-functionality
grammar G2 extends G1 { C = D; }
(if extension is AST-conservative, so that build doesn’t fail)

G1Mill [Link] g1.C

CD
static [Link] cBuilder() «gen»
instantiates

instantiates [Link] g2.C

Dd Dd
G2Mill C build() void setD(D d)
static [Link] cBuilder()
CBuilder setD(D d) // …
// …

44 Software Engineering | RWTH Aachen

Composed Mill to Retrieve Builders - 2

• To make composition work we have a strong recommendation:

 Do not use constructors directly!

 Do not use builders directly, but only through mill!

• Conservative extensions allow to reuse given functions

 such as pretty print, CoCos, visitors, calculations, etc.

• Some restrictions apply, because Java has no multiple inheritance

45 Software Engineering | RWTH Aachen

Composing Cocos of Languages

• One CoCo is defined locally on one AST node

• Consequence:
 unchanged parts of a language can reuse their CoCos.
 conservative extension also conserves reuse of CoCos.

• Additionally:
 new CoCos that cross language borders?
 or relax (and rewrite) some CoCos?

• CoCos are highly diverse:

 no general guideline for CoCos, only best practices:
 Best practice: write CoCos in a decomposed, localized form
(each CoCo on one AST node and calculate intermediate results if needed)

46 Software Engineering | RWTH Aachen

Visitor Reuse on Language Composition

• Goals: Reuse of visit / endVisit / traverse of

MyG1Visitor2 for the MyG2Visitor2
• Approach: several visitor objects (for each language
component)

«gen» «hc»
grammar G1 { MG CD
G1Visitor2 … MyG1Visitor …
A = "a";
+ visit(ASTA n) + visit(ASTA n)
}

grammar G2 extends G1 { «gen» «hc»

B = "b";
} G2Visitor2 … MyG2Visitor …
+ visit(ASTB n) + visit(ASTB n)

47 Software Engineering | RWTH Aachen

Visitor Reuse on Language Composition

• G2Traverser extends G1Traverser and … «gen»

CD
• manages all visitors of the G1 and G2 grammar «interface»
G1Traverser …
• Principle: Delegation of the visit methods
+ handle(ASTA n)
+ traverse(ASTA n)
+ visit(ASTA n)

«gen»
grammar G1 { MG «interface»
A = "a"; G2Traverser …
} + handle(ASTB n)
+ traverse(ASTB n) «hc»
+ visit(ASTB n)
grammar G2 extends G1 { MyG1Visitor2 …
B = "b"; «gen» + visit(ASTA n)
} G2Traverser
Implementation … «hc»
+ add4G1(G1Visitor2 n)
MyG2Visitor2 …
+ add4G2(G2Visitor2 n)
+ visit(ASTB n)

48 Software Engineering | RWTH Aachen

Language Composition and Late Binding

• Modular, incremental generation:

 each grammar can be mapped separately

grammars .java .class .jar

G1 G1
AST, builder, mill,
G1 AST … G1
visitor, etc.
grammar language
G1 G1
parser parser

Pipeline of generation

49 Software Engineering | RWTH Aachen

Language Composition and Late Binding

• Modular, incremental generation:

 each grammar can be mapped separately
 late binding for all language constituents (except parser)
 reuse of pre-compiled sublanguages as black-box
grammars .java .class .jar
G1 G1
AST, builder, mill,
G1 AST … G1
visitor, etc.
grammar language
G1 G1
parser parser
extends
G2 G2
AST, builder, mill,
G2 AST … G1+2
visitor, etc.
grammar language
G1+2 G1+2
generate parser compile parser build a component

50 Software Engineering | RWTH Aachen

Summary

• Language composition, aggregation, inheritance and conservative extension facilitate reuse

• Reuse must/should be black box

(without modification or even recompilation)

• Reusable are:
 Grammars for concrete and abstract syntax
 the generated classes
 Handcoded extensions against a grammar
 Visitors, CoCos, Symbol management

• Libraries as reusable assets

51 Software Engineering | RWTH Aachen