SOFTWARE
TECHNOLOGY
Editor: Christof Ebert
Vector Consulting Services
[Link] t@[Link]
Requirements
Engineering Tools
Juan M. Carrillo de Gea, Joaqun Nicols,
Jos L. Fernndez Alemn, Ambrosio Toval, Christof Ebert, and Aurora Vizcano
Requirements engineering is the disciplined and systematic approach to elicit, specify,
analyze, commit, validate, and manage requirements while considering user, technical,
economic, and business-oriented needs and objectives. It spans the entire lifecycle,
often involving distributed teams and supply chains. Tools facilitate consistency and
efficiency in managing requirements. Finding out which tool is suitable for given needs
isnt easy. Our article aims to provide a brief overview on requirements engineering
tools. I look forward to hearing from both readers and prospective authors about
this column and the technologies you want to know more about. Christof Ebert
REQUIREMENTS ENGINEERING (RE) tools are quickly
evolving. The demand for flexibility, lean and agile development, worldwide collaboration, and advanced software and
systems ecosystems is changing how we manage requirements. For instance, agile teams are less document-centric
cycle. Their RE tool should assure service for generations of
software.
RE tools are adapting to these demands with changes to
their design and architecture. Traditionally, RE tools are proprietary and well maintained by their vendors; theyre often
oriented toward distinct environments
and niche markets (for example, automotive, medical, and defense), developDistributed development teams need
ment processes (for example, agile development, product management, and
to easily and comprehensively access
prototyping), or utilization settings (for
requirements and specifications
example, local versus global software
development). This alone is enough reawith traceability throughout the life cycle.
son to evaluate RE tools and technologies with different use cases.
and more code-orientedthey expect brief requirements directly related to code changesso their RE tool should be
lightweight. On the other hand, distributed development
teams need to easily and comprehensively access requirements and specifications with traceability throughout the life
Background
To evaluate currently relevant RE tools, we surveyed their
vendors and compared the results with user experiences from
typical RE use cases. Out of 94 vendors, 37 responded, representing the global marketplace: the US (21 tools); Can-
86 I E E E S O F T W A R E | P U B L I S H E D B Y T H E I E E E C O M P U T E R S O C I E T Y
0 74 0 -74 5 9 / 1 0 / $ 2 6 . 0 0 2 0 1 0 I E E E
ada (4); the UK (3); Germany (2); and
France, Israel, Japan, New Zealand,
Russia, Spain, and Switzerland (1 each).
The tools worldwide licensing varies: two have from 0 to 100 active licenses; nine have from 101 to 1,000; 13
have from 1,001 to 10,000; and nine
have more than 10,000. Four vendors
didnt answer this question.
The average cost per license is typically above $1,000 (17 tools). Five tools
are in the $501 to $1,000 range; four
are in the $100 to $500 range, and
four cost less than $100. Seven vendors
didnt answer this question.
Most tools require Windows (28
tools), although Web-based clients are
quite common to facilitate distributed
and collaborative access to resources
(9 tools). Other OSs, such as Linux (8),
Unix (6), and the Mac OS (4), have a
more limited presence.
Vendors typically charge for the
tools and keep the licenses proprietary
(34 tools), but a few offer other licensing, such as proprietary and free, open
source and free, or open source and
nonfree (1 each).
Tool Evaluation
We based the evaluation on typical use
cases and on ISO/IEC TR 24766:2009,
a framework for evaluating RE tool
capabilities. The document is a type 2
technical report (that is, theres a future but not immediate possibility of
an agreement on an international standard). It supplements the more general
ISO/IEC 14102:2008 standard, which
focuses on evaluating computer-aided
software engineering tools. According
to ISO/IEC TR 24766, a total of 157
RE tool capabilities fall into six major
categories (see Table 1).
We refined the questionnaire to comprise 126 questions organized in eight
feature categoriessix from ISO/IEC
TR 24766 plus two more: modeling and
traceability. We also added questions to
represent a meaningful set of features
(for example, traceability across a tools
TABLE 1
SOFTWARE TECHNOLOGY
The number of
requirements engineering tool features.
Category of tool capability
No. of features
Requirements elicitation
37
Requirements analysis
36
Requirements specification
16
Requirements verification and validation
34
Requirements management
17
Other capabilities
17
Total
157
boundaries, specific reporting facilities, and concrete modeling and specification languages). Furthermore, we
added 20 questions regarding the tools
basic administrative information (for
example, tool name, vendor name, and
current version). So, the final questionnaire had 146 questions.
Because the vendors are distributed
worldwide, we designed a Web-based
survey using LimeSurvey ([Link]), an open source survey application. Survey results are accessible
online (see Table 2 for links to our survey and others on RE tools).
Of the studys 37 participants, some
important vendors (for example, IBM
and IRqA) didnt address some sections
of the questionnaire. Some also showed
low commitment. So, we discarded incoherent or incomplete answers and
added new questions in favor of critical and lacking aspects of ISO/IEC TR
24766 to produce high-quality data.
The responses could contain bias because representatives tend to rank their
own tools positively. To counteract this
effect, the questionnaires preface carefully explained the research purpose
and protocol. To draw valid conclusions, we used triangulation: three researchers performed and validated the
statistical work, and three other researchers revised the findings.
Ranking Summary
Table 3 shows the participants and the
tools score in each category. The score
in the Global column represents the
level of accomplishment on the basis of
the entire ISO/IEC TR 24766 (that is,
when considering the distinct categories of features all together). Because
some tools didnt participate in all categories, we performed this calculation
on only those that did.
The RE tools scores in concrete use
scenarios can help determine their suitability for certain tasks. We assessed the
tools performance in three scenarios.
Next, we summarized the results and
highlighted helpful features along with
the percentage of tools supporting them.
These features represent the less accomplished capabilities in our scenarios
that is, capabilities that have a smaller
percentage of tools supporting them.
Scenario 1
This scenario addressed development
projects in which determining user
needs is criticalfor instance, those
involving various types of stakeholders
or geographically distributed stakeholders, or those with insufficient time for
requirements workshops. In such situations, ensuring that the final product
meets specifications and fulfills expectations is important. This scenarios
J U LY/A U G U S T 2 0 1 1
| IEEE
S O F T W A R E 87
TABLE 2
SOFTWARE TECHNOLOGY
Databases with requirements engineering tool surveys.
Database source
URL
Our tools survey (evolving)
[Link]/giisw/EN/re-tools-survey
37
Ian Alexander
[Link]
67
Alarcos Research Group
[Link]
International Council on Systems Engineering
[Link]/ProductsPubs/products/[Link]
34
Ludwig Consulting Services
[Link]/Requirements_Management_Tools.html
40
QAGuild
[Link]
Volere
[Link]/[Link]
71
@WEBO
[Link]/[Link]
41
activities included requirements elicitation and verification and validation
(V&V).
The tools support elicitation by
storing and managing elicitation
templatesfor example, quality
function deployment or goal-question-metrics (59 percent);
storing and managing elicitation
checklists (62 percent);
storing and managing prioritization
forms (59 percent); and
providing Object Management
Group Requirements Interchange
Format (ReqIF) compatibility for
exchanging requirements information (27 percent)ReqIF (formerly
RIF) defines an open, nonproprietary exchange format.
The tools support V&V by
generating exception reports on
verification plan cases with no verification procedures and on verification procedures not linked to verification plan cases (59 percent);
generating exception reports on validation plan cases with no validation procedures and on validation
procedures not linked to validation
plan cases (62 percent);
providing a standard format for in-
terfacing to verification tools (59
percent);
providing a standard format for interfacing to validation tools (59 percent); and
implementing built-in requirements
checksthat is, requirements verification capabilities that determine
whether requirements are complete, consistent, correct, modifiable, ranked, traceable, unambiguous, understandable, or verifiable
(51 percent).
The most capable tools for both
requirements elicitation and requirements V&V are Cognition Cockpit,
Cradle, QPack, and Reqtify. MKS Integrity and Polarion Requirements
stand out for elicitation; Aligned Elements, Case Spec, GMARC, IRqA,
PACE, ReqMan, and TraceCloud stand
out for V&V.
Scenario 2
This scenario refers to organizations
that want to establish a strong basis
for design and implementation, including modeling and requirements specification. The tools support modeling by
providing storage and display of
business-process-modeling notation
(46 percent),
88 I E E E S O F T W A R E | W W W. C O M P U T E R . O R G / S O F T W A R E
No. of tools
goal models (38 percent),
Systems Modeling Language artifacts (32 percent), and
data-flow diagrams (46 percent).
The tools support specification by
inspecting the document through
spell checking, grammar checking,
data dictionaries, and acronym tables (59 percent);
generating a finished specification,
including page security markings,
graphics or figures, user-definable
tables, and indexes (68 percent); and
synchronizing changes between
the RE tool and the formatted
documentcreating a complete
loop between the two (41 percent).
The most capable tools for both
modeling and requirements specification are Cognition Cockpit, Cradle,
and PACE. Reqtify stands out for modeling; GMARC, inteGREAT, MKS Integrity, QPack, Doors, TraceCloud,
and VisibleThread stand out for requirements specification.
Scenario 3
The last scenario is related to organizations seeking a high level of project control and quality assurance.
Achieving these is difficult by any
Price range
(single seat)
Global
Other tool
capabilities
Traceability
Management
Verification &
validation
Modeling
Vendor
Analysis
Tool
Specification
Requirement engineering tools' scores and prices.*
Elicitation
TABLE 3
SOFTWARE TECHNOLOGY
Acclaro DFSS
Axiomatic Design Solutions
++
n/a
++
n/a
n/a
n/a
$$$$
Aligned Elements
Aligned
++
++
++
$$$$
Avenqo PEP
Avenqo, Germany
++
++
--
++
$$$
Blueprint
Blueprint Software Systems
++
n/a
++
n/a
$$$$
Bright Green Projects
Bright Green
++
++
++
n/a;
Caliber RM
Micro Focus
++
n/a
++
n/a
n/a
Cameo Requirements+
No Magic
++
$$$
CASE Spec
Goda Software
n/a
n/a
n/a
n/a
++
++
++
n/a
n/a
Cognition Cockpit
Cognition
++
++
++
++
++
++
++
++
++
$$$$
Cradle
3SL
++
++
++
++
++
++
++
++
++
$$$$
GMARC
Computer System Architects
++
++
++
++
++
++
$$$$
inteGREAT
eDev technologies
++
++
++
++
++
++
++
$$$$
IRqA
Visure Solutions
n/a
n/a
n/a
n/a
++
++
++
n/a
n/a
jUCMNav
jUCMNav
n/a
n/a
n/a
n/a
n/a
Leap SE
Leap Systems
$$
MacA&D / WinA&D
Excel Software
n/a
n/a
n/a
n/a
$$$$
MKS Integrity
MKS
++
++
++
++
++
++
++
$$$$
PACE
ViewSet
++
++
++
++
++
++
++
++
$$$$
Polarion Requirements
Polarion Software
++
++
++
++
++
++
++
$$$
Psoda
Psoda
++
++
++
++
++
QFDcapture
International TechneGroup
--
--
$$$$
QPack
Orcanos
++
++
n/a
++
++
++
n/a
$$$
RaQuest
SparxSystems Japan
$$
Rational Doors
IBM Rational
++
n/a
n/a
n/a
n/a
n/a
$$$$
ReqMan
RequirementOne
++
++
++
Reqtify & Requirement
Central
Dassault Systemes
++
++
++
++
++
++
++
++
$$$$
* For the scores, ++ = very high, + = high, 0 = medium, - = low, and -- = very low. For prices, $$$$ > $1,000, $$$ = $501-$1,000, $$ = $100-$500, and $ < $100, free = , free version available with limitations = . n/a = not applicable.
J U LY/A U G U S T 2 0 1 1
| IEEE
S O F T W A R E 89
Price range
(single seat)
Global
Other tool
capabilities
Traceability
Management
Verification &
validation
Modeling
Vendor
Analysis
Tool
Specification
Requirement engineering tools scores and prices.*
Elicitation
TABLE 3, CONTD
SOFTWARE TECHNOLOGY
Rational Requirements
Composer
IBM Rational
n/a
n/a
$$$$
RTIME
QAvantage
++
++
$$
Rational RequisitePro
IBM Rational
n/a
n/a
n/a
n/a
$$$$
RMTrak
Prometeo Technologies
--
$$
Rommana
Rommana Software
n/a
n/a
n/a
n/a
$$$
SpiraTeam
Inflectra
++
--
++
TestTrack RM
Seapine Software
++
n/a
TopTeam Analyst
TechnoSolutions
++
++
$$$$
TraceCloud
TraceCloud
++
++
++
++
TrackStudio
TrackStudio
++
$;
VisibleThread
On-premise/On-demand
VisibleThread
++
++
n/a
n/a
n/a
n/a
n/a
$$$$
* For the scores, ++ = very high, + = high, 0 = medium, - = low, and -- = very low. For prices, $$$$ > $1,000, $$$ = $501-$1,000, $$ = $100-$500, and $ < $100, free = , free version available with limitations = . n/a = not applicable.
means other than requirements V&V
and traceability. We already addressed
V&V; the tools support traceability by
generating reports comparing current and previous versions when
a source document is updated (57
percent);
tracing across the tools boundaries
(57 percent);
tracing text to graphics (54 percent);
tracing graphics to graphics (49
percent);
tracing elements in graphics (43
percent);
tracing tables and table cells (38
percent); and
generating reports of traceability
attributesfor example, category,
number approved or unapproved,
number of changes, and number
pending (62 percent).
The most capable tools for both requirements V&V and traceability are
CASE Spec, Cognition Cockpit, Cradle, GMARC, and Reqtify. inteGREAT
stands out for traceability.
Discussion
Most tools obtained high or very high
scores in requirements elicitation (88
percent), analysis (76 percent), specification (77 percent), V&V (82 percent),
traceability (79 percent), and other capabilities (73 percent). However, a relevant set obtained medium, low, or very
low scores in modeling (42 percent)
and requirements management (39 percent). This information suggests that a
margin for improvement still exists for
modeling and requirements management, although weve found that generally, current tools cover RE well.
Moving to more fine-grained con-
90 I E E E S O F T W A R E | W W W. C O M P U T E R . O R G / S O F T W A R E
cerns, these RE tools dont extensively
support a small percentage of relevant
features. Although they deal well with
requirements elicitation, few of them
(for example, Caliber RM, Cognition
Cockpit, and Cradle) support ReqIF to
fill the gap in effectively sharing, communicating, and collaborating across
different tools. Support to ReqIF is
needed because companies usually
dont work on the same requirements
repository or use the same RE tools.1
Other tool capabilities, including
tool administrative information, GUI,
and data integration features, are well
covered. However, regarding data integration, only a few tools (for example,
CASE Spec, inteGREAT, and MKS Integrity) support database federation.
This feature provides users a virtual
data warehouse that eliminates the
need to transfer the data, provides ac-
SOFTWARE TECHNOLOGY
cess to live data and functions, and employs a single arbitrarily complex query
to efficiently combine data from multiple sources of different types.
As we noted, these tools dont completely support requirements management. For instance, they poorly support the open data model. This model
facilitates tool interaction with external
components by ensuring a standard format for the applications fundamental
data structures. This allows runtime access to the information without requiring a complex protocol. A few tools (for
example, Bright Green Projects, PACE,
and ReqMan) provide this functionality
by implementing the required features.
With the increasing complexity of
software development and products,
the need is growing for RE tools integrated in product life cycle management and application life-cycle management (PLM/ALM) architectures.2 The
software product life cycle treats RE in
an orderly fashion and is continuously
managed through an appropriate set of
tools (for example, MKS Integrity, Polarion ALM, and QPack ALM-suite).
he RE tools market is changing fast. Classic tools that used
to dominate the market are increasingly complex and difficult to use.
Many expensive tools aren't sufficiently
open to the tools of other vendors, such
as for modeling or traceability. This encourages newcomers to introduce interesting capabilities, especially for collaboration. We recommend carefully
evaluating the requirements of the RE
tool for your own environment. Be prepared to pay for what you expect. Cheap
tools don't deliver sophisticated features.
In the future, we plan to explore
whether current RE tools adequately
support features aimed at global software development environments, and
how these tools deal with the difficulties of distributed and collaborative
work.
Acknowledgments
We thank the surveys participants for their
invaluable help. The Pegaso/Pangea project
(TIN2009-13718-C02-01-02), Origin Integrated Project (IDI-2010043 (1-5), and Englobas Project (PII2I09-0147-8235) have funded
this research.
References
1. M.R. Monteiro, C. Ebert, and M. Recknagel,
Improving the Exchange of Requirements
and Specifications between Business Partners,
Proc. 17th IEEE Intl Requirements Eng.
Conf. (RE 09), IEEE CS Press, 2009, pp.
253260.
2. C. Ebert, Requirements Engineering: Management, Encyclopedia of Software Eng.,
P.A. Laplante, ed., Taylor & Francis, 2010,
pp. 932948.
JOAQUN NICOLS is an associate professor at
the Software Engineering Research Group, Faculty of
Computer Science, University of Murcia. Contact him
at jnr@[Link].
JOS L. FERNNDEZ ALEMN is an associate
professor at the Software Engineering Research
Group, Faculty of Computer Science, University of
Murcia. Contact him at aleman@[Link].
AMBROSIO TOVAL is a full professor at the
Software Engineering Research Group, Faculty of
Computer Science, University of Murcia. Contact him
at atoval@[Link].
CHRISTOF EBERT is managing director at Vector
Consulting Services. He is a senior member of the
IEEE and is on IEEE Softwares editorial board.
Contact him at [Link]@[Link].
JUAN M. CARRILLO DE GEA is a research as-
AURORA VIZCANO is an associate professor in
sistant at the Software Engineering Research Group,
Faculty of Computer Science, University of Murcia.
Contact him at jmcdg1@[Link].
the Alarcos Research Group, Escuela Superior de Informtica, University of Castilla-La Mancha. Contact
her at [Link]@[Link].
PURPOSE: The IEEE Computer Society is the worlds largest association of computing professionals and is the
leading provider of technical information in the field. Visit our website at [Link].
OMBUDSMAN: Email help@[Link].
Next Board Meeting: 2327 May 2011,
Albuquerque, NM, USA
Violet S. Doan; Director, Products & Services: Evan
Butterfield; Director, Sales & Marketing: Dick Price
EXECUTIVE COMMITTEE
President: Sorel Reisman*
President-Elect: John W. Walz;* Past President: James
D. Isaak;* VP, Standards Activities: Roger U. Fujii;
Secretary: Jon Rokne (2nd VP);* VP, Educational
Activities: Elizabeth L. Burd;* VP, Member &
Geographic Activities: Rangachar Kasturi; VP,
Publications: David Alan Grier (1st VP);* VP, Professional
Activities: Paul K. Joannou;* VP, Technical & Conference
Activities: Paul R. Croll; Treasurer: James W. Moore,
CSDP;* 20112012 IEEE Division VIII Director: Susan
K. (Kathy) Land, CSDP; 20102011 IEEE Division V
Director: Michael R. Williams; 2011 IEEE Division
Director V Director-Elect: James W. Moore, CSDP*
COMPUTER SOCIETY OFFICES
Washington, D.C.: 2001 L St., Ste. 700, Washington,
D.C. 20036-4928
Phone: +1 202 371 0101 Fax: +1 202 728 9614
Email: [Link]@[Link]
Los Alamitos: 10662 Los Vaqueros Circle, Los Alamitos,
CA 90720-1314 Phone: +1 714 821 8380 Email:
help@[Link]
Membership & Publication Orders
Phone: +1 800 272 6657 Fax: +1 714 821 4641
Email: help@[Link]
Asia/Pacific: Watanabe Building, 1-4-2 MinamiAoyama, Minato-ku, Tokyo 107-0062, Japan Phone:
+81 3 3408 3118 Fax: +81 3 3408 3553 Email:
[Link]@[Link]
*voting member, nonvoting member of the Board of Governors
BOARD OF GOVERNORS
Term Expiring 2011: Elisa Bertino, Jose CastilloVelzquez, George V. Cybenko, Ann DeMarle, David S.
Ebert, Hironori Kasahara, Steven L. Tanimoto
Term Expiring 2012: Elizabeth L. Burd, Thomas M.
Conte, Frank E. Ferrante, Jean-Luc Gaudiot, Paul K.
Joannou, Luis Kun, James W. Moore
Term Expiring 2013: Pierre Bourque, Dennis J. Frailey,
Atsuhiro Goto, Andr Ivanov, Dejan S. Milojicic, Jane
Chu Prey, Charlene (Chuck) Walrad
EXECUTIVE STAFF
Executive Director: Angela R. Burgess; Associate
Executive Director, Director, Governance: Anne
Marie Kelly; Director, Finance & Accounting: John
Miller; Director, Information Technology & Services:
Ray Kahn; Director, Membership Development:
IEEE OFFICERS
President: Moshe Kam; President-Elect: Gordon W.
Day; Past President: Pedro A. Ray; Secretary: Roger
D. Pollard; Treasurer: Harold L. Flescher; President,
Standards Association Board of Governors: Steven
M. Mills; VP, Educational Activities: Tariq S. Durrani;
VP, Membership & Geographic Activities: Howard
E. Michel; VP, Publication Services & Products:
David A. Hodges; VP, Technical Activities: Donna
L. Hudson; IEEE Division V Director: Michael R.
Williams; IEEE Division VIII Director: Susan K. (Kathy)
Land, CSDP; President, IEEE-USA: Ronald G. Jensen
revised 5 May 2011
J U LY/A U G U S T 2 0 1 1
| IEEE
S O F T W A R E 91
