MODULE 10: PROTOTYPING - TEST AND VALIDATE YOUR
PRODUCT OR SERVICE
Lecturer: Siemon Smid, Susan Vrona Bejina & Claudia Muresan
Duration: 3 hours (1h30 e-learning + 1h30 webinar)
OBJECTIVE
WHAT YOU’LL LEARN
At the end of this module, the
At the end of this module, the participants
participants should know how to
should know:
create a prototype and understand
its function and its relevance. • How to create a mock-up from
scratch.
• How the prototype should enable
you to test your assumptions with
your potential users.
ADVICE • How the prototype can help you to
present your idea more clearly and
Please feel free to join us on the to reduce your risks.
forum to stay updated on the • The importance of stakeholders in
course, interact with the speakers the testing and validation process.
and ask any question you may • Why it is important to have a
have. You can find it here: balanced team to develop a good
product.
[Link]
/forum/[Link]?id=568
2
Introduction and Learning Objectives
ACTIVITIES SUPPORTING MATERIAL
• Video: “Introduction to • Video transcript in Appendix A
Prototyping phase” with
Siemon Smid 4m35s
Definition of objectives and specifications
ACTIVITIES SUPPORTING MATERIAL
• Video: “Steve Balmer selling • Video “Steve Balmer selling you
you Windows 1.0” 1m05s Windows 1.0”
• Exercise: “Differences and
similarities between Windows • Video: “A Brief Overview of the
1.0 and latest versions?” 3m00s Scrum Framework”
• Video: “A Brief Overview of the
Scrum Framework” 2m21s
• Quiz 5m00s
3
How to build a prototype
ACTIVITIES SUPPORTING MATERIAL
• Video: “How did we create • Video transcript in Appendix B
FaST Prototype?” with Susan
Vrona Bejina 12m53s
• Exercise: “Create your own
prototype” 30m00s
• Quiz: “Prototyping” 5m00s
Testing and validating the prototype with the client
and Critical Points
ACTIVITIES SUPPORTING MATERIAL
• Video: “Testing and validating • Video transcripts in
the prototype” with Siemon Appendix C, D & E
Smid 3m54s
• Reading: “Pig and Chicken
• Video: “Interview to Claudia Fable”
Muresan” 6m59s
• Slides: “Stakeholder Analysis
and Pig and Chicken Fable” 10m00s
• Exercise: Stakeholder
Analysis Quiz 5m00s
• Video: “Team Analysis” with
Siemon Smid 2m36s
4
Summary
ACTIVITIES SUPPORTING MATERIAL
Video: “Summary” with Siemon • Video transcript in Appendix F
Smid 1m07s
5
APPENDIX A: “INTRODUCTION TO PROTOTYPING PHASE” WITH
SIEMON SMID [TRANSCRIPT]
My name is Siemon Smid. I am the Director of the in International Business Development at
PwC in Luxembourg and take an active part in the development of Copernicus products and
services.
Prototyping is an essential element for the success of new Copernicus products and services.
Here are some questions that anyone thinking about developing a prototype should consider:
• Do you know how to build a prototype?
• Do you know how to keep the costs of prototyping under control?
• Do you know how to test and validate a prototype?
If you don’t know the answers to these questions, then stay tuned.
Today, I will talk about how you can build a prototype, how you can keep its costs under control
and how you can assure it fulfils the expectations of its potential users. I can tell you, from
personal experience, that prototyping is a major challenge. Even after my more than 40 years
of experience in innovation projects for industry and the public sector in Europe, the Middle
East and Asia, I find successful prototyping daunting. The difference between what I had to
work (with) 40 years ago and what I have now, is that many good methodologies and practices
have been introduced. I will talk about some of those methods later in this module.
To make it easier to understand how these methods work in practice, I am going to give you
some insights by showing how we here at PwC build prototypes. In the previous video, Susan
Coleman gave you answers to some main questions in ideation. She explained:
• How do you identify your audience? By using customer journeys and personas.
• How do you create a list of “pains”? By using painstorming and design thinking
principles.
• How can you be sure that you are solving the right problems for the right customers?
• And most importantly for prototyping, she introduced solution modelling.
Solution modelling is all about taking the outputs of the painstorming and beginning to run
“what if” scenarios around them. As a result, you will have created all the different ways the
pain points could be dealt with. So, by now you should be able to choose your preferred
solutions from the set of all the ways you could address the painpoints.
6
Now YOU are ready to move on to the next phase of the creative process: Let’s create your
prototype!
During this module, you will learn how to create a prototype and understand its functions and
its relevance. We will illustrate the creation of a prototype using an example out of the daily
practice of PwC. As we speak, we are developing a new app for the European Commission. This
app uses Copernicus data to optimise the usage of nutrients in farming.
Is this something new?
No, it isn’t, but what is new is that we are developing an app which can be used by all farmers
in all regions in Europe allowing them to conform to the post 2020 Common Agricultural Policy.
But you still might think that our app competes with other agricultural apps which are already
on the market. I want to convince you that our app is unique and does not compete with
commercial service providers. We will integrate the basic app into a platform on which
commercial developers can access data and build their own value-added farming apps.
Now, let’s come back to the prototype of the app.
Before starting with developing the tool, we created a prototype of the app. This prototype
was part of a feasibility study we finished in the beginning of 2019. Now you might want to
know what kind of app this is actually is. The App is called the FaST tool which stands for
Farming Sustainability Tool. In this module, you will learn how to build a prototype along the
lines we used to build the FaST tool. In this module, the FaST tool prototyping will be presented
by my colleague Susan Vrona Beijina from our data lab in Toulouse and by Claudia Muresan,
the project owner from the European Commission.
After this intro and recap, let's start with our module.
Are you ready?
7
APPENDIX B: “HOW DID WE CREATE FAST PROTOTYPE?” WITH
SUSAN VRONA BEJINA [TRANSCRIPT]
Hi, my name is Susan Vrona Béjina, and I’m a Senior Manager at PwC in Technology Consulting
Data & Analytics. I started my career as a research biochemist, but I switched to software
development during the internet boom. Over the years I have developed an expertise in
applying Agile methodology to product development. But my real passion is putting the user
experience first in the development of digital platforms and new products and services.
Now that you know a little bit more about prototyping from Simeon’s presentation, I’d like to
share with you a real-world example of a prototype that uses Copernicus data.
This prototype is called FaST. It was introduced by Siemon earlier in this Module.
So, what exactly is FaST? FaST stands for the Farm Sustainability Tool. The purpose of FaST is
to help EU farmers to farm in a sustainable way. I was a member of the team that delivered the
FaST prototype and I’m going to tell you how we did the prototyping of FaST. The first tool we
used in developing FaST, was the test card.
What exactly is a test card? A test card allows you to create a high-level description of your
product and its contents in 6 easy steps. The 1st is to name your product.
The second step is to state the goals of your product that will allow your customers or clients
to get things done. The next thing you need to define are metrics. Metrics will be used to
determine if your product delivers on the user’s goals. An example of a metric would be: The
farmer can create a new trend management plan. Another could be: The farmer can save the
new trend plan or the farmer can send the new trend management plan to the paying agency.
You get the idea.
The next step in filling in your test card is to define the product’s target group. These are your
customers or the users of your product. We expand our understanding by expressing the traits,
intentions and needs of our users through the development of personas. Susan Coleman
introduced the concept of personas in Module 9.
The big picture is the desired user experience. Using your personas, imagine a day in the life of
a given persona. Trace their journey through the activities that they perform to achieve their
goals. This is the Big Picture. It determines the required functional and non-functional aspects
of your product. An example of a functional aspect would be “Communicate with Farmers”. A
non-functional aspect might be the need to connect an existing email system in order to put
this functionality into place.
8
Finally, we fill in the product details. The product details are made up of a list of user stories. A
user story states which persona wants to do what and why and will be used to define the first
iteration of your product prototype.
Let's make more concrete what I’ve just described by examining the test card for the FaST
platform.
The European Commission wanted to create a level playing field for EU Member States for
nutrient management planning, that is, the application of the appropriate amount of fertiliser
to the farmer’s crops. And they wanted to do this using a common platform. The platform would
also allow member states to communicate with the farm community. And, it would support the
needs of researchers wanting data on sustainable farming.
Finally, a prototype of the platform would allow the European Commission to communicate
about the FaST with the future users of the platform. The users are:
• Farmers & farm advisors
• Paying agency administrators
• Researchers / scientists
I think everyone understands what farmers and researchers are… But what are paying Agency
Administrators? Paying Agency Administrators are the people within the Member States who
are helping farmers to meet their obligations for sustainable farming...
Back to our test card. Once our test card was developed, describing the context for the FaST
platform and the future users of the FaST, we developed personas which captured the sought
after gains, the existing pains and the goals for the different users of the system in more detail.
There were listed in the FaST test card in our example.
But we actually came up with the list of personas by interviewing the different stakeholders
about FaST. The personas we came up with were:
• The cost-conscious farmer
• The informed farmer
• The technology minimalist farmer
• The happy paying agencies
• And the data enthusiasts
Just the names of the personas captured the different ways the users of the platform needed
to interact with FaST to achieve their goals. You may have noticed that we had several personas
for our farmers. Why so many?
9
Each persona captures a different point of view for the farmer. For example, the technology
minimalist farmer means, we needed to take into account the farmer’s ease with technology
when designing our prototype and of course, for the final product.
Now, let’s look at the happy member states and paying agencies persona in more detail. We
can see that the paying agency administrator wants an efficient way to communicate with the
farmer while at the same time to ensure compliance for nutrient management via a digital
platform.
Equipped with this information of the paying agency persona, we translated our understanding
into individual user stories for each desired piece of functionality, while paying attention to the
sought-after gains and avoiding or removing the indicated pains. The paying agency worker
wanted the ability to communicate with one-to-many farmers in order to diffuse important
information. This was captured by the following user story: “As a paying agency administrator,
I want to be able to communicate with one-to-many farmers in order to keep them up to date.
For example, I want to communicate deadlines for compliance or regulation updates.”
As soon as the first user stories for FaST were developed, we were ready to start iterating on
the FaST prototype. The first step was to create a storyboard using an online tool. This tool is
called ‘Figma’.
The storyboard was a way to test how the user story could be accomplished within the FaST
platform. The storyboard was very high level, but it allowed the stakeholders to validate what
the core features would be for the platform.
Once the high-level features were agreed upon, the next iteration of the prototype was a more
detailed version, this time showing the step by step workflow for the first user stories. You
might wonder: When do the potential users get to interact with a physical prototype in a hands-
on way?
In our case, we showed each iteration of the developing prototype to the stakeholders. This
allowed us to test and improve our understanding of the user’s needs and to correct ergonomic
or design errors and to expose any shortcomings or functional gaps.
Perhaps you have another question in your head right now: “While all this theoretical work is
going on, do you have an IT or dev team building the platform?” The answer to your questions
is: “It depends”. You can continue to iterate in a ‘low fidelity’ (that means not real) way on your
prototype as long as you feel you are getting to the needed level of understanding to start
constructing your product.
10
Your technical team will also be developing an understanding of what they need to do to build
the product. They will probably start developing their technical blueprint, create basic technical
foundations or perform other set up activities. At some point however, your prototype can be
completely thrown away, so you can start developing your product. In fact, it is rare that you
can completely reuse your prototype assets because they may not consist of the same elements
of code that will work in your final product.
That being said, you have definitely saved money, time and resources while iterating on your
prototype.
In the case of the FaST, the final iteration of the prototype looked very much like a completed
product and it is still used today to educate the future user base on the capabilities of FaST,
while the final product is (now) under development. I hope this FaST prototype explanation
gives you a more concrete understanding of what a prototype could be and how to go about
building one.
What do you say, ready to try prototyping yourself? We’ve defined an exercise for you with
text from an article regarding the monitoring of the environmental effects of deforestation,
using Copernicus data.
Follow the steps indicated in the instructions! You can use a digital prototyping tool referred to
earlier in my presentation, called Figma. If you can’t access Figma and would like a more guided
approach, you can work out the exercise using paper and pen or another tool and compare your
paper prototype against the example solution we have provided.
A paper prototype is a valid form of prototyping! Just remember: There are no wrong solutions!
Just creative approaches to using Copernicus data to create prototypes for new products and
services!
In summary, you have learned three things to take away from this prototyping example:
• You have developed a deeper understanding of prototyping techniques,
• You have seen examples of each prototyping technique applied to a real-world
example in FaST
• And you have a better understanding of what a prototype is and how it differs from a
fully developed product
Thanks for spending time with me today, I’ve really enjoyed it!
11
And I hope you have too!
12
APPENDIX C: “TESTING AND VALIDATING THE PROTOTYPE” WITH
SIEMON SMID [TRANSCRIPT]
In the previous video by Susan, YOU learned how to create a prototype on FIGMA. We
explained that by using the example of the Farming Sustainability Tool.
How important do you think it is to check and validate your prototype with your client? Let’s
hear it from Oisin McGrath from DroneSARPilot, who you saw in Module 8:
From an early stage, during the beta test versions of the DroneSAR software, there were a
number of Rescue Agencies involved. The Irish Coast Guard were early stage users of
DroneSAR and provided continual feedback on features that would be helpful, necessary, and
also reported back with any bugs that were in the software. Having honest feedback and more
importantly an operational input to the features of DroneSAR was hugely important to the
success of the product. The product has been regularly updated based on input from various
teams, various terrain types and for various emergency response scenarios.
To keep it simple: With a prototype, you do a reality check to determine if your product is really
fit for purpose.
What do you think are the most famous prototype checking events? Do you know?
The most famous prototype presentations are done by car makers at car shows, like the
Frankfurt Motor Show. To take advantage of such prototype demonstrations to your potential
customers, you need to know in which context you will ask them to evaluate your prototype.
Will you, for instance, ask them to test it as a private user or on behalf of an organisation? Will
you ask them their personal opinion, or from the point of a user group? This is exactly what car
makers are doing at car shows. Therefore, it is crucial to know WHO your potential customers
are and what relation they have to other customers.
It is also essential to know where they are located. Their location influences their perception of
your prototype. A customer in Berlin might have completely different expectations compared
to a customer with the same profile in Tokyo. And what do you think about the BENEFITS of
having potential customers test and validate your product? Of course, one of YOUR benefits is
to get feedback from people inside and outside the development team to IDENTIFY FLAWS in
your prototype early on so that you can IMPROVE the prototype for future checking iterations.
This helps you (to) SPEED UP the development process and keep development costs under
control. But first of all, you need to create a prototype that can be tested and validated, or at
least a device to show the prototype. With a prototype in your hand, you can create a test plan,
to define what is going to be tested and how it is going to be tested.
13
The great thing about FIGMA is that you can create a test plan, too. There are different types
of testing that you need to consider before validating your prototype. The first option is to
present your prototype to real users in a real context. This will help you (to) get more realistic
feedback. You can also walk through a digital service with users to get their impressions.
Alternatively, you can quickly test prototypes with internal stakeholders to clarify ideas or
concepts.
Another way to get objective feedback is giving participants clear instructions for tasks to solve
and integrate their test feedback into the next generation of prototypes.
As you can see, there are many types of testing that you can use to validate your prototype and
get your final product validated.
14
APPENDIX D: “TEAM ANALYSIS” WITH SIEMON SMID [TRANSCRIPT]
The success of your prototype not only depends on tests and validations with customers.
Another critical consideration is the amount of human resources that (are) is required to create
a prototype, that is your development team.
It is very important to build a good combination of competences in your team, because the
BEST IDEAS emerge when DIFFERENT PERSPECTIVES are expressed. Most often the team
that is looking to develop new products or services includes people from: IT, Sales, Marketing,
Product Management and R&D.
If you already have a team, you must EVALUATE if your team possesses the necessary
competences and experiences. If not, you will need to bring someone into your team to cover
the missing elements. One way to do this could be to do a search in LinkedIn for someone who
has the skills you need and who might be interested in joining your team.
Which roles do you think are essential in your innovation team?
If we think in terms of a team of 8 people, you need a CEO, a Chief Executive Officer, who
oversees leading, managing and developing the business. You need a CTO, a Chief Technology
Officer, who is the person in charge of developing the technical Earth observation products and
services. You also need a CMO, Chief Marketing Officer, who oversees the marketing, pricing
and sales. Don’t forget about the CDO, the Chief Data Officer, who is responsible for designing
the user experience with the products and the user interface. You also need a CBO, a Chief
Brand Officer, who oversees the business and ensures (the) values of the company are
maintained. And OF COURSE, you need an EARTH OBSERVATION EXPERT who knows how
to use Copernicus data and services.
Several of these roles can be assumed by the same person.
In addition, there are other types of functions that every team should have, such as timekeeper,
secretary, messenger, reporter or party starter, that are equally or more important than those
previously mentioned.
Now you have all the pieces that will allow you to create the team that will develop a successful
prototype.
Are you ready to make it work?
15
APPENDIX E: “INTERVIEW WITH CLAUDIA MURESAN” [TRANSCRIPT]
So far, we have presented prototype development from the point of view of developers. Now
it is time to show the prototype development from the perspective of the customer. We will
talk with Claudia Muresan, the promotor of FaST and policy officer at the European
Commission. We will ask her about her experiences during the prototype process and the
importance of her advice during that process.
Hello Claudia. Thank you for joining us! What was your role during the development of the
FaST?
My role in the European Commission is to make sure that EU Copernicus satellite data find real
use cases and live up to their potential for different economic sectors and for different EU
policies. Given the importance of satellite data for agricultural applications, our efforts are
particularly focused on the Common Agricultural Policy.
Our strategy is to work with high-impact large-scale projects, such as the Farm Sustainability
Tool and platform. So… The European Commission proposes this tool as a significant innovative
element in the new Common Agricultural Policy, that is the policy after 2020. The Farm
Sustainability Tool will be a mobile application offered by member states to all farmers that
receive EU subsidies. This tool will integrate Copernicus data, so satellite data and information
coming from the Copernicus services with data from many other repositories and databases, so
that farmers have a clear picture of their farm, of the environmental and administrative
obligations. It will simplify the farmers’ relations with the public authorities, help them comply
with applicable legislation, enable many types of on-farm decision support applications,
especially on environment and climate.
A first real use case mentioned in the Common Agricultural Policy (CAP) legislation is nutrient
management, that is helping farmers to choose the type, quantity of fertilisers, to use them at
the right time and to use them for the right crops. The idea is that more data would translate
in less chemical input, with good effects on the environment and also on the farm’s
competitiveness. So, a win-win solution.
Who are the stakeholders to be involved in the test and validation phase of the prototype for
FaST?
In the 1st place, member states are obliged by the common legislative framework to have a
system that gives the farmer this tool. On the other hand, the European Commission is obliged
to offer support to member states introducing this change.
16
In the Member States, it is the Paying Agencies that are responsible for giving subsidies to
farmers after they made sure that the farmers comply with a set of environmental rules and
conditions.
The tool will therefore enable the communication between Paying Agencies and farmers, while
also helping farmers to comply more easily with the environmental legislation.
Besides the fertilisation plans, this tool will make it possible to add other modules such as on-
farm carbon counting, pesticides or water management, irrigation and so on.
For the system to be a success, it needs to provide real value both for the farmers and for the
member state’s agencies that will use it.
We, in the Commission, are helping on this. We started an action, in January 2020, we kicked
off the development of a working FaST tool with the Paying Agencies from three member
states, where this tool will be customised with input also from the final users that are like
farmers researchers helping in the system and so on.
The plan is to scale up the platform for all interested member states, with the same approach
applied to the first three.
How can a mock-up help you to understand the final product?
What is very special about the Farm Sustainability Tool is that the European Commission
described the tool in its basic functionalities in the legislation. This gives certainty to the
Member States as to what is expected but leaves a lot of room for innovation, customisation,
as well as for taking into account local farmers and their needs, within a common framework.
The aim is for the FaST tool to be developed as a user centric application, to be fine-tuned to
national, regional and farm characteristics.
The European Commission provided a mock-up or a uprototype and the blueprint of an IT
architecture for the entire platform. The mock-up made it possible to show the intentions of
the Commission explicitely towards the member states, towards the Paying Agencies, towards
the farmers and the public.
People tend to simplify prototypes and leave certain parts out. In this case, what could have
been left out?
We actually didn’t leave anything out, we added some features to the prototype that will not
be in the final product.
We asked that anyone should be able to experience, not only the use of the first instance of
the application but also to be able to envisage or imagine how the further modules and the
further uses could develop.
17
We made available a demonstrator online for both Android and IOS phones. And in this
demonstrator we included examples of further services supported by the platform – developed
both by public and private actors.
The idea was to show possible future developments, and also show the flexibility and the
modularity of our approach.
What advice can you give to public authorities who want to develop a prototype on the basis
of the Copernicus data?
I would say, start with your policy objectives before thinking about digital technologies or
design, and then create a workflow that will clearly allows for the input of all relevant
stakeholders. Then, think in functional terms, so that you leave space for innovation and for
future-proofing your solution rather than describing technology.
Describe only the essential elements and functionalities of what you need. You don’t have to
describe how it will work. Leave that up the developers. Use their creativity to get a state-of-
the -art tool that will last.
Thank you very much for joining us and providing us you very nice insights.
Thank you and enjoy the MOOC!
18
APPENDIX F: “SUMMARY” WITH SIEMON SMID [TRANSCRIPT]
That’s all I want to present to you all in this module. We will talk more about prototyping during
the Webinar. Before I end, I would like to summarise the most important points that you should
keep in mind from today’s presentations.
We hope that after this hour, you feel able to create a prototype and that you agree with us
that it doesn't have to be something EXPENSIVE, DIFFICULT or SOPHISTICATED. A simple
mock-up like FaST can be the DIFFERENCE between the SUCCESS or FAILURE of your product
or service.
Don't be afraid of receiving feedbacks about your product during this phase.
This phase is just for THAT. FEEDBACK! You need feedback so that you may CORRECT and
CHANGE the prototype. TEST and VALIDATE your prototype with customers and
stakeholders. They will give you the KEY that will make your service UNIQUE.
And if you are a stakeholder, be generous and constructive with your advice and experience.
Your OPINION is essential for the SUCCESS of the product.
Thank you very much for your attention and I will see you in the Webinar.
19