Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

Multi-Agent Visualization for Explaining Federated Learning

Xiguang Wei1 , Quan Li1 , Yang Liu1,2 , Han Yu4 , Tianjian Chen1 and Qiang Yang1,3
1
AI Group, WeBank
2
The Joint NTU-WeBank Research Centre of Eco-Intelligent Applications (THEIA)
3
Department of Computer Science and Engineering, Hong Kong University of Science and Technology
4
School of Computer Science and Engineering, Nanyang Technological University (NTU)
{xiguangwei,forrestli,yangliu,tobychen}@[Link], [Link]@[Link], qyang@[Link]

Abstract
As an alternative decentralized training approach,
Federated Learning enables distributed agents to
collaboratively learn a machine learning model
while keeping personal/private information on lo-
cal devices. However, one significant issue of this
framework is the lack of transparency, thus obscur-
ing understanding of the working mechanism of
Federated Learning systems. This paper proposes
a multi-agent visualization system that illustrates
what is Federated Learning and how it supports
multi-agents coordination. To be specific, it allows
users to participate in the Federated Learning em- Figure 1: System pipeline. Users are involved in the car racing game
powered multi-agent coordination. The input and play. Their view observations and actions are used to train two kinds
output of Federated Learning are visualized simul- of AI models, which are local AI and FL. Both AI models can drive
taneously, which provides an intuitive explanation one or more cars to race in the game. Their data are then fed into the
of Federated Learning for users in order to help visualization system for further demonstration.
them gain deeper understanding of the technology.
Despite its wide applicability in industrial, medical, and
1 Introduction financial scenarios [Liu et al., 2018; Huang and Liu, 2019;
Huang et al., 2018; Hankz Hankui Zhuo and Lin, 2019;
With advancement in data collection techniques and high-
Kumar et al., 2017], FL has its own problems. The most
efficiency computing devices, data-driven machine learning
significant one is the lack of transparency behind their be-
has become the mainstream of engineering nowadays. Con-
haviors, which leaves users, e.g., collaboration partners and
ventionally, in these data-driven systems, a centralized ap-
customers, with little technical background in this area very
proach is adopted by traditional machine learning which re-
confused. The concerns about the non-transparent nature of
quires the training data from different sources to be aggre-
FL have hampered its wider adoption [Du et al., 2018].
gated on a single machine or in a datacenter. This centralized
In this work, we showcase a platform for intuitive demon-
training approach, however, is privacy-intrusive. In many
stration and explanation of how a typical FL system works.
applications, users have to sacrifice their privacy by sharing
To be specific, we built a multi-agent visualization platform
their personal data to train a better machine learning model.
to illustrate what is FL and how it supports the privacy-
Recently, with several cases regarding privacy violation and
preserving multi-agent coordination. The platform consists
harsher requirements by the General Data Protection Reg-
of three parts: 1) a controllable racing game based on multi-
ulation (GDPR) by the European Union [Regulation, 2016;
agent box cars; 2) AI models running behind the game, and
Voigt and Von dem Bussche, 2017], data privacy has become
3) an intuitive visualization system for the explanation of the
a hot issue in today’s society. As an alternative decentralized
training and inference processes of this game.
training approach, Federated Learning (FL) enables users to
collaboratively learn a machine learning model while keep-
ing all the personal data that may contain private informa- 2 Introduction to FL
tion on their local devices [Konečnỳ et al., 2016; McMahan FL was first proposed in by Google as “a specific category
and Ramage, 2017; Yang et al., 2019; Konečnỳ et al., 2015; of distributed machine learning approaches which trains ma-
Bonawitz et al., 2019]. In such a case, users can benefit from chine learning models using decentralized data residing on
a well-trained machine learning model without sharing their end devices such as mobile phones” [McMahan and Ram-
sensitive personal data. age, 2017]. Lenovo built hardware to simulate industrial pro-

6572
 Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

Figure 2: Our multi-agent visualization system to explain FL. The left figure shows the real-time gameplay of four cars controlled by real
players, and the right figure demonstrates our visualization system, which consist of: (A,B) Player View which includes a sensor view based
on point cloud data, a loss function view and an action view; (C) a FL Overview; (D) Player Ranking View; (E) Aggregated Accuracy and
Client Contribution View, and (F) AI View.

cesses in factories [Rojek, 2018], and the objective was to of the corresponding player, while the federated local model
predict how the pressure inside the hardware. Cloudera Fast keeps updating itself by communicating model information
Forward Labs showcased an interactive simulation prototype, with all clients following the FL protocols.
Turbofan Tycoon that leverages visualizations to communi-
cate the advantages of Federated model which makes more 3.3 Visualization
accurate predictions about when a turbofan will fail [Mike,
2018]. However, one challenge they faced is that the case of We develop three main visualizations to demonstrate the lo-
preventative maintenance is “not necessarily everyone’s idea cal view and the performance of local models in terms of a
of an exciting or approachable topic”. They further indicated local model and a federated local model for each player, the
that if FL is simulated in a video game scene, lots of niche federated aggregation model, and the AI model activities, re-
technical details can be a source of entertainment, which also spectively. Specially, we design 1) a player view which con-
motivates our work. sists of a sensor view (point cloud data based to visualize the
observation and surroundings of the car), a loss view to illus-
trate the loss function and an action view to show the current
3 The Demonstration System actions of the player; 2) a FL overview to illustrate model up-
3.1 Racing Game dates and communications between clients and the server, a
We designed simple rules and interactive inputs to facilitate player ranking view and an aggregated accuracy view to show
user involvement in the racing game. To be specific, in the the performance of FL and contributions of each client, and 3)
box car racing game, each player can control a box-like car an AI view which is similar to the player view but it shows the
and the one who first reaches the finish line wins. During the observation and the actions of AI-controlled cars. Users can
racing, randomly generated obstacles will appear in the rac- select any two real players in the player view and their data
ing track to slow down the racing. In each round of the game, will be simultaneously visualized in the system. Similarly,
four cars are controlled by real players, while the other six are we can select any one of the other six AI model-controlled
controlled by AI models. During each game play, as shown cars in the AI view for observation.
in Fig.2, we use X and Y to denote the observations and ac-
tions of each player, respectively, which will be recorded for 4 Conclusions and Future Work
further training of AI models. These data will be fed into the
subsequent visualization system so that players can have an The system demonstrated in this paper is a promising edu-
intuitive overview and understanding of the generated input cational tool for FL. In the future, we plan to generalize this
and output to feed the AI models. framework to other FL scenarios and incorporate hardware
components for better illustration.
3.2 AI Models
We train AI models to control cars in the game play. A multi- Acknowledgements
layer perceptron (MLP) is leveraged to generate the AI mod-
els. As shown in Fig. 1, given the recorded data for each This research is supported, in part, by the Nanyang Assistant
player, two variants of AI models, i.e., a local AI model and Professorship (NAP) and the Joint NTU-WeBank Research
a federated local model, are generated and trained for each Centre of Eco-Intelligent Applications (THEIA), Nanyang
player. The local model leverages purely the recorded data Technological University, Singapore.

6573
 Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

References -september-at-ai-summit-in-san-francisco-9ac6462d7f8e,
[Bonawitz et al., 2019] Keith Bonawitz, Hubert Eichner, 2018.
Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, [Voigt and Von dem Bussche, 2017] Paul Voigt and Axel
Vladimir Ivanov, Chloe Kiddon, Jakub Konecny, Stefano Von dem Bussche. The eu general data protection regu-
Mazzocchi, H Brendan McMahan, et al. Towards fed- lation (gdpr). A Practical Guide, 1st Ed., Cham: Springer
erated learning at scale: System design. arXiv preprint International Publishing, 2017.
arXiv:1902.01046, 2019. [Yang et al., 2019] Qiang Yang, Yang Liu, Tianjian Chen,
[Du et al., 2018] Mengnan Du, Ninghao Liu, and Xia Hu. and Yongxin Tong. Federated machine learning: Concept
Techniques for interpretable machine learning. arXiv and applications. ACM Transactions on Intelligent Systems
preprint arXiv:1808.00033, 2018. and Technology (TIST), 10(2):12, 2019.
[Hankz Hankui Zhuo and Lin, 2019] Qian Xu Qiang Yang
Hankz Hankui Zhuo, Wenfeng Feng and Yufeng Lin.
Federated reinforcement learning. arXiv preprint
arXiv:1901.08277, 2019.
[Huang and Liu, 2019] Li Huang and Dianbo Liu. Patient
clustering improves efficiency of federated machine learn-
ing to predict mortality and hospital stay time using
distributed electronic medical records. arXiv preprint
arXiv:1903.09296, 2019.
[Huang et al., 2018] Li Huang, Yifeng Yin, Zeng Fu, Shifa
Zhang, Hao Deng, and Dianbo Liu. Loadaboost: Loss-
based adaboost federated machine learning on medical
data. arXiv preprint arXiv:1811.12629, 2018.
[Konečnỳ et al., 2015] Jakub Konečnỳ, Brendan McMahan,
and Daniel Ramage. Federated optimization: Distributed
optimization beyond the datacenter. arXiv preprint
arXiv:1511.03575, 2015.
[Konečnỳ et al., 2016] Jakub Konečnỳ, H Brendan McMa-
han, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh,
and Dave Bacon. Federated learning: Strategies for
improving communication efficiency. arXiv preprint
arXiv:1610.05492, 2016.
[Kumar et al., 2017] Saurabh Kumar, Pararth Shah, Dilek
Hakkani-Tur, and Larry Heck. Federated control with hi-
erarchical multi-agent deep reinforcement learning. arXiv
preprint arXiv:1712.08266, 2017.
[Liu et al., 2018] Yang Liu, Tianjian Chen, and Qiang Yang.
Secure federated transfer learning. arXiv preprint
arXiv:1812.03337, 2018.
[McMahan and Ramage, 2017] Brendan McMahan and
Daniel Ramage. Federated learning: Collaborative ma-
chine learning without centralized training data. Google
Research Blog, 3, 2017.
[Mike, 2018] Mike. Federated learning: distributed machine
learning with data locality and privacy. [Link]
[Link]/2018/11/14/[Link],
2018.
[Regulation, 2016] Protection Regulation. Regulation (eu)
2016/679 of the european parliament and of the council.
REGULATION (EU), 679, 2016.
[Rojek, 2018] Marcin Rojek. Devices learning from
each other? see it live this september at ai summit
in san francisco! [Link]
devices-learning-from-each-other-see-it-live-this\

6574