ARTICLE

Overcoming the Current Knowledge Gap of

Algorithmic “Collusion” and the Role of
Computational Antitrust

Renato Nazzini* and James Henderson**

Abstract. Digital markets are evolving rapidly, and pricing algorithms are becoming
prevalent. While they provide many benefits, there is a real threat of new harms and
new challenges for antitrust authorities. Computational modelling has
demonstrated these risks by showing that in many instances self-learning pricing
algorithms lead to collusive outcomes. However, so far there has been woefully little
empirical research into the dynamics of pricing algorithms. To provide context for
this threat, we first review the usage and types of algorithmic pricing systems and
critically examine the established taxonomy of algorithm-based collusion scenarios.
We then describe how cartel screening techniques can be applied to algorithmic
systems and the consequential logistical challenges and uncertainties. We propose
action points needed to fill the knowledge gap.

* Professor of Law, Dickson Poon School of Law, King’s College London.

** PhD, Barrister, Research Assistant, Dickson Poon School of Law, King’s College London.
2 Stanford Computational Antitrust VOL. IV

I. Introduction

Algorithms are playing an increasingly important role in many markets. In the

“E-commerce Sector Inquiry” the European Commission reported that the majority
of online retailers track online prices from their competitors, and, in doing so, two
thirds use automated software.1 In 2016, Chen et al. analyzed the top 1,641 best-
selling products on the Amazon Marketplace and identified that 500 sellers used
automated pricing software, based on the high frequency of price changes and their
correlation with other sellers. These could only be achieved using automated systems.
According to these authors, the sellers who used automated pricing received more
positive feedback from consumers and won the Buy Box more frequently than their
non-algorithmic counterparts, suggesting algorithmic sellers obtained higher sales
volume and revenue.2 On Amazon the “Buy Box” is a prominent section on product
pages that allows users to add an item to their cart or make an instant purchase. When
there are multiple potential sellers, an algorithm is used by Amazon to select which
one will be featured based on price, seller rating, and other factors such as order defect
rates and inventory volume.3 The Buy Box is responsible for 80-90% of sales, so being
consistently selected and winning the Buy Box gives sellers a significant competitive
edge. The UK Competition and Market Authority (CMA) reports that for large
Amazon sellers (over $1,000,000 in annual revenue) automated pricing software is
considered essential.4

The adoption of algorithms is not limited to e-commerce. Automated

(algorithmic) pricing is being implemented in many business areas. In general, there
are three main situations where automated pricing is beneficial.5 Firstly, in areas
such as insurance or credit, the cost to serve customers can vary considerably but can
be estimated algorithmically using observable data. Secondly, in areas where demand
fluctuates rapidly such as taxi fares, airline fares or hotel room pricing, algorithms

1
Eur. Comm’n, Final report on the E-commerce Sector Inquiry, at 31, COM (2017) 229 final (May 10, 2017),
[Link]
fc19cbd0ab5d_en?filename=2017_ecommerce_SI_final_report_en.pdf.
2
Le Chen et al., An Empirical Analysis of Algorithmic Pricing on Amazon Marketplace, PROCEEDINGS OF THE
25TH INTERNATIONAL CONFERENCE ON WORLD WIDE WEB (2016),
[Link]
3
Emily Sullivan, Winning the Amazon Buy Box [Algorithm Tips for 2024] (2024),
[Link]
4
Competition & Mkts. Aut., Pricing Algorithms — Economic Working Paper on the Use of Algorithms to
Facilitate Collusion and Personalised Pricing 18 (2018),
[Link]
746353/Algorithms_econ_report.pdf.
5
Oxera Consulting LLP, When Algorithms Set Prices: Winners and Losers, DISCUSSION PAPER, 2 (2017).
3 “Knowledge Gap of Algorithmic Collusion” 2024

can be used to react rapidly to the changing conditions.6 Finally, automated pricing is
convenient for vendors who have a wide range of products to price.

Pricing algorithms used by sellers thus react rapidly to changes in the market
environment,7 enable consistent pricing strategies, and reduce costs by automating
decisions.8 In this way, pricing algorithms can improve the allocation of resources
and are consistent with a dynamic, well-functioning market.9 Automated data
collection and processing allows organization to make faster and better decisions and
markets more efficient. Pricing algorithms can provide better reaction to demand
and reduce information asymmetry and “mispricing” by producers. This enables
better inventory management and reduces the risk of perishable stocks going to
waste.10

However, by increasing supply side market transparency and the ability of sellers
to react to each other’s pricing, these algorithms can also contribute to supra-
competitive equilibria11 if they “collude” by aligning prices instead of competing.12
Algorithms can make collusive agreements more stable and, potentially, easier to
initiate in the first place.

A stark example of pricing algorithmics enabling better coordination and

signaling can be seen in the Albertan wholesale electricity market.13 In 2013 Alberta’s
Market Surveillance Administrator (MSA) flagged up concerns that firms were
“tagging” their otherwise anonymous bids to reveal their identities and allow firms
to coordinate prices.14 Machine learning algorithms had been able to predict the
identity of a firm with an average accuracy of 86%. Following the MSA report
suppliers appeared to randomize their bids, removing any overt patterns in price

6
OECD, Algorithms and Collusion: Competition Policy in the Digital Age 16 (2017),
[Link]
7
Robert M Weiss & Ajay K Mehrotra, Online Dynamic Pricing: Efficiency, Equity and the Future of E-
Commerce, 6 VA. JL & TECH. 11 (2001).
8
Competition & Mkts. Aut., supra note 4, at 21.
9
OECD, Algorithmic Competition, OECD Competition Policy Roundtable Background Note 10-11 (2023),
[Link]
10
Competition & Mkts. Aut., supra note 4, at 20.
11
Pricing above what can be sustained in a competitive market.
12
Shen Li, Claire Chunying Xie & Emilie Feyler, Algorithms & Antitrust: An Overview of EU and National
Case Law, CONCURRENCES E-COMPETITIONS ALGORITHMS & COMPETITION (2021),
[Link]
antitrust-an-overview-of-eu-and-national-case-law.
13
David P Brown et al., Information and Transparency: Using Machine Learning to Detect Communication
between Firms, 3 STAN. COMPUTATIONAL ANTITRUST 199 (2023).
14
MKT. SURVEILLANCE ADMINISTRATOR, COORDINATED EFFECTS AND THE HISTORICAL TRADING REPORT:
DECISION AND RECOMMENDATION 8, 8-15 (2013).
4 Stanford Computational Antitrust VOL. IV

decimals. Nevertheless, despite an initial drop in accuracy, within three months the
algorithm obtained average accuracy of 82%.15

However, is important to not lose sight of the social and consumer welfare
enhancing effect that algorithms can provide, as blanket bans or other heavy-handed
interventions risk doing more harm than good. Even algorithmically driven supra-
competitive coordination may not lead to a reduction in consumer welfare.
O’Connor and Wilson found that algorithms designed to reduce consumer demand
uncertainty would expand the scope for collusion in situations where it would not
otherwise be sustainable. This was because more accurate data collection and
processing would allow companies to better differentiate between low sales volumes
from demand shocks and those from firms undercutting an agreed cartel price.
However, these systems would also make it easier to identify when there are greater
payoffs for defecting. The authors found the overall effect on consumer welfare was
ambiguous, as there were many instances where collusion was still possible, but
companies could no longer sustain monopolistic prices.16

Algorithmic pricing may also lead to price discrimination in the form of

personalized pricing. This is the practice of charging different customers different
prices not justified by differences in costs, but instead based on observable features.17
This typically improves social welfare, however the general effect of price
discrimination on consumer welfare is ambiguous.18 Companies use personalized
pricing to try and capture as much consumer surplus as possible, but it can also
intensify competition and lower overall prices.19 By collecting personal data about
consumers, algorithms can allow for even granular pricing schemes based on an
individual’s estimated willingness to pay.20

Models based on the assumption that firms are able to use tracking devices to
collect data on their own customers show an increase in aggregate consumer
surplus.21 However, Dubé and Misra found that algorithmic personalized pricing

15
Brown et al., supra note 13.
16
Jason O’Connor & Nathan E. Wilson, Reduced Demand Uncertainty and the Sustainability of Collusion:
How AI Could Affect Competition, 54 INFORMATION ECON. & POL'Y (2021).
17
Christopher Townley et al., Big Data and Personalized Price Discrimination in EU Competition Law, 36
YEARBOOK EUR. L. 683 (2017).
18
See Frederik Zuiderveen Borgesius & Joost Poort, Online Price Discrimination and EU Data Privacy Law,
40 J. CONSUMER POL'Y 347 (2017).
19
James C Cooper et al., Does Price Discrimination Intensify Competition-Implications for Antitrust, 72
ANTITRUST L.J. 327 (2004).
20
Haggai Porat, Algorithmic Personalized Pricing in the United States: A Legal Void, CAMBRIDGE HANDBOOK
ON PRICE PERSONALIZATION AND THE LAW (forthcoming).
21
Chongwoo Choe et al., Pricing with Cookies: Behavior-Based Price Discrimination and Spatial Competition,
64 MGMT. SCIENCE 5669 (2018).
5 “Knowledge Gap of Algorithmic Collusion” 2024

instead reduced total consumer surplus by 23% compared to uniform pricing, but
over 60% of customers benefited from lower prices.22 Personalized Pricing is an area
of considerable debate,23 which we do not explicitly address in this paper.

There is some debate as to whether the current US or EU competition regimes

adequately address all instances of algorithmic collusion, particularly when the
collusion arises autonomously from algorithms interacting, without intentional
conduct, awareness, or communication between human competitors.24 For the
purposes of this paper, we leave open the issue of whether this kind of autonomous
algorithmic conduct is unlawful. Instead we focus on the logistical problems of
detecting and analyzing algorithmic pricing patterns, the current lack of empirical
research in this area, and the technical and legal tools deployed by regulators.

This article is structured as follows. In Section II, we examine the usage of and
types of algorithmic pricing systems, distinguishing between the relatively simple
“rules based” systems and the more technologically sophisticated “machine learning”
systems, and, in the case of the latter, “deep learning” systems. In section III we
critically examine the now established taxonomy of algorithm-based collusion
scenarios and argue it is more useful to divide these groups into two main categories,
human relatable conduct, and purely automated conduct. In section IV and V we
describe how screening techniques designed to detect collusion can be applied to
algorithmic systems: section IV discusses the methods used to detect algorithmic
conduct, while section V examines screening indicia. In section VI we look at the
challenge of collecting the volume of data necessary to understand algorithm pricing
software. We discuss what has been done to date and its limitations and offers
suggestions for what needs to be done. Finally, in Section VII, we discuss the
techniques for auditing algorithms themselves and some of the technical difficulties
regulators face in doing so and we propose appropriate solutions.

22
Jean-Pierre Dubé & Sanjog Misra, Personalized Pricing and Consumer Welfare, 131 J. OF POL. ECON. 131
(2023).
23
See OECD, Personalised Pricing in the Digital Era (2018), [Link]/daf/competition/personalised-
[Link].
24
See Joseph E Harrington, Developing Competition Law for Collusion by Autonomous Artificial Agents, 14
J. OF COMPETITION L. & ECON. 331 (2018); Stefan Thomas, Harmful Signals: Cartel Prohibition and Oligopoly
Theory in the Age of Machine Learning, 15 J. COMP. L. & ECON. 159 (2019). Cf. Nicolas Petit, Antitrust and
Artificial Intelligence: A Research Agenda, 8 J. EUR. COMPETITION L. & PRACTICE 361 (2017); Cento
Veljanovski, Pricing Algorithms as Collusive Devices, 53 INT’L REV. INTELLECTUAL PROPERTY & COMPETITION
L. (2022).
6 Stanford Computational Antitrust VOL. IV

II. Algorithmic Pricing Systems in Practice

Many online marketplaces, such as Amazon, eBay, Shopify, Walmart and Google
Shopping, provide inbuilt tools for automated price adjustments by implementing
pricing rules with pre-set triggers. For example, on the Amazon Marketplace, a
vendor could create a price rule designed to automatically undercut the Buy Box price
(the price of the current winner of the Buy Box) by a fixed amount until they win the
Buy Box or reach a specified minimum.25

There is also a growing market for third party repricing services that can offer
more sophisticated or finer pricing controls, such as ChannelEngine,
RepricerExpress and [Link].26 These services allow for greater flexibility, such
as price-matching to specific competitors, or switching between multiple pricing
strategies depending on market conditions. These services often advertise
themselves on their ability to more reliably win the Buy Box while maximizing profit
margins.

Pricing algorithms can be divided into two broad categories: “fixed” or “rule-
based” algorithms that depend on human-selected rules and parameters, and those
that instead rely on machine learning techniques that automatically change and
adapt over time in an attempt to maximize the seller’s long-term profits.27 Examples
of the former include Repriceit or ChannelMAX. Examples of the latter include
Feedvisor and WisePricer.28

There are three main types of rule-based pricing.29 Competition-based pricing is

the most commonly used strategy and allows sellers to create rules that will adjust
their selling price based on the actions of their competitors, such as the price
matching or undercutting techniques described above. The second type, sales-based
pricing, depends instead on changes in sales volume. For example, Amazon’s inbuilt
sales tools allow sellers to impose pricing rules that will automatically decrease prices
if sales volumes drop below a certain threshold.30 Finally, there are time-based pricing
features rules where price changes are dependent on the time of day, or day of the

25
AMAZON, AUTOMATE PRICING,
[Link]
26
Qiaochu Wang et al., Algorithms, Artificial Intelligence and Simple Rule Based Pricing (2022),
[Link]
27
Emilio Calvano et al., Algorithmic Pricing: What Implications for Competition Policy?, 55 REV. INDUS.
ORG. 155 (2019).
28
Dana Popescu, Repricing Algorithms in E-Commerce, Working Paper No. 2015/75/TOM INSEAD
(2015), [Link]
29
Wang et al., supra note 26.
30
AMAZON, CREATE A SALES-BASED PRICING RULE,
[Link]
7 “Knowledge Gap of Algorithmic Collusion” 2024

week. For example, RepricerExpress features a “sleep mode” which can be used to
reset prices to a pre-set maximum overnight in an attempt to reset any pricing wars
against competitors using pricing algorithms set to undercut the seller.31 Table 1
provides an overall summary of the available third-party repricing systems.

Algorithm Type Provider Example

Competition-based Alpharepricer, Aura, Setting prices to
ChannelAdvisor, marginally undercut the
ChannelMAX, [Link], lowers price on the market
SellerEngine,
RepricerExpress, Repriceit,
SellerActice

Sales-based Alpharepricer, Decreasing prices if the

ChannelMAX, SellerEngine, volume of sales drop
SellerActive

Time-based ChannelMAX Resetting prices to

RepricerExpress, Repriceit maximum values during
low sale periods

Machine Learning Alpharepricer, Aura, Black box decision making

based Feedvisor, [Link], rules
WisePricer

Table I: Repricing Algorithm Types.

There are three main categories of machine learning systems: (a) supervised
learning, where the algorithm uses a sample of labelled data to learn a general rule
that maps inputs to outputs; (b) unsupervised learning, where the algorithm attempts
to identify correlations and patterns from unlabelled data; and (c) reinforcement
learning, where an algorithm performs actions in a dynamic environment and learns
through trial and error.32

Third party commercial repricing software providers typically do not divulge the
machine learning techniques used, but most experimental computer science

31
REPRICEREXPRESS, USING THE SLEEP MODE TO AVOID A PRICE WAR, [Link]
mode.
32
Competition & Mkts. Aut., supra note 4, at 11.
8 Stanford Computational Antitrust VOL. IV

literature uses reinforcement learning.33 To function, a reinforcement learning

algorithm must receive data about the state of the environment, be able to take
actions that then affect the state and have a goal relating to said state. When the
algorithm executes actions, it receives feedback in the form of a “reward signal” and
based on this the algorithm seeks to learn what actions maximize the expected
cumulative reward. Many reinforcement learning algorithms involve estimating
“value functions”, which is the expected long-term reward of a given action. Use of
value functions allows the algorithm to learn the benefit of taking actions that offer
little (or even negative) immediate reward, but which have a larger long-term
payoff.34 Rather than being provided with hardcoded rules such as “always undercut
the cheapest rival by X% down to the pre-set minimum price,” the algorithm develops
its own decision-making rules.

An important subset of machine learning is “deep learning”. While traditional

machine learning algorithms can only be applied to linearly separable data, deep
learning algorithms can learn any arbitrary function.35 Deep learning algorithms
involve a “neural net” composed of multiple layers of simple processing units that
mimic the behaviour of human neurons.36 There will be an input layer, one or more
hidden layers (defined as a layer that is neither input or output), and an output layer.
To qualify as “deep learning,” the network must have at least two hidden layers.

The input layer does not process information, the output of each neuron is simply
the value of the data stored. Each input layer neuron then sends this value of each of
the first hidden layer neurons. Each hidden layer neuron then processes this
information and sends an output value to each of the neurons on the next layer, and
so on.37 Each connection between neurons has an associated weight, which is adjusted
as the network learns, and the output of each neuron depends on the weighted sum of
all inputs. The values of the output layer neurons will have some meaning which
corresponds to the task the network is designed to perform, but the output of neurons
in the hidden layer may not have any meaningful interpretation.38 Because of this, it
can be difficult to interpret the decision-making process of a deep learning

33
See Ludo Waltman & Uzay Kaymak, Q-learning Agents in a Cournot Oligopoly Model, 32 J. ECON.
DYNAMICS & CONTROL 3275 (2008); Emilio Calvano et al., Artificial Intelligence, Algorithmic Pricing, and
Collusion, 110 AM. ECON. REV. 3267 (2020).
34
RICHARD S. SUTTON & ANDREW G. BARTO, REINFORCEMENT LEARNING: AN INTRODUCTION 6-13 (ed. 2018).
35
MICHAEL A NIELSEN, NEURAL NETWORKS AND DEEP LEARNING ch. 4 § 25 (2015); Yann LeCun et al., Deep
learning, 521 NATURE 436 (2015).
36
IAN GOODFELLOW et al., DEEP LEARNING pt 1.2 (2016).
37
See HOWARD B DEMUTH, et al., NEURAL NETWORK DESIGN 44-48 (2014); Saurabh Karsoliya,
Approximating Number of Hidden Layer Neurons in Multiple Hidden Layer BPNN architecture, 3 INT’L J.
ENGINEERING TRENDS & TECH. 714 (2012).
38
JOHN D KELLEHER, DEEP LEARNING 67-76 (2019).
9 “Knowledge Gap of Algorithmic Collusion” 2024

algorithm. 39 Given a particular set of inputs, the outputs or decision reached can be
observed, but it can be difficult to determine how the network reached this outcome,
or even which parts of the input data most strongly influenced the final decision. As
such, deep learning networks are often described as opaque “black boxes” that “hide
their internal logic to the user.”40

III. Legal Taxonomy of Potential Collusive Scenarios

While algorithmic pricing promises many advantages, since 2015 legal scholars
and policy makers have expressed concerns that algorithmic pricing software may
also facilitate collusive behavior.41

There are several mechanisms that have been proposed to explain how and why
algorithms could lead to collusive outcomes. They generally fall into four main
categories. Firstly, algorithms make it easier and cheaper to monitor a collusive
agreement, and respond more rapidly to any deviations. Secondly, algorithms can
more reliably implement a collusive agreement, with a reduced risk of errors or
agency slack.42 With a greater volume of information about demand conditions and
competitor prices, firms are less likely to confuse a period of low demand with a cartel
partner cheating. Improved analytical power also allows better demand prediction, as
well as predicting rival actions.43 Thirdly, algorithms may be able to signal more
effectively, by being able to send signals indicating a short term commitment to a
particular pricing strategy that are either too brief or are sent at periods of low
demand and so do not impact sales, but that can be detected by monitoring
algorithms.44 They may also be designed to react predictably, in a way that can reduce
strategic uncertainty.45

Ezrachi and Stucke identified four scenarios in which algorithms could lead to
collusion.46 The first, “messenger”, is when algorithms are used to more reliably

39
Madalina Busuioc, Accountable Artificial Intelligence: Holding Algorithms to Account, 81 PUBLIC ADMIN.
REV. 825 (2021).
40
Riccardo Guidotti et al., A Survey of Methods for Explaining Black Box Models, 51 ACM COMPUTING
SURVEYS 1 (2018).
41
See Salil K Mehra, Antitrust and the Robo-Seller: Competition in the Time of Algorithms, 100 MINN. L. REV.
1323 (2015). OECD, supra note 9.
42
Competition & Mkts. Aut., supra note 4, at 23-25.
43
Michal Gal, Limiting Algorithmic Coordination, 38(1) BERKELEY TECH. L. J. (forthcoming).
44
OECD, supra note 9, at 24-32.
45
Competition and Markets Authority, supra note 4, at 25; Autorité de la Concurrence &
Bundeskartellamt, Algorithms and Competition 38-39 (2019),
[Link]
lautorite-et-le-bundeskartellamt-publient-une.
46
ARIEL EZRACHI & MAURICE E. STUCKE, VIRTUAL COMPETITION 36-37 (2016).
10 Stanford Computational Antitrust VOL. IV

implement and monitor explicit cartel schemes. The second, “Hub and Spoke”,
involves firms relying on a common third-party provider of pricing algorithms.
“Predictable Agent” posits firms unilaterally adopting algorithms deliberately
designed to facilitate tacit collusion, while “Digital Eye” goes one step further and
raises the prospect that self-learning algorithms tasked with the goal of profit
maximisation may autonomously and independently converge on collusive
outcomes without ever being explicitly programmed to do so.

However, from a legal and operational perspective, we consider that it is necessary

to divide algorithmic collusion scenarios into two main categories: human-relatable
conduct, and purely automated conduct. The former covers situations where
algorithms are used to facilitate or coordinate traditional collusive practices and
incorporates the “Messenger” scenario. The latter covers collusion in the absence of
any prior or ongoing communication between human representatives and
incorporates the “Predictable Agent” and “Digital Eye” scenarios. Depending on the
context, the “Hub and Spoke” scenario may straddle the line between the two. This
taxonomy is based on the principles of legal liability and attribution, and allows for a
clearer categorization for policy making and law enforcement purposes.

The Topkins case in the US is a clear example of human relatable conduct. Here,
several sellers conspired to fix the price of posters on the Amazon Marketplace and
agreed to adopt pricing algorithmic software to implement the scheme. One of the
competitors programmed an algorithm to find the lowest third party price offered
by a third party and set their price just below that, while the conspirators had an
algorithm programmed to match their co-conspirator’s price.47

In the UK Posters case, Trod and GB Eye also agreed not to undercut one another
for prices on posters and frames. Both sellers configured their algorithm to compete
aggressively against sellers outside of the scheme and rapidly respond to changes in
market conditions but would deliberately ignore each other’s prices.48

In 2018 the European Commission fined Asus €63,522,000 for imposing a fixed
or minimum release price for online retailers for a range of consumer electronics.49
The Commission found that Asus relied on internal software monitoring tools to
identify retailers that were selling their products below the desired level. The
Commission also highlighted how the use of automatic pricing software by retailers

47
United States v Topkins , No. CR 15-00201, 2015 (N.D. Cal. Apr. 30, 2015); Salil K Mehra, US v. Topkins:
Can Price Fixing Be Based on Algorithms? 7 J. EUR. COMPETITION L. & PRACTICE 470 (2016).
48
Decision of the Competition & Mkts. Aut. in case no. 50223: Trod Ltd/GB Eye Ltd (Aug. 12, 2016).
49
Eur. Comm'n, Antitrust: Commission Fines Four Consumer Electronics Manufacturers for Fixing Online
Resale Prices (2018), [Link]
11 “Knowledge Gap of Algorithmic Collusion” 2024

amplified the effect of Asus’s interventions. The pricing algorithms used by the
retailers were designed to price match, so by targeting the lowest pricing retailers
Asus could prevent a more general price erosion.

The “Hub and Spoke” category includes situations which are similar to, but would
not necessarily actually qualify as, classic hub-and-spoke cartel arrangements.50 The
case of Eturas in the EU has been cited as an example of an algorithmic “Hub and
Spoke” situation under Ezrachi and Stucke’s taxonomy.51 However, while it did
demonstrate similar structure and technical implementation, this case was not
legally considered to be an instance of conventional hub-and-spoke collusion.52 In
Eturas, Lithuanian travel agents used a common third-party booking software. The
administrator of the software then proposed a rule that would limit the maximum
allowable discount that could be applied via the booking system. The European Court
of Justice found that this would constitute a concerted practice under Article 101
TFEU if it could be shown that the travel agencies were aware of the message.53

Ezrachi and Stucke confine the “Hub and Spoke” scenario to instances where
competitors all use the same algorithms to determine market prices or market
changes.54 However the potential range of situations in which algorithmic collusion
involves a third party is much broader. While the third party could be the provider of
a common algorithm, it could also provide a means of exchanging data, or even a
common data pool. Third parties could also coordinate the algorithms of competitors
in other ways, such as an external consultant that advises companies in the same
market on the design and use of algorithms.55 The common feature is that there is no
direct communication or contact between the competitors.

Whether this behavior would amount to human relatable conduct would depend
on the awareness of the parties, as set out by the ECJ in Eturas. This division is also
adopted by the Autorité de la Concurrence and Bundeskartellamt, which distinguish
between situations where competitors knowingly coordinate via a third party, and

50
A hub-and-spoke agreement occurs when a horizontal agreement is implemented without any direct
communication between the competitors but is facilitated by agreements with a vertically related
common third party. See RICHARD WHISH & DAVID BAILEY, COMPETITION LAW 337-340 (7th ed. 2012).
51
EZRACHI & STUCKE, supra note 46, at 52-53.
52
Opinion of AG Szpunar, Case C-74/14, "Eturas" UAB and others v. Lietuvos Respubilkos
Konkutencijos Taryba, ECLI:EU:C:2015:493, ¶ 65 (July 16, 2015).
53
Case C-74/14, "Eturas" UAB and others v. Lietuvos Respublikos Konkutencijos Taryba,
ECLI:EU:C:2016:42 (Jan. 21, 2016).
54
Ariel Ezrachi & Maurice E. Stucke, Artificial Intelligence & Collusion: When Computers Inhibit
Competition, 5 U. ILL. L. REV. 1776 (2017).
55
Autorité de la Concurrence & Bundeskartellamt, supra note 45, at 31.
12 Stanford Computational Antitrust VOL. IV

those where they are unaware of the coordination, in that they do not know or could
not reasonably foresee it.56

The CMA considers that scenarios where sellers use the same algorithm or data
pool to determine prices present the most immediate risk57 but to date there have
been no successful enforcement actions. However, at the time of writing there are
several ongoing investigations and lawsuits alleging third-party driven behavior. In
Gibson v. MGM it is alleged that hotels in the Las Vegas strip used third party
software to aggregate pricing strategy information, keeping room rental rates
artificially high.58 RealPage, a provider of a price setting algorithm for property
owners, is currently under investigation by the United States Department of Justice
over allegations that its software allows users to coordinate pricing. The software
works by collecting information from users, including what rents they are able to
charge tenants, which is then used to recommend prices. RealPage states that this
data is aggregated and anonymized and denies any anti-competitive conduct.59

Purely automated conduct has not yet been tested in enforcement practice, but a
growing body of theoretical studies and computer simulations suggest that collusive
outcomes are a real possibility under certain market conditions.60 Few papers have
identified algorithmic collusion in an empirical setting, although in a study of the
German retail gasoline market Assad et al. found that the adoption of pricing
algorithms in a duopoly led to a margin increase of 28% when both rivals adopted
algorithmic pricing, while when only one station adopted an algorithm there was no
increase.61 Brown and MacKay relied on modelling to show that the adoption of
algorithmic pricing by the five large online over-the-counter allergy drug retailers in
the United States led to a profit increase of 9.6% and a 4.1% reduction on consumer
surplus compared to a non-algorithmic counterfactual.62

It is important to note, however, that if a competitor created an algorithm that

was deliberately intended to collude, even a self-learning one such as the algorithm

56
Id. at, 32.
57
Competition & Mkts. Aut., supra note 4, at 31.
58
Richard Gibson et al. v. MGM Resorts Int’l et al., 2:23-cv-00140-MMD-DJA (D. Nev. Oct. 24, 2023).
59
Heather Vogel, Department of Justice Opens Investigation into Real Estate Tech Company Accused of
Collusion with Landlords, PROPUBLICA (2022).
60
Bruno Salcedo, Pricing Algorithms and Tacit Collusion, MANUSCRIPT, PENNSYLVANIA STATE UNIVERSITY
(2015); Calvano et al., supra note 33; Timo Klein, Autonomous Algorithmic Collusion: Q-Learning under
Sequential Pricing, 52 RAND J. ECON. 538 (2021).
61
Stephanie Assad et al., Algorithmic Pricing and Competition: Empirical Evidence from the German Retail
Gasoline Market (2020) CESifo Working Paper No. 8521 (2020).
62
Zach Y. Brown & Alexander MacKay, Competition in Pricing Algorithms, 15 AM. ECON. J. 109 (2023).
13 “Knowledge Gap of Algorithmic Collusion” 2024

demonstrated by Meylahn and den Boer,63 this could constitute human related
conduct.

It would appear, looking back, that Ezrachi and Stucke’s fears have not been
realized. As noted above, there have been relatively few enforcement actions since
their book’s publication. We postulate that this due to a combination of two main
reasons.

Firstly, much of the technology was, at the time, speculative. Even today machine
learning-based re-pricing systems are still in their infancy, although, as discussed in
Section II, they are becoming increasingly commercially available. For the most part,
companies are only now in a position to implement the systems that could lead to
automated collusion.

Secondly, there is good reason to believe that detection will be extremely difficult.
As will be discussed below in Section VI, analysis of algorithmic pricing systems
requires the collection and processing of large quantities of pricing data. This
presents obstacles for both regulators and academics. There have only been a handful
of empirical studies, discussed in Section IV, exploring the behavior of pricing
algorithms. While several authorities worldwide have developed systems for
automatically collecting price data, most are limited to daily updates at most,64 which
therefore cannot capture high frequency price changes.

This is on top of the fact that cartel detection is inherently quite difficult. It has
been estimated that the overall detection rate for cartels since World War 2 is
between 10 and 30%.65 A cartel screen can only flag up concerning behavior that has
been previously identified from discovered and successfully prosecuted cartels.
However, the set of discovered cartels may not be a representative sample of the
overall population of cartels.66 In the EU, between 1998 and 2017, over 90% of
prosecutions came from leniency applications67 and, as Schinkel points out, there is
reason to believe that cartels successfully identified via leniency applications are

63
Janusz M Meylahn & Arnoud V. den Boer, Learning to Collude in a Pricing Duopoly, 24 MANUFACTURING
& SERVICE OPERATIONS MGMT. 2577 (2022).
64
See Thibault Schrepel & Teodora Groza, The Adoption of Computational Antitrust by Agencies: 2nd
Annual Report, 3 STAN. COMPUTATIONAL ANTITRUST 55 (2023).
65
John M. Connor, Cartel Detection and Duration Worldwide, 2 COMPETITION POL’Y INT’L: ANTITRUST
CHRONICLE (2011).
66
Joseph E. Harrington Jr. & Yanhao Wei, What Can the Duration of Discovered Cartels Tell Us About the
Duration of All Cartels?, 127 ECON. J. 1977 (2017).
67
Jerome De Cooman, Outsmarting Pac-Man with Artificial Intelligence, or Why AI-Driven Cartel Screening
Is Not a Silver Bullet, 14(4) J. EUR. COMPETITION L. & PRACTICE 186 (2023).
14 Stanford Computational Antitrust VOL. IV

liable to be the weakest, least stable cartels.68 More sophisticated cartels that are
resilient enough, or have otherwise developed ways to become leniency resistant, will
not be detected. As will be discussed in Section V, there is reason to believe that
algorithmic collusion may be even more challenging to uncover.

IV. Detection of Algorithmic Pricing

Detection of algorithmic collusion is a two-step process. Firms do not necessarily

announce they are using pricing algorithms, so the first step may be to identify their
usage. This is important because while many of the traditional methods for cartel
screening can be adapted for algorithmic sellers, as will be discussed below,
algorithmic sellers appear to display unique behavioral dynamics.

As noted above, there have been very few papers that examine the behavior of
real-world pricing algorithmics. To the authors’ knowledge there are only three main
papers that do so.

The seminal paper by Chen et al. represents the first major attempt to detect
algorithmic sellers, in this case on the Amazon Marketplace.69 The authors operated
with the assumption that algorithmic sellers would update their prices more
frequently, and that their prices would be more strongly correlated to the prices of
other sellers. After all, a seller seeking to offer the lowest price for a given product
must be setting their price relative to the competitor with the current lowest price. As
such they examined prices pegged to the lowest price, second lowest or that of the first
party (i.e., Amazon). The authors were unable to use the Amazon Marketplace Web
Services API, as it was both heavily rate limited and did not return the identity of the
third-party sellers, and so they resorted to web scraping.

This technique was subsequently adapted by Wieting and Sapi to analyse [Link],
the largest online marketplace in the Netherlands and Belgium.70 They decided that
frequency of changes was the most reliable indicator of algorithmic pricing, with
price correlation serving as a robustness check for two reasons. First, because the data
they used only covered a small sample of a seller’s product range, meaning that a high
number of observed prices likely implied orders of magnitude more changes across
the entire product portfolio. Doing this would be impractical without automated

68
Maarten Pieter Schinkel, Balancing Proactive and Reactive Cartel Detection Tools: Some Observations,
OECD POLICY ROUNDTABLES: EX OFFICIO CARTEL INVESTIGATIONS AND THE USE OF SCREENS TO DETECT
CARTELS 263 (2013), [Link]
69
Chen et al., supra note 2.
70
Marcel Wieting & Geza Sapi, Algorithms in the Marketplace: An Empirical Analysis of Automated Pricing
in E-Commerce (NET Institute Working Papers 21-06, 2021),
[Link]
15 “Knowledge Gap of Algorithmic Collusion” 2024

repricing tools. Second, correlations with other prices may simply fail to detect
algorithms not reliant on a price-correlation strategy and cannot be relied upon at all
in monopoly markets.

Finally, Assad et al. were able to use a Quandt-Likelyhood Ratio test, which tests
for a structural break for each period in some interval of time,71 to estimate if and
when German gasoline retailers adopted algorithmic pricing, based on the fact that
trade publications reported mass adoption occurred beginning in 2017.72 They did
this by testing for structural breaks at each station for each week in a large window
around the time of supposed adoption, relying on the number of daily price changes,
the average size of price changes and the response time of a station’s price update
given a rival’s price change. As with Chen et al., the authors assume that the adoption
of algorithmic pricing will correspond to more frequent updates and faster reaction
to competitor behavior.

From this, it appears that there is little difficulty in detecting the use of
algorithmic pricing. The main obstacle, as discussed below in section VI, is the sheer
volume of data that algorithmic pricing systems generate and that must be studied if
their behavior is to be quantified.

V. Cartel Screens

Once algorithmic pricing has been identified, it is a matter of quantifying the

algorithms’ behavior and flagging up any activity that could indicate collusion or
other harmful practices. These indications will not definitively demonstrate
wrongdoing, which requires an agreement to fix trading conditions, but can serve as
a trigger for a more detailed investigation by regulators.

Based on the limited research available and previous studies of cartel behavior, it
appears possible to identify patterns that indicate supra-competitive prices
consistent with collusive behavior. These patterns include:

● Low price variance, which can occur when it is costly or difficult to

coordinate price changes73 or when buyers start to become suspicious of the
presence of a cartel following a period of price rises.74

71
See Richard E Quandt, Tests of the Hypothesis that a Linear Regression System Obeys Two Separate
Regimes, 55 J. AM. STATISTICAL ASS’N 324 (1960).
72
Assad et al., supra note 61.
73
Rosa M. Abrantes-Metz et al., A Variance Screen for Collusion, 24 INT’L J. IND. ORG. 467 (2006).
74
Joseph E. Harrington & Joe Chen, Cartel Pricing Dynamics with Cost Variability and Endogenous Buyer
Detection, 24 INT’L J. IND. ORG 1185 (2006).
16 Stanford Computational Antitrust VOL. IV

● Increased price uniformity across firms and the reduction in discounts, to

simplify the functioning and monitoring of the cartel agreement.75
● A negative correlation between price and demand. During periods of high
demand, the pay-off for cartel numbers defecting and undercutting the
cartel price is higher, so the cartel is only stable with lower prices.76
● Sharp increases in high price-cost margins. High-cost margins alone are
evidence of market power, and do not imply collusion, but sharp increases
(absent any exogenous factors like a spike in demand) may be difficult to
explain without the existence of a cartel.77
● Prices going up quickly and remaining high for a relevant period, with
temporary sudden drops followed by prices going up again to previous
levels, suggests collusion with periods of cheating followed by successful
punishment and re-establishing of collusion. This can occur when a
collusive agreement breaks down (for example, because of a new entrant)
and is then restored.78
● A sharp and steady price increase following a steep decline. This can be
attributed to the formation of a cartel in reaction to an event that caused a
sharp decline in prices.79

In relation to all the above patterns, it is important to control for exogenous

factors that can explain the behavior in ways other than collusion, such as variations
in cost, in demand, or in external factors such as regulation or taxation, geopolitical
shocks, or changes in trade or customs rules.

For human relatable conduct, it can be expected that many previously identified
indicators would still be relevant, but the indicators may be altered to be harder to
detect. For example, as alluded to previously, algorithms may allow firms to
distinguish more accurately between periods where demand is low and when a cartel
partner is cheating. This could improve cartel stability, and therefore reduce
instances of the sharp decline and price restoration pattern associated with a
breakdown of the cartel and subsequent punishment periods. However, modelling by
Miklós-Thal and Tucker suggests that better predictive power may undermine cartel
stability by increasing the temptation to undercut prices during periods of high

75
OECD, Ex officio cartel investigations and the use of screens to detect cartels, OECD Competition Policy
Roundtable — Crisis Cartels 29 (2011), [Link]
76
J. E. Harrington, Detecting Cartels, HANDBOOK OF ANTITRUST ECONOMICS, 26-29 (2008).
77
Id. at 20-22.
78
Edward J. Green & Robert H. Porter, Noncooperative Collusion under Imperfect Price Information, 52
ECONOMETRICA: J. ECONOMETRIC SOC’Y 87 (1984).
79
OECD, supra note 75, at 55.
17 “Knowledge Gap of Algorithmic Collusion” 2024

demand.80 O’Conner and Wilson reach similar conclusions and show that cartels may
need to resort to longer punishment periods and sub-monopoly pricing to maintain
stable collusion.81 These may require adjusting or reformulating screening patterns
accordingly.

A further complication is that algorithms may reliably implement more

sophisticated cartel arrangements. For example, the Autorité de la Concurrence and
Bundeskartellamt propose that algorithms could even be used to attempt to
deliberately conceal cartel behaviour by being programmed to implement different
prices during periods of low demand or being set to occasionally generate periods of
price heterogeneity or instability.82 If this were true, we might see less of a trend
towards price homogeneity, and as such, that indicator may cease being an effective
screen.

To date, however, there has been relatively little empirical or modelling work on
the potential impact of algorithms.83 More generally, there has also been relatively
little examination or modelling of the pricing patterns associated with algorithmic
pricing systems and, in particular, which of these patterns might signify unlawful
collusive behavior.

Wieting and Sapi identified five price patterns that were associated with
repricing software but could not definitively ascertain whether any of the five
patterns could be attributed to collusive behavior:84

1. Jitters: rapid transitory increases or decreases in price

2. Rockets and feathers: pricing shooting up rapidly and then gradually
decreasing, often reaching the starting point
3. Balloons and rocks: price increases slowly up to a point, then falls rapidly
often to the starting point
4. Alternating prices: pricing jumps up or down for longer but transitory
periods between two values.
5. Random jumps: pricing changes frequently in a seemingly random manner

80
Jeanine Miklós-Thal & Catherine Tucker, Collusion by Algorithm: Does Better Demand Prediction
Facilitate Coordination between Sellers?, 65 MGMT. SCIENCE 1455 (2019).
81
Jason O’Connor & Nathan E. Wilson, Reduced Demand Uncertainty and the Sustainability of Collusion:
How AI Could Affect Competition, 54 INFO. ECON. & POL’Y 100882 (2021).
82
Autorité de la Concurrence & Bundeskartellamt, supra note 45, at 28.
83
See Assad et al., supra note 61.
84
Wieting & Sapi, supra note 70, at Sec. 5.3.
18 Stanford Computational Antitrust VOL. IV

Price jitters were also documented by Chen et al. but the jitters were attributed to
malfunctions—“Transitory inconsistencies in Amazon’s infrastructure, rather than
actual price changes by sellers.”85 However, Wieting and Sapi found this explanation
unconvincing and attribute the price jitters to actual pricing behavior for several
reasons, the most critical being that there are products where the jitters led to a
reaction by other actors, such as a change in the Buy Box seller.86 The authors
speculate that these jitters may be acting as a form of signaling, with a downward
jitter indicating a firm’s ability to reduce prices and punish deviating rivals.
Downward jitters, where prices drop very briefly before returning to the previous
baseline, are particularly concerning as they suggest that the firms in question are
selling substantially above-cost most of the time. However, upward jitters could also
signal to competitors an intention to raise prices, as Byrne and Roos documented for
the Australian petrol market.87 Further work would be needed to determine whether
these patterns are harmless noise or intentional conduct consistent with cartel-like
behavior.

Rockets and feather patterning was observed by Wieting and Sapi 11% of the
time, and both are consistent with the classic collusion patterns described above, as
well as the pricing patterns seen by Calvano et al.88 and Klein.89 In the absence of an
innocent explanation such as unexpected cost shocks (unlikely to change within the
timeframe examined), the authors suggest rockets and feather patterning are most
likely due to algorithmic collusion, be it tacit or otherwise.

A rockets and feathers-type pattern was also identified by Musolff and was
attributed to vendors adopting repricing software designed to undercut competitors
in an attempt to win the Buy Box. The software is programmed to reset prices when
they get too low or at a specific time of day, typically at night when sales are lowest.90
The net result is pricing cycles reminiscent of Edgeworth price cycles, first proposed
by Maskin and Tirole, but not driven by the same Markov perfect equilibria behavior
(optimum pricing strategies that depend only on the current state of the system).
Edgeworth cycles are a form of tacit collusion characterized by a slow decline in prices

85
Chen et al., supra note 2, at 4.
86
Wieting & Sapi, supra note 70, at 20.
87
David P Byrne & Nicolas De Roos, Learning to Coordinate: A Study in Retail Gasoline, 109 AM. ECON. REV.
591 (2019).
88
Emilio Calvano, et al., Algorithmic Collusion with Imperfect Monitoring, 79 INT’L J. IND. ORG. 102712
(2021).
89
Klein, supra note 60; Wieting & Sapi, supra note 70.
90
Leon Musolff, Algorithmic Pricing Facilitates Tacit Collusion: Evidence from E-Commerce, EC '22:
Proceedings of the 23rd ACM Conference on Economics and Computation (2022),
[Link]
19 “Knowledge Gap of Algorithmic Collusion” 2024

as firms take turns undercutting each other until both firms reach marginal cost.91 At
this point, the firms then enter a “war of attrition,” each waiting and hoping a
competitor will raise prices first. When one firm eventually relents, the others will
then raise their prices to slightly undercut this new higher price, and the cycle
repeats.92 According to Maskin and Tirole, once prices have dropped far enough,
competitors switch from undercutting prices to pricing at marginal cost, and then
randomize between resetting prices or keeping them unchanged in the hope that
their rival might be the one to reset.93 In the observed behavior, the minimum price
is typically higher than the marginal price, and resetting occurs more frequently and
deterministically, either once a pre-set level is reached (with no war of attrition) or at
a set time (such as resetting every night during hours when sale probabilities are
lowest regardless of whether the minimum was reached).94

The little empirical and economic research carried out so far suggests that, while
previous models of cartel behavior are unlikely to become totally obsolete and our
current understanding of cartelized markets will remain important in guiding
further research, understanding how “collusion” works in the world of algorithms is
still at its infancy. The next stage is, therefore, wide-spread and systematic analysis of
markets affected by algorithmic pricing.

VI. Analyzing Algorithmic Pricing: Automation and the Data

Problem

In this section we review issues that regulators face when dealing data generated
with algorithmic systems. The sheer volume of price changes and pricing data
generated by algorithmic pricing software makes it impractical to audit without
relying on automated systems. In 2013, it was reported that Amazon implemented
more than 2.5 million price changes per day, fifty times more than Best Buy and
Walmart during the same period.95

In analyzing [Link], Wieting and Sapi performed two crawls. The first covered
2,840 products over a 30-day period and recorded 2,437,557 price changes, an
average of 28 changes per product per day.96 However, on average, crawl frequency

91
Eric Maskin & Jean Tirole, A Theory of Dynamic Oligopoly, II: Price Competition, Kinked Demand Curves,
and Edgeworth Cycles, 56 ECONOMETRICA: J. ECONOMETRIC SOC’Y 571 (1988).
92
Id.
93
Id.
94
Musolff, supra note 90, at 24-25.
95
Profitero, Profitero Price Intelligence: Amazon Makes more than 2.5 million Daily Price Changes (2013),
[Link]
5-million-price-changes-every-day.
96
Wieting & Sapi, supra note 70, at 12.
20 Stanford Computational Antitrust VOL. IV

was only once every two hours.97 The second covered 1,949 products over a different
30-day period and recorded 17,066,561 changes, an average of 292 changes per
product per day with a crawl frequency of approximately 30 minutes.98 Musolff
relied on a near complete set of notifications for a single third-party repricing
company which recorded 1,331,657,526 changes over a 577-day period, covering
859,823 products with three changes per product per day.99

Because of this volume of data, antitrust authorities are turning to technological

tools to address the demands of digital markets.100 However, there many legal and
technical issues that must be overcome regarding the reliability of these tools as
evidence and the ability for regulators to share data amongst themselves needed to
build reliable tools.

The CMA has been an early adopter with the creation of the Data, Technology and
Analytics (DaTA) unit in February 2019.101 Other authorities such as the US Federal
Trade Commission, and EU Directorate General for Competition have followed
suit.102 A notable success by the DaTA Unit is the in-house development of a tool to
detect retail price maintenance (RPM) schemes by identifying anomalous patterns in
scraped price data. The idea for the tool arose after an investigation in the musical
instruments sector, where the CMA found that firms were using price monitoring
software to determine compliance with RPM schemes.103 The CMA intends to use this
tool to monitor other sectors for suspicious pricing activity.104

A handful of other antitrust agencies have made similar tools.105 The Columbian
Superintendence of Industry and Commerce’s “Sabueso” Project uses automated bots
to monitor and analyze information about available goods on online retailers. Bots
designed to simulate customers harvest product data. This is then supported by

97
Id.
98
Id.
99
Musolff, supra note 90, at 6.
100
See Thibault Schrepel & Teodora Groza, The Adoption of Computational Antitrust by Agencies: 2021
report 2 STAN. COMPUTATIONAL ANTITRUST 78 (2022); Schrepel & Groza, supra note 64.
101
Steven Hunt, The Technology-Led Transformation of Competition and Consumer Agencies: The
Competition and Markets Authority’s Experience, COMPETITION & MKTS. AUT. (2022),
[Link]
1085931/The_technology_led_transformation_of_competition_and_consumer_agencies.pdf.
102
Competition and Markets Authority, Compendium of Approaches to Improving Competition in Digital
Markets (2021), [Link]
improving-competition-in-digital-markets.
103
Simon Nicols, Restricting Resale Prices: How We're Using Data to Protect Customers, COMPETITION &
MKTS. AUT. (2020), [Link]
how-were-using-data-to-protect-customers/.
104
Id.
105
For a comprehensive overview, see Schrepel & Groza, supra note 64.
21 “Knowledge Gap of Algorithmic Collusion” 2024

machine learning systems used to identify identical products across different stores,
as different retailers use dissimilar names and descriptions.106

The Greek Data Analysis and Economic Intelligence Platform (DAECI) collects
data from e-katanalotis (Market Observatory) for the prices of foods and common
household goods, OKAA (Central Markets and Fisheries Organization) and Eurostat
for the price of fruit, vegetables, meats, and fish, and [Link] for fuel prices. 107
In 2023 it started collecting product data from online retailers directly, with daily
updates for over 60,000 products.108 Similarly, the Armenian Competition
Protection Authority has created an “e-Compete” platform designed to collect daily
prices of selected goods via the State Revenue Committee databases.109 While this will
no doubt help detect instances of price fixing, daily snapshots will likely be unable to
capture algorithmic pricing dynamics and, depending on when the snapshot is taken,
could miss elevated pricing behavior. Equally, however, more frequent updates
would require significantly more resources and data storage capabilities.

To reliably identify collusion in a given market with supervised learning, the AI

must be trained on a dataset from the same market containing labelled instances of
collusive and competitive behavior.110 Datasets from other markets can be used, but
the effectiveness of transposed models depends heavily on the comparability
between markets.111 If a sufficient volume of suitable data is unavailable, then
training will be flawed, and the screen will likely be unreliable.112 The same problem
arises with unsupervised learning, where the dataset is unlabeled and the AI seeks to
identify outliers that are most dissimilar from the “norm,”113 as outliers can only be
identified when there is sufficient data to establish a baseline.

Problems can arise even when an appropriate level of data is available. In 2017, the
CMA released the “Screening for Cartels” (SfC) tool, designed to flag instances of
potential bid-rigging.114 The tool was made available to be freely disseminated and

106
Superintendence of Industry and Commerce, Digital Evidence Gathering in Cartel Investigations – Note
from Columbia, (OECD, Latin American and Caribbean Competition Forum, 2020),
[Link]
107
Schrepel & Groza, supra note 64, at 97.
108
Schrepel & Groza, supra note 64, at 99.
109
Schrepel & Groza, supra note 64, at 60.
110
Rosa M Abrantes-Metz & Albert Metz, Can Machine Learning Aide in Cartel Detection?, ANTITRUST
CHRONICLE, COMPETITION POL’Y INT’L (2018).
111
Joseph E Harrington & David Imhof, Cartel Screening and Machine Learning, 2 STAN. COMPUTATIONAL
ANTITRUST 134 (2022).
112
Id.
113
Ai Deng, Cartel Detection and Monitoring: A Look Forward, 5 J. ANTITRUST ENFORCEMENT 488 (2017).
114
Albert Sanchez-Graells, ‘Screening for Cartels’ in Public Procurement: Cheating at Solitaire to Sell Fool’s
Gold?, 10 J. EUR. COMPETITION L. & PRACTICE 199 (2019).
22 Stanford Computational Antitrust VOL. IV

replicated for procurers in the UK and other jurisdictions. The tool, as released, was
based on data from over 100 tenders, involving nearly 500 bids. However, there is no
reliable centralized repository of procurement data. While individual regulators who
decided to adopt the tool could train on further data, to this date any subsequent
improvements could not be shared with others. Each parallel version would evolve in
a different way. This is, of course, assuming the screens even evolved at all. In all
likelihood, in the absence of a centralized repository, any single operator would be
unlikely to provide enough data and carry out analyses over a sufficient number of
tenders so as to meaningfully refine the system. Consequently, the CMA withdrew
the SfC from use on January 20th, 2020.

The Danish Competition and Consumer Authority (DCCA) developed Bid

Viewer, also designed to detect bid rigging and unusual patterns in public
procurement. This tool was developed in collaboration with the Spanish and Swedish
and other national authorities. These collaborators can share code, data and
methodologies. However, there is still no single data source with procurement data
available and the DCCA sees data acquisition as the largest obstacle.115

Such repositories can and should be created. The CMA DaTA team has created
LEDA (which stands for “LEDA is an Environment for Data Analysis”) a platform for
creating what is known as a data lake—a centralized system for storing and accessing
large quantities of raw data. This required developing the infrastructure necessary to
ingest, curate and process sensitive data at scale, and acquired over 160 Terabytes of
data across over 130 million objects at minimal cost between 2019 and 2022.116
Following the DCCA’s initiative with Bid Viewer, the authors suggest that greater
international collaboration is necessary, particularly as national competition
authorities seek to deal with global digital firms who operate in borderless markets.
The challenges national competition authorities face are very similar and
international collaboration can help alleviate the difficulties of acquiring in-house
technical expertise and share the cost of developing new technologies.117
Furthermore, international cooperation may allow for more and better data sets to
be collected and pooled from different jurisdictions. International organizations
such as the Organisation for Economic Cooperation and Development (OECD) or
international cooperation networks such as the International Competition Network
(ICN) could play a vital role of coordination on technology transfer and data sharing
relevant to algorithmic pricing analysis. The OECD’s Competition Committee has
held best practice roundtables on Algorithms in 2017 and 2023 and developed

115
Danish Competition and Consumer Authority, Data Screening Tools for Competition Investigations
– Note by Denmark (OECD, 136th OECD Working Party 3 meeting 2022).
116
Hunt, supra note 101 at 24.
117
Hunt, supra note 101 at 45-46.
23 “Knowledge Gap of Algorithmic Collusion” 2024

Council Recommendations on enhancing agency cooperation.118 The ICN has

recently created of the role of ICN Vice Chair Digital Coordination119 and the
Technologist Group, designed to act as a forum for discussion and knowledge sharing
among agency technologists, data scientists and other digital experts.

The Hellenic Competition Commission argues that competition authorities

should share cartel data from existing screens to create a training dataset for machine
learning.120 The CMA also notes that code sharing, including data pipeline, scraping,
tools and analysis, could be a game changer for regulators and the benefits of
international cooperation in these areas have the potential to be much higher than
other forms of knowledge sharing among regulators.121 In the area of bid rigging the
Australian Competition and Consumer Commission has been internally advocating
for more centralized, detailed and standardized collection of procurement data
within Australia and believes there would be benefits to standardization of such data
collection on a global scale.122 This should apply equally to the development of
standardization of data collection and formatting to facilitate the development and
training of algorithmic screens in other market areas.

However, pooling national datasets is not necessarily straightforward. Huber,

Imhof and Ishii studied the transferability of national screening datasets using Swiss
and Japanese datasets. 123 Seven different models were used, relying on screens based
on the variance, asymmetry and uniformity of bids. Two models relied on all the
screens, but with one also including the number of bids and contract values; another
two relied on all the screens but subject to demeaning (centering the screens within
countries such that they have a zero mean), while the remaining three each relied on
just two categories of screens.
Models trained and tested on just the Japanese dataset had a detection rate of 93
to 97% However, models trained on one national dataset and tested on the other
proved less effective. Ensemble models (which rely on the weighted average of six

118
Competition & Mkts. Aut., Compendium of Approaches to Improving Competition in Digital Markets, 47-
48 (2023), [Link]
improving-competition-in-digital-markets.
119
Australian Competition & Consumer Comm’n, ACCC Chair Rod Sims appointed to International
Competition Network Role (2021), [Link]
appointed-to-international-competition-network-role.
120
Hellenic Competition Comm’n, Computational competition law and economics - an inception report
(2021), [Link]
[Link].
121
Hunt supra note 101, at 45-46.
122
Australian Competition and Consumer Comm’n, Data Screening Tools for Competition Investigations
(OECD, 136th OECD Working Party 3 meeting 2022).
123
Martin Huber et al., Transnational Machine Learning with Screens for Flagging Bid-Rigging Cartels, 185
J. ROYAL STAT. SOC’Y SERIES A: STATISTICS IN SOC’Y 1074 (2022).
24 Stanford Computational Antitrust VOL. IV

different algorithms: bagged decision trees, Bayesian additive regression trees,

random forest, lasso regression, support vector machines and neural nets) had an
overall detection rate of 82% to 88%. However, these models had a significance
imbalance, as much as 20%, in the detection rate for competitive tenders and
collusive tenders. Models based on just random forest were significantly less effective
across the board with overall detection rates dropping to between 58% and 62%.

This performance drop was attributed to differing institutional dynamics in the

tendering process across the two countries. For example, the coefficient of variance
in collusive Swiss tenders is generally higher than in collusive Japanese tenders, likely
due to additional cost estimate information available to Japanese firms. This means
that the bid pattern for competitive Japanese tenders is comparable to that of
collusive Swiss tenders.

However certain specific subsets of predictors, such as bid asymmetry, seemed

robust to differing national dynamics. Relative comparators were also quite
effective. While in absolute terms all Japanese tenders had less variance than Swiss
tenders, collusive bids in both countries still have lower variance compared to the
national baseline. By applying the process of demeaning, to reduce the institutional
differences across countries, the authors were able to obtain detection rate of
between 85% and 90% for the ensemble models. This strongly suggests that
international, multilateral cooperation, whilst not without challenges, is likely to be
the way forward.

In 2019, a survey of OECD and ICN members reported that many do not have any
legal restrictions on sharing authority confidential information (information held
by an authority that is not in the public domain and while not prohibited by statute
from sharing, is considered confidential or sensitive) between regulators and that
doing so would be valuable. All the respondents reported that there is no difficulty
with sharing publicly available information.124

One potential solution would be to rely on synthetic data to avoid privacy

concerns.125 This involves the creation of artificial datasets derived from the original
that reproduces its structure and characteristics. Ideally this provides training data
and allows for valid statistical inferences to be made, without needing to disclose
sensitive or confidential information. This typically relies on altering the data, such
as by swapping the values of a few variables, adding random minor perturbations, or

OECD/ICN, Report on International Co-operation in Competition Enforcement 176 (2021).

124

125
Eur. Data Protection Supervisor, Synthetic Data (2022), [Link]
publications/publications/techsonar/synthetic-data_en.
25 “Knowledge Gap of Algorithmic Collusion” 2024

coarsening the variables by using larger categories (such as grouping by county or

state rather than zip code).126 These techniques are already widely used in healthcare
to avoid revealing data about individual patients.127

However, this field is relatively new and there are many technical obstacles to
overcome. There is no standard measure for the utility of synthetic data,128 and the
strategies for how best to generate synthetic data for use in machine learning (and
how best to tune models) is still an area of active research.129

Synthetic data can be created without direct use of collected data using a data
simulator. This has the potential to reduce training costs, as data collection and
processing tends to be complex and labor intensive. 130 Unfortunately, these
simulators can only generate data based on an existing model, and so the quality of
the generated data depends on how well the underlying phenomenon is understood
and at the moment the empirical dynamics of pricing algorithms is poorly
understood.

VII. Analyzing Algorithmic Pricing: Empirical and Technical

Audits

Having discussed the issues of collecting data, in this section we review how
pricing algorithms can be analyzed. In order to understand whether algorithmic
pricing leads to collusive outcomes, it is first of all necessary to develop adequate tools
to verify how the algorithms behave on the market. We discuss here the tools
available and whether competition authorities are already equipped to use them or
should be given new powers to this end.

Broadly speaking there are two main kinds of audits. Empirical audits attempt to
measure the effect of an algorithm by observing inputs and outputs,131 while a
technical audit examines the underling code or data.

126
Trivellore E. Raghunathan, Synthetic Data, 8 ANN. REV. STATISTICS AND ITS APPLICATION 129 (2021).
127
See Synthetic Data, CPRD [Link]
128
Joshua Snoke et al., General and Specific Utility Measures for Synthetic Data, 181 J. ROYAL STAT. SOC’Y
SERIES A: STATISTICS IN SOC’Y 663 (2018).
129
Fida K Dankar & Mahmoud Ibrahim, Fake It Till You Make It: Guidelines for Effective Synthetic Data
Generation, 11 APPLIED SCIENCES 2158 (2021).
130
Michal Gal & Orla Lynskey, Synthetic Data: Legal Implications of the Data-Generation Revolution 109
IOWA L. REV. (forthcoming).
131
Competition & Mkts. Aut., Auditing Algorithms: The Existing Landscape, Role of Regulators and Future
Outlook (2022), [Link]
processing-workstream-spring-2022/auditing-algorithms-the-existing-landscape-role-of-regulators-
and-future-outlook.
26 Stanford Computational Antitrust VOL. IV

As discussed above, there is relatively little empirical research on the pricing

patterns caused by algorithmic systems, and even fewer on algorithmic driven
collusion.132 However, trying to investigate harm caused by algorithms does have one
potential major advantage. For an algorithm, there is no difference between
simulated test data and actual market data. Given identical inputs, an algorithm
should respond in identical ways. Therefore, it is, in principle, easier to model the
behavior of an algorithm more accurately without the need for simplifying
assumptions and approximations for how it will act in a given situation.

An empirical audit was used in the European Commission’s investigation of

Google’s search engine, where it was found that it was giving preferential treatment
to its own comparison-shopping service, promoting it in search results at the expense
of competitors.133 Google had implemented the “Panda” algorithm , which they claim
was designed to reduce the rankings of sites with low quality content and promote
sites with unique, informative and original content. The Commission was able to
show that this algorithm downranked competing comparison-shopping services,
while Google’s own Shopping service was exempted.134 Critically, as part of their
evidence, the Commission was able to simulate the effect of swapping the order of
generic search results search queries to demonstrate that the same search result
received more traffic when higher rated, demonstrating that Google’s downranking
harmed competitors. This result may seem intuitive but proving it was an essential
part of the case. The Commission’s empirical analysis involved using 5.2 Terabytes of
search results, the equivalent of 1.7 billion queries.

While this does include analysis of historical data on inputs, it also includes
testing the algorithm by submitting specific simulated queries. This can be done on
the live system. For example, Chen et al. were able to analyze how Uber’s surge pricing
algorithm worked by emulating 43 copies of the Uber smartphone app over a period
of four weeks.135 Alternatively, this exercise can be performed in a “sandbox”,
running an isolated copy of the algorithm in a controlled environment. This avoids
the risk that the algorithm might learn from the test input data, for example,
repeated searches for a specific product might be interpreted as increased interest,
leading the pricing algorithm to raise prices. Sandboxing also potentially allows for
the temporary freezing of some of the algorithm’s parameters, allowing for a higher

132
See Wieting & Sapi, supra note 70; Assad et al., supra note 61.
133
See Comm’n Decision AT.39740, Google Search (Shopping) (2017).
134
Eur. Comm’n, Statement by Commissioner Vestager on Commission Decision to Fine Google €2.42 Billion
for Abusing Dominance as Search Engine by Giving Illegal Advantage to Own Comparison Shopping Service
(2017), [Link]
135
Le Chen et al., Peeking Beneath the Hood of Uber (Proceedings of the 2015 internet measurement
conference, 495-508, 2015), [Link]
27 “Knowledge Gap of Algorithmic Collusion” 2024

degree of control at the cost of the algorithm demonstrating non-realistic behavior

due to the artificial setting.136

One of the main limitations of the empirical audit technique is that while it can
identify potentially problematic behavior, it will not typically reveal the cause of the
behavior in the algorithmic code or how to address it. In order to do so, it is possible
to go one stage further and conduct a technical audit.137 With a technical audit not
only can an algorithm’s behavior be accurately tested, but it is possible to “read its
mind” by analyzing the underlying code. While regulators can only infer the
reasoning and decision-making processes of human actors, the decisional parameters
of an algorithm can be determined precisely.138 Attempts to obfuscate collusive
conduct by generating periods of apparent price instability as discussed above could
hinder detection attempts, but the code of the system would reveal those efforts
through a technical audit.

Technical audits do have their downsides, however, as direct code analysis is not
necessarily straightforward.139 The source code can be extensive, complex, or lacking
in documentation. Algorithms based on machine learning tend to be “black boxes,”
where the decision-making processes and the precise relevance of input parameters
can be opaque. These systems can be more readily understood through an empirical
audit.140

Harrington suggested that certain algorithmic features or decision rules may be

more liable to lead to anti-competitive outcomes and therefore proposed a research
program in which learning algorithms could be tested in a simulated market under a
range of conditions.141 By examining when and under what circumstances
supracompetitive or competitive prices emerge, it may be possible to identify what
properties are present for supracompetitive prices but not competitive prices.142 This
could ease the burden associated with technical audits, as it may be possible to
establish certain classes of algorithms that could either be prohibited or presumed to
be anticompetitive if they have no plausible rationale other than to facilitate an anti-
competitive outcome.143 A full technical audit would only be required if none of these
features are present. There is some evidence to suggest that asynchronous learning

136
Autorité de la Concurrence & Bundeskartellamt, supra note 45, at 72.
137
Competition & Mkts. Aut., supra note 102.
138
Michal Gal, Algorithms as Illegal Agreements, 34 BERKELEY TECH. L. J. 68 (2019).
139
See Autorité de la Concurrence & Bundeskartellamt, supra note 45, at 70.
140
Id. at 71-73.
141
Harrington, supra note 24.
142
Id.
143
Id.
28 Stanford Computational Antitrust VOL. IV

models, where the AI only learns about the result of the action it took, is more liable
to lead to near monopoly pricing while synchronous learning, where the AI also
learns the result of alternative actions it could have made, leads to competitive
pricing.144

Regulators could be given new powers that could require companies to assist in
testing algorithmic systems, both to assist in the development of more effective
algorithmic screens, and to aid in follow up investigations once potentially
problematic conduct has been detected. For example, the current Digital Markets,
Competition and Consumers Bill will give the CMA new investigative powers in
relation to the digital markets regime, including powers to require a person to obtain,
generate, collect, or retain specified information or to conduct a specified
demonstration or test.145 This would give the CMA the power to require an
undertaking to demonstrate how an algorithm operates or undertake empirical
audits or sandbox testing of the algorithm and report the outcomes.

Tsoukalas has suggested that New Competition Tool (NCT) should be resurrected
to address the threat of algorithmic collusion.146 Currently the Commission has no
remedial powers following a sector inquiry. The NCT was a proposal, seemingly
modelled after the UK’s market investigation system, to grant the Commission the
power to impose structural or behavioral remedies following an investigation. 147 The
NCT was abandoned following consultation, and not included in the Commission’s
proposal for the Digital markets Act.148

Under the Enterprise Act 2002 the CMA is granted the power to impose market
wide remedies independent of any individual infringement proceedings.149 The
CMA may launch an investigation if it has reasonable grounds for suspecting that
certain features of the market “prevents, restricts or distorts competition in
connection with the supply or acquisition of any goods or services in the United
Kingdom or a part of the United Kingdom.”150 If there is a finding of an “adverse
effect on competition,” the CMA has the power to take such action “as it considers to

144
John Asker et al., The Impact of Artificial Intelligence Design on Pricing, J. ECON. & MGM. STRATEGY
(2023).
145
Digital Markets, Competition and Consumers HC Bill (2022-23) § 350 §§ 68.
146
Vasileios Tsoukalas, Should the New Competition Tool be Put Back on the Table to Remedy Algorithmic
Tacit Collusion? A Comparative Analysis of the Possibilities under the Current Framework and under the NCT,
Drawing on the UK Experience, 13 J. EUR. COMPETITION L. & PRACTICE 234 (2022).
147
Eur. Comm’n, New Competition Tool (2020), [Link]
your-say/initiatives/12416-Single-Market-new-complementary-tool-to-strengthen-competition-
enforcement_en.
148
Eur. Comm’n, Proposal for a regulation of the European Parliament and of the Council on contestable and
fair markets in the digital sector (Digital Markets Act) COM (2020) 842 final (Dec. 15, 2020).
149
Enterprise Act 2002, s 138(2).
150
Id. at s 131(1).
29 “Knowledge Gap of Algorithmic Collusion” 2024

be reasonable and practicable to remedy, mitigate or prevent” it as well as resultant

“detrimental effects on customers.”151 This flexible tool allows the CMA to address
problems that might otherwise be difficult to remedy in a targeted and evidence-
based way. In the context of algorithmic collusion, these powers would circumvent
the question of whether autonomous algorithmic conduct amounts to an
infringement and allow for prospective remedies designed to make the market less
susceptive to this form of collusion in the future.152

One extreme option would be to outright ban algorithmic pricing under certain
conditions. In August 2023 the Italian government adopted a legislative degree
seeking to ban algorithmic pricing for domestic flights from Sicily or Sardinia when
sold during peak-demand seasons and if the resultant ticket price was 200% higher
than the average fare. It also sought to ban algorithmic personalised pricing based on
profiling. Such heavy-handed measures would likely have likely disincentivise
further investments in algorithmic systems, which, in general bring about
significant efficiency gains. Indeed, following an intervention by the European
Commission the Italian government has instead transferred the matter to Italian
Competition Authority (ITA) to oversee, rather than imposing a ban.153 Separately,
the ITA launched a market inquiry into possible airline price fixing, but the
investigation was closed on January 2024 without any finding of infringement.154

Another possibility, in light of the risks of automated conduct, would be to

require sellers or third-party providers to test and train algorithms in a sandbox to
ensure, as much as practically possible, that they are not prone to collusive outcomes.
This may be analogous to the testing regime required for algorithmic trading on the
financial markets within the EU. Art 17(1) of the Markets in Financial Instruments
Directive II (MiFID II) requires algorithms to be tested in a simulated market to
ensure that they do not behave in an unintended manner or “contribute to disorderly
trading conditions” before they can be used on a trading venue.155 Article 48(6)
MiFID II further obligates all trading venue operators to require their participants to
both carry out “appropriate testing of algorithms” and to provide “environments to
facilitate such testing.”156

151
Id. at s 138(2).
152
Francisco Beneke & Mark-Oliver Mackenrodt, Remedies for Algorithmic Tacit Collusion, 9 J. ANTITRUST
ENFORCEMENT 152 (2021).
153
Angelo Amante & Keith Weir, Italy's Government Dilutes Plan to Cap Airfares To Islands, REUTERS
(2024), [Link]
control-air-fares-islands-2023-09-19/.
154
Italian Competition Authority, ICA’s Bulletin No. 1 of 2 January 2024.
155
Community Delegated Regulation (EU) 2017/589 of July 19, 2016 on the Organisational
Requirements of Investment Firms Engaged in Algorithmic Trading O.J. (L 87/417).
156
Id.
30 Stanford Computational Antitrust VOL. IV

This provision is quite flexible, and seeks to be principle-based, rather than

prescriptive. Instead of explicitly setting out the testing regime and the exact
measures to be taken, which would risk the regulations becoming obsolete as newer
technologies and techniques are developed, it sets out high level guidelines and their
stated intent. The trading venue is obligated to provide a testing environment, which
may be carried out internally or by a third party.157 This could be implemented for
certain markets or online platforms if algorithmic collusion becomes a major
concern.

For example, Abada and Lambin found that some instances of apparent
“algorithmic collusion” by reinforcement learning algorithms in a simulated energy
market were due to insufficient exploration of the parameter space. The algorithms
would converge on a supra-oligopolistic price but under testing the authors found it
would punish both pro-competitive and pro-collusive deviations from it. Abada and
Lambin found this could be addressed with improvements to the training regime
with the inclusion of a maverick firm designed to bid aggressively when the other
players appeared to reach a collusive outcome but otherwise bid conservatively led to
a reduction in collusive outcomes and a commensurate improvement in overall social
welfare.158 More research is needed, but this suggests that requiring the inclusion of
this kind of maverick in the training environment could prevent collusive outcomes.

A tiered, risk-based approach, similar to that envisioned by the EU Artificial

Intelligence Act would likely be in order.159 This would require identifying which
market sectors or categories of system are most likely to be at risk. It is already well
documented that certain market features, such as homogenous products or high
barriers to market entry make collusion more likely and are therefore most at risk.160
Similarly, it would almost certainly be disproportionate to require extensive testing
of every algorithmic pricing system. Instead, the regime should only be limited to
algorithms with large user bases or companies with significant market power.

157
Patrick Raschner, Algorithms Put to Test: Control of Algorithms in Securities Trading Through Mandatory
Market Simulations?, Eur. Banking Inst. Working Paper Series No. 87 (2021),
[Link]
158
Ibrahim Abada & Xavier Lambin, Artificial Intelligence: Can Seemingly Collusive Outcomes Be Avoided?,
69 MGMT. SCIENCE 4973 (2023).
159
Eur. Comm’n, Proposal for a Regulation of the European Parliament and of the Council laying down
harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain Union legislative
acts, COM (2021) 206 final (Apr. 21, 2021).
160
OECD, supra note 6, 20-22.
31 “Knowledge Gap of Algorithmic Collusion” 2024

VIII. Conclusion

Digital markets are evolving rapidly and the use of pricing algorithms is
becoming increasingly prevalent. While they provide many benefits, there is a real
risk that they will bring new harms and new challenges for regulators seeking to
prevent anti-competitive behavior.

This paper did not address the question of what types of algorithmic pricing are
or may be antitrust violations. It also does not address the issue of whether tacit
collusion, which is not generally considered a violation in the absence of direct or
indirect human contact, should become a concern if achieved through the means of
algorithmic pricing. In the authors’ view, the first and most important step at this
stage is to understand the functioning of markets affected by algorithmic pricing in
order to verify whether the impact on the economy of new forms of “collusion” is
sufficiently severe to warrant a rethink of the law or even the introduction of new
regulatory tools.

To date, while preliminary evidence appears to suggest that pricing algorithms

have an impact on observed patterns of pricing behavior, there has been
comparatively little research into this area.161 More research is needed, not just to
detect what patterns can be attributed to collusive outcomes, but what regular
unlawful algorithmic behavior looks like.

Going forward, the authors consider the following five action points to be of
crucial importance in developing the knowledge that is needed for policy-making in
this area:

1. While there have been many theoretical studies, more empirical and
economic research is needed on the behavior and characteristics of actual
implementations of algorithmic pricing. In the first place, this could be
driven by competition authorities. To this end, competition authorities,
possibly leveraging on the existing frameworks of the OECD or the ICN,
should cooperate internationally to share technologies and pool data, so that
effective algorithms can be developed and trained to review and analyze
algorithmic prices.
2. Legal barriers to the sharing of technology and data internationally among
competition authorities for this purpose should be removed, while retaining
adequate safeguards if information is sensitive or confidential, or if third-

161
See Wieting & Sapi, supra note 70.
 32 Stanford Computational Antitrust VOL. IV

party rights are otherwise affected. Consistent standards for the collection
and formatting of data should also be developed.
3. Competition authorities should be given adequate investigative powers to
run empirical and technical audits, including enhanced data gathering
powers to this effect. This would be particularly important in relation to
“deep learning” algorithms, which do not respond to any predefined set of
rules.
4. Data and outcomes obtained by competition authorities, and, indeed, other
public authorities, should be made available, with appropriate safeguards, to
independent, academic researchers, whose work would prove crucial in
complementing, expanding upon, and verifying any research carried out by
competition authorities. Furthermore, public enforcers may not have the
resources to exploit the full potential of the data they have or to follow all
clues, which is instead the task of independent academia. This could be akin
to the data access and scrutiny provisions Digital Services Act, which grants
vetted researchers access to data from very large online platforms and very
large online search engines for the purposes of research that contributes to
the detection, identification and understanding of systemic risks.162
5. In the short term, in light of the risks of automated conduct, a possible
solution would be to require sellers or third-party providers to test
algorithms above a certain user threshold in a sandbox to ensure, as much as
practically possible, that they are not prone to collusive outcomes. A
solution for certain markets or online platforms analogous to the testing
regime required for algorithmic trading on the financial markets could be
implemented if algorithmic collusion were to become a major concern. 163

162
Regulation (EU) 2022/2065 of the European Parliament and of the Council of Oct. 19, 2022, on a
Single Market for Digital Services and Amending Directive 2000/31/EC (Digital Services Act), O.J. (L
277), Art 40.
163
Commission Delegated Regulation (EU) 2017/589 of July 19, 2016, Supplementing Directive
2014/65/EU of the European Parliament and of the Council with regard to Regulatory Technical
Standards Specifying the Organisational Requirements of Investment Firms Engaged in Algorithmic
Trading,O.J. (L 87), Art 17(1)(d).

Stanford Computational Antitrust Project Director: Dr. Thibault Schrepel // Editor-in-Chief: Teodora Groza

Editors: Thaiane Abreu, Eleanor Liu, Helena Mao Li, María Manuela Palacio Villarreal, Kirill Ryabtsev, Björn ten Seldam, Michael Wang, Glen Williams

Academic Outreach Chair: Aleksandra Wierzbicka // Executive Operations Chair: Alex Sotropa

Hosted by Stanford University CodeX Center