Linux schedulers – overview

continued
CFS, MuQSS, sched_ext, EEVDF

Other topics related to process scheduling

 Table of contents

• Linux schedulers – overview (continued)

– CFS
– BFS
– MuQSS
– Sched_ext
– EEVDF
• Other topics related to process scheduling
– CPU Idle Loop
– Thermal pressure
– Energy Aware Scheduling (EAS)
– Arm [Link] CPU chip
– Core scheduling

2
 Completely Fair Scheduler
Ingo Molnar, April 2007
Ingo Molnar, April 2007
I wrote the first line of code of the CFS patch this week, 8am Wednesday morning, and released it
to lkml 62 hours later, 22pm on Friday.
I’d like to give credit to Con Kolivas for the general approach here: he has proven via RSDL/SD that
‘fair scheduling’ is possible and that it results in better desktop scheduling. Kudos Con!
The CFS patch uses a completely different approach and implementation from RSDL/SD. My goal
was to make CFS’s interactivity quality exceed that of RSDL/SD, which is a high standard to
meet 
18 files changed, 1454 inserstions(+), 1133 deletions(-)

Schedulers: the plot thickens, [Link] J. Corbet, April 2007

The new scheduler which got into the game so abruptly caused a big storm on Linux mailing lists.
Ultimately, CFS was incorporated into the 2.6.23 kernel, and Con Kolivas gave up work on the
Linux kernel (temporarily).
3
 Completely Fair Scheduler

• A completely new concept for the scheduler.

• Trying to implement Processor Sharing (there is no state in which one process has gotten more
of the CPU than another, since all tasks are running on it at once, so all processes have a fair
share of the CPU).
• On a real CPU unfairness will inevitably be nonzero when there are more tasks than CPUs.
When one task is running on a CPU, this increases the amount of CPU time that the CPU owes
to all other tasks. CFS schedules the task with the largest unfairness onto the CPU first.
• Unfairness is measured by the virtual runtime.
• In practice, the virtual runtime of a task is its actual runtime normalized to the total number of
running tasks.
• Virtual time flows at a priority-dependent speed.
• Instead of queues of tasks – a red-black tree, sorted after the virtual runtime and the selection
of the task, that run for the shortest time period.
4
 Completely Fair Scheduler

• It is also possible to group tasks and share processor time fairly among defined „entities” –
process groups.
• When implementing CFS, the code was reorganized to separate sections responsible for the
scheduling policy (the struct sched_class has been created, there is a pointer to this structure
in struct task_struct).
• Like the O(1) scheduler, CFS maintains separate data structures for each CPU. This reduces the
wait for the lock to be removed, but requires explicit processor load balancing.
• When a new task is created, it is assigned the minimum current vruntime (min_vruntime).

• With CFS, tasks have no concept of timeslice but struct task_struct {

...
rather run until they are no longer the most unfairly
const struct sched_class *sched_class;
treated task. To reduce context switching overhead, ...
CFS divides time into a minimum granularity. }

5
 Completely Fair Scheduler

Red-black tree
for CFS scheduler
process selection O(1) insertion
O(log(n))

• When a task has finished running on the CPU, all of the other tasks in the tree need to have their
unfairness increase.
• To prevent having to update all of the tasks in the tree the scheduler maintains a per-task vruntime
statistic.
• This is the amount of total nanoseconds that the task has spent running on a CPU weighted by its
niceness.
• Thus, instead of updating all other tasks to be more unfair when a task has finished running on the
CPU, we update the leaving task to be more fair than others by increasing its virtual runtime.
• The scheduler always selects the most unfairly treated task by selecting the task with the lowest
vruntime.

[Link] 6
 Completely Fair Scheduler
sched_latency_ns – the time when one era should take place, i.e. all tasks from the queue should
be completed. The default is 20 ms.
sched_min_granularity_ns – minimum time, which on average should be given to a task in an era.
The default is 4 ms.

epoch = max(sched_latency_ns; sched_min_granularity_ns * nr_running)

ideal_slice = epoch * (weight/∑weight)

Variable quantum lengths, but a fixed period of rotation of the era, guarantees small delays. Under
heavy load, the quanta are not shortened below the minimum value, at the expense of
responsiveness.

Processes with different priorities receive different weights. The virtual time is scaled with these
weights – the scheduler takes into account the differences in nice values of processes. Priority
weights are allocated as geometric progression:
prio_to_weight[n] ≈ prio_to_weight[n+1] * 1.25
Processes with a given priority difference will always receive CPU time in a constant proportion.
7
 Completely Fair Scheduler
weights of processes

A process with nice -20 will get about 6000 times more CPU time than a process with nice 19 (for
comparison: 160 time more than in the old scheduler) 8
CFS scheduler doesn’t deal with tasks, but with
scheduler entities of type struct sched_entity.
Sched entity may represent a task or a queue of
entities of type struct cfs_rq (which is referenced by
field my_q), thus allowing to build hierarchies of
entities and allocate resources to task groups
(cgroups).
Processor run queue, represented by type struct rq,
contains field cfs which is instance of struct cfs_rq
and contains queue of all high-level entities.
Each entity has cfs_rq pointer which points to CFS
runqueue to which that entity belongs.

In this example processor run queue has two scheduler entities: one CFS queue with single task
(which refers to top-level cfs_rq through parent pointer) in it and one top-level task.

Source: [Link] 9
 Why one scheduler for all is not enough?

• Sched_ext: pluggable scheduling in the Linux kernel, Kernel Recepies, Oct. 5, 2023, David Vernet.
– Scheduling is a notoriously difficult problem. An effective scheduler should fully utilize a system, while
also optimizing for cache locality, while also accounting for real time constraints, while also accounting
for battery life and power management, while also ensuring fairness, etc.
– The landscape of the tech industry has changed a lot in the last 15 years. Back in the late 2000s, cores
were typically homogeneous, and were spaced further apart from one another. Modern systems are by
comparison much more complex. Heterogeneous architectures are the norm for mobile devices, and
are becoming more common in x86. Cache hierarchies are also less uniform, with Core Complex (CCX)
chips having multiple shared L3 caches within a single socket.
– Use cases have evolved as well. Applications such as mobile and VR have latency requirements to avoid
missing deadlines that impact user experience, and stacking workloads in data centers is constantly
pushing the demands on the scheduler in terms of workload isolation and resource distribution.
– While CFS is a great scheduler, there are opportunities to continue to improve it for such use cases.
With sched_ext, we can easily experiment and find scheduling algorithms that address these use cases
by allowing developers to implement scheduling policies in BPF programs.
18
How to schedule tasks on a CPU with hybrid cores?

Crafting a Linux kernel scheduler in Rust – Andrea Righti (December 2024) 19

 Introduction – A short story of sched_ext
• Patchset v1: 2022-11-30, patchset v2: 2023-01-27
• Kernel Report 2023: The extensible scheduler class (write complete CPU schedulers in BPF)
• It allows users to write a custom scheduling policy using BPF without modifying the
kernel code.
• BPF provides a safe kernel programming environment.
• BPF verifier ensures that the custom scheduler has neither a memory bug nor an David Vernet
infinite loop.
• BPF scheduler can be updated without reinstalling the kernel and rebooting a server.
• Developed by engineers from Meta and Google.
• Why: easy experimentation, faster scheduler development, ad hoc schedulers for
special workloads.
• Why not: added maintenance burden, benchmark gaming, vendors may require
specific schedulers, ABI concerns, redirection of work on core scheduler.
Peter Zijlstra
• Rejected by scheduler maintainer (Peter Zijlstra). Changwoo Min’s
• September 30, 2024: Linus has released 6.12-rc1 – sched_ext got merged. introduction to the
sched_ext scheduling class
• [Link]
20
 Sched_ext: pluggable scheduling using BPF
• A new scheduling class, called SCHED_EXT, that can be selected with a sched_setscheduler() call like
most others.
• SCHED_EXT is placed between the idle class (SCHED_IDLE) and the completely fair scheduler
(SCHED_NORMAL) in the priority stack. As a result, no SCHED_EXT scheduler can take over the system.

• The BPF-written scheduler is global to the system as a whole.

• If there is no BPF scheduler loaded, then any processes that have been put into the SCHED_EXT class will
be run as if they were in SCHED_NORMAL.
• Once a BPF scheduler is loaded, it will take over the
responsibility for all SCHED_EXT tasks.
• A BPF program implementing a scheduler will manage
a set of dispatch queues, each of which may contain
runnable tasks that are waiting for a CPU to execute on.
• The BPF side of the scheduler is mostly implemented as
a set of callbacks to be invoked via an operations
structure, each of which informs the BPF code of an
event or a decision that needs to be made.
21
 Sched_ext: interfaces

A sched_ext scheduler consists of four layers,

bottom to top:
– core kernel scheduler,
– sched_ext framework,
– BPF scheduler, and
– BPF’s user-space counterpart.

These layers interact with each other through

(relatively) well-defined interfaces.

Sched_ext: scheduler architecture and

interfaces (part 2)

22
 Sched_ext: pluggable scheduling using BPF
• LSF/MM/BPF Summit 2024, 13 May, More features and use cases for sched_ext, David Vernet.
• LPC, Vienna, 18-20 Sept., 2024 – First public gathering of sched_ext community, many presentations
– Sched_ext at LPC 2024, Jonathan Corbet, Sept. 26, 2024
– „Hey, pssst, try this.” The underground culture around custom CPU schedulers.
– The current status and future potential of sched_ext, David Vernet.
• Scheduling with superpowers: Using sched_ext to get big perf gains, Kernel recepies, Oct. 1, 2024,
David Vernet.
– Since last year the project has grown significantly; both in terms of its technical capabilities, as
well as in the number of contributors and users of the project.
– sched_ext now runs at massive scale at Meta, and will also soon run as the default scheduler on
Steam Deck devices (the year of Linux gaming is upon us at last)!
– Some cutting edge sched_ext schedulers enable great performance on certain workloads.
– New features available in sched_ext, like cpufreq integration, which can improve both
datacenter and handheld workloads.

23
 Quotes of the week – posted by J. Corbet on Jan 2024
I ended up writing a Linux scheduler in Rust using sched_ext during Christmas break, just for
fun. I'm pretty shocked to see that it doesn't just work, but it can even outperform the
default Linux scheduler (EEVDF) with certain workloads (i.e., gaming).
Writing a Linux scheduler in Rust that runs in user-space, OSPM, 30 May 2024
Crafting a Linux kernel scheduler that runs in user-space using Rust, LPC, 18-20 Sept, 2024

Andrea Righi introduced scx_rustland, a framework to write CPU schedulers for Linux that run Andrea Righi
as user-space programs. Principal System
Software Engineer at
Initially a project aimed at teaching operating systems concepts to undergraduate students, it
NVIDIA
led Righi to appreciate the convenience of the change/build/run workflow to modify the
kernel's behaviour without rebooting.
This effort was able to prove that user-space
scheduling is not only possible, but can even
reach respectable performance, such as video
gaming at 60 frames per second while compiling
the Linux kernel at the same time.

scx_rustland_core architecture 24
 Earliest Eligible Virtual Deadline First (EEVDF)
• An EEVDF CPU scheduler for Linux, Jonathan Corbet, March 9, 2023.
• Completing the EEVDF scheduler, Jonathan Corbet, April 11, 2024.
• Work by Peter Zijlstra.
Peter Zijlstra
• (Oct 30, 2023) Merged as an option for the 6.6 kernel.
• It should provide better performance and fairness while relying less on fragile heuristics. The merge
message notes that there may be some rare performance regressions with some workloads, and that
work is ongoing to resolve them.
• One place where there is a desire for improvement is in the handling of latency.
• Scheduling algorithm is not new; it was described in this 1995 paper by Ion Stoica and Hussein Abdel-
Wahab.
• (April 05, 2024) Peter Zijlstra posted a patch series intended to finish the EEVDF work. Beyond some
fixes, this work includes a significant behavioral change and a new feature intended to help latency-
sensitive tasks.
• The amount of CPU time given to any two processes (with the same nice value) will be the same, but the
low-latency process will get it in a larger number of shorter slices.
• Currently a default scheduler (replaced CFS, which retired after serving 15+ years).
25
 Earliest Eligible Virtual Deadline First (EEVDF)
• There are many constraints beyond the fair allocation of CPU time that are placed on the scheduler.
– It should maximize the benefit of the system's memory caches.
– It should preserve battery life (power management).
– Improvement in the handling of latency is required.
• CFS does not give processes a way to express their latency requirements; nice values (priorities) can be
used to give a process more CPU time, but that is not the same thing.
• For each process, EEVDF calculates the difference between the time that process should have gotten (a
task's virtual run time) and how much it actually got (its actual running time); that difference is called lag.
• For any process with a negative lag, there will be a time in the future where the time it is entitled to
catches up to the time it has actually gotten and it will become eligible again; that time is deemed the
eligible time.
• The virtual deadline is the earliest time by which a process should have received its due CPU time. This
deadline is calculated by adding a process's allocated time slice to its eligible time. A process with a 10ms
time slice, and whose eligible time is 20ms in the future, will have a virtual deadline that is 30ms in the
future.
• EEVDF will run the process with the earliest virtual deadline first.
• Processes with shorter time slices will have closer virtual deadlines and, as a result, to be executed first.
26
 Earliest Eligible Virtual Deadline First (EEVDF)
1. CPU-bound tasks (A, B, and C) start at the same time. Before any of them runs, they will all have a
lag of zero:
A: 0ms B: 0ms C: 0ms
2. Since none of the tasks have a negative lag, all are eligible. If the scheduler picks A to run first
with, for example, a 30ms time slice, and if A runs until the time slice is exhausted, the lag
situation will look like this:
A: 10-30=-20ms B: 10-0=10ms C: =10-0=10ms
Over those 30ms, each task was entitled to 10ms of CPU time. A got 30ms, so it accumulated a lag
of -20ms; the other two tasks, which got no CPU time at all, ended up with 10ms of lag.
3. Task A is no longer eligible, so the scheduler will have to pick one of the others next. If B is given
(and uses) a 30ms time slice, the situation becomes:
A: -20+(10-0)=-10ms B: 10+(10-30)=-10ms C: 10+(10-0)=20ms
Each task has earned 10ms of lag corresponding to the CPU time it was entitled to, and B burned
30ms by actually running. Now only C is eligible, so the scheduler's next decision is easy.
4. The sum of all the lag values in the system is always zero

27
 Earliest Eligible Virtual Deadline First (EEVDF)
• Process Virtual Runtime (Vi) is a timer that only ticks when a process is actually using the CPU.
• Virtual System Time (Vsys) moves forward at a rate based on the combined weight of all active processes.
• A process is considered eligible if its Virtual Runtime (Vi) is less than or equal to the Virtual System Time
(Vsys):
Vi <= Vsys
• Virtual deadline (D) is the point in the future at which a process should complete its current turn to
maintain perfect fairness. It is calculated based on the process's weight (w) and the size of the time
quanta (request – r) that the process requests:
Di = Vi + ri/wi
• Lag is the difference between the amount of service a process should have received (its fair share) and
the amount of service it actually received:
Lagi = (Vsys – Vi) * wi

Task is eligible to run if its Lag >= 0.

28
 Earliest Eligible Virtual Deadline First (EEVDF)
Real time 0 ms

29
 Earliest Eligible Virtual Deadline First (EEVDF)
Real time 30 ms

30
 Earliest Eligible Virtual Deadline First (EEVDF)
Real time 60 ms

31
 Earliest Eligible Virtual Deadline First (EEVDF)
Real time 90 ms

Real time 120 ms

32
Earliest Eligible Virtual Deadline First (EEVDF)

33
 Earliest Eligible Virtual Deadline First (EEVDF)
• Any task can request shorter time slices, which will cause it to be run sooner and, possibly,
more frequently. If, however, the requested time slice is too short, the task will find itself
frequently preempted and will run slower overall.
• A task can use the sched_setattr() system call, passing the desired slice time (in nanoseconds)
in the sched_runtime field of the sched_attr structure. The allowed range for time slices is
100µs to 100ms.
• EEVDF allows one task to preempt another if its virtual deadline is earlier. This provides more
consistent timings for short-time-slice tasks, while slowing long-running tasks slightly.
• When a task sleeps, it is normally removed from the run queue so that the scheduler need not
consider it. In EEVDF an ineligible process that goes to sleep will be left on the queue, but
marked for "deferred dequeue". Since it is ineligible, it will not be chosen to execute, but its
lag will increase according to the virtual run time that passes. Once the lag goes positive, the
scheduler will notice the task and remove it from the run queue. The result of this
implementation is that a task that sleeps briefly will not be able to escape a negative lag
value, but long-sleeping tasks will eventually have their lag debt forgiven. A positive lag value
is, instead, retained indefinitely until the task runs again.
34
Other topics related to process scheduling
 Role of CPU Idle loop
• Intel - CPU idle time management, Rafael J. Wysocki
• When there are no other tasks to run on a CPU, the idle task runs on it.
• The idle task’s code is the idle loop.
• The idle loop calls into cpuidle to allow the CPU to be put into an energy-saving state (if this
makes sense).
• Cpuidle uses a governor to decide which idle state to put the CPU into (and whether of not to
stop the scheduler tick on it).
• Three cpuidle governors are available (in the mainline), but 2 of them are practicaly relevant
(menu and teo).
• Idle state parameters that are used by the governors for making decisions are the target
residency and the exit latency.

36
 CPU Idle Loop
CPU Idle Loop Rework, Rafael J. Wysocki (Intel), 2018.
What’s a CPU to do when it has nothing to do?, Tom Yates, October 2018.
Although increasingly deep idle states consume decreasing amounts of power, they have
increasingly large costs to enter and exit. It is in the kernel's best interests to predict how long a
CPU will be idle before deciding how deeply to idle it. This is the job of the idle loop. The
scheduler then calls the governor, which does its best to predict the appropriate idle state to
enter.
He reworked the idle loop for kernel 4.17 so that the decision about stopping the tick is taken
after the governor has made its recommendation of the idle state.

37
 CPU Idle Loop
CPU Idle Loop Rework, Rafael J. Wysocki (Intel), 2018.

High-level CPU idle time management control flow But there is a CPU scheduler tick timer ..
Original idle loop design issue

jmd@students$ cat /sys/devices/system/cpu/cpuidle/current_governor_ro

menu 38
 CPU Idle Loop

Before

Short idle duration prediction problem

After

Redesigned idle loop (Linux 4.17 and later)

CPU Idle Loop Rework, Rafael J. Wysocki (Intel), 2018. 39
 Idle power (Intel OTC Server Power Lab)

The green line is with the old

idle loop, the red is with the
new: power consumption is
less under the new scheme,
and moreover it is much more
predictable than before.

CPU Idle Loop Rework, Rafael J. Wysocki (Intel), 2018.

40
 CPU Idle Loop

• Improving idle behavior in tickless systems, Marta Rybczyńska, December 2018.

Linux currently provides two cpu idle governors, ladder and menu. Wysocki implemented the third, timer
events oriented (TEO) – similar to menu, but takes into cosideration different factors.
• Fixing SCHED_IDLE, Viresh Kumar, November 2019 – The 5.4 kernel release includes a few improvements to
the existing SCHED_IDLE scheduling policy that can help users improve the scheduling latency of their high-
priority (interactive) tasks if they use the SCHED_IDLE policy for the lowest-priority (background) tasks.
• Intel - CPU idle time management, Rafael J. Wysocki, Intel, OSPM 20, May 31, 2024. Problems with governors
– they don’t take latency into account.
• Many presentations on OSPM, March 18-20, 2025.

41
 Energy Model of devices • The Energy Model (EM) framework serves as an interface
between drivers knowing the power consumed by devices at
various performance levels, and the kernel subsystems willing
Energy models of devices
to use that information to make energy-aware decisions.
• The source of the information about the power consumed by
devices can vary from one platform to another.
• In order to avoid each and every client subsystem to re-
implement support for each and every possible source of
information, the EM framework serves as an abstraction layer
which standardizes the format of power cost tables in the
kernel.
• The power values might be expressed in micro-Watts or in an
abstract scale.
• In case of CPU the EM framework manages power cost tables
per performance domain in the system. A performance domain
is a group of CPUs whose performance is scaled together.
• Performance domains generally have a 1-to-1 mapping with
CPUFreq policies.

42
 Energy Aware Scheduling
• Energy Aware Scheduling (EAS) gives the scheduler the ability to predict the impact of its decisions on
the energy consumed by CPUs.
• EAS relies on an Energy Model of the CPUs to select an energy efficient CPU for each task, with a
minimal impact on throughput.
• EAS operates only on heterogeneous CPU topologies (such as Arm [Link] and other multi-core
SoCs ) because this is where the potential for saving energy through scheduling is the highest.
• Definitions
– energy = [joule] (resource like a battery on powered devices)
– power = energy/time = [joule/second] = [watt]
• The goal of EAS is to minimize energy, while still getting the job done. That is, we want to
– maximize: performance [inst/s] / power [W] which is equivalent to
– minimizing: energy [J] / instruction.
• It is essentially an alternative optimization objective to the current performance-only objective for the
scheduler. This alternative considers two objectives: energy-efficiency and performance.
• The use-cases where EAS can help the most are those involving a light/medium CPU utilization.
43
 Energy Aware Scheduling
• Researched since 2013. In 2019 added to Linux 5.0.
• Energy Aware Scheduling (EAS) on ARM wiki. Arm, Linaro and key partners are contributing
jointly to the development of EAS. Energy-Aware Scheduling Project on [Link].
• An Unbiased Look at the Energy Aware Scheduler, Vitaly Wool, Embedded Linux Conference,
2018.
• Evaluating vendor changes to the scheduler, Jonathan Corbet, May 2020.
The benchmark results for each of these patches were remarkably similar. They all tended to
hurt performance by 3-5% while reducing energy use by 8-11%.
• Saving frequency scaling in the data center, J. Corbet, May 2020.
Frequency scaling — adjusting a CPU's operating frequency to save power when the workload
demands are low — is common practice across systems supported by Linux. It is, however,
viewed with some suspicion in data-center settings, where power consumption is less of a
concern and there is a strong emphasis on getting the most performance out of the hardware.
• Imbalance detection and fairness in the CPU scheduler, J. Corbet, May 2020.
44
 Scheduling – thermal pressure
Telling the scheduler about thermal pressure, Marta Rybczyńska, May 2019.
Even with radiators and fans, a system's CPUs can overheat. When that happens, the kernel's thermal
governor will cap the maximum frequency of that CPU to allow it to cool. The scheduler, however, is not
aware that the CPU's capacity has changed; it may schedule more work than optimal in the current
conditions, leading to a performance degradation.
The solution adds an interface to inform the scheduler about thermal events so that it can assign tasks better
and thus improve the overall system performance.
The term thermal pressure means the difference between the maximum processing capacity of a CPU and the
currently available capacity, which may be reduced by overheating events.
The two approaches, the thermal pressure approach and energy-aware scheduling (EAS), have different
scope: thermal pressure is going to work better in asymmetric configurations where capacities are
different and it is more likely to cause the scheduler to move tasks between CPUs.
The two approaches should also be independent because thermal pressure should work even if EAS is not
compiled in.
Enhancements and adjustments of the thermal control subsystem, Rafael J. Wysocki , Linux Plumbers
Conference (LPC), September 19, 2024.
45
 Scheduling – Arm [Link] CPU chip
Scheduling for asymmetric Arm systems, Jonathan Corbet, November 2020.
The [Link] architecture placed fast (but power-hungry) and slower (but more power-efficient) CPUs in the
same system-on-chip (SoC); significant scheduler changes were needed for Linux to be able to properly
distribute tasks on such systems.
Putting tasks on the wrong CPU can result in poor performance or excessive power consumption, so a lot of
work has gone into the problem of optimally distributing workloads on [Link] systems.
When the scheduler gets it wrong, though, performance will suffer, but things will still work.
Future Arm designs, include systems where some CPUs can run both 64-bit and 32-bit tasks, while others are
limited to 64-bit tasks only. The result of an incorrect scheduling choice is no longer a matter of
performance; it could be catastrophic for the workload involved.
• What should happen if a 32-bit task attempts to run on a 64-bit-only CPU?
• Kill the task or
• recalculate the task's CPU-affinity mask?

Other asymmetric architectures:

• Apple M4 MAX: 12-P cores and 4 E-cores
• Intel Core Ultra 9: 8 P-cores and 16 E-cores
Cortex A57/A53 MPCore [Link] CPU chip
 Core scheduling
Core scheduling, Jonathan Corbet, February 2019.
SMT (simultaneous multithreading) increases performance by turning
one physical CPU into two virtual CPUs that share the hardware; while
one is waiting for data from memory, the other can be executing. Sharing
a processor this closely has led to security issues and concerns for years,
and many security-conscious users disable SMT entirely.

On kernels where core scheduling is enabled, a core_cookie field is added to the task structure. These
cookies are used to define the trust boundaries; two processes with the same cookie value trust each
other and can be allowed to run simultaneously on the same core. (Peter Zijlstra)
Completing and merging core scheduling, Jonathan Corbet, May 2020.
A set of virtualization tests showed the system running at 96% of the performance of an unmodified
kernel with core scheduling enabled; the 4% performance hit hurts, but it's far better than the 87%
performance result measured for this workload with SMT turned off entirely.
The all-important kernel-build benchmark showed almost no penalty with core scheduling, while turning
off SMT cost 8%.
47
 Core scheduling
Core Scheduling Looks Like It Will Be Ready For Linux 5.14 To Avoid Disabling SMT/HT, Michael Larabel,
May 2021.
Core scheduling lands in 5.14, Jonathan Corbet, 2021.
Core scheduling should be effective at mitigating user-space to user-space and user-to-kernel attacks
when the functionality is properly used. But the default kernel policy will not change over how tasks
are scheduled but is up to the administrator for identifying tasks that can or cannot share CPU
resources.

[Link] 48
 The future of hyperthreading?
(Oct 2024) Hyperthreading is becoming less prominent in mainstream CPUs,
with Intel removing it from their latest P-cores (Performance cores) in favor
of hybrid designs, citing better power efficiency, die space savings for more
dedicated cores, and security benefits, though it's still useful and present in
some high-end Intel and AMD server/enthusiast chips.
Chipmakers like Apple and Qualcomm never used it, favoring efficiency for
mobile devices, and Intel is now adopting similar strategies with its P-core/E-
core (Efficient core) architecture, making SMT less essential for overall [Link]
performance gains.

(Oct 2025) Now Intel states it will take steps to halt and reverse its declining market share in PC and server
processors—literally to revitalize the “Intel x86 ecosystem.” Among these revitalization attempts, Lip-Bu
Tan mentions reintroducing Hyper-Threading. He cites its removal as a mistake that cost Intel market share
and revenue, explicitly stating it created a “competitive disadvantage” and that its return will help “close
performance gaps.”
…we are reintroducing simultaneous multi-threading (SMT). Moving away from SMT put us at a
competitive disadvantage. Bringing it back will help us close performance gaps.

49
 Linux news – April 2025

• Cache awareness for the CPU scheduler, Jonathan Corbet, April 29, 2025 (work in progress)
– If a process has multiple threads, those threads are likely to be sharing memory and could
benefit from running within the same cache domain.
– A per-CPU array is added to mm_struct to keep track of how much time threads using that
mm_struct spend on each CPU in the system. This data decays over time, so recent usage is
more strongly represented than usage in the distant, forgotten past (a few tens of milliseconds
ago, say).
– When the time comes to wake a thread that had been waiting for some event, the scheduler
goes to that per-CPU array and determines which CPU has spent the most time executing
threads from the same process. If the thread of interest has been running elsewhere, it will be
moved to the selected CPU, where it will be closer to the other threads and, with luck, benefit
from sharing cache space with them.

50
 Conclusions

Scheduler performance varies

dramatically according to hardware and
workload, and as a result we strongly
encourage Linux distributions to take an
increased level of responsibility for
selecting appropriate default schedulers
that best suit the intended usage of the
system.

53