Memory Management in Rust Explained

0% found this document useful (0 votes)
116 views14 pages
Memory thinking with Rust

Memory Thinking for Rust

Slides with Descriptions and Source Code Illustrations

Dmitry Vostokov
Software Diagnostics Services

OpenTask
2

Memory Thinking for Rust: Slides with Descriptions and Source Code
Illustrations

Published by OpenTask, Republic of Ireland

Copyright © 2024 by OpenTask

Copyright © 2024 by Dmitry Vostokov

Copyright © 2024 by Software Diagnostics Services

Copyright © 2024 by Dublin School of Security

All rights reserved. No part of this book may be reproduced, stored in a


retrieval system, transmitted in any form or by any means, or used for
training artificial intelligence systems without the prior written permission
of the publisher.

OpenTask books are available through booksellers and distributors


worldwide. For further information or comments, send requests to
press@[Link].

Product and company names mentioned in this book may be trademarks of


their owners.

A CIP catalog record for this book is available from the British Library.

ISBN-13: 978-1912636662 (Paperback)

Revision 1.00 (June 2024)


3

Table of Contents

Table of Contents 3

Preface 15

About the Author 16

Introduction 17

Prerequisites 18

Training Goals 19

Warning 20

Training Principles 21

Schedule 22

Training Idea 23

General Rust Aspects 24

What We Do Not Cover 26

Linux Rust Aspects 27

Windows Rust Aspects 28

Why Rust? 29

My Genealogy of Rust 31

Rust Mastery Process 32

Thought Process 33
4

Philosophy of Unsafe Pointers 34

A General Pointer Concept 35

Pointer 36

Pointer Dereference 37

Many to One 38

Many to One Dereference 39

Invalid Pointer 40

Invalid Pointer Dereference 41

Wild (Dangling) Pointer 42

Pointer to Pointer 43

Pointer to Pointer Dereference 44

Naming Pointers and Entities 45

Names as Pointer Content 46

Pointers as Entities 47

Unsafe Rust Code Examples 48

Unsafe Pointer 49

Unsafe Pointer Dereference 51

Unsafe Many to One 53

Unsafe Many to One Dereference 55


5

Invalid Pointer 57

Invalid Pointer Dereference (Alignment) 58

Invalid Pointer Dereference (Access Violation) 59

Wild (Dangling) Pointer 60

Unsafe Pointer to Pointer 62

Unsafe Pointer to Pointer Dereference 64

Philosophy of Values 66

Values and Owners 67

Moving Values 69

Copying Values 71

Dropping Values 73

Ownership Tree 75

Ownership Tree and Drops 77

Partial Drops 81

Multiple Owners (not in Rust) 82

Multiple Owners and Drops 83

Owners vs. Pointers 84

Rust: A Copernican Revolution 85

Values Revolve around Pointers 86


6

Owners Revolve around Values 87

Rust Philosophy of Values 88

Restricted Ownership 89

Value Lifetime 91

Owner Lifetime 94

Rust Philosophy of Pointers 96

Types of Pointers 97

References as Pointer Types 98

References as Addresses 100

Borrowing References 103

Reference Lifetime 105

x64 Disassembly Review (WinDbg) 107

x64 CPU Registers 108

Instructions and Registers 109

Memory and Stack Addressing 110

Memory Cell Sizes 111

Memory Load Instructions 112

Memory Store Instructions 113

Flow Instructions 114


7

Function Parameters 115

Struct Function Parameters 116

x64 Disassembly Review (GDB AT&T Flavor) 117

x64 CPU Registers 118

x64 Instructions and Registers 119

Memory and Stack Addressing 120

x64 Memory Load Instructions 121

x64 Memory Store Instructions 122

x64 Flow Instructions 123

x64 Function Parameters 124

x64 Struct Function Parameters 125

ARM64 Disassembly Review 126

A64 CPU Registers 127

A64 Instructions and Registers 128

Memory and Stack Addressing 129

A64 Memory Load Instructions 130

A64 Memory Store Instructions 131

A64 Flow Instructions 132

A64 Function Parameters 133


8

A64 Struct Function Parameters 134

Memory Storage 135

Memory Regions 136

Dynamic Virtual Memory 137

Static Memory 138

Rust Static Memory Values 139

Rust Static Memory References 140

Stack Memory 145

Thread Stack Frames 146

Local Value Lifecycle 147

Rust Stack Memory Values 148

Rust Stack Memory References 149

Heap Memory 152

Rust Heap Memory Values 153

Useful WinDbg Commands 156

Useful GDB Commands 157

Memory and Pointers 158

Mental Exercise 159

Debugger Memory Layout 160


9

Memory Dereference Layout 161

Names as Addresses 162

Addresses and Entities 163

Addresses and Structures 164

Pointers to Structures 165

Arrays 166

Arrays and Pointers to Arrays 167

Fat Pointers 168

Array Slices 169

String Literals (UTF-8) 173

Byte Strings 176

Vectors 178

Vector Slices 179

Strings 183

String Slices 184

C-Strings 187

C-String Slices 188

Basic Types 191

Bytes, References, and Pointers 192


10

u32, Pointers, and References 195

Little-Endian System 198

u64 and Pointers 200

Size 203

Alignment 209

Entity Conversion 211

Conversion through Pointers 212

Safe Conversion 215

Forcing 217

Tuples 219

Structs 222

Tuple-like Structs 223

Newtypes 227

Named-field Structs 230

Reference/Pointer to Struct 235

Ref/Ptr to Struct Dereference 236

Dereference with Replacement 237

Many Ref/Ptr to One Struct 240

Many to One Dereference 241


11

Ref/Ptr to Ref/Ptr to Struct 242

Ref/Ptr to Ref/Ptr Dereference 243

Memory and Structs 248

Addresses and Structs 249

Struct Field Addresses 250

Ref/Ptr to Structs 254

Ref/Ptr to Struct and Fields 255

External Struct Alignment 259

Internal Struct Alignment (WinDbg) 260

Internal Struct Alignment (GDB) 261

Source Code and Symbols 264

Conceptual Layer (Modules) 265

Logical Layer (Crates) 266

Physical Layer (Source Files) 267

Name Isolation 268

Functions 271

Pointers to Functions 272

References to Functions 275

Function Pointer Types 277


12

Struct Function Fields 278

Associated Functions 280

Pointers to Associated Functions 282

Type-associated Functions 284

Trait Functions 286

Trait Objects 287

vtable Memory Layout 289

Trait Object Memory Layout 290

Boxed Trait Object Layout 296

Struct Constructors 300

Struct Destructor 303

Struct Clone 308

Struct Copy 310

Parameters by Value 312

Parameters by Ref/Ptr 316

self 319

Closures and Captures 321

Closure Struct 322

A64 Closure Struct Example 324


13

Captures (Borrowing) 325

Captures (Borrowing) x64 Linux 326

Captures (Move) 328

Captures (Move) x64 Linux 329

Pinning 331

Use Cases 332

Rust Books 338

Common questions

Powered by AI

Rust's memory model offers significant advantages for systems programming by providing memory safety without a garbage collector, avoiding common errors related to manual memory management seen in languages like C/C++. It achieves this by ensuring that every memory access through borrowing and ownership is safe. However, this model can introduce complexity, particularly with lifetimes and borrowing rules, which may increase the learning curve for developers transitioning from languages with less stringent memory management requirements. Despite this, it often results in clearer and more robust code .

Rust's philosophy of values centers around the idea that values themselves dictate the memory allocation pattern. Values in Rust have a clear lifecycle starting from ownership, moving, copying, and eventually dropping. This dictates memory allocation as Rust automatically manages memory via its ownership model. Values are allocated on the stack or heap depending on their size and usage context, with ownership directly influencing when memory gets deallocated, thus avoiding leaks and ensuring efficient memory use .

Lifetimes in Rust specify the validity scope of references, ensuring that references do not outlive the data they point to. This ties closely with Rust's borrowing mechanism, which allows references to be taken to values without transferring ownership. With lifetimes, the compiler checks if the references are valid throughout their usage, effectively preventing use-after-free errors. This strict lifetime management coupled with borrowing allows Rust to guarantee memory safety and prevent data races without runtime overhead .

Rust's approach to pointers, particularly through concepts of safe and unsafe pointers, significantly enhances both safety and efficiency. Safe pointers and the borrowing system allow for memory access without the common pitfalls associated with raw pointers, such as dereferencing null or dangling pointers. Unsafe pointers exist but are restricted and must be clearly marked, ensuring the programmer's intention and additional scrutiny. This helps in maintaining efficiency close to C/C++ while preventing many common errors associated with pointer manipulation .

In Rust, the ownership model is central to its memory management strategy, ensuring memory safety without a garbage collector. Each value in Rust has a single owner, and the ownership can be transferred but cannot be shared with multiple owners simultaneously. This is different from traditional pointer-based languages where multiple pointers can refer to the same memory location leading to potential data races. Rust enforces strict rules at compile time to prevent issues related to data races, invalid pointers, and memory leaks by using a combination of ownership, borrowing, and lifetimes .

Rust is fundamentally designed around the principles of safety, concurrency, and efficiency. The ownership model provides safety by preventing data races and ensuring memory safety without garbage collection. Concurrency is facilitated through the aliasing XOR mutability principle, which allows safe concurrent data access. Efficiency is ensured through zero-cost abstractions and compile-time optimizations. These principles are reflected in its syntax and semantics through features like lifetimes, borrowing, and the strict delineation of safe and unsafe code blocks, allowing developers to write robust and performant code .

Rust's approach to thread safety is heavily reliant on its ownership and borrowing models, which prevent data races by design. Unlike traditional languages where thread safety often depends on runtime checks or developer-determined synchronization mechanisms, Rust enforces compile-time guarantees that shared data is immutable unless explicitly intended. Additionally, Rust's 'Send' and 'Sync' traits ensure that types are safe to be sent across threads and accessed from multiple threads, respectively. These compile-time assurances result in more robust concurrent applications with fewer runtime errors .

Using unsafe code in Rust, especially with pointers, can introduce risks similar to those found in unmanaged languages like C/C++. These include null pointer dereferencing, data races, and memory corruption. However, Rust mitigates these risks by confining unsafe code to blocks expressly marked as 'unsafe', thus clearly signaling potential issues to developers. Additionally, Rust's powerful static analysis tools and compiler checks help catch errors even in these unsafe blocks, requiring the developer to ensure that their code maintains safety guarantees manually .

Rust's zero-cost abstractions provide high-level programming constructs without incurring runtime penalties, crucial for systems programming where performance is critical. This is achieved by using compile-time checks and optimizations which ensure that abstractions like iterators or option types have no overhead compared to equivalent manual implementations. Consequently, programmers can write safe, maintainable code while still achieving the performance levels necessary for system-critical applications, combining the best of high-level language safety with low-level language performance .

Rust ensures safety with references by enforcing that all references are valid during their use through the concept of lifetimes, preventing issues such as dangling or null references. Rust's borrowing rules further enhance this by limiting how references can be used simultaneously. Unsafe references become necessary when interfacing with low-level system calls or performing optimizations that require bypassing Rust's borrow checker. Despite being marked as 'unsafe', developers must still ensure soundness through testing and careful review .

You might also like