Scribd
Upload
2 views4 pages

Parallel Computing Complete Notes

Parallel computing addresses the limitations of traditional computing by utilizing multiple cores to perform tasks simultaneously, thus overcoming issues related to increased power and heat from faster processors. It requires careful programming to manage coordination, communication, synchronization, and load balancing among cores. This approach is essential for tackling modern computational challenges in fields like AI and big data, offering significant performance improvements despite its complexity.

Uploaded by

Faiza Asad
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
2 views4 pages

Parallel Computing Complete Notes

Parallel computing addresses the limitations of traditional computing by utilizing multiple cores to perform tasks simultaneously, thus overcoming issues related to increased power and heat from faster processors. It requires careful programming to manage coordination, communication, synchronization, and load balancing among cores. This approach is essential for tackling modern computational challenges in fields like AI and big data, offering significant performance improvements despite its complexity.

Uploaded by

Faiza Asad
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Why Parallel Computing - Complete

Detailed Notes
WHY PARALLEL COMPUTING – COMPLETE EXPLANATION

1. Physics Limits in Computing

Earlier, computers became faster by increasing clock speed and reducing transistor size.

However:

Faster CPU → more power → more heat → system becomes unstable.

So, we cannot keep increasing processor speed.

2. Solution: Multicore Processors

Instead of one fast processor, we use multiple cores working simultaneously.

Example:

1 worker → slow

4 workers → fast

3. Need for Parallel Programming

Even if a system has multiple cores, serial programs use only one core.

Example:

for(i = 0; i < n; i++) {

sum += arr[i];

Solution: Divide work and run simultaneously.

4. Requirements of Parallel Programming


Coordination:

All cores must work together properly.

Communication:

Cores must share data.

Synchronization:

Cores may need to wait for each other.

Load Balancing:

Work must be equally distributed.

5. Parallel Programs: Complex but Faster

They are complex due to coordination and communication, but much faster because tasks
run simultaneously.

6. Serial vs Parallel Code

Serial:

int sum = 0;

for(int i = 0; i < n; i++) {

sum += compute_next_value(i);

Parallel:

Each core computes part:


int local_sum = 0;

for(int i = my_start; i < my_end; i++) {

local_sum += compute_next_value(i);

Naive Method:

Master collects all results → slow

Better Method (Tree Reduction):

Combine results step by step:

Step 1: 0←1, 2←3, 4←5, 6←7

Step 2: 0←2, 4←6

Step 3: 0←4

7. Why We Need More Computing Power

Modern problems like AI, climate modeling, and big data require huge computation.

8. Growth of Computing

Earlier: 50% performance increase per year

Now: ~20%

9. Parallel Computing Concept

Multiple processors work together simultaneously.

10. Multi-core Systems

Dual-core, Quad-core, Octa-core systems.


11. Data vs Task Parallelism

Data Parallelism: Same task on different data

Task Parallelism: Different tasks on same data

12. Types of Systems

Shared Memory: All cores share memory

Distributed Memory: Each core has its own memory

13. Key Terms

Concurrent: Tasks overlap

Parallel: Tasks run simultaneously

Distributed: Multiple systems collaborate

14. Final Conclusion

Parallel computing solves performance limitations using multiple processors.

It requires smart programming but provides significant speed improvement.

You might also like