Introduction to Parallel Computing

[Nytro]

[h=1]Introduction to Parallel Computing[/h]

[TABLE=width: 100%]

[TR]

[TD]Author: Blaise Barney, Lawrence Liver[/TD]

[TD=align: right][/TD]

[/TR]

[/TABLE]

[h=2]Table of Contents[/h]

[LIST=1]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Abstract"]Abstract[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Overview"]Overview[/URL]
[LIST=1]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Whatis"]What is Parallel Computing?[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#WhyUse"]Why Use Parallel Computing?[/URL]
[/LIST]

[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Concepts"]Concepts and Terminology[/URL]
[LIST=1]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Neumann"]von Neumann Computer Architecture[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Flynn"]Flynn's Classical Taxonomy[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Terminology"]Some General Parallel Terminology[/URL]
[/LIST]

[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#MemoryArch"]Parallel Computer Memory Architectures[/URL]
[LIST=1]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#SharedMemory"]Shared Memory[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DistributedMemory"]Distributed Memory[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#HybridMemory"]Hybrid Distributed-Shared Memory[/URL]
[/LIST]

[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Models"]Parallel Programming Models[/URL]
[LIST=1]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#ModelsOverview"]Overview[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#ModelsShared"]Shared Memory Model[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#ModelsThreads"]Threads Model[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#ModelsMessage"]Distributed Memory / Message Passing Model[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#ModelsData"]Data Parallel Model[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Hybrid"]Hybrid Model[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#SPMD-MPMD"]SPMD and MPMP[/URL]
[/LIST]

[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Designing"]Designing Parallel Programs[/URL]
[LIST=1]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignAutomatic"]Automatic vs. Manual Parallelization[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignUnderstand"]Understand the Problem and the Program[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignPartitioning"]Partitioning[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignCommunications"]Communications[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignSynchronization"]Synchronization[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignDependencies"]Data Dependencies[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignLoadBalance"]Load Balancing[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignGranularity"]Granularity[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignIO"]I/O[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignCosts"]Limits and Costs of Parallel Programming[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#DesignPerformance"]Performance Analysis and Tuning[/URL]
[/LIST]

[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#Examples"]Parallel Examples[/URL]
[LIST=1]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#ExamplesArray"]Array Processing[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#ExamplesPI"]PI Calculation[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#ExamplesHeat"]Simple Heat Equation[/URL]
[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#ExamplesWave"]1-D Wave Equation[/URL]
[/LIST]

[*][URL="https://computing.llnl.gov/tutorials/parallel_comp/#References"]References and More Information[/URL]
[/LIST]

[TABLE=width: 100%]

[TR]

[TD=bgcolor: #98ABCE] Abstract

[/TD]

[/TR]

[/TABLE]

This tutorial is the first of eight tutorials in the 4+ day "Using LLNL's Supercomputers" workshop. It is intended to provide only a very quick overview of the extensive and broad topic of Parallel Computing, as a lead-in for the tutorials that follow it. As such, it covers just the very basics of parallel computing, and is intended for someone who is just becoming acquainted with the subject and who is planning to attend one or more of the other tutorials in this workshop. It is not intended to cover Parallel Programming in depth, as this would require significantly more time. The tutorial begins with a discussion on parallel computing - what it is and how it's used, followed by a discussion on concepts and terminology associated with parallel computing. The topics of parallel memory architectures and programming models are then explored. These topics are followed by a series of practical discussions on a number of the complex issues related to designing and running parallel programs. The tutorial concludes with several examples of how to parallelize simple serial programs.

Tutorial:

https://computing.llnl.gov/tutorials/parallel_comp/