Parallel computing

Parallel computing is the simultaneous execution of the same task (split up and specially adapted) on multiple processors in order to obtain faster results.

Parallel software

A huge number of software systems have been designed for programming parallel computers, both at the operating system and programming language level. These systems must provide mechanisms for partitioning the overall problem into separate tasks and allocating tasks to processors. Such mechanisms may provide either implicit parallelism -- the system (the compiler or some other program) partitions the problem and allocates tasks to processors automatically (also called automatic parallelizing compilers) -- or explicit parallelism where the programmer must annotate his program to show how it is to be partitioned. Most of the current implementations of parallelizing compilers only support single-level parallelism, as opposed to multi-level parallelism (also called nested parallelism), which allows threads already running in parallel to spawn further parallelism. It is also usual to provide synchronisation primitives such as semaphores and monitors to allow processes to share resources without conflict.

Related Topics:
Operating system - Implicit parallelism - Compiler - Explicit parallelism - Single-level parallelism - Multi-level parallelism - Semaphores - Monitors

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

Load balancing attempts to keep all processors busy by moving tasks from heavily loaded processors to less loaded ones.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

Communication between tasks is usually done with threads communicating via shared memory or with message passing, either of which may be implemented in terms of the other.

Related Topics:
Shared memory - Message passing

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

Well known parallel software problem sets are:

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

• Embarrassingly parallel problems
• Grand Challenge problems