Parallel computing

Parallel computing is the simultaneous execution of the same task (split up and specially adapted) on multiple processors in order to obtain faster results.

Inter-thread communication

Parallel computers are theoretically modeled as Parallel Random Access Machines (PRAMs). The PRAM model ignores the cost of interconnection between the constituent computing units, but is nevertheless very useful in providing upper bounds on the parallel solvability of many problems. In reality the interconnection plays a significant role.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

The processors may either communicate in order to be able to cooperate in solving a problem or they may run completely independently, possibly under the control of another processor which distributes work to the others and collects results from them (a "processor farm").

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

Processors in a parallel computer may communicate with each other in a number of ways, including shared (either multiported or multiplexed) memory, a crossbar, a shared bus or an interconnect network of a myriad of topologies including star, ring, tree, hypercube, fat hypercube (an hypercube with more than one processor at a node), a n-dimensional mesh, etc. Parallel computers based on interconnect network need to employ some kind of routing to enable passing of messages between nodes that are not directly connected. The communication medium used for communication between the processors is likely to be hierarchical in large multiprocessor machines. Similarly, memory may be either private to the processor, shared between a number of processors, or globally shared. Systolic array is an example of a multiprocessor with fixed function nodes, local-only memory and no message routing.

Related Topics:
Topologies - Routing - Systolic array

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

Approaches to parallel computers include:

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

• Multiprocessing
• Computer cluster
• Parallel supercomputers
• Distributed computing
• NUMA vs. SMP vs. massively parallel computer systems
• Grid computing