Parallel Processing

Pipeline and Vector Processing

Definition and Purpose

Parallel processing is a computing technique where multiple calculations or processes are carried out simultaneously.

This approach aims to significantly improve the speed and efficiency of executing tasks. It is particularly effective for complex problems that can be broken down into smaller, independent parts.

Challenges in Parallel Processing

While parallel processing leads to significant performance improvements, it also introduces several challenges:

  • Synchronization between processes
  • Managing shared resources
  • Ensuring that tasks are divided and coordinated effectively

Processor Organization with Multiple Functional Units

A system designed for parallel execution may feature a processor with multiple functional units. This architecture includes processor registers whose output is routed to various specialized units.

Processor with Multiple Functional Units

Format with Mode

Functional Units

Adder-subtractor
Integer multiply
Logic unit
Shift unit
Incrementer
Floating-point add-subtract
Floating-point multiply
Floating-point divide

The results from these functional units typically loop back to the processor registers.

Classification of Parallel Processing (Flynn's Taxonomy)

Parallel processing systems can be classified in various ways. One widely recognized way to consider the organization of a computer system is by the number of instructions and data items manipulated simultaneously, as proposed by M. J. Flynn.

The fundamental sequence of operation is to fetch an instruction from memory and execute it.

Key concepts:

  • Instruction Stream: The sequence of instructions fetched from memory
  • Data Stream: The operations performed on the data within the processor

Parallel processing can occur in the instruction stream, in the data stream, or both.

Flynn's Four Categories

SISD
Single Instruction, Single Data

Traditional sequential processor executing one instruction on one data element at a time.
SIMD
Single Instruction, Multiple Data

Same instruction executed on multiple data elements simultaneously. Vector processing follows this principle.
MISD
Multiple Instruction, Single Data

Multiple instructions operate on the same data stream. Rare in practice.
MIMD
Multiple Instruction, Multiple Data

Multiple processors execute different instructions on different data independently.
Note on SIMD: Vector processing is related to the SIMD (Single Instruction, Multiple Data) principle, as a single vector instruction operates on multiple data elements simultaneously. SIMD extensions (like SSE and AVX in x86 processors) are examples of modern processors incorporating vector processing principles.

Interactive Challenge

Test your understanding of parallel processing concepts!

Start Learning Pipelining