Vector Operations

Understanding High-Performance Computing with Vectors

๐Ÿ“Š What are Vector Operations?

Many scientific problems require arithmetic operations on large arrays of numbers. These numbers are usually formulated as vectors and matrices of floating-point numbers.

A vector V of length n is an ordered set of a one-dimensional array of data items. It can be represented as:

V = [Vโ‚ Vโ‚‚ Vโ‚ƒ ... Vโ‚™] (row vector)

The element Vแตข of vector V is written as V(i) where the index i refers to a memory address or register where the number is stored.

๐Ÿ”„ Scalar vs. Vector Processing

Aspect Scalar Processing Vector Processing
Operations One element at a time Multiple elements simultaneously
Loop Overhead High (fetch, execute per iteration) Minimal (single instruction)
Instruction C(I) = B(I) + A(I) C(1:100) = A(1:100) + B(1:100)
Efficiency Lower for large datasets Higher for large datasets

๐Ÿ’ป Scalar Processing Example

Traditional C loop processing elements sequentially:

for (i = 1; i <= 100; i++) {
    C[i] = B[i] + A[i];
}

This requires the following machine language sequence:

   Initialize I = 0
20 Read A(I)
   Read B(I)
   Store C(I) = A(I) + B(I)
   Increment I = I + 1
   If I โ‰ค 100 go to 20
   Continue

This loop reads a pair of operands from arrays A and B, performs floating-point addition, updates the control variable, and repeats 100 times.

โšก Vector Processing Example

A computer capable of vector processing eliminates the overhead with a single vector instruction:

C(1:100) = A(1:100) + B(1:100)

The vector instruction includes:

  • Initial address of the operands
  • Length of the vectors
  • Operation to be performed

The addition is done with a pipelined floating-point adder, significantly improving performance.

๐Ÿ—๏ธ Vector Instruction Format

A typical vector instruction format (Figure 9-11):

Operation
Code
Base Address
Source 1
Base Address
Source 2
Base Address
Destination
Vector
Length

This is essentially a three-address instruction with three fields specifying base addresses and an additional field for vector length.

Implementation Options:

  • Memory-based: Vectors reside in memory, accessed during operation
  • Register-based: Store all operands in registers prior to addition. Base address and length specify a group of CPU registers.

๐ŸŽฏ Benefits of Vector Processing

  • Reduced Loop Overhead: Eliminates time to fetch and execute loop control instructions
  • Single Instruction Execution: One composite instruction handles entire vector operation
  • Pipelined Operations: Floating-point operations executed in parallel pipeline stages
  • Improved Performance: Significant speedup for large-scale scientific computations
  • Simplified Programming: Higher-level abstraction for array operations

๐ŸŽจ Interactive Vector Visualization

Vector A:

Vector B:

Result C = A + B:

Start Learning Matrix Multiplication Using Vector Processing