6. Parallel Patterns

6.1 Reductions and Maps

Standard patterns.

Reduction always achieves $O(\log n)$ parallel time.
Map achieves $O(\log n)$ (using divide and conquer).

Here, datastructures matter. Implementing a map or a reduction on a LinkedList incurs a lot of overhead, since we need to iterate over all elements in the array before the target.

Using a tree is beneficial (trees are as flexible as lists, if we don’t know the size in advance). This gets us $O({\log }_2(n))$ indexing instead of $O(n)$ iterating through list.

Task-Graphs:

With map we don’t coalesce the results together at the bottom. With reduce we have to as we output a single output.

6.2 Prefix Sum

1. Up: Build a binary tree
   • Root has sum of the range $[\mathtt{x},\mathtt{y})$
   • If a node has sum of $[\mathtt{lo},\mathtt{hi})$ and $\mathtt{hi}>\mathtt{lo}$,
     • Left child has sum of $[\mathtt{lo},\mathtt{middle})$
     • Right child has sum of $[\mathtt{middle},\mathtt{hi})$
     • A leaf has sum of $[\mathtt{i},\mathtt{i}+1)$, i.e., $\mathtt{input}[\mathtt{i}]$

This is an easy fork-join computation: combine results by actually building a binary tree with all the range-sums. The Tree is built bottom-up in parallel.

Analysis: $O(n)$ work, $O(\log n)$ span

2. Down: Pass down a value fromLeft
   • Root given a $\mathtt{fromLeft}$ of $0$
   • Node takes its $\mathtt{fromLeft}$ value and
     • Passes its left child the same $\mathtt{fromLeft}$
     • Passes its right child its $\mathtt{fromLeft}$ plus its left child’s $\mathtt{sum}$ (as stored in part 1)
   • At the leaf for array position $\mathtt{i}$: $\mathtt{output}[\mathtt{i}]=\mathtt{fromLeft}+\mathtt{input}[\mathtt{i}]$

This is an easy fork-join computation: traverse the tree built in step 1 and produce no result.

• Leaves assign to $\mathtt{output}$
• Invariant: $\mathtt{fromLeft}$ is sum of elements left of the node’s range

Analysis: $O(n)$ work, $O(\log n)$ span

6.3 Pack

Pack is built on top of prefix sum.

Example Given an array input, produce an array output containing only elements such that f(element) = true (so with a filter / condition).

We execute the following steps to get our result:

Note that:

• the first two steps can be combined into one pass
  • just different base case for prefix sum (if f(element) increase by one)
  • Has no effect on asymptotic complexity though
• Can also combine third step into down pass of prefix sum

Analysis $O(n)$ work, $O(\log n)$ span.

Parallel pack is the basis for parallel quicksort. We just pack on values bigger than the pivot and then smaller and rearrange.