8. Shortest Path Algorithms (BFS, Dijkstra's)

We are searching for the shortest walk in a graph. We can make the observation that the shortest walk is always a path (counting distance as the length of the walk, if we count edge-costs, then there might be negative ones).

Distance in a directed Graph

In the directed Graph $G=(V,E),n=|V|$, we define $\text{dist}(u,v)=\text{Shortest Length of a walk from }u\text{ to }v$

For a shortest path problem that has additional state, such as cheats or similar, look at 0. Layered Graphs for an explanation.

Shortest Path - BFS

Runtime: $O(|V|+|E|)$.

If we want to find the shortest path from a vertex to another, we will need to determine the distances to all other reachable nodes, lest we overlook a “shortcut”.

The algorithm BFS (Breadth-First-Search) explores the graph in layers, first going to all adjacent vertices, then those adjacent to those, etc… We can easily disregard cycles (already visited nodes), as they would imply longer distances.

The tree at the bottom is called a shortest path tree. It represents the shortest distance from a vertex to all other reachable ones.

Levels

We can easily see here that there are multiple nodes at the same distance from $A$ as others.

Distance Levels

We can call this a level $k$ (of vertices at the same distance): $S_k:=\{v\in V|\text{dist}(s,v)=k\}$

If there’s an edge from an $S_k$ to an $S_{k^{\prime }}$, then $k^{\prime }\le k+1$ as it increases distance by at most $1$ (and in an undirected graph $k-1\le k^{\prime }\le k+1$, because it can also go back).

Algorithm How can we recursively compute the set $L_k$, given $L_0,L_1,\dots ,L_{k-1}$? We loop over the vertices, checking the following conditions for $v\in L_k$:

1. We have to check that the vertex is not in previous levels ($v\notin L_0\cup L_1\cup \cdots \cup L_{k-1}$). If $v$ was already in a level closer to $s$ (start vertex), then that would be a shorter path.
2. Successor of a vertex in $L_{k-1}$ We have to make sure $v$ is actually reachable in that distance $k$, which by our previous formula is the case if there is an edge from a $u\in L_{k-1}$ to $v$.

Implementation with Queues

Using a queue (FIFO) for the implementation of BFS makes the algorithm more efficient and easier to implement.

We maintain the list of levels “in the queue” instead of explicitly.

def bfs_shortest_paths(G, s):
  """Computes shortest path distances using BFS with a queue."""
  dist = {v: float('inf') for v in G}
  
  dist[s] = 0
  q = deque([s])  # Initialize queue with the source vertex
  parent = {} #dictionary to maintain shortest path tree
  
  while not q.empty():
    u = q.dequeue()
    
    for v in G[u]: # get adjacent nodes
        if dist[v] == float('inf'): # check v was not visited yet
          dist[v] = dist[u] + 1 # set distance
          parent[v] = u # u is parent of v
          
          q.enqueue(v)
          
  return dist, parent # return distances and the shortest path tree

1. Initialisation:
   • Set the distance to all vertices to $\infty$ in the d[v] array. Set the d[s] = 0.
   • Initialise a Queue $Q$ with $s$
   • Set the dictionary parent = {}
2. Exploration:
   • Dequeue the first element in the queue $v$
   • For all adjacent nodes $u$ with distance $=\infty$ (not visited yet):
     • Set the distance d[u] = d[v] + 1
     • add $u$ to the queue
     • Set the parent[u] = v.
3. Return: We return the distances and the shortest path tree

Runtime

The while loop runs over all vertices (exactly once as we check to prevent visiting multiple times), and in each operation we take $O(1+\mathrm{deg}(u))$.

The runtime of BFS is $O(|V|+|E|)$, we have constant time enqueue/dequeue and no set operations (needed without queue).

Enter and Leave Order

Same as with pre-/postordering, we can use enter-/leave-ordering here:

• enter step at which vertex $v$ is first encountered.
• leave step at which vertex $v$ is dequeued

We can observe that:

• enter[v] < leave[v] This is fundamental
• enter order $=$ leave order: Within a given level, vertices enter the queue in the same order as they leave it in, this is due to the FIFO nature of a queue. Not true across different levels
• Epochs (see Levels)
  • $t_k=\min \{\text{leave}[v]|\text{dist}(s,v)\ge k\}$ is defined as the first time a vertex with distance $k$ is dequeued
  • $t_0=1\le t_1\le \cdots \le t_{n-1}\le t_n=\infty$ are the epochs in the execution of BFS
  • $R_k:=\{v|t_k\le \text{leave}[v]<t_{k+1}\}$
  • We then have $R_k=L_k$

We can prove BFS is correct using induction.

Checking for Bipartite Graph using BFS

We can use BFS to check if a graph is bipartite by checking if it can be two-coloured.

While traversing the tree, in each layer, we colour all vertices with the same. If we then encounter a vertex with the same colour during traversal, it’s not two-colourable.

public boolean isBipartite(int[][] graph) {
	int[] c = new int[graph.length];
	Queue<Integer> Q = new LinkedList<Integer>();
 
	int s = 0;
 
	c[s] = 1;
	Q.add(s);
 
	while (Q.size() > 0) {
		int u = Q.remove();
 
		for (int v : graph[u]) {
			if (c[v] == 0) {
				c[v] = (c[u] % 2) + 1;
 
				Q.add(v);
			} else {
				if (c[v] == c[u]) return false;
			}
		}
 
		if (Q.size() == 0) {
			for (int i = 0; i < c.length; i++) {
				if (c[i] == 0) {
					Q.add(i);
					c[i] = 1;
					break;
				}
			}
		}
	}
 
	return true;
}

Cheapest Walks in Weighted Graphs

Cost of a Walk

In a weighted graph ($G=(V,E,c)$) each edge is assigned a cost/weight. The cost of a walk is the sum of the weight of it’s edges: ${\sum }_{i=0}^{l-1}c(v_i,v_i+1)$.

The cost of the cheapest path between $u,v$ is denoted as $d(u,v):=\min \{c(W)|W\text{ walk from }u\text{ to }v\}$.

Negative Costs If we introduce the possibility of negative weights, there could be a cycle with a total negative cost. This means that we can’t have a “cheapest” path anymore as we can arbitrarily reduce the cost by just traversing the cycle more often.

A cheapest path in a weighted graph (without negative cycles) is the optimal substructure: any subpath is itself the cheapest path between it’s endpoints.

Triangle Inequality

The triangle inequality holds in a weighted graph: $d(u,v)\le d(u,w)+d(w,v)$

This holds as if the path through $w$ was actually cheaper, then $d(u,v)$ would be wrong.

Cheapest path recursively

We can define the cheapest path from $u$ to $v$ recursively thanks to the triangle inequality $d(u,v)=\{\begin{array}{ll}0& \text{if }u=v\\ {\min }_{(u,w)\in E}\{c(u,w)+d(w,v)\}& \text{if }u\ne v\text{ and }(u,w)\in E\\ \infty & \text{otherwise}\end{array}$

The cheapest path from $u$ to $v$ is either $0$ if $u=v$ or the minimum cost among all paths that go from $u$ to some neighbour $w$ and then continue to $v$ from $w$.

Implementation

Toposort - DAG

Restrictions No cycles Runtime $O(|V|+|E|)$

In an acyclic graph, topological sorting is already an algorithm that gives us the most-efficient order to calculate the cost in.

Go overt he array and for each set the distance of it’s neighbours: if d[u] + c(u, v) < d[v] update d[v] to d[u] + c(u, v). We can do this as if there’s a path from $u$ to $v$, we have already calculated distance to $u$ before we go to $v$ (as it’s toposorted).

Cheapest Path Weighted Graph - Dijkstra’s

We assume that there are no negative edge-weights in the following.

We need to carefully consider the order of computation to avoid infinite loops or illegal access to values. Dijkstra’s implicitly sorts the vertices by distance from the source and then uses only the previous ones in the calculation!

For Dijkstra’s, we needed to prove that for $v_k$, the cheapest path from the source $s$ to $v_k$ is correctly computed by the recurrence $d(s,v_k)={\min }_{(v_i,v_k)\in E,i<k}\{d(s,v_i)+c(v_i,v_k)\}$Vertices are considered in increasing order of their distances from the source. To guarantee this, we maintain a set $S$ of vertices with definitely determined distance to the source.

From this state, we would now choose the vertex $v^{\ast }$ with the minimum $d_s(v^{\ast })$ value, in this case it would be from $v_3$ using the edge with cost $5$ for a total of $15$: $d_s(v^{\ast })={\min }_{u\in S}\{d(s,u)+c(u,v^{\ast })\}$ This works as the distance $d_s(v^{\ast })$ is equal to the true shortest path distance $d(s,v^{\ast })$.

Proof

1. Assume for contradiction that $d(s,v^{\ast })\ne d_s(v^{\ast })$. This implies that there is a cheaper path from $s$ to $v^{\ast }$ that the one found by $d_s(v^{\ast })$.
2. Alternative Path Let $W$ be such a cheaper path from $s$ to $v^{\ast }$. Since $v^{\ast }\notin S$, this path $W$ must leave the set $S$ at some point. Let $u$ be the last vertex in $S$ along the path $W$. Let $v$ be the first vertex outside $S$ along $W$ (note that $v$ might be $v^{\ast }$ itself).
3. Cost $c(W)=d(s,u)+c(u,v)+d(v,v\ast )$
4. Non-negative edge-costs since all edge costs are non-negative $d(v,v^{\ast })\ge 0$. Therefore $c(W)\ge d(s,u)+c(u,v)$.
5. Contradiction
   1. We know that $d_s(v)$ is the minimal path going from any node in $S$ to $v$, therefore $d(s,u)+c(u,v)\ge d_s(v)$.
   2. …
   3. Since we chose $v^{\ast }$ such that $d_s(v^{\ast })$ is minimal (first vertex we considered in greedy algo), we have $d_s(v^{\ast })\le d_s(v)\le c(W)$.
   4. This contradicts our assumption that $W$ is a cheaper path.

We conclude that $d(s,v^{\ast })=d_s(v^{\ast })$.

Algorithm

Restrictions No negative edge-weights Runtime $O((|E|+|V|)\log |V|)$

def dijkstra(graph, start):
    # Initialize distances: all nodes are at "infinity" except the start node (0 distance to itself)
    distances = {node: infinity for each node in graph}
    distances[start] = 0
    
    # Create a priority queue to process nodes in the order of their distance from the start
    priority_queue = [(0, start)]  # (distance, node)
 
    while priority_queue is not empty:
        # Get the node with the smallest distance from the queue
        current_distance, current_node = pop the smallest item from priority_queue
        
        # For each neighboring node:
        for neighbor, weight in graph[current_node]:
            # Calculate the total distance to the neighbor
            distance = current_distance + weight
            
            # If this new distance is shorter than the previously known distance:
            if distance < distances[neighbor]:
                # Update the distance to this neighbor
                distances[neighbor] = distance
                
                # Add the neighbor to the queue for further exploration
                push (distance, neighbor) into priority_queue
 
    return distances

Use a MinHeap (a priority queue basically) as an efficient data structure in which we store the distances. Then we can quickly find the vertex with the currently cheapest cost and iterate.

The runtime is calculated from $O(n+(\#\text{extract-min}+\#\text{decrease-key})\cdot \log n)$ which gives $O((n+m)\cdot \log n)$.

Why the restriction on negative edge-weights?

The reason Dijkstra’s algorithm does not work on graphs with negative edges is that we exclude vertices from analysis once we visited them. As the distance to a vertex is not modified once we reached it, negative edges would violate the “safety property” we are exploiting.

In a non-negative graph, the triangle inequality holds and thus we mark visited vertices as safe. No path can be longer but cheaper. With negative edge-weights this no longer applies.

When is it better to use Dijkstra’s with an Array instead of a Priority Queue

We can also implement Dijkstra’s using an array instead of a priority queue. Runtime:

• extract_min takes $O(|V|)$ with an array ($O(\log |V|)$ in a MinHeap).
• decrease_key takes $O(1)$ in an array ($O(\log |V|)$ in a MinHeap) which is done maximum $|E|$ times for $O(|E|)$ total runtime.

Therefore, the array implementation takes $O(|V{|}^2+|E|)=O(|V{|}^2)$ for $|E|=\Theta (|V{|}^2)$ (there are at most $|V{|}^2$ edges in a graph).

The MinHeap version takes $O((|V|+|E|)\log |V|)$.

In cases where the graph is very dense, i.e. $|E|>\frac{|V{|}^2}{\log |V|}$, it makes more sense to use Dijkstra’s with an array.

Negative Edge-Weights Allowed - Bellman-Ford

Restrictions Negative edge weights allowed Runtime $O(|V|\times |E|)$ Detects negative weight cycles but can’t handle them (obviously, as cost is undefined there).

The idea of Bellman-Ford is to sort the vertices in the recursion by the number of edges in the cheapest path. We define the $S_l$: $S_{\le l}:=\{v\in V|\exists \text{cheapest path with }\le l\text{ Kanten}\}$ und $S_{\le 0}=\{s\}$ and $S_{\le n-1}=V$.

We then establish the recursion $\forall v\in S_{\le l}\setminus \{s\}$ $d(s,v)=\min \{d(s,u)+c(u,v)|u\to v,u\in S_{\le l-1}\}$The difficulty is calculating $S_{\le l}$, but we can calculate $S_{\le l}$ good bounds.

Initialise the distances as $d[v]=\{\begin{array}{ll}0& \text{if }v=s\\ c(s,v)& \text{if }s\to v\\ \infty & \text{otherwise}\end{array}$ then we have $l$-good bounds $d[v]\{\begin{array}{ll}=d(s,v)& \text{if }v\in S_{\le l}\\ \ge d(s,v)& \text{otherwise}\end{array}$ We then improve the bounds from $l-1$ to $l$ by iterating over $v\in V$ and setting $d[v]=\min \{d[v],{\min }_{u\to v}d[u]+c(u,v)\}$We have $d[u]=d(s,u)$ for $u\in S_{\le l-1}$ thus this new $d[v]=d(s,v)$ for $v\in S_{\le l}$.

We have to perform this “relaxation” $n-1$ times, as this is the longest possible path from $s$ to $t$ for a connected graph with $n$ vertices.

Algorithm

It’s quicker to implement the edge-based approach, which iterates over all edges in each loop and then updates the distances immediately.

In this example, we start with $A$, thus $d[A]=0$ and the rest is $\infty$. In the first relaxation step, we do: Edge Based:

• $A\to B$: cost $3$ and $d[A]+c(A,B)=0+3<\infty$, thus $d[B]=3$.
• $A\to C$: cost 1, thus $d[C]=0+1=1$
• $B\to D$: cost 2 and $d[D]=d[B]+c(B,D)=3+2=5$
• …
• $F\to C$ : cost $-11$ $d[C]=d[F]-11=9-11=-2<d[C]=1$ thus we update $d[C]=-2$.

In the second relaxation step, we can improve on $d[E]$:

• $C\to E$: cost 5 and $d[C]=-2$, thus $d[E]=-2+5=3<6$ Vertex Based:
• $A\to C$: cost 1, thus $d[C]=0+1=1$
• $F\to C$: cost -11, but we have $d[F]=\infty$ thus we don’t update
• …
• $D\to F$ now $d[F]=9$ In the second relaxation step, we can improve on $d[C]$:
• $F\to C$: cost -11 and $d[F]=9$, thus $d[C]=9-11=-2<1$

It depends on the order and method if one of these is faster, but they both iterate over all edges and thus find the same result.

EdgeBasedBellmanFord(G, s):
  Initialize distances: d[s] = 0, d[v] = ∞ for all v ≠ s
 
  for i = 1 to |V| - 1:  // Main relaxation loop
    for each edge (u, v) in E:
      if d[u] + c(u, v) < d[v]:
        d[v] = d[u] + c(u, v)
 
  for each edge (u, v) in E:  // Negative cycle detection
    if d[u] + c(u, v) < d[v]:
      return "Negative cycle detected"
 
  return d // Shortest path distances

We iterate over all edges in the “relaxation” thus the time complexity of that step is $O(m)$ (the actual check is $O(1)$). As we relax $n-1$ (or $n$ for negative cycle check) times, the total runtime is $O(n\cdot m)$.

Negative Cycle detection with Bellman-Ford

We relax the edges one more time after $n-1$ times. If the distance to an edge decreased, there’s a negative cycle reachable from $s$.