Max flow intuition: If \(\color{purple}s\) is a faucet, \(\color{Blue}t\) is a drain, and \(\color{purple}s\) connects to \(\color{Blue}t\) through a network of pipes \(E\) with capacities \(\color{green}c(e)\), what is the maximum amount of water which can flow from the faucet to the drain?
Network Flow
flow / capacity
Assignment of values \(\color{red}f(e)\) to edges
“Amount of water going through that pipe”
Capacity constraint
\({\color{red}f(e)} \le {\color{green}c(e)}\)
“Flow cannot exceed capacity”
Flow constraint
\(\forall v \in V - \{ {\color{purple}s},{\color{Blue}t}\}\), \(\text{inflow}(v)=\text{outflow}(v)\)
\(\text{inflow}(v)=\sum_{x \in V}f(x,v)\)
\(\text{outflow}(v)=\sum_{x \in V}f(v,x)\)
Water going in must match water coming out
Flow of \(G\): \(|f|=\text{outflow}({\color{purple}s})-\text{inflow}({\color{purple}s})\)
Net outflow of \(\color{purple}s\)
3 in this example
Maximum Flow Problem
Of all valid flows through the graph, find the one that maximizes:
Greedy choice: saturate highest capacity path first
Greedy Approach
Greedy choice: saturate highest capacity path first
Greedy Approach
Greedy choice: saturate highest capacity path first
Flow: 20
Greedy Approach
Greedy choice: saturate highest capacity path first
Maximum flow: 30
Observe: highest capacity path is not saturated in optimal solution
Residual Graphs
Given a flow \(f\) in graph \(G\), the residual graph \(G_{\color{red}f}\) models additional flow that is possible
Forward edge for each edge in \(G\) with weight set to remaining capacity \({\color{green}c(e)}-{\color{red}f(e)}\)
Models additional flow that can be sent along the edge: flow to add
Backward edge by flipping each edge \(e\) in \(G\) with weight set to flow \({\color{red}f(e)}\)
Models amount of flow that can be removed from the edge: flow to remove
Flow \(\color{red}f\) in \(G\)
Residual graph \(G_{\color{red}f}\)
Residual Graphs
Given a flow \(f\) in graph \(G\), the residual graph \(G_{\color{red}f}\) models additional flow that is possible
Forward edge for each edge in \(G\) with weight set to remaining capacity \({\color{green}c(e)}-{\color{red}f(e)}\)
Models additional flow that can be sent along the edge: flow to add
Backward edge by flipping each edge \(e\) in \(G\) with weight set to flow \({\color{red}f(e)}\)
Models amount of flow that can be removed from the edge: flow to remove
Flow \(\color{red}f\) in \(G\)
Residual graph \(G_{\color{red}f}\)
Residual Graphs Example
Flow Graph
Residual Graph
Residual Graphs
Consider a path from \({\color{purple}s} \rightarrow {\color{Blue}t}\) in \(G_{\color{red}f}\) using only edges with positive (non-zero) weight Consider the minimum-weight edge along the path: we can increase the flow by \(w(e)\)
Send \(w(e)\) flow along all forward edges (these have at least \(w(e)\) capacity)
Remove \(w(e)\) flow along all backward edges (these contain at least \(w(e)\) units of flow)
Observe: Flow has increased by \(w(e)\)
Residual Graphs
Consider a path from \({\color{purple}s} \rightarrow {\color{Blue}t}\) in \(G_{\color{red}f}\) using only edges with positive (non-zero) weight Consider the minimum-weight edge along the path: we can increase the flow by \(w(e)\)
Send \(w(e)\) flow along all forward edges (these have at least \(w(e)\) capacity)
Remove \(w(e)\) flow along all backward edges (these contain at least \(w(e)\) units of flow)
Observe: Flow has increased by \(w(e)\)
Residual Graphs
Consider a path from \({\color{purple}s} \rightarrow {\color{Blue}t}\) in \(G_{\color{red}f}\) using only edges with positive (non-zero) weight Consider the minimum-weight edge along the path: we can increase the flow by \(w(e)\)
Send \(w(e)\) flow along all forward edges (these have at least \(w(e)\) capacity)
Remove \(w(e)\) flow along all backward edges (these contain at least \(w(e)\) units of flow)
Observe: Flow has increased by \(w(e)\)
Ford-Fulkerson Algorithm
Define an augmenting path to be an \({\color{purple}s} \rightarrow {\color{Blue}t}\) path in the residual graph \(G_{\color{red}f}\) (using edges of non-zero weight)
Ford-Fulkerson max-flow algorithm:
Initialize \({\color{red}f(e)}=0\) for all all \(e \in E\)
Construct the residual network \(G_{\color{red}f}\)
While there is an augmenting path \(p\) in \(G_{\color{red}f}\):
Let \(c=\min_{e \in E}c_f(e)\) where \(c_f(e)\) is the weight of edge \(e\) in the residual network \(G_{\color{red}f}\)
Add \(c\) units of flow to \(G\) based on the augmenting path \(p\)
Update the residual network \(G_{\color{red}f}\) for the updated flow
Ford-Fulkerson approach: take any augmenting path (will revisit this later)
Ford-Fulkerson Example
Increase flow by 1 unit
Initially:\({\color{red}f(e)}=0\) for all \(e \in E\)
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
Increase flow by 1 unit
Residual graph \(G_{\color{red}f}\)
Ford-Fulkerson Example
No more augmenting paths
Residual graph \(G_{\color{red}f}\)
Maximum flow: 4
Ford-Fulkerson Running Time
Define an augmenting path to be an \({\color{purple}s} \rightarrow {\color{Blue}t}\) path in the residual graph \(G_{\color{red}f}\) (using edges of non-zero weight)
Ford-Fulkerson max-flow algorithm:
Initialize \({\color{red}f(e)}=0\) for all all \(e \in E\)
Construct the residual network \(G_{\color{red}f}\)
While there is an augmenting path \(p\) in \(G_{\color{red}f}\):
Let \(c=\min_{e \in E}c_f(e)\) where \(c_f(e)\) is the weight of edge \(e\) in the residual network \(G_{\color{red}f}\)
Add \(c\) units of flow to \(G\) based on the augmenting path \(p\)
Update the residual network \(G_{\color{red}f}\) for the updated flow
How many iterations are needed?
For integer-valued capacities, min-weight of each augmenting path is 1, so number of iterations is bounded by \(|f^*|\), where \(|f^*|\) is max-flow in \(G\)
For rational-valued capacities, can scale to make capacities integer
For irrational-valued capacities, algorithm may never terminate!
For graphs with integer capacities, running time of Ford-Fulkerson is
\[O(|f^*| \cdot |E|)\]
Highly undesirable if \(|f^*| >> |E|\) (e.g., graph is small, but capacities are \(\approx 2^{32}\))
As described, algorithm is not polynomial-time!
Initialization:\(O(|E|)\)
Construct residual network:\(O(|E|)\)
We only care about nodes reachable from the source \(s\), so the number of nodes that are “relevant” is at most \(|E|\)
Finding augmenting path in residual network:\(O(|E|)\) using BFS/DFS
Ford-Fulkerson Running Time
Define an augmenting path to be an \({\color{purple}s} \rightarrow {\color{Blue}t}\) path in the residual graph \(G_{\color{red}f}\) (using edges of non-zero weight)
Ford-Fulkerson max-flow algorithm:
Initialize \({\color{red}f(e)}=0\) for all all \(e \in E\)
Construct the residual network \(G_{\color{red}f}\)
While there is an augmenting path \(p\) in \(G_{\color{red}f}\):
Let \(c=\min_{e \in E}c_f(e)\) where \(c_f(e)\) is the weight of edge \(e\) in the residual network \(G_{\color{red}f}\)
Add \(c\) units of flow to \(G\) based on the augmenting path \(p\)
Update the residual network \(G_{\color{red}f}\) for the updated flow
How many iterations are needed?
For integer-valued capacities, min-weight of each augmenting path is 1, so number of iterations is bounded by \(|f^*|\), where \(|f^*|\) is max-flow in \(G\)
For rational-valued capacities, can scale to make capacities integer
For irrational-valued capacities, algorithm may never terminate!
For graphs with integer capacities, running time of Ford-Fulkerson is
\[O(|f^*| \cdot |E|)\]
Highly undesirable if \(|f^*| >> |E|\) (e.g., graph is small, but capacities are \(\approx 2^{32}\))
As described, algorithm is not polynomial-time!
Initialization:\(O(|E|)\)
Construct residual network:\(O(|E|)\)
We only care about nodes reachable from the source \(s\), so the number of nodes that are “relevant” is at most \(|E|\)
Finding augmenting path in residual network:\(O(|E|)\) using BFS/DFS
Ford-Fulkerson Running Time
Define an augmenting path to be an \({\color{purple}s} \rightarrow {\color{Blue}t}\) path in the residual graph \(G_{\color{red}f}\) (using edges of non-zero weight)
Ford-Fulkerson max-flow algorithm:
Initialize \({\color{red}f(e)}=0\) for all all \(e \in E\)
Construct the residual network \(G_{\color{red}f}\)
While there is an augmenting path \(p\) in \(G_{\color{red}f}\):
Let \(c=\min_{e \in E}c_f(e)\) where \(c_f(e)\) is the weight of edge \(e\) in the residual network \(G_{\color{red}f}\)
Add \(c\) units of flow to \(G\) based on the augmenting path \(p\)
Update the residual network \(G_{\color{red}f}\) for the updated flow
How many iterations are needed?
For integer-valued capacities, min-weight of each augmenting path is 1, so number of iterations is bounded by \(|f^*|\), where \(|f^*|\) is max-flow in \(G\)
For rational-valued capacities, can scale to make capacities integer
For irrational-valued capacities, algorithm may never terminate!
For graphs with integer capacities, running time of Ford-Fulkerson is
\[O(|f^*| \cdot |E|)\]
Highly undesirable if \(|f^*| >> |E|\) (e.g., graph is small, but capacities are \(\approx 2^{32}\))
As described, algorithm is not polynomial-time!
Initialization:\(O(|E|)\)
Construct residual network:\(O(|E|)\)
We only care about nodes reachable from the source \(s\), so the number of nodes that are “relevant” is at most \(|E|\)
Finding augmenting path in residual network:\(O(|E|)\) using BFS/DFS
Ford-Fulkerson Running Time
Define an augmenting path to be an \({\color{purple}s} \rightarrow {\color{Blue}t}\) path in the residual graph \(G_{\color{red}f}\) (using edges of non-zero weight)
Ford-Fulkerson max-flow algorithm:
Initialize \({\color{red}f(e)}=0\) for all all \(e \in E\)
Construct the residual network \(G_{\color{red}f}\)
While there is an augmenting path \(p\) in \(G_{\color{red}f}\):
Let \(c=\min_{e \in E}c_f(e)\) where \(c_f(e)\) is the weight of edge \(e\) in the residual network \(G_{\color{red}f}\)
Add \(c\) units of flow to \(G\) based on the augmenting path \(p\)
Update the residual network \(G_{\color{red}f}\) for the updated flow
How many iterations are needed?
For integer-valued capacities, min-weight of each augmenting path is 1, so number of iterations is bounded by \(|f^*|\), where \(|f^*|\) is max-flow in \(G\)
For rational-valued capacities, can scale to make capacities integer
For irrational-valued capacities, algorithm may never terminate!
For graphs with integer capacities, running time of Ford-Fulkerson is
\[O(|f^*| \cdot |E|)\]
Highly undesirable if \(|f^*| >> |E|\) (e.g., graph is small, but capacities are \(\approx 2^{32}\))
As described, algorithm is not polynomial-time!
Initialization:\(O(|E|)\)
Construct residual network:\(O(|E|)\)
We only care about nodes reachable from the source \(s\), so the number of nodes that are “relevant” is at most \(|E|\)
Finding augmenting path in residual network:\(O(|E|)\) using BFS/DFS
Ford-Fulkerson Running Time
Define an augmenting path to be an \({\color{purple}s} \rightarrow {\color{Blue}t}\) path in the residual graph \(G_{\color{red}f}\) (using edges of non-zero weight)
Ford-Fulkerson max-flow algorithm:
Initialize \({\color{red}f(e)}=0\) for all all \(e \in E\)
Construct the residual network \(G_{\color{red}f}\)
While there is an augmenting path \(p\) in \(G_{\color{red}f}\):
Let \(c=\min_{e \in E}c_f(e)\) where \(c_f(e)\) is the weight of edge \(e\) in the residual network \(G_{\color{red}f}\)
Add \(c\) units of flow to \(G\) based on the augmenting path \(p\)
Update the residual network \(G_{\color{red}f}\) for the updated flow
How many iterations are needed?
For integer-valued capacities, min-weight of each augmenting path is 1, so number of iterations is bounded by \(|f^*|\), where \(|f^*|\) is max-flow in \(G\)
For graphs with integer capacities, running time of Ford-Fulkerson is
\[O(|f^*| \cdot |E|)\]
Highly undesirable if \(|f^*| >> |E|\) (e.g., graph is small, but capacities are \(\approx 2^{32}\))
As described, algorithm is not polynomial-time!
Initialization:\(O(|E|)\)
Construct residual network:\(O(|E|)\)
We only care about nodes reachable from the source \(s\), so the number of nodes that are “relevant” is at most \(|E|\)
Finding augmenting path in residual network:\(O(|E|)\) using BFS/DFS
Worst Case Ford-Fulkerson
Increase flow by 1 unit
Worst Case Ford-Fulkerson
Increase flow by 1 unit
Worst Case Ford-Fulkerson
Increase flow by 1 unit
Worst Case Ford-Fulkerson
Increase flow by 1 unit
Worst Case Ford-Fulkerson
Increase flow by 1 unit
Worst Case Ford-Fulkerson
Increase flow by 1 unit
Worst Case Ford-Fulkerson
Increase flow by 1 unit
Worst Case Ford-Fulkerson
Increase flow by 1 unit
Observation: each iteration increases flow by 1 unit
Total number of iterations:\(|f^*|=200\)
Ford-Fulkerson Running Time
Define an augmenting path to be an \({\color{purple}s} \rightarrow {\color{Blue}t}\) path in the residual graph \(G_{\color{red}f}\) (using edges of non-zero weight)
Ford-Fulkerson max-flow algorithm:
Initialize \({\color{red}f(e)}=0\) for all all \(e \in E\)
Construct the residual network \(G_{\color{red}f}\)
While there is an augmenting path \(p\) in \(G_{\color{red}f}\):
Let \(c=\min_{e \in E}c_f(e)\) where \(c_f(e)\) is the weight of edge \(e\) in the residual network \(G_{\color{red}f}\)
Add \(c\) units of flow to \(G\) based on the augmenting path \(p\)
Update the residual network \(G_{\color{red}f}\) for the updated flow
How many iterations are needed?
For integer-valued capacities, min-weight of each augmenting path is 1, so number of iterations is bounded by \(|f^*|\), where \(|f^*|\) is max-flow in \(G\)
For rational-valued capacities, can scale to make capacities integer
For irrational-valued capacities, algorithm may never terminate!
For graphs with integer capacities, running time of Ford-Fulkerson is
\[O(|f^*| \cdot |E|)\]
Highly undesirable if \(|f^*| >> |E|\) (e.g., graph is small, but capacities are \(\approx 2^{32}\))
As described, algorithm is not polynomial-time!
Initialization:\(O(|E|)\)
Construct residual network:\(O(|E|)\)
We only care about nodes reachable from the source \(s\), so the number of nodes that are “relevant” is at most \(|E|\)
Finding augmenting path in residual network:\(O(|E|)\) using BFS/DFS
Ford-Fulkerson Running Time
Define an augmenting path to be an \({\color{purple}s} \rightarrow {\color{Blue}t}\) path in the residual graph \(G_{\color{red}f}\) (using edges of non-zero weight)
Ford-Fulkerson max-flow algorithm:
Initialize \({\color{red}f(e)}=0\) for all all \(e \in E\)
Construct the residual network \(G_{\color{red}f}\)
While there is an augmenting path \(p\) in \(G_{\color{red}f}\):
Let \(c=\min_{e \in E}c_f(e)\) where \(c_f(e)\) is the weight of edge \(e\) in the residual network \(G_{\color{red}f}\)
Add \(c\) units of flow to \(G\) based on the augmenting path \(p\)
Update the residual network \(G_{\color{red}f}\) for the updated flow
How many iterations are needed?
For integer-valued capacities, min-weight of each augmenting path is 1, so number of iterations is bounded by \(|f^*|\), where \(|f^*|\) is max-flow in \(G\)
For rational-valued capacities, can scale to make capacities integer
For irrational-valued capacities, algorithm may never terminate!
For graphs with integer capacities, running time of Ford-Fulkerson is
\[O(|f^*| \cdot |E|)\]
Highly undesirable if \(|f^*| >> |E|\) (e.g., graph is small, but capacities are \(\approx 2^{32}\))
As described, algorithm is not polynomial-time!
Initialization:\(O(|E|)\)
Construct residual network:\(O(|E|)\)
We only care about nodes reachable from the source \(s\), so the number of nodes that are “relevant” is at most \(|E|\)
Finding augmenting path in residual network:\(O(|E|)\) using BFS/DFS
Can We Avoid This?
Edmonds-Karp Algorithm:choose augmenting path with fewest hops
When we’ve maximized max flow, we’ve minimized min cut (and vice-versa), so we can check when we’ve found one by finding the other
Example: Maxflow/Mincut
Flow Graph \(G\)
Residual graph \(G_{\color{red}f}\)
\(|f|=4\)
No more augmenting paths
\(||{\color{magenta}S}, {\color{skyblue}T} ||=4\)
Idea: When there are no more augmenting paths, there exists a cut in the graph with cost matching the flow
Proof: Maxflow/Mincut Theorem
If \(|f|\) is a max flow, then \(G_{\color{red}f}\) has no augmenting path
Otherwise, use that augmenting path to “push” more flow
Define \({\color{magenta}S}=\) nodes reachable from source node \(\color{purple}s\) by positive-weight edges in the residual graph
\({\color{skyblue}T}=V-{\color{magenta}S}\)
\({\color{magenta}S}\) separates \({\color{purple}s},{\color{Blue}t}\) (otherwise there’s an augmenting path)
Flow Graph \(G\)
Residual graph \(G_{\color{red}f}\)
Proof: Maxflow/Mincut Theorem
To show: \(||{\color{magenta}S},{\color{skyblue}T}||=|f|\)
Weight of the cut matches the flow across the cut
Consider edge \(\color{forestgreen}(u,v)\) with \(u \in {\color{magenta}S},v \in {\color{skyblue}T}\)
\({\color{red}f(u,v)}={\color{forestgreen}c(u,v)}\) because otherwise \({\color{forestgreen}w(u,v)}>0\) in \(G_{\color{red}f}\), which would mean \(v \in {\color{magenta}S}\)
Consider edge \(\color{brown}(y,x)\) with \(y \in {\color{skyblue}T}, x \in {\color{magenta}S}\)
\({\color{red}f(y,x)}=0\) because otherwise the back edge \({\color{brown}w(y,x)}>0\) in \(G_{\color{red}f}\), which would mean \(y \in {\color{magenta}S}\)
Flow Graph \(G\)
Residual graph \(G_{\color{red}f}\)
Proof Summary
The flow \(|f|\) of \(G\) is upper-bounded by the sum of capacities of edges crossing any cut separating source \({\color{purple}s}\) and sink \({\color{Blue}t}\)
When Ford-Fulkerson terminates, there are no more augmenting paths in \(G_{\color{red}f}\)
When there are no more augmenting paths in \(G_{\color{red}f}\) then we can define a cut \({\color{magenta}S}=\) nodes reachable from source node \({\color{purple}s}\) by positive-weight edges in the residual graph
The sum of edge capacities crossing this cut must match the flow of the graph
Therefore this flow is maximal
Bipartite Matching
Readings in CLRS 4th edition: chapter 25
Edge-Disjoint Paths
Given a graph \(G=(V,E)\), a start node \({\color{purple}s}\) and a destination node \({\color{Blue}t}\), give the maximum number of paths from \({\color{purple}s}\) to \({\color{Blue}t}\) which share no edges
Edge-Disjoint Paths
Given a graph \(G=(V,E)\), a start node \({\color{purple}s}\) and a destination node \({\color{Blue}t}\), give the maximum number of paths from \({\color{purple}s}\) to \({\color{Blue}t}\) which share no edges
Set of edge-disjoint paths of size 3:
Edge-Disjoint Paths
Given a graph \(G=(V,E)\), a start node \({\color{purple}s}\) and a destination node \({\color{Blue}t}\), give the maximum number of paths from \({\color{purple}s}\) to \({\color{Blue}t}\) which share no edges
Set of edge-disjoint paths of size 4:
How could we solve this?
Edge-Disjoint Paths
Make \({\color{purple}s}\) and \({\color{Blue}t}\) the source and sink, give each edge capacity 1, find the max flow.
Set of edge-disjoint paths of size 4
Max flow = 4
Edge-Disjoint Paths
Make \({\color{purple}s}\) and \({\color{Blue}t}\) the source and sink, give each edge capacity 1, find the max flow.
Set of edge-disjoint paths of size 4
Max flow = 4
Vertex-Disjoint Paths
Given a graph \(G=(V,E)\), a start node \({\color{purple}s}\) and a destination node \({\color{Blue}t}\), give the maximum number of paths from \({\color{purple}s}\) to \({\color{Blue}t}\) which share no vertices
Vertex-Disjoint Paths
Given a graph \(G=(V,E)\), a start node \({\color{purple}s}\) and a destination node \({\color{Blue}t}\), give the maximum number of paths from \({\color{purple}s}\) to \({\color{Blue}t}\) which share no vertices
Not a vertex-disjoint path!
Vertex-Disjoint Paths
Given a graph \(G=(V,E)\), a start node \({\color{purple}s}\) and a destination node \({\color{Blue}t}\), give the maximum number of paths from \({\color{purple}s}\) to \({\color{Blue}t}\) which share no vertices
Not a vertex-disjoint path!
How could we solve this?
Vertex-Disjoint Paths
Idea: Convert an instance of the vertex-disjoint paths problem into an instance of edge-disjoint paths
Make two copies of each node, one connected to incoming edges, the other to outgoing edges
Compute Edge-Disjoint Paths on new graph
Maximum Bipartite Matching
Maximum Bipartite Matching
Maximum Bipartite Matching
Maximum Bipartite Matching
Maximum Bipartite Matching
Given a graph \(G=(L,R,E)\)
A set of left nodes, right nodes, and edges between left and right
Find the largest set of edges \(M \subseteq E\) such that each node \(u \in L\) or \(v \in R\) is incident to at most one edge
Maximum Bipartite Matching
How could we solve this?
Maximum Bipartite Matching Using Max Flow
Make \(G=(L,R,E)\) a flow network \(G=(V,E)\) by:
Adding in a source and sink to the set of nodes:
\(V'=L \bigcup R \bigcup \{ {\color{purple}s},{\color{Blue}t}\}\)
Adding an edge from source to \(L\) and from \(R\) to sink:
\(E'=E \bigcup \{u \in L | ({\color{purple}s},u)\}\)\(\bigcup \{v \in R | (v,{\color{Blue}t})\}\)
Make each edge cap 1:
\(\forall e \in E', c(e)=1\)
Maximum Bipartite Matching Using Max Flow
Make \(G=(L,R,E)\) a flow network \(G=(V,E)\) by:
Make \(G\) into \(G'\)
\(\color{red}\Theta(L+R)\)
Compute Max Flow on \(G'\)
\(\color{red}\Theta(E \cdot V)\) since \(|f| \le L\)
Return \(M\) as “middle” edges with flow 1
\(\color{red}\Theta(L+R)\)
Total: \(\Theta(E \cdot V)\)
Reductions
Readings in CLRS 4th edition: N/A (reductions are covered in CLRS, but not in a context we are studying in CS 3100)
Reductions
Algorithm technique of supreme ultimate power
Convert instance of problem A to an instance of Problem B
Convert solution of problem B back to a solution of problem A
Reductions
Shows how two different problems relate to each other
MOVIE TIME!
MacGyver’s Reduction
Problem we don’t know how to solve
Problem we do know how to solve
Opening a door
Solution for \(A\):
Keg cannon battering ram
Reduction
Lighting a fire
How?
Solution for \(B\):
Alcohol, wood, matches
Bipartite Matching Reduction
Problem we don’t know how to solve
Problem we do know how to solve
Bipartite Matching
Solution for \(A\):
Reduction
Max Flow
Ford-Fulkerson
Solution for \(B\):
Edge Disjoint Paths Reduction
Problem we don’t know how to solve
Problem we do know how to solve
Edge Disjoint Paths
Solution for \(A\):
Max Flow
Ford-Fulkerson
Solution for \(B\):
Vertex Disjoint Paths Reduction
Problem we don’t know how to solve
Problem we do know how to solve
Vertex Disjoint Paths
Solution for \(A\):
Merge these back
Edge Disjoint Paths
(another reduction)
Solution for \(B\):
Vertex Disjoint Paths Big Picture
Problem we don’t know how to solve
Problem we still don’t know how to solve
Problem we do know how to solve
Vertex Disjoint Paths
Solution for \(A\):
Merge these back
Edge Disjoint Paths
Solution for \(B\):
Use edges with flow
Max Flow
Ford-Fulkerson
Solution for \(C\):
Reductions for Algorithms
Create an algorithm for a new problem by using one you already know!
More algorithms = More opportunities!
The problem you reduced to could itself be solved using a reduction!
In General: Reduction
Problem we don’t know how to solve
Solution for \(A\):
Map instances of problem \(A\) to instances of problem \(B\)
Any instance of \(A\) can be mapped to some instance of \(B\)
Map solutions of problem \(B\) to solutions of problem \(A\)
Reduction
Problem we do know how to solve
Using any algorithm for \(B\)
Solution for \(B\):
Another use of Reductions
Problem \(A\)
Suppose I know a worst-case lower-bound of \(\Omega(f(n))\) for \(A\)
\(\Omega(f(n))\)
Solution for \(A\):
Map instances of problem \(A\) to instances of problem \(B\)
Then this entire path must be \(\Omega(f(n))\)
Map solutions of problem \(B\) to solutions of problem \(A\)
Reduction
Some algorithm for \(B\)
Solution for \(B\)
Worst Case Lower Bound
Definition:
A worst case lower bound on a problem is an asymptotic lower bound on the worst case running time of any algorithm which solves it
If \(f(n)\) is a worst case lower bound for problem \(A\), then the worst-case running time of any algorithm which solves \(A\) must be \(\Omega(f(n))\)
i.e. for sufficiently large values of \(n\), for every algorithm which solves \(A\), there is at least one input of size \(n\) which causes the algorithm to do \(\Omega(f(n))\) steps
Examples:
\(n\) is a worst-case lower bound on finding the minimum in a list
\(n^2\) is a worst-case lower bound on matrix multiplication
Worst case lower bound Proofs
Opening a door
Problem \(A\)
Problem \(B\)
Lighting a fire
Alcohol, wood, matches
Algorithm for \(B\)
Algorithm for \(A\)
Keg cannon battering ram
\(A\) is not harder than problem \(B\): \(A \le B\)
The name “reduces” is confusing: it is in the opposite direction of the making
Proof of Lower Bound by Reduction
To show: \(Y\) is slow
We know \(X\) is slow (by a proof) (e.g., \(X =\) some way to open the door)
Assume \(Y\) is quick [toward contradiction] (\(Y =\) some way to light a fire)
Show how to use \(Y\) to perform \(X\) quickly
\(X\) is slow, but \(Y\) could be used to perform \(X\) quickly conclusion: \(Y\) must not actually be quick
Reduction Proof Notation
Problem \(A\)
Problem \(B\)
Algorithm for \(B\)
With \(O(f(n))\) overhead
Algorithm for \(A\)
\(A\) is not harder than problem \(B\)
\(A \le B\)
If \(A\)requires time \(\Omega(f(n))\) time then \(B\)also requires time \(\Omega(f(n))\)
\(A \le_{f(n)}B\)
Two Ways to Use Reductions
Suppose we have a “fast” reduction from \(A\) to \(B\)
A “fast” algorithm for \(B\) gives a fast algorithm for \(A\)
Then \(\color{red}A\) is fast
If \(\color{orange}B\) is fast
If we have a worst-case lower bound for \(A\), we also have one for \(B\)
If \(\color{red}A\) is slow
Then \(\color{orange}B\) is slow
Bipartite Matching Reduction
Problem we don’t know how to solve
Problem we do know how to solve
Bipartite Matching
Solution for \(A\):
Reduction
Max Flow
Ford-Fulkerson
Solution for \(B\):
If this is fast
Then this is fast
If this is slow
Then this is slow
Worst-case Lower Bound Using Reductions
Closest pair of points
D&C algorithm: \(\Theta(n \log n)\)
Can we do better?
Idea: Show that doing closest pair in \(o(n \log n)\) enables an impossibly fast algorithm for another problem
Reductions for Lower-Bound Proofs
Problem we know is “hard”
Problem \(A\)
and this must be slow,
“Hard” means this must be slow
Solution for \(A\):
Quickly map instances of problem \(A\) to instances of problem \(B\)
If this is quick
Map solutions of problem \(B\) to solutions of problem \(A\)
Reduction
and this is quick
Problem we want to show is “hard”
then this can’t be fast!
Some algorithm for \(B\)
Solution for \(B\)
Reductions for Lower-Bound Proofs
Problem we know is \(\Omega(n \log n)\)
Problem \(A\)
Then this can be done in \(o(n\log n)\)
\(\Omega(n \log n)\)
Solution for \(A\):
Quickly map instances of problem \(A\) to instances of problem \(B\)
Map solutions of problem \(B\) to solutions of problem \(A\)
Reduction
Problem we want to show is \(\Omega(n \log n)\)
Closest pair of points
Some algorithm for \(CPP\)
If this could be done in \(o(n \log n)\)
Solution for \(B\)
A “Hard” Problem: Element Uniqueness
113
901
555
512
245
800
018
121
True
103
801
401
323
255
323
999
101
False
Input:
A list of integers
Output:
True if all values are unique, False otherwise
Can this be solved in \(O(n \log n)\) time?
Yes! Sort, then check if any adjacent elements match
def check_uva_userid1(what): chars =list(what.lower())# check first character (must be a letter)iflen(chars) ==0: returnFalseifnot chars[0].isalpha(): returnFalse chars.pop(0)# check second character (must be a letter)iflen(chars) ==0: returnFalseifnot chars[0].isalpha(): returnFalse chars.pop(0)# return true if of the form lliflen(chars) ==0: returnTrue# check optional 3rd letterif chars[0].isalpha(): chars.pop(0)# return true if of the form llliflen(chars) ==0: returnTrue# check digitiflen(chars) ==0: returnFalseifnot chars[0].isdigit(): returnFalse chars.pop(0)# check first letter after the digitiflen(chars) ==0: returnFalseifnot chars[0].isalpha(): returnFalse chars.pop(0)# return true if of the form lldl or llldliflen(chars) ==0: returnTrue# check second letter after the digitifnot chars[0].isalpha(): returnFalse chars.pop(0)# return true if of the form lldll or llldlliflen(chars) ==0: returnTrue# check third letter after the digitifnot chars[0].isalpha(): returnFalse chars.pop(0)# return true if of the form lldlll or llldllliflen(chars) ==0: returnTrue# if there is more input, then it's not a valid useridreturnFalse
Each arrow assumes that it pops (consumes) one character
Double circled states are final states
If the end of input is in a final state, the string is accepted
As code
def check_uva_userid2(what):# returns True if the passed parameter is a valid UVA userid, else False state_table = [ # from each state, where to go on an 'l', 'd', and 'o', respectively [], # we numbered our states from 1, so we burn spot 0 [2, None, None], # state 1 goes to 2 on a letter [3, None, None], # state 2 goes to 3 on a letter [4, 5, None], # state 3 goes to 4 on a letter and 5 on a digit [None, 5, None], # state 4 goes to 5 on a digit [6, None, None], # state 5 goes to 6 on a letter [7, None, None], # state 6 goes to 7 on a letter [8, None, None], # state 7 goes to 8 on a letter [None, None, None], # no transitions out of state 8 ] final_states = [3, 4, 6, 7, 8] chars =list(what.lower()) state =1whilelen(chars) >0:# which is whether it's a letter (0), digit (1), or other (2) which =0if chars[0].isalpha() else1if chars[0].isdigit() else2# get the state transition, and verify it's not None next_state = state_table[state][which]if next_state isNone: returnFalse# transition to that state and pop the input character state = next_state chars.pop(0)# we should have ended in a final statereturn state in final_states