Algorithm 1 Efficient LLM Partitioning and Allocation
Require: Set of models M, Set of devices D, Network latency matrix L
Ensure: Optimized allocation of models to devices
1: Sort models M by size or priority
2: for each model m ∈ M do
3: Identify devices Dfit ⊆ D that can fit m entirely
4: if Dfit ̸= ∅ then
5: Assign m to device d∗ = arg max FLOPS(d)
d∈Dfit
6: continue
7: end if
8: Partition m into layers {l1 , l2 , ..., ln }
9: Initialize empty current group G = ∅, previous device dprev = ∅
10: for each layer li in m do
11: Append li to G
12: Compute required resources for G
13: Identify feasible devices Dgroup ⊆ D
14: if Dgroup = ∅ then
15: if dprev = ∅ then
16: Abort: No allocation possible for li
17: end if
18: Allocate G \ {li } to dprev
19: Reset G = {li }
20: end if
21: Select optimal device:
−FLOPS(d)
d∗ = arg min + L(dprev , d)
d∈Dgroup 1GPU (d)
22: Assign G to d∗
23: Update dprev = d∗
24: end for
25: end for
1 Proposed Algorithm
1
Algorithm 2 Efficient LLM Partitioning and Allocation
Require: Set of models M, Set of devices D, Network latency matrix L
Ensure: Optimized allocation of models to devices
1: Sort models M by size or priority
2: for each model m ∈ M do
3: Identify devices Dfit ⊆ D that can fit m entirely
4: if Dfit ̸= ∅ then
5: Assign m to device d∗ = arg max FLOPS(d)
d∈Dfit
6: continue
7: end if
8: Partition m into layers {l1 , l2 , ..., ln }
9: Initialize empty current group G = ∅, previous device dprev = ∅
10: for each layer li in m do
11: Append li to G
12: Compute required resources for G
13: Identify feasible devices Dgroup ⊆ D
14: if Dgroup = ∅ then
15: if dprev = ∅ then
16: Abort: No allocation possible for li
17: end if
18: Allocate G \ {li } to dprev
19: Reset G = {li }
20: end if
21: Select optimal device:
−FLOPS(d)
d∗ = arg min + L(dprev , d)
d∈Dgroup 1GPU (d)
22: Assign G to d∗
23: Update dprev = d∗
24: end for
25: end for
2
Algorithm 3 Efficient LLM Partitioning and Allocation with Partition Con-
straint
Require: Set of models M, Set of devices D, Network latency matrix L, Max-
imum partitions Pmax
Ensure: Optimized allocation of models to devices
1: Sort models M by size or priority
2: for each model m ∈ M do
3: Identify devices Dfit ⊆ D that can fit m entirely
4: if Dfit ̸= ∅ then
5: Assign m to device d∗ = arg max FLOPS(d)
d∈Dfit
6: continue
7: end if
8: Partition m into layers {l1 , l2 , ..., ln }
9: Initialize empty current group G = ∅, previous device dprev = ∅
10: Set partition count P = 0
11: for each layer li ∈ m do
12: if P ≥ Pmax then
13: Abort: Partitioning exceeds allowed limit Pmax
14: end if
15: Append li to G
16: Compute required resources for G
17: Identify feasible devices Dgroup ⊆ D
18: if Dgroup = ∅ then
19: if dprev = ∅ then
20: Abort: No allocation possible for li
21: end if
22: Allocate G \ {li } to dprev
23: Reset G = {li }
24: Increment partition count P = P + 1
25: end if
26: Select optimal device:
∗ −FLOPS(d)
d = arg min + L(dprev , d)
d∈Dgroup 1GPU (d)
27: Assign G to d∗
28: Update dprev = d∗
29: end for
30: end for
3
Algorithm 4 Task-to-Expert Allocation
Require: Incoming task T , Set of experts E, Set of nodes N , Load matrix L,
Latency matrix Λ
Ensure: Assign task T to an optimal expert-node pair
1: Extract task type τ (T ), complexity c(T ), and required model type
2: Identify relevant experts ET ⊆ E that match τ (T )
3: if ET = ∅ then
4: Abort: No suitable expert found
5: end if
6: Initialize best assignment (e∗ , n∗ ) ← (∅, ∅), minimum cost Cmin ← ∞
7: for each expert e ∈ ET do
8: for each node n hosting e do
9: Compute expected execution time:
c(T )
Texec =
FLOPS(n)
10: Compute communication latency:
Tcomm = Λ(nclient , n)
11: Compute total cost:
C = Texec + Tcomm + L(n)
12: if C < Cmin then
13: Update best assignment (e∗ , n∗ ) ← (e, n)
14: Update Cmin ← C
15: end if
16: end for
17: end for
18: if (e∗ , n∗ ) = (∅, ∅) then
19: Abort: No feasible allocation found
20: else
21: Assign task T to expert e∗ on node n∗
22: Update node load: L(n∗ ) ← L(n∗ ) + c(T )
23: end if