SLALOM

SGX Hardware Mode — Complete Setup Guide

From-Scratch Installation • Build • Run • Origami Roadmap

⚠ Proof-of-concept only — do NOT use for real secure data!

0 — How Slalom Works (Quick Mental Model)
Before touching any command, understand what you are building:
Slalom's key insight: In a DNN, ~98% of compute is linear (Conv, Dense). These are easy to verify. Only ~2% is non-linear
(ReLU, Pooling). Slalom splits work along that line.

Component Runs Where Operation Why

Conv / Dense (linear) Untrusted GPU Fast matmul on blinded data GPU is 50× faster than SGX
ReLU / Pooling (non-linear) SGX Enclave (trusted) Activation in secure memory Non-linear ops are cheap
Freivalds' check SGX Enclave (trusted) Verify GPU result with random vec Probabilistic integrity proof
Blinding / Unblinding SGX Enclave (trusted) x̃ = x + r → ỹ - u = y Input privacy via stream cipher

Two eval scripts exist in the repo: (1) [Link] — verifiable inference (GPU computes, enclave verifies). (2)
eval_slalom.py — private + verifiable inference (blinding added). Use (2) for full Slalom.
1 — Prerequisites & Version Matrix
The original Slalom paper was built with TensorFlow 1.8.0 from source. This is the single biggest compatibility constraint.
Everything else must align to it.

Component Required Version Notes

OS Ubuntu 16.04 LTS (paper) / 18.04 (tested working) Bare metal — no VM for SGX HW mode
CPU Intel Skylake (6th gen) or later with SGX support Check: grep sgx /proc/cpuinfo
Intel SGX Driver linux-sgx-driver 2.x Kernel module — must match kernel version
Intel SGX SDK 2.6 – 2.16 (match driver) Source: [Link]/intel/linux-sgx
Intel SGX PSW (AESM) Same version as SDK Provides aesmd service for HW attestation
GCC / G++ 7.x (paper used 7.5.0) g++ 8+ may have ABI issues with TF 1.8
Python 2.7 or 3.6 (paper used 2.7) Python 3.6 also confirmed working
TensorFlow 1.8.0 — built FROM SOURCE with GPU + CUDA pip TF will NOT have right CUDA headers
Bazel 0.13.1 – 0.15.x (for TF 1.8 build) Exact version matters for TF 1.8 source build
CUDA Toolkit 9.0 – 9.2 (recommended for TF 1.8) CUDA 10.x may work with patches
cuDNN 7.x (matching CUDA version) Required for TF GPU build
GPU Any NVIDIA GPU with CUDA support Paper used Titan XP; Quadro P1000 works
Keras 2.1.6 (standalone, not [Link]) Install via pip after TF source build

⚑ Why TF must be built from source

Slalom's App/ directory contains cuda_fmod.[Link] — a custom CUDA kernel. This file needs
TensorFlow's internal CUDA kernel headers (cuda_kernel_helper.h, etc.). These headers are
only present when TF is built from source. pip TF does NOT include them, which is why
cuda_fmod.so cannot be compiled from a pip install of TensorFlow.
2 — Step-by-Step Build (Fresh Machine)
Phase A — System & SGX Setup

STEP A1 Verify SGX hardware support

# Check SGX flag is present in CPU:
grep -m1 sgx /proc/cpuinfo
# Check BIOS: Intel SGX must be set to 'Enabled' (not 'Software Controlled') in BIOS
# Verify bare metal (SGX HW mode does NOT work in VMs):
systemd-detect-virt # must print: none

STEP A2 Install SGX Driver (kernel module)

git clone [Link]
cd linux-sgx-driver
sudo apt-get install linux-headers-$(uname -r)
make
sudo mkdir -p '/lib/modules/'$(uname -r)'/kernel/drivers/intel/sgx'
sudo cp [Link] '/lib/modules/'$(uname -r)'/kernel/drivers/intel/sgx'
sudo sh -c "cat /etc/modules | grep -Fxq 'isgx' || echo 'isgx' >> /etc/modules"
sudo /sbin/depmod
sudo /sbin/modprobe isgx
# Verify driver loaded:
lsmod | grep isgx # must show isgx module

STEP A3 Install SGX SDK & PSW

# Download pre-built installer from Intel's release page:
# [Link] (choose version matching driver)

# Install PSW first (provides aesmd service):

sudo dpkg -i libsgx-enclave-common_*.deb
sudo dpkg -i libsgx-enclave-common-dev_*.deb

# Install SDK:
chmod +x sgx_linux_x64_sdk_*.bin
sudo ./sgx_linux_x64_sdk_*.bin # install to /opt/intel/sgxsdk

# Source SDK environment (add this to ~/.bashrc):

source /opt/intel/sgxsdk/environment

# Start AESM service:

sudo systemctl enable aesmd && sudo systemctl start aesmd
sudo systemctl status aesmd # must show: active (running)

Phase B — TensorFlow 1.8 From Source

⚑ Why this step is mandatory

TF 1.8 source build is non-negotiable. cuda_fmod.[Link] needs TF's internal CUDA headers.
Plan ~2–4 hours for the Bazel build on a typical workstation.

STEP B1 Install Bazel 0.13.1

 # Bazel 0.13.1 is the version compatible with TF 1.8:
wget [Link]
chmod +x bazel-0.13.1-installer-linux-x86_64.sh
./bazel-0.13.1-installer-linux-x86_64.sh --user
export PATH=$PATH:$HOME/bin # add to ~/.bashrc
bazel version # must print: Build label: 0.13.1

STEP B2 Install CUDA 9.0 + cuDNN 7

# Install CUDA 9.0 toolkit (exact version matters):
sudo dpkg -i cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64.deb
sudo apt-key add /var/cuda-repo-9-0-local/[Link]
sudo apt-get update && sudo apt-get install cuda-9-0

# Install cuDNN 7 for CUDA 9.0 (download from [Link]):

sudo dpkg -i libcudnn7_7.*.deb
sudo dpkg -i libcudnn7-dev_7.*.deb

# Export paths (add to ~/.bashrc):

export CUDA_HOME=/usr/local/cuda-9.0
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH

nvcc --version # must show release 9.0

STEP B3 Clone TF 1.8 and configure

git clone [Link]
cd tensorflow
git checkout v1.8.0

./configure
# Answer the configure prompts:
# Python binary: /usr/bin/python2.7 (or python3.6 path)
# Python library: leave default
# ROCm support: N
# CUDA support: Y
# CUDA SDK version: 9.0
# CUDA location: /usr/local/cuda-9.0
# cuDNN version: 7
# cuDNN location: /usr/local/cuda-9.0
# CUDA compute capabilities: 6.1 (for Quadro P1000 / GTX 1080 series)
# Use 5.2 for Maxwell, 6.1 for Pascal, 7.0 for Volta
# MPI support: N
# XLA support: N
# All other options: N (defaults)

STEP B4 Build TensorFlow pip package

# This takes 1–4 hours depending on CPU:
bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

# Build the wheel:

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

# Install it:
pip install /tmp/tensorflow_pkg/tensorflow-1.8.0-*.whl
 # Verify:
python -c "import tensorflow as tf; print(tf.__version__); print([Link].is_gpu_available())"
# Expected: 1.8.0 / True

Phase C — Clone and Build Slalom

STEP C1 Clone the Slalom repository

git clone [Link]
cd slalom

# Install Python dependencies:

pip install keras==2.1.6
pip install numpy scipy h5py pillow

# Verify Eigen is present in Include/:

ls Include/eigen3_sgx/ # must exist (it is bundled in the repo)
ls Include/eigen/ # also needed — clone standard Eigen here if missing:
# git clone [Link] Include/eigen

STEP C2 Build the SGX Enclave (slalom/ directory)

source /opt/intel/sgxsdk/environment

# Production SGX HW mode (what you want for real hardware):

cd slalom
make clean
make SGX_MODE=HW SGX_DEBUG=0 SGX_PRERELEASE=0

# Expected last lines of output:

# LINK => App/slalom_app
# SIGN => [Link]
# The project has been built in hardware mode.

# Verify enclave signed:

ls -la [Link] # must exist, non-zero size

# Quick HW mode check (Enclave id must be a small positive integer, e.g. 2):
./slalom_app 2>&1 | head -3
# Expected: Enclave id: 2
# (Simulation mode would show a huge random number like 642759035715586)

STEP C3 Build the TF custom ops (App/ directory)

cd App

# Step 1 — build CPU version of TF ops:

make SGX_MODE=HW
# Produces: ../lib/slalom_ops.so

# Step 2 — compile cuda_fmod.[Link] (needs TF from-source CUDA headers):

TF_INC=$(python -c 'import tensorflow as tf; print([Link].get_include())')
TF_LIB=$(python -c 'import tensorflow as tf; print([Link].get_lib())')
 nvcc -std=c++11 -c -o cuda_fmod.cu.o cuda_fmod.[Link] \
-I$TF_INC -I$TF_INC/external/nsync/public \
-I/usr/local/cuda/include \
--expt-relaxed-constexpr -fPIC \
-D GOOGLE_CUDA=1 -x cu -Xcompiler -fPIC

g++ -std=c++11 -shared -fPIC -o cuda_fmod.so cuda_fmod.cu.o \

-L/usr/local/cuda/lib64 -lcudart -lcuda \
-L$TF_LIB -ltensorflow_framework

mv cuda_fmod.so ../App/cuda_fmod.so

# Step 3 — build SGX+GPU version of TF ops:

# In App/Makefile_hardware, ensure this flag is present:
# App_Cpp_Flags := ... -D_GLIBCXX_USE_CXX11_ABI=0
# Then:
make -f Makefile_hardware
# Produces: ../lib/slalom_ops_sgx.so

STEP C4 Fix the Python module symlink

cd slalom/python

# Remove any wrong symlink:

rm -f python_link python

# Create correct self-referential symlink:

ln -s . python

# Verify:
ls -la python # must show: python -> .

# The reason: scripts use 'import [Link]'.

# Python needs a folder named 'python' in the path.
# The self-symlink provides this when running from slalom/ dir.

STEP C5 Verify all required libraries are present

ls -la slalom/lib/

# All five files must exist:

# slalom_ops.so (TF ops, CPU build) ~76 KB
# slalom_ops_sgx.so (TF ops, SGX+GPU build) ~80 KB
# [Link] (SGX DNN native lib) ~3.1 MB
# enclave_bridge.so (Python/SGX bridge) ~23 KB
# cuda_fmod.so (CUDA fmod kernel) ~40 KB

ls slalom/[Link] # Signed enclave — must exist

3 — Running Slalom (Python Eval Path)
⚑ Always use eval_slalom.py — not slalom_app
slalom_app is a C++ test harness with a known host-side malloc bug for VGG19.
It is NOT the main inference path. Use eval_slalom.py for all real experiments.

3.1 — Setup Export Variables (Every Terminal Session)

# Source SGX environment:
source /opt/intel/sgxsdk/environment
# or: source ~/sgxsdk/environment

# CUDA libraries:
export CUDA_HOME=/usr/local/cuda
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH

# TF library path (adjust for your Python install):

TF_LIB=$(python -c 'import tensorflow as tf; print([Link].get_lib())')
export LD_LIBRARY_PATH=$TF_LIB:$LD_LIBRARY_PATH

# PYTHONPATH — must be set from the slalom/ directory:

cd /path/to/slalom
export PYTHONPATH=$PWD:$PYTHONPATH

# Verify AESM is running:

sudo systemctl status aesmd

3.2 — Mode A: Verifiable Inference (Integrity Only)

GPU computes, SGX enclave verifies using Freivalds' algorithm. No blinding — faster but no input privacy.
# From the slalom/ directory:

# With preprocessing secrets (fastest, recommended):

python -m [Link] vgg_16 sgxdnn \
--batch_size=1 --max_num_batches=4 \
--verify_preproc --use_sgx

# On-the-fly verification (slower):

python -m [Link] vgg_16 sgxdnn \
--batch_size=1 --max_num_batches=4 \
--verify --use_sgx

# Also works with mobilenet or mobilenet_sep:

python -m [Link] mobilenet sgxdnn \
--batch_size=4 --max_num_batches=4 \
--verify_preproc --use_sgx

3.3 — Mode B: Private + Verifiable Inference (Full Slalom)

This is the complete Slalom protocol: inputs are blinded before leaving the enclave, GPU computes on blinded data,
Freivalds verifies integrity. Matches the paper's privacy+integrity results.
# Full Slalom with blinding (privacy) + integrity:
python -m [Link].eval_slalom vgg_16 \
--batch_size=1 --max_num_batches=4 \
--blinding --integrity --use_sgx

# Privacy only (no Freivalds check, just blinding):

python -m [Link].eval_slalom vgg_16 \
--batch_size=1 --max_num_batches=4 \
--blinding --use_sgx

# For MobileNet (smaller model, faster iteration):

python -m [Link].eval_slalom mobilenet \
--batch_size=4 --max_num_batches=8 \
 --blinding --integrity --use_sgx

3.4 — Expected Output (Successful Run)

Using TensorFlow backend.
Loading model...
Model loaded successfully.
Enclave id: 2
filters initialized

Running batch 0 of 4...

Batch time: [Link] sec
Running batch 1 of 4...
...
Throughput: XX.X images/sec
Mean latency: [Link] sec

⚑ What the numbers mean

Enclave id: 2 → Real SGX hardware mode confirmed.
Throughput with --use_sgx --verify_preproc → compare to paper's Figure 3.
VGG16 paper result: ~20 images/sec with verify_preproc; ~13 images/sec with privacy+integrity.

3.5 — Baseline Comparisons to Reproduce Paper Results

# Baseline 1: Fully in enclave (no GPU offload):
python -m [Link] vgg_16 sgxdnn \
--batch_size=1 --max_num_batches=4 --use_sgx

# Baseline 2: Fully on GPU (no security, fastest):

python -m [Link] vgg_16 sgxdnn \
--batch_size=8 --max_num_batches=4

# Record the images/sec for each and compute speedup:

# Speedup = Slalom_throughput / Baseline1_throughput
# Paper reports: 10-20x speedup for VGG16 verify_preproc vs baseline
4 — Known Issues & Fixes
# Error / Symptom Root Cause Fix
1 Error 4102 slalom_app malloc() allocates filter array Ignore slalom_app entirely. Use eval_slalom.py as
(SGX_ERROR_INVALID_PARAMETER) in sized for VGG16 but VGG19 JSON has more described in Section 3. The enclave itself works fine
slalom_app layers — host-side heap corruption passes (Enclave id: 2 proves it).
bad pointer to enclave.
2 cuda_fmod.so missing / build fails cuda_fmod.[Link] needs TF internal CUDA Build TF 1.8.0 from source (Phase B). After source
headers only present in from-source TF build, the CUDA headers are in TF_INC and nvcc can
build. pip TF does not include them. compile cuda_fmod.[Link].
3 undefined symbol: TF 1.8 pip was compiled with old ABI (- Add -D_GLIBCXX_USE_CXX11_ABI=0 to App_Cpp_Flags
_ZN10tensorflow16FormatFromString D_GLIBCXX_USE_CXX11_ABI=0). in App/Makefile_hardware, then rebuild
(ABI mismatch) slalom_ops_sgx.so compiled with GCC7 slalom_ops_sgx.so.
defaults uses new ABI.
4 ModuleNotFoundError: No module Scripts import '[Link]'. Python cd into slalom/ dir. Create self-symlink: cd python &&
named 'python' needs a folder called 'python' in ln -s . python. Export:
PYTHONPATH. Running from wrong directory PYTHONPATH=$PWD:$PYTHONPATH from slalom/ dir.
breaks this.
5 undefined reference to ocall_start_clock [Link] declares 4 ocall functions. Add stub implementations to enclave_bridge.cpp: void
/ ocall_print_string sgx_edger8r generates calls to them in ocall_start_clock(){} void ocall_print_string(const char*
Enclave_u.c but enclave_bridge.cpp never s){printf("%s",s);} double ocall_get_time(){return 0.0;}
implemented them. void ocall_end_clock(const char* s){}
6 SGX_ERROR_SERVICE_UNAVAILABLE — The AESM daemon (aesmd) manages SGX sudo systemctl start aesmd. Also ensure PSW
AESM not running hardware provisioning. If it is not running, no (Platform Software) is installed correctly.
enclave can be created in HW mode.
7 Enclave id shows huge random number This indicates SGX_MODE=SIM was used Rebuild with: make SGX_MODE=HW. Check Makefile
(e.g. 642759035715586) accidentally — simulation mode, not to ensure HW flag propagates to all subdirs.
hardware mode.
8 [Link] not found during Python eval [Link] is built as part of the main Enclave Run make SGX_MODE=HW from the slalom/ root
make target, not App/ separately. Missing if directory (not from App/). Check: ls lib/[Link] —
enclave was not built. should be ~3.1 MB.
9 Segfault in TF graph construction at Stale slalom_ops_sgx.so compiled without - rm lib/slalom_ops_sgx.so && make -f
relu_slalom op D_GLIBCXX_USE_CXX11_ABI=0 causes Makefile_hardware in App/ dir. Ensure ABI flag is in
symbol mismatch at TF graph load time. Makefile_hardware.
10 free(): invalid pointer / Aborted (core After the enclave is destroyed, the host-side Harmless if the enclave runs correctly before the crash.
dumped) after slalom_app exits cleanup in enclave_bridge.cpp calls free() on Use Python eval path instead — this crash does not
a filter array that was incorrectly allocated. affect eval_slalom.py.
5 — Origami Roadmap (After Slalom Works)
Once Slalom's eval_slalom.py runs cleanly, you have the foundation for Origami. Here is the conceptual difference and what
to do next.

Slalom Origami
Layer handling ALL layers split: linear→GPU (blinded), non-linear→SGX. First K layers: split like Slalom (blinded GPU + SGX). Layer K+1 onwards:
Repeated for every layer. entire computation on GPU, no blinding.
Privacy Full privacy for all layers via blinding Privacy only up to partition layer K; later layers exposed but input
guarantee cannot be reconstructed
Performance ~10–11x over pure SGX (for VGG19) ~15x over pure SGX (for VGG19) — saves blinding overhead for later
layers
Key parameter None (all layers treated equally) Partition point K — chosen empirically using c-GAN adversary (paper:
K=6 for VGG16/19)
Repo [Link]/ftramer/slalom [Link]/cjbaq-origami/origami_inference

Origami Execution Steps

STEP O1 Run Slalom as baseline (measure throughput)

python -m [Link].eval_slalom vgg_19 \
--batch_size=1 --max_num_batches=4 \
--blinding --integrity --use_sgx
# Record: images/sec (this is Slalom/Privacy baseline from Origami paper Fig. 9)

STEP O2 Switch to Origami repo and build

cd /path/to/origami_inference
# The repo already uses Slalom's SGXDNN library internally.
# Build same as Slalom (Phase C above) — same make commands.
# Origami's Python scripts are in python/slalom/scripts/

STEP O3 Run Origami with partition at layer 6

# Origami runs layers 1-6 with Slalom blinding, layers 7-19 directly on GPU:
PYTHONPATH=$PWD:$PYTHONPATH python python/slalom/scripts/eval_origami.py vgg_19 \
--batch_size=1 --max_num_batches=4 \
--blinding --use_sgx --partition_layer 6

# Layer 6 is the Origami paper's safe partition point for VGG-16/19.

# At layer 6 the c-GAN adversary cannot reconstruct the original input.
# Expected speedup over Slalom: ~1.3-1.5x (15.1x vs 11x over pure SGX baseline)

STEP O4 Compare results to paper Table / Figure 9

# Compare your measured speedups to Origami paper Figure 9:
# Baseline 2 (pure SGX): 1.0x
# Slalom/Privacy: 11x (VGG19)
# Origami: 15.1x (VGG19)

# Your GPU (Quadro P1000) has less memory than GTX 1080 Ti used in paper.
# Absolute images/sec will differ but speedup ratios should be similar.
6 — Quick Reference Card
Critical Paths
SGX env: source /opt/intel/sgxsdk/environment
Build enclave: make SGX_MODE=HW SGX_DEBUG=0 (from slalom/)
Build ops: cd App && make SGX_MODE=HW (CPU ops)
cd App && make -f Makefile_hardware (SGX+GPU ops)
Run Slalom: cd slalom && PYTHONPATH=$PWD:$PYTHONPATH \
python -m [Link].eval_slalom vgg_16 \
--batch_size=1 --max_num_batches=4 --blinding --integrity --use_sgx
HW check: ./slalom_app 2>&1 | head -1 → 'Enclave id: 2' (not a large random number)
AESMD check: sudo systemctl status aesmd → 'active (running)'

Flags Reference
Script Flag Effect
[Link] --use_sgx Run CPU side inside SGX enclave (real HW mode)
[Link] --verify Enable Freivalds' check (on-the-fly, no preprocessing)
[Link] --verify_batched Batched Freivalds — efficient for larger memory
[Link] --verify_preproc Freivalds with preprocessed secrets — fastest integrity
eval_slalom.py --use_sgx Run non-linear ops and verification inside SGX enclave
eval_slalom.py --blinding Enable input privacy via additive stream cipher blinding
eval_slalom.py --integrity Enable Freivalds integrity check in addition to blinding
make SGX_MODE=HW Build for real hardware SGX (not simulation)
make SGX_MODE=SIM Build for simulation mode (no real enclave, for debugging)
make SGX_DEBUG=1 Enable SGX debug mode (adds debug output, slower)