This repository contains the code artifact for Rethinking ReLU Replacement in MLP Classifiers: From Approximation to Decision-Preserving, a decision-aware post-training replacement method for ReLU activations in single-hidden-layer MLP heads and encrypted CKKS inference experiments.

The artifact has two main parts:

1. Plaintext experiments for fitting and evaluating quadratic ReLU replacements, classical polynomial baselines, calibration/test agreement diagnostics, and HE artifact export.
2. CKKS experiments for encrypted inference on selected MLP heads using Microsoft SEAL 4.1-compatible CKKS.

The DINOv2 and Qwen3 experiments evaluate encrypted classification-head inference on encrypted feature vectors. They do not evaluate the full DINOv2 or Qwen3 backbone under FHE. In the intended frozen-backbone workflow, the client computes features locally, encrypts the feature vector, and sends the encrypted feature vector to the server-side CKKS evaluator.

A typical checkout is organized as follows.

The quad4fhe-0.1.0-new.tar.gz archive is included so that users can reproduce the plaintext experiments without separately packaging the local quad4fhe library.

The FHE_CKKS/thirdparty/SEAL-4.1-bs/ directory is kept as a convenience copy of the SEAL-compatible library used in the original experiments. If you already have a compatible Microsoft SEAL 4.1 installation, you may use that instead by setting SEAL_DIR when configuring CMake.

The plaintext experiments require a CUDA-capable GPU for the larger DINOv2 and Qwen3 runs.

The CKKS experiments reported in the paper were run on:

Different CPUs, compiler flags, SEAL builds, and thread counts can change latency numbers.

The recommended environment is a CUDA-enabled Conda environment.

Then install the local Quad4FHE package:

Verify the installation:

If you run the notebooks from Jupyter, restart the kernel after installing or reinstalling quad4fhe.

Some datasets are included in this repository for convenience. If your local checkout does not contain them, place the expected files in the same folder as the corresponding notebook.

Examples:

CIFAR-10 and CIFAR-100 are not automatically downloaded by the repository setup. Download them manually and place the dataset files in the same folder as the corresponding notebook:

The notebooks use official CIFAR train/test partitions and then create internal validation/calibration splits where needed.

For DINOv2 experiments, first check that the DINOv2 model and image data are available locally. The results JSON files expect paths similar to:

If the cached feature file does not exist, the notebook will compute features using the DINOv2 backbone. This can take time and requires a GPU.

For Qwen3 experiments, make sure the Hugging Face datasets and model are accessible before launching long runs. The notebooks use models/datasets such as:

A quick connectivity check is:

If you use a local mirror or pre-downloaded model, update the paths in the notebooks accordingly.

Each dataset folder contains autosave notebooks. Run the full-train and small-pool notebooks from their own directory. For example:

The notebooks automatically save logs and structured outputs under:

For example:

The combined dataset-level JSON is saved as:

The most important diagnostic fields are:

For binary RCH runs, mu_grid, mu_p, and mu_n are also reported.

The plaintext notebooks export HE artifact folders containing MLP weights, polynomial coefficients, plaintext reference logits, and test features needed by the C++ CKKS programs.

The CKKS scripts expect the relevant artifact folders to be under FHE_CKKS/ unless you edit the shell scripts.

For the main CKKS experiments, copy or move these folders into FHE_CKKS/:

For small-pool encrypted experiments or custom scripts, you may also need these folders:

You can either copy them:

or edit run_exp_otto_all.sh, run_exp_fgvc_all.sh, and run_exp_massive_all.sh to point to the artifact folders in their original locations.

The repository keeps a local SEAL-compatible tree at:

Build and install it locally:

Find the generated SEAL CMake config:

The resulting directory is passed to FHE_CKKS as SEAL_DIR. Depending on the exact SEAL build, it is usually one of:

If you already have Microsoft SEAL 4.1 installed elsewhere, you can skip this step and use your existing SEAL_DIR.

From FHE_CKKS/:

If CMake cannot find SEAL, replace SEAL_DIR with the path found in the previous step.

The build should generate:

From FHE_CKKS/, run:

The scripts first run a 200-sample sanity check with 8 threads and then run the full sweeps. The full-sweep scripts use 32 threads in the reported configuration.

The C++ programs print:

The paper reports no bootstrapping for these selected CKKS experiments.

• Plaintext experiments can be sensitive to GPU type, PyTorch/CUDA version, and deterministic settings.
• CKKS latency depends strongly on CPU, compiler, SEAL build, -march=native, and thread count.
• The full experiments use fixed seeds in the notebooks where specified, but the artifact does not claim multi-seed statistical confidence intervals.
• DINOv2 and Qwen3 experiments depend on external model/data availability and may require prior Hugging Face downloads.
• The CKKS experiments cover encrypted MLP-head inference only, not full encrypted image/text backbones.

This repository includes original code, third-party code, raw/processed datasets, and references to external pretrained models.

• The repository-level LICENSE applies to the original Quad4FHE code unless a file or subdirectory states otherwise.
• FHE_CKKS/thirdparty/SEAL-4.1-bs/ retains its own upstream license and notices.
• Dataset files retain their original dataset licenses and terms of use.
• Hugging Face models and datasets retain their own licenses and terms.

Before redistributing this repository, check that retaining the bundled datasets and third-party source tree is compatible with your intended distribution channel.

If you use this code, please cite the associated paper and this code artifact. A CITATION.cff file is included for GitHub citation support.