Model Weight Encryption and Protection: Rethinking the Cost of Encryption at Rest and Defining Protection Boundaries

Status: Draft (2026-10-07). Encryption/decryption speed, integrity checks, and the path of placing decrypted weights into a memory volume to boot a vLLM server (0.6B model) are values measured directly by the author. Confidential computing could not be tested directly due to a lack of hardware, so paper and vendor documentation descriptions were reviewed and cited. Loading large models and multiple GPUs was not measured (unmeasured). Writing Rule: Measured data is the default, and unmeasured items are marked as 「Unmeasured」.

Table of Contents

• 1. Define the Question First: What Are We Protecting Weights From?
• 2. Methodology
• 3. Cost: How Much Slower and Larger Does It Get?
  • Additional Time Added to Cold Starts
• 4. Integrity: Detecting Tampering and Swapping
  • Ciphertext is Indistinguishable from Plaintext
  • How Much Does It Increase When Inserted into the Serving Engine Loading Path?
  • CPU Supplemental Tests: Disabling Acceleration, Parallelization
• 5. What Cannot Be Blocked: Weights During Execution Are Plaintext
• 6. Where to Store the Keys
• 7. Potential Interview Questions
• External References
• Unverified Items
• Bibliography
• Verified Versions and Dates

1. Define the Question First: What Are We Protecting Weights From?

The phrase "encrypting the model" is meaningless unless threats are defined. Weights are exposed through multiple paths, and encryption only protects against some of them.

This article measures the costs and effects of the first and third rows (encryption at rest and integrity) and explains why the fourth row remains unblocked.

2. Methodology

3. Cost: How Much Slower and Larger Does It Get?

• File size practically does not increase. Only a 12-byte nonce, 16-byte tag, and 4-byte length are appended per chunk.
• Decryption is at 36-44% of plaintext read speed(0.78-0.95 vs 2.15 GB/s). However, pure computation is 1.8-2.2 GB/s , meaning the file path in this test was slowed down by I/O and Python-side processing rather than computation. This distinction was not broken down (estimated).
• Increasing threads yields small gains. Computation increased by only 22% from 1 to 4 threads (1.81 to 2.21), and the file path by 22% (0.78 to 0.95). Initially, it was assumed this was because threads shared the same memory bandwidth, but the "CPU Supplemental Tests" below proved this assumption wrong.
• The CPU had hardware acceleration (AES instructions). Speed without acceleration was measured by disabling it via environment variables in the "CPU Supplemental Tests" below.

Additional Time Added to Cold Starts

Applying a decryption throughput of approx. 0.9 GB/s directly, the additional time by model size is as follows (calculated based on file path values from this test).

140GB is an extrapolated calculation and was not directly measured. Since 70B models are typically split across multiple GPUs, parallelizing decryption per GPU distributes the time (parallel tests splitting processes are in "CPU Supplemental Tests" below). Compared to server preparation time in Article 14 (35-55 seconds for reboot, of which weight loading accounts to 4-6 seconds), a few seconds of decryption is a noticeable ratio for smaller models, but gets buried in model download and loading times for large models.

4. Integrity: Detecting Tampering and Swapping

GCM checks for tampering using tags alongside encryption (NIST SP 800-38D). Two tests were conducted directly.

• If chunk numbers are not included in AAD, chunk swapping is not detected. Since each chunk is valid under its own tag, information binding its position must be included in the authentication scope. This was verified directly. When four chunks were encrypted and two were swapped, decryption succeeded without errorwhen AAD was absent (with contents differing from the original text), and resulted in InvalidTag when chunk numbers were included in AAD.
• Attacks truncating the end of the file to completely remove chunkswere not covered in this test. Verified directly: for a file with the last chunk removed, if AAD contains only the chunk number, 3 chunks are decrypted without error, meaning tags failed to indicate the truncation. If the total chunk count is included in AAD, modifying the total count in the header caused all chunks to result in InvalidTag , whereas leaving the header intact triggered the step comparing the read count against the total count (code assert) to catch the truncation. In other words, both binding the total count to AAD and verifying the count are required.
• Nonces are randomized 12-byte values per chunkNIST SP 800-38D Section 8.2.2 requires a randomized field of 96 bits or more for the randomized IV construction, and Section 8.3 limits the total number of authenticated encryption invocations with the same key to 2 to the 32nd power or less. Divided into 64MiB chunks, this limit allows approximately 256PiB per key (2 to the 32nd power × 64MiB, author's calculation). Since a 140GB model has about 2,100 chunks, it remains well within the limit until encrypting such models about 2 million times with the same key, and the probability of nonce collision within a single file is roughly 2 to the -75th power by the birthday paradox approximation (author's calculation). Because the limit counts encryption invocations, key rotation procedures repeating re-encryption with the same key accumulate invocations. Decryption is not counted.
• File identifiers should also be included in AAD (author's deduction, not a NIST requirement). AAD in this article contains only chunk numbers and total chunk counts. If multiple files are encrypted with the same key, swapping chunks of the same number between different files still passes tag verification. Either file-specific data encryption keys (DEKs) must be used or file identifiers must be included in AAD, which aligns with the DEK recommendation in Section 6. Verified directly: between two files created with the same key, when chunk 1 of file A was placed into chunk 1's position in file B, decryption succeededwhen AAD had only (chunk number, total count), yielding file A's contents for chunk 1. Adding file identifiers to AAD resulted in InvalidTag .

Ciphertext is Indistinguishable from Plaintext

Byte entropy of the first 1MiB (including headers) was 6.34 bits/byte for plaintext (safetensors, bf16 weights) and 8.00 bits/byte for ciphertext. While weight files retain statistical structures that can provide clues via compression or pattern analysis, ciphertext is equivalent to randomness. Since encrypted files cannot be compressed either, compression must be done before encryption .

How Much Does It Increase When Inserted into the Serving Engine Loading Path?

We placed decrypted weights into a memory volume (/dev/shm) and booted a vLLM server along that path. The model is Qwen3-0.6B (safetensors 1.50GB) configured with a single 8GB-class GPU, 64MiB chunks, and chunk numbers and total chunk counts included in AAD. Server preparation time spans from launching the server process to receiving a health check response, and decryption time is not included in this value.

The interpretation is as follows:

• The steady-state overhead is simply the decryption time. From the second boot onward, server preparation time was at the same level for both plaintext and decryption paths (around 28 seconds), preceded by about 2.3 seconds of decryption. Cold starts increased by about 8% for this model.
• First boots were lengthy for both. The first plaintext startup time of 68.0 seconds appears to be because the compilation result was not in the cache, and the first startup time of 49.4 seconds on the decryption path is presumed to be due to recreating the cache because it was the first time encountering that model path (the cause was not verified through logs). Since it can take longer each time the path or configuration changes, this effect may be greater than the decryption cost.
• Tampering is rejected before the server boots up. This assumes a configuration where the server does not start if the decryption stage fails (the startup script checks the exit code of the decryption).
• This test was conducted using only a single 0.6B model and a single GPU. Values for a 140GB model or multiple GPUs were not measured (unmeasured).

CPU Supplement Test: Disabling Acceleration, Parallelization

The remaining questions from the test above were re-measured using only the CPU (ephemeral pod, one node of a 22-core server CPU, cryptography 50.0.2, 64MiB chunk, memory-only). Since another workload was running concurrently on this node, the absolute values for the same test fluctuated by about 40% (decryption for a single process came out to 1.20 GB/s, and one of the thread tests was 2.00 GB/s). Therefore, only the ratios are meaningful.

• Without AES instructions, decryption becomes 4 to 7 times slower. For a 140GB model, the extrapolation above (approx. 70 seconds) becomes around 5 minutes (calculated). Most general cloud instances have acceleration, but if you are in an environment where CPU flags are masked due to virtualization settings, you need to check (grep -m1 -o aes /proc/cpuinfo).
• Threads stalled at 2, but processes scaled as they were increased. The initial memory bandwidth estimation was wrong. Compared to this node's memory bandwidth, 2.5 GB/s is a very small value, and when split into processes, it rose up to 6 GB/s at 8 processes. This test did not break down why thread parallelism is blocked within a single Python process (cause unverified, likely an interpreter-side limitation). Practically, a configuration that decrypts by dividing chunks across processes (or a separate loader process per GPU) is correct.
• Limitations: This is a single test on a single node and excludes file I/O. The actual time to load across multiple GPUs was not measured (unmeasured).

5. What Cannot Be Prevented: Weights Are Plaintext During Execution

For the server to use the model, it must load the decrypted weights into memory (including GPU memory). From that moment on, root on the node or a compromised process on the same node can read the memory. Standard GPUs do not encrypt memory, so this threat cannot be prevented solely by encryption-at-rest.

An NVIDIA tech blog (2023-05-31) explains that H100 GPUs provide confidential computing features that work alongside CPU trusted execution environments (AMD SEV-SNP, Intel TDX). This article could not directly test this feature (lack of datacenter GPUs). A blog post from the same company on 2023-08-03 describes the protection method as follows: the CPU TEE encrypts the virtual machine's memory, the GPU is isolated in hardware, and data between the CPU and GPU passes through an encrypted bounce buffer. On the other hand, it states that HBM on the GPU package is considered secure against casual physical attack tools and is therefore not encrypted. Thus, calling confidential computing "GPU memory encryption" is inaccurate; what it blocks is plaintext exposure on the host side and access by the host administrator. The same post also states that the bandwidth between the CPU and GPU is capped by CPU encryption performance at around 4 GB/s, and in this article's path of loading weights at startup, this value may dictate the loading time (vendor description, directly unmeasured). How to securely deliver keys (releasing keys after attestation) is also a separate design issue.

Therefore, practical layers are stacked as follows:

1. Storage Side: Encrypt weights and keep keys in a KMS or secret management tool. Bucket permission errors do not immediately lead to leaks.
2. Loading Side: Receive keys and decrypt at startup. If you use a memory-based volume (emptyDir of medium: Memory) so that decrypted plaintexts are not left on disk, plaintexts will not remain on the node disk. Since this volume consumes RAM equal to the model size, it must be calculated by including it in the node memory and pod memory limits (calculated, not directly tested).
3. Execution Side: Node access control (SSH, debug pods prohibited), pod security settings, and confidential computing nodes if necessary.
4. Output Side: Rate limiting and abnormal query detection. You can build a surrogate model by collecting outputs even without stealing weights.

6. Where to Store the Keys

• Do not store them in the same place as the images or weight repository. Otherwise, a single breach exposes both.
• Keys should be received from an external KMS when the pod starts (authenticated by the pod's identity) and should not be left in plaintext as environment variables for long.
• A procedure to rotate keys (re-encryption) is required. Re-encrypting a 3.4GB file at 0.56 GB/s takes about 6 seconds (calculated), but large models take longer, so using a scheme where a key wrapping the file (DEK) is kept separately and that key is wrapped by a master key reduces key rotation costs.

7. Potential Interview Questions

• What is protected when model weights are encrypted? Storage, backup, and registry leaks, as well as tampering (if GCM is used). Memory dumps during execution cannot be prevented.
• Why use GCM? It performs encryption and tamper detection simultaneously. Flipping a single byte causes decryption to be rejected.
• What should be noted when encrypting in chunks? Put chunk numbers into the additional authenticated data to prevent reordering attacks, include the total chunk count in the authenticated data and verify the read count to prevent truncation attacks, and include the file identifier to prevent chunks from different files from mixing (all three verified directly), ensuring nonces do not overlap.
• How much does decryption increase cold starts? In this test, it was about 0.7 to 0.9 GB/s, making a 7GB model take about 8 to 10 seconds (extrapolated), and for the 0.6B model, about 2.3 seconds of decryption were added to about 28 seconds of server preparation. The calculation itself is at the 2 GB/s level.
• How is a running model protected? Node access control and confidential computing (CPU TEE and GPU isolation, PCIe section encryption. HBM itself is not encrypted). This article only verified papers and vendor documentation.

External References

Claims that could not be measured directly or where direct measurement was insufficient were verified using the materials below. "Abstract only" means only the abstract, not the main text, was read.

There are also materials that could not be accessed. RAND's model weight protection report (2024) could not be opened because access to the original text was blocked, so figures regarding the frequency of leak incidents are not included in this article.

Unverified Items

• Confidential computingperformance cost and actual protection scope. It could not be measured due to a lack of hardware and can only be estimated based on the scope of external sources below (inference generally in single-digit percentages, up to about 20% depending on conditions, and a 45-70% difference in configurations swapping models).
• Loading time for large models and multiple GPUs. Only the value for a single 0.6B model was measured; parallel decryption or tensor parallel configurations were not measured.
• Decryption methods via GPU, parallel decryption time when actually dividing and loading a large model across multiple GPUs (the CPU supplement test only measured up to process parallelism).
• Call latency and failure behavior of Key Management Systems (KMS).

References

• NIST SP 800-38D, Recommendation for Block Cipher Modes of Operation: The CMAC Mode for Authentication: https://csrc.nist.gov/pubs/sp/800/38/d/final
• cryptography Library AEAD documentation: https://cryptography.io/en/latest/hazmat/primitives/aead/
• NVIDIA Technical Blog, Protecting Sensitive Data and AI Models with Confidential Computing (2023-05-31): https://developer.nvidia.com/blog/protecting-sensitive-data-and-ai-models-with-confidential-computing/
• NVIDIA Technical Blog, Confidential Computing on NVIDIA H100 GPUs for Secure and Trustworthy AI (2023-08-03): https://developer.nvidia.com/blog/confidential-computing-on-nvidia-h100-gpus-for-secure-and-trustworthy-ai/
• Zhu et al., Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Study: https://arxiv.org/abs/2409.03992
• Carlini et al., Stealing Part of a Production Language Model: https://arxiv.org/abs/2403.06634
• Parts 14 (Server Startup Time) and 8 (Safetensors Format and Model Loading) of this series

Verified Versions and Dates

As of October 7, 2026. cryptography Libraries (AESGCM), Python 3.12 inside the container, Qwen3-1.7B safetensors (speed test), and Qwen3-0.6B (server loading path test, vLLM 0.31.0).