The Problem: A Service That Holds What Nobody Should Read

Evgoras is a psychometric assessment platform. Organizations send clinical and HR instruments to participants; participants answer them. Think about what that means for a moment: the database holds people's answers to depression and anxiety screenings, alongside their names and emails. Under GDPR this is special-category data, the tier with the strictest processing requirements in the regulation. A leaked backup, a misconfigured replica, or a stolen database credential should never be enough to read any of it.

The architecture puts all of that behind one service: a FastAPI application we call the vault. Participant PII, raw test responses, scored results, assistant chat messages, biometric proctoring payloads: every sensitive read and write flows through it. The rest of the platform (the Nuxt BFF, the AI gateway) only ever sees what the vault decides to return.

The vault is Python end to end, except for the roughly 140 lines that actually touch key material. Those are Rust. This post explains why the split exists, how the encryption is actually constructed, and what each layer of the design protects against.

Why Python, and Why Not Only Python

We chose Python for the vault for the usual reasons: FastAPI is productive, the scientific ecosystem carries the scoring engine, and WeasyPrint renders the PDF reports. When a scoring model changes or a new instrument type lands, we want to iterate in hours, not days.

The encryption core has exactly the opposite requirements. It should be small, boring, auditable, and it should almost never change.

To be fair to the alternative: pyca/cryptography is an excellent library, and since version 3.4 its internals are themselves written in Rust. Using it directly is the right call for most applications. What we wanted was a property it cannot give us by design, because it is a general-purpose library: a surface small enough that a security review can read all of it. Our entire crypto module is one Rust file exposing seven functions:

What Envelope Encryption Actually Is

Before the key hierarchy makes sense, it helps to understand the problem it solves.

The naive approach is one key that encrypts every row in the database. It works, until you ask two questions. What happens when the key leaks? Everything is exposed at once. And what does key rotation look like? Re-encrypting every row in every table, in one migration, under pressure.

Envelope encryption is the standard answer, used by every cloud KMS. The idea: a master key never encrypts data directly. Instead it protects a set of data keys, and each data key encrypts one bounded scope of data. Compromising a data key exposes only its scope; rotating the master key means re-protecting a handful of data keys instead of terabytes of rows.

There are two ways to get data keys from a master key. You can generate them randomly and store them encrypted under the master key, which is what AWS KMS does. Or you can derive them deterministically from the master key plus a context string, which means there is nothing extra to store or back up. We chose derivation:

VAULT_MASTER_ENCRYPTION_KEY          (32 bytes, lives in the secret manager)
    │
    derive_key(master, salt = key_id)     Argon2id
    │
    ├── org:{org_id}:v1          → PII data key (participant records)
    └── session:{session_id}:v1  → response data key (answers, chat, proctoring)

The context string is the interesting part. Every encrypted row stores its key_id in a sibling column, and that same string is the derivation salt. To decrypt a row, you read its key_id, re-derive the key, and open the ciphertext. This gives you three properties for free:

• No key table. There is nothing to protect, replicate, or back up besides the master key itself. The database can hold ten thousand distinct data keys without holding a single key byte.
• Scoped blast radius. PII derives from the organization id, so one derived key opens one organization's contact records. Test responses derive from the session id, so one derived key opens a single participant's single sitting. Neither gets you anywhere near the platform.
• Rotation as a planned migration. The v1 suffix is part of the context. Rotating means writing new rows as v2 and re-encrypting old ones on a schedule. Old rows still name the exact key that opens them, so there is never a moment where the system cannot tell which key a row needs.

The Primitives, and Why Each One

AES-256-GCM: encryption that notices tampering

Plain encryption hides data; it does not protect it from modification. Flip a bit in a ciphertext encrypted with an unauthenticated mode and it decrypts happily into corrupted plaintext, which your application may then trust. AES-GCM is an authenticated mode: encryption produces the ciphertext plus a 16-byte tag computed over it, and decryption verifies the tag before returning a single byte. A tampered row does not decrypt into garbage; it refuses to decrypt at all.

GCM has one famous sharp edge: the nonce, a 12-byte value that must never repeat under the same key. Reusing a nonce in GCM does not just weaken the encryption, it breaks it catastrophically, leaking both plaintext relationships and the authentication key. Our construction removes the caller from that decision entirely: the nonce is generated from the OS random source inside the encrypt function, and travels with the ciphertext as base64(nonce || ciphertext || tag). There is no API through which a caller could supply, and therefore repeat, a nonce.

Argon2id: a key derivation function that costs something

A key derivation function turns one secret plus a context into a new key. The fast option is HKDF, which is what you would pick if derivation speed mattered. We deliberately picked the slow one: Argon2id, winner of the Password Hashing Competition, designed to be expensive in both CPU and memory so that brute-forcing it on GPUs is uneconomical.

Why pay that cost for keys derived from a random 32-byte master? Defense in depth. Against an attacker who has the full master key, the KDF choice is irrelevant, they have everything. But real leaks are messier than that: partial key material, a weaker secret accidentally fed into the hierarchy, an offline attacker guessing candidate masters against a known ciphertext. A memory-hard KDF turns each guess from nanoseconds into a deliberate, measurable expense.

Constant-time comparison: not leaking secrets through the clock

Comparing a secret token with == returns early at the first mismatched byte, which means the time the comparison takes reveals how many leading bytes were correct. Timing attacks exploit exactly this, and they work over networks. The Rust core compares every byte unconditionally and folds the differences with XOR, so a wrong first byte and a wrong last byte cost identical time.

Zeroization: keys should not outlive their use

When a Python object is garbage-collected, its memory is reclaimed, not erased. Key material can sit in freed heap pages until something overwrites them, visible to anyone who can read process memory or a core dump. Rust's zeroize crate overwrites buffers when they drop, and the compiler is prevented from optimizing the "pointless" write away. It is a small guarantee, but it is one Python fundamentally cannot make.

The Rust Core

With the concepts in place, the actual code is almost anticlimactic. The encrypt path, close to verbatim:

#[pyfunction]
fn encrypt(plaintext: &[u8], key: &[u8]) -> PyResult<String> {
    if key.len() != 32 {
        return Err(PyValueError::new_err("Key must be 32 bytes"));
    }
    let cipher = Aes256Gcm::new_from_slice(key)?;

    let mut nonce_bytes = [0u8; 12];
    rand::thread_rng().fill_bytes(&mut nonce_bytes);

    let ciphertext = cipher.encrypt(Nonce::from_slice(&nonce_bytes), plaintext)?;

    // nonce (12) || ciphertext + tag
    let mut combined = Vec::with_capacity(12 + ciphertext.len());
    combined.extend_from_slice(&nonce_bytes);
    combined.extend_from_slice(&ciphertext);
    Ok(BASE64.encode(&combined))
}

There is nothing clever in it, which is the point. Decryption failures return a deliberately unhelpful "Decryption failed": the error message will not help an attacker distinguish a bad tag from a bad nonce.

Every dependency in the crate's Cargo.toml is pinned with =, down to the RNG. Cryptographic code is the one place where an automatic minor-version bump is not a free upgrade; it is an unreviewed change to the most sensitive code path in the system.

PyO3: What the Boundary Looks Like

PyO3 is the bridge that lets Rust functions appear as a native Python module. The #[pyfunction] attribute generates the glue: Python bytes arrive as &[u8], Rust errors become Python exceptions, and from the caller's side the whole thing is one import:

import evagora_crypto

ciphertext = evagora_crypto.encrypt(data, key)

No FFI ceremony, no serialization layer, no sidecar process. The compiled extension loads into the interpreter like any C extension, and calling into it costs roughly a function call.

Building it is where the real decisions live. maturin compiles the crate into a Python wheel, and our Docker build does it in a stage of its own, using maturin's official manylinux image so the resulting binary is compatible with the deployment base image. The extension targets a specific interpreter version rather than the stable ABI, so the wheel is built for exactly the Python that will load it.

That builder image is pinned to an exact version, and the Dockerfile documents why:

This stage compiles the crypto module that every piece of candidate PII is encrypted with, so a silent toolchain bump here is the last place we want one.

The Python Side: Policy, Not Primitives

The split is not "Rust does security, Python does the rest." Python owns the security policy, and the policy layer is where most real-world encryption failures actually happen. Not broken math, but a fallback that silently stores plaintext, a "temporary" debug path that skips the check, an error handler that swallows the wrong exception.

Three policies do the work here.

Fail closed, loudly. If the Rust module failed to import, or the master key is missing, malformed, or the wrong length, every request that needs crypto returns HTTP 503. A vault that cannot encrypt is down, not degraded. The tempting alternative, log a warning and store the row anyway, converts a visible outage into an invisible data exposure, which is a far worse trade.

Tolerate broken rows, never broken configuration. Rows created before the crypto layer existed, or corrupted since, decrypt to None with a logged warning instead of crashing a listing endpoint for everyone. But the distinction is explicit in the code: only per-row problems are swallowed, while configuration errors re-raise as 503s:

def try_decrypt_json(ciphertext, key_id):
    if not ciphertext or not key_id or ":" not in key_id:
        return None          # legacy placeholder rows
    try:
        return decrypt_json(ciphertext, key_id)
    except HTTPException:
        raise                # missing key/module is still a 503
    except Exception:
        logger.warning("Failed to decrypt payload for key_id=%s", key_id)
        return None

One bad row is that row's problem. A missing master key is everyone's problem, and it should page someone.

Cache the expensive thing, on purpose. Argon2id being intentionally slow is a feature at derivation time and a tax at request time. Deriving on every request would add that cost to every read, so derived keys sit behind an lru_cache and each org or session pays the derivation once per process lifetime. The trade is explicit: cached keys reside in process memory for the worker's lifetime, which widens the window an attacker with memory access could exploit. We accepted that with eyes open, and it appears again in the limits below.

What This Design Does Not Do

Encryption at rest gets oversold constantly, so here is the honest list:

• It is not an HSM. The master key exists in process memory, and cached data keys live beside it. An attacker with code execution inside the vault process wins. The design targets the far more common failure modes: leaked backups, exposed database credentials, misconfigured replicas, and any path to the data that bypasses the application.
• It does not replace access control. Authorization is a separate layer (application checks plus row-level security in the database) and is enforced whether or not a row is encrypted. Encryption limits what a stolen copy of the data is worth; it does not decide who may ask for it.
• It is not what most applications need. If your threat model does not include special-category personal data, pyca/cryptography with Fernet, or your cloud provider's KMS, is less code and fewer moving parts. The hybrid earns its complexity here because the data is exactly the kind whose leak cannot be walked back.

Lessons Learned

1. Split the codebase by the cost of being wrong

The Python/Rust boundary is not about performance; AES was never our bottleneck. Most of the vault can afford bugs: a scoring regression is annoying, visible, and fixed by the next deploy. A nonce reuse or a timing leak is silent, invisible, and unrecoverable once the data has left. The parts that change often belong in the language that changes fastest. The part that must never be wrong belongs in the language that makes wrong hardest.

2. The policy layer is where encryption fails in practice

The primitives were the easy 140 lines. Deciding what happens when decryption fails, when the key is missing, when a legacy row predates the scheme, that is where a design quietly leaks plaintext or quietly loses data. Write those policies as deliberately as the crypto.

3. Remove decisions instead of documenting them

Every place the API could have offered a choice (nonce supply, key size, wire format), it offers none. The safest configuration option is the one that does not exist.

4. A slow KDF forces you to be explicit about caching

Choosing Argon2id meant confronting where derived keys live and for how long, a question a fast KDF would have let us ignore. The uncomfortable question turned out to be the valuable one.

5. Pin the toolchain that builds your crypto

Reproducibility matters most exactly where review matters most. A floating builder tag is an unreviewed dependency on whatever ships tomorrow.

Conclusions

The hybrid is not a compromise between two languages; it is an assignment of responsibilities. Python owns everything that benefits from speed of change: routes, tenancy, scoring, policy. Rust owns the seven functions where a mistake cannot be observed, cannot be hotfixed after the fact, and cannot be undone.

Envelope encryption with derived keys keeps the blast radius of any single compromise bounded to one organization's records or one participant's session, makes rotation a planned migration instead of an emergency, and stores zero key material outside the secret manager. The primitives underneath are deliberately unremarkable: AES-256-GCM so tampering is detected, Argon2id so derivation costs an attacker something, constant-time comparison so the clock stays quiet, zeroization so keys do not linger in freed memory.

None of it is novel cryptography, and that is the highest compliment this kind of code can receive. The novelty budget went where it belongs: into drawing the boundary, and into making sure the code on the sensitive side of it is small enough that anyone can read all of it.

Handling sensitive data and wondering where encryption actually belongs in your stack? Let's talk.