Security

Security overview

The trust model in one page.

Security architecture

K-Lake runs entirely inside your infrastructure. The components are the API, the web console, a fleet of stateless workers, an optional MCP server, and a managed database. Your identity provider and your content sources stay under your control: K-Lake reads from sources read-only and federates sign-in to your IdP.

Improvements on prior approaches

K-Lake was designed to close gaps common in conventional indexing and enterprise-search products.

Memory-safe foundation

The core services and every source connector are written in Rust. This is a deliberate security decision.

• Memory safety by construction. Rust's ownership model eliminates buffer overflows, use-after-free and data races at compile time — the vulnerability classes behind a large share of CVEs in systems software.
• No garbage collector, so performance is predictable, resource cleanup is deterministic, and there is no GC-pause attack surface.
• Strong typing and exhaustive error handling reduce the logic errors that lead to unsafe states.

Multi-tenant isolation

Every source, user, token and indexed file belongs to exactly one tenant. Isolation is not a matter of careful application coding — it is enforced by the database.

Each request runs in a database session bound to its authenticated tenant and principal. Row-level security policies attached to every table restrict visibility to that tenant's rows. A bug in application code cannot widen this boundary, because the boundary lives below the application.

Authorisation and security trimming

Beyond tenant isolation, K-Lake enforces per-file access control. A caller sees a file only if the file's own source-native permissions — NTFS or NFSv4 ACLs, POSIX ownership, S3 policy — grant their identity access. Native principals are correlated to your IdP identities by directory enrichment, then enforced, again, by row-level security at the database layer.

1. The client presents an OIDC bearer token or a personal access token.
2. The API validates the token with your identity provider and resolves the identity.
3. The IdP returns a verified principal and its group memberships.
4. The API queries in a database session bound to that tenant and principal.
5. Row-level security filters by tenant and per-file grants.
6. Only authorised rows are returned — and every access is audited.

Because trimming is enforced where the data lives, the same guarantee applies uniformly across the console, CLI, REST API and AI access over MCP. There is no admin code path that bypasses it.

Identity and access management

• Federated sign-in. Users authenticate through your own identity provider over OIDC and OAuth 2.1, and SAML where applicable. K-Lake does not become a parallel identity store — it consumes verified identities from providers such as Microsoft Entra ID, Okta, Google or Keycloak.
• Personal access tokens. For programmatic and CLI use, scoped tokens are minted per user, shown once and revocable.
• AI access over MCP. The MCP server is a full OAuth 2.1 authorisation server, following the relevant RFCs for protected resources and audience restriction. Assistants connect as the individual user, inherit that user's per-file trimming, and receive a citation for every fact — so answers are grounded and verifiable, and nothing is exported to the model provider in bulk.

Encryption

Credentials are never written to logs or shell history — secrets are read from stdin or the environment, never from command-line flags.

Auditability and observability

• Audit trail. Every privileged operation — sign-in, source change, crawl, search, file open, token mint and AI query — is recorded with the acting principal, tenant and resource, supporting forensic review and access reporting.
• Structured logs and metrics. Services emit structured logs and Prometheus-compatible metrics that integrate with your existing SIEM and monitoring stack. All of it is scraped and stored by you, inside your boundary — see Telemetry.
• Verifiable retrieval. Every search result and AI citation links back to the original file and re-checks the caller's permissions at open time, so an auditor can confirm exactly where any answer came from.

Supply chain and vulnerability management

• Daily security report. A rolling daily report publishes automated vulnerability scan results across every shipped container image. [Mirror URL under zegress.ai to be inserted before publishing.]
• Signed images. Release images are cryptographically signed, so you can verify provenance and integrity before deployment.
• Minimal images. Containers ship only what each service needs, reducing both attack surface and patch burden.

Deployment and data residency

• Runs in your cluster. K-Lake deploys into your own Kubernetes environment; all content and metadata stay within your boundary.
• Air-gap capable. K-Lake can run with no internet egress at all, including fully on-premise AI retrieval over MCP, so regulated and disconnected environments are first-class.
• Offline licensing. Licensing is enforced offline. There is no licence-server call-home.

Data protection and compliance

Zegress.Ai is registered with the UK Information Commissioner's Office under registration {{ICO_REGISTRATION}}.

Because K-Lake is self-hosted in your own infrastructure, your content and metadata remain under your control and within your data-residency boundary. Zegress does not process or receive your indexed data.

Reporting a vulnerability

If you believe you have found a security issue, email [email protected] or contact your Zegress representative. Please don't disclose publicly until we have confirmed a fix.

Security trimming

Per-file access control, enforced where the data lives.

Per-file security trimming means a caller sees only the files their source-native permissions already grant them — the file's own owner, group or ACL. Someone who can't read a file on the NAS can't see it, its metadata or its extracted content through K-Lake either.

Enforcement lives in row-level security, so it applies everywhere automatically: listings, file detail and content search are trimmed by the same predicate. There is no separate "filter the search results" code path to keep in sync.

Trimming is fail-closed and strictly opt-in. A source you don't configure for it keeps source-level visibility. A source you do configure hides any file whose grants haven't been computed yet — K-Lake would rather hide a file you could read than leak one you can't.

How it works

Both halves of the question — who is this caller, and who may read this file — collapse into one namespace, the principal ref, of the form <kind>:<scope>:<value>. Each distinct ref is interned to an integer principal ID.

• File side. When a file is indexed, K-Lake derives the set of principal refs that may read it from its POSIX mode bits and, if captured, its ACL, then records a read-grant for each granted principal.
• Caller side. When a caller hits the API, their IdP claims — object ID, email, UPN, group memberships — are turned into principal refs, resolved to a principal ID set and handed to the datastore.

A file is authorised for a caller when the two sets intersect. The same predicate applies to listings, metadata and content, ANDed on top of the existing tenant and source-visibility checks. It short-circuits in this order:

1. Tenant admin, or source owner or editor — unrestricted admin path.
2. Source not in per_file mode — source-level visibility applies.
3. File is world- or everyone-readable (grants_state = 2) — visible.
4. Caller holds a read grant on the exact file — visible.
5. fail_closed is off and grants aren't computed yet — visible, the fail-open fallback.
6. Otherwise — hidden.

Principal-ref kinds

The same ref often appears on both sides — a POSIX group on the file and the caller's resolved group — and that's the common case. When the two sides are in different namespaces, such as an on-premise AD SID on the file against an Entra group object ID in the token, they're bridged by directory enrichment, which builds the alias graph.

Enabling it on a source

The trim policy lives in a security_trim block inside the source's config, read by both the workers and the authorisation predicate.

Set it through the dedicated endpoint, which merges the block into the source config without disturbing anything else. The change takes effect within about 30 seconds.

zegress-control --api-url "$ZEGRESS_URL" --api-token "$ZEGRESS_TOKEN" \
  source security-trim --source-id 'smbfs://nas.corp/finance' \
  --mode per_file --fail-closed true

--fail-closed is optional — omit it to change the mode while leaving the stored value untouched.

curl -X POST -H "Authorization: Bearer $ZEGRESS_TOKEN" \
     -H "Content-Type: application/json" \
     "$ZEGRESS_URL/api/sources/<urlencoded-source-id>/security-trim" \
     -d '{ "mode": "per_file", "fail_closed": true }'

The response echoes the stored policy. Authorisation is tenant-admin or the source's owner or editor; an unauthorised caller gets 404 rather than 403, because the row isn't visible under row-level security in the first place. In the console, the source detail page has a Per-file security trimming panel that posts to the same endpoint.

Prerequisite for ACL-based trimming

POSIX grants are derived automatically from the mode, UID and GID the crawl already captures. ACL grants are not. The source must first capture the raw ACL bytes through a metadata-capture cap — ntfs_acl for SMB and SMBFS, nfs4_acl for NFS. Without the cap there are no ACL bytes to derive grants from, and an ACL-controlled file falls back to its POSIX bits, commonly leaving grants_state = 0 and therefore hidden under fail-closed.

zegress-control --api-url "$ZEGRESS_URL" --api-token "$ZEGRESS_TOKEN" \
  source meta-caps --source-id 'smbfs://nas.corp/finance' --caps ntfs_acl

# NFS export
zegress-control --api-url "$ZEGRESS_URL" --api-token "$ZEGRESS_TOKEN" \
  source meta-caps --source-id 'nfs://nas.corp/export/data' --caps nfs4_acl

After enabling a cap on a source that's already indexed, run a backfill so existing files get their ACL bytes, and therefore their grants, computed:

zegress-control source backfill-meta --source-id 'smbfs://nas.corp/finance'

POSIX and ACL grants

K-Lake derives read-grants from whichever permission model the source exposes. Each cap that produced a grant is tracked against that grant, so re-crawling one model's grants never clobbers another's.

POSIX — automatic

ACL — requires the cap

Derivation is deliberately conservative: it emits only grants it is certain of, and any parse error or malformed field yields no grants at all.

• NFSv4 (nfs4_acl, RFC 7530 §5.11 wire format). Allow-read ACEs become grants. OWNER@ and GROUP@ resolve to the file's POSIX owner and group principals; named whos become nfs4who refs for directory enrichment to alias later. An EVERYONE@ allow-read marks the file world-readable, and a deny-read on EVERYONE@ yields no grants.
• NTFS (ntfs_acl, parsed from the self-relative security descriptor, MS-DTYP §2.4.6). DACL allow-read ACEs become sid::<SID> grants, and the well-known Everyone SID S-1-1-0 marks the file world-readable. Deny-overrides-allow is resolved at compute time, so reads stay a positive set-intersection.

Grants state and fail-closed behaviour

Each file's grants_state records whether grants have been computed and what they say. It's how the read-time predicate distinguishes "no grants yet" from "grants present" without scanning the full grant set on every hit.

Correlating callers to files

POSIX trimming works out of the box when the caller's IdP groups resolve to the same names or IDs as the file owners. Cross-directory cases — an on-premise AD SID on a Windows ACL against an Entra object ID in an OIDC token, or a numeric POSIX GID against a name-based IdP group — need the two namespaces bridged.

That bridging is directory enrichment. It materialises the alias graph from declared operator mappings, POSIX name resolution, Entra Graph discovery including group-membership and overage resolution, and AD/LDAP correlation for unlinked on-premise directories. Only high-confidence alias edges expand grants, so a fuzzy match can never produce a false grant.

Limitations and management surface

• Set through the console, CLI or API. The Per-file security trimming panel, the source security-trim command and the POST /sources/:id/security-trim endpoint all write the same config block.
• ACL trimming needs the capture cap first. Enable ntfs_acl or nfs4_acl and backfill before trusting ACL-controlled files to be visible. POSIX-only trimming needs no cap.
• Cross-directory trimming needs directory enrichment. Without the alias graph, only same-namespace grants — typically POSIX — will match.
• Fail-closed means hidden until computed. Plan a backfill as part of any cutover to per_file.
• The trim layer is consumed by both the API and MCP. Both resolve the caller's principal set into the same authorisation context, so a caller sees the same trimmed view through either surface.

Directory enrichment

Tying caller identities to file identities, so trimming can decide.

When a caller signs in, your identity provider hands K-Lake a set of claims: an object ID, an email, a UPN, some group IDs. When K-Lake indexes a file, the file carries a different kind of identity entirely — a POSIX UID or GID, or an NTFS and Active Directory SID. These two namespaces rarely match on their own. Directory enrichment ties them together, so security trimming can decide whether a given caller may see a given file.

The correlation problem

A caller's token yields refs like oid:<issuer>:<guid> and upn:[email protected]. A file's ACL yields refs like sid::S-1-5-21-… or posixgid:<source>:5000. Trimming authorises a caller for a file only when the caller's refs and the file's grant refs land on the same principal. Directory enrichment's job is to discover the edges that make that true — that an Entra group is the on-premise SID, that a POSIX group is a named directory group.

Every identity, caller-side or file-side, is normalised to a principal ref of the form <kind>:<scope>:<value>. Emails and names are lowercased; SIDs are globally unique, so they use the empty scope.

The alias graph and fail-closed confidence

Correlation is stored as an alias graph — bidirectional edges between two interned principals. At trim time, K-Lake takes the caller's refs and expands them across the graph to reach every principal they are equivalent to, then checks those against the file's grants.

Correlation strategies

K-Lake builds the alias graph from four edge sources. Use any combination; pick by what your environment can offer.

Declared mappings

The deterministic, immediately usable path. You state from/to/confidence triples in the tenant's principal mappings; each becomes a bidirectional edge, high-confidence unless you mark it medium. Good for one-off equivalences you already know, or for promoting a correlation you've verified by hand.

POSIX names

For per-file-trimmed NFS and POSIX sources, K-Lake resolves the UID and GID set captured during crawling to names — via the worker host's name-service resolution, for instance on an AD-joined worker — and links the POSIX group ref to the named group ref a name-based IdP produces. This runs automatically; there's no separate command.

Entra Graph discovery

For Entra ID, K-Lake uses Microsoft Graph with app-only client credentials to discover three kinds of edge:

• Groups. Each group's object ID is aliased to its security identifier and, when the group is AD-synced, its on-premise security identifier. Whichever SIDs are present become edges to the caller's group ref. This is how a file ACL'd to a group resolves to a caller's group membership.
• Users. Each synced user's on-premise security identifier is aliased to their UPN or email ref, bridging files a user owns to the caller's UPN. Because it reads the already-correct cloud UPN straight from Graph, no UPN rewrite is needed for a properly synced tenant.
• Membership, and overage resolution. For each SID-bearing group, K-Lake reads its transitive user members and adds a directed member-to-group edge. This handles JWT group overage: when a user belongs to more than roughly 200 groups, Entra omits the groups claim from the token — but the caller's UPN ref still reaches their groups' grants through these pre-synced edges, with no per-request Graph call. The edges are directed, so two members of the same group never gain access to each other's personal files.

The app registration needs the Graph application permissions Group.Read.All, GroupMember.Read.All and User.Read.All — all read-only — plus admin consent.

This works when the SIDs on the files belong to the same Entra tenant, or its AD-synced on-premise side. It does not bridge a SID living in a directory Entra has never seen.

AD and LDAP correlation for unlinked directories

The common hard case: the file server's AD is a separate directory from a cloud-only Entra tenant. The two share no SIDs and use different UPN suffixes, so Graph SID-bridging alone cannot connect them — Entra has never heard of the on-premise SID.

LDAP correlation solves it. K-Lake binds to the on-premise domain controller over LDAPS, enumerates users and groups, and aliases each AD principal's object SID to the caller-side ref the Entra token actually carries. The key setting is a UPN-suffix rewrite: an on-premise UPN is rewritten to the cloud UPN, producing a high-confidence edge. The mail attribute, where present, is likewise high confidence. A sAMAccountName match is a medium fallback — recorded, never granting on its own.

Choosing a strategy

• Same or AD-synced Entra tenant — Entra Graph.
• Separate, unlinked on-premise AD against a cloud-only Entra tenant — AD/LDAP with a UPN rewrite, or declared mappings for the handful of groups you care about. Graph alone will not connect them.
• NFS or POSIX with a name-based IdP — POSIX names, which run automatically, plus declared mappings for anything names don't cover.

Command line

Discover Entra groups once

Runs Graph discovery immediately and materialises all three edge sets without storing any config. Use it to validate credentials or do a one-shot sync.

export ZEGRESS_GRAPH_CLIENT_SECRET='<app-client-secret>'
zegress-control --postgres-url "$ZEGRESS_POSTGRES_URL" \
  directory sync-graph \
  --tenant <tenant-slug> \
  --entra-tenant <entra-tenant-id> \
  --client-id <app-registration-client-id>

Store Entra config for automatic discovery

Stores the non-secret config and encrypts the client secret at rest. After this the worker runs Graph discovery automatically — you don't run sync-graph again.

export ZEGRESS_GRAPH_CLIENT_SECRET='<app-client-secret>'
export ZEGRESS_MASTER_KEY='<base64-master-key>'
zegress-control --postgres-url "$ZEGRESS_POSTGRES_URL" \
  directory set-graph \
  --tenant <tenant-slug> \
  --entra-tenant <entra-tenant-id> \
  --client-id <app-registration-client-id>

Store AD/LDAP config

Stores the non-secret config and encrypts the bind password at rest. --name-scope sets the scope for name fallback refs, typically the NetBIOS domain. --upn-rewrite is repeatable, each as from=to.

export ZEGRESS_LDAP_BIND_PASSWORD='<ad-bind-password>'
export ZEGRESS_MASTER_KEY='<base64-master-key>'
zegress-control --postgres-url "$ZEGRESS_POSTGRES_URL" \
  directory set-ldap \
  --tenant <tenant-slug> \
  --url 'ldaps://dc.corp.example:636' \
  --bind-user 'CORP\svc-zegress' \
  --base-dn 'DC=corp,DC=example' \
  --name-scope CORP \
  --upn-rewrite corp.example=example.co.uk

Run AD/LDAP correlation once

Runs correlation immediately using the tenant's stored config. Useful straight after set-ldap, to materialise edges without waiting for the next refresh.

export ZEGRESS_MASTER_KEY='<base64-master-key>'
zegress-control --postgres-url "$ZEGRESS_POSTGRES_URL" \
  directory sync-ldap --tenant <tenant-slug>

There is no POSIX sync command. POSIX name resolution and declared mappings run only inside the workers' automatic refresh, which you trigger implicitly by storing config and letting it run.

Automatic refresh

Once Graph or LDAP config is stored, the workers keep the alias graph fresh on their own, running a directory-sync refresh roughly every ten minutes. Each refresh runs declared mappings and POSIX name resolution for every tenant, always — and Entra Graph and AD/LDAP discovery for every tenant with stored config, but only when the worker has the master key set, since it needs the key to decrypt stored secrets.

Managing directory enrichment

A candid view of what each surface can do:

In practice the surfaces compose: set the non-secret config over the API or CLI, then store the credential once with set-graph or set-ldap. The API response reports whether each secret is present, so you can tell if a block is still waiting on one.

curl -X PATCH -H "Authorization: Bearer $ZEGRESS_TOKEN" \
     -H "Content-Type: application/json" \
     "$ZEGRESS_URL/api/tenants/<tenant-slug>/directory" \
     -d '{
       "graph": { "entra_tenant_id": "<entra-tenant>", "client_id": "<app-client-id>" },
       "ldap":  { "url": "ldaps://dc.corp.example:636", "bind_user": "CORP\\svc-zegress",
                  "base_dn": "DC=corp,DC=example", "name_scope": "CORP",
                  "upn_rewrite": { "corp.example": "example.co.uk" } },
       "principal_mappings": [
         { "from": "oid:<issuer>:<group-guid>", "to": "sid:CORP:S-1-5-21-…", "confidence": "high" }
       ]
     }'

This endpoint merges only the blocks you send, leaving issuer, audience and sibling blocks untouched — unlike a wholesale tenant patch, which replaces the config entirely. A has_graph_secret of false means the Graph config is in place but inert until you store the encrypted client secret.