Skip to content

Comparing and fingerprinting persisted schemas #8123

Open

Description

@tenkoverse

What is the problem this feature would solve?

I’d like to persist a code defined Effect schema in a database, then later compare the schema in code against the stored version to detect changes and reject unexpected differences.

The comparison should tolerate differences in layout while detecting changes to the schema’s structure or constraints. Stable fingerprints would also be useful for this workflow.

The smallest example I’ve tested is reordering struct fields. Both direct comparison and SHA-256 hashing of the serialized representations treat these as different:

import * as NodeCrypto from "@effect/platform-node/NodeCrypto"
import { assert, it } from "@effect/vitest"
import {
  Crypto,
  Effect,
  Encoding,
  Schema,
  SchemaRepresentation,
} from "effect"

const A = Schema.Struct({
  name: Schema.String,
  active: Schema.Boolean,
})

const B = Schema.Struct({
  active: Schema.Boolean,
  name: Schema.String,
})

const represent = (schema: Schema.Constraint) =>
  SchemaRepresentation.toJson(
    Schema.toRepresentation(schema),
  )

it.effect("reordering fields changes representations and fingerprints", () =>
  Effect.gen(function* () {
    const crypto = yield* Crypto.Crypto

    const fingerprint = (representation: Schema.Json) =>
      crypto.digest(
        "SHA-256",
        new TextEncoder().encode(JSON.stringify(representation)),
      ).pipe(Effect.map(Encoding.encodeHex))

    const a = represent(A)
    const b = represent(B)
    const hashA = yield* fingerprint(a)
    const hashB = yield* fingerprint(b)

    // Both assertions pass with the current behavior.
    assert.notDeepEqual(a, b)
    assert.notStrictEqual(hashA, hashB)
  }).pipe(Effect.provide(NodeCrypto.layer)),
)

Verified with Effect and @effect/platform-node 4.0.0-rc.112 on Node 26.8.1.

For this use case, I’d want these schemas to compare as unchanged and produce the same fingerprint.

What is the feature you are proposing to solve the problem?

Support for reliably comparing and fingerprinting persisted schema representations, with documented guarantees about which differences are significant.

Field ordering is the smallest example motivating the request, rather than the entire scope. I’m interested in what guarantees are practical, which schema features could be supported, and where explicit limitations would be necessary.

An equivalence API or a canonical representation might support this, but I don’t have a particular API or implementation in mind. Is there an existing recommended approach, or would supporting this workflow fit within Effect’s scope?

What alternatives have you considered?

No response

Activity

  1. tenkoverse commented on Sep 9, 2026

    @tenkoverse
    Author

    @gcanti 👀

  2. spencerbeggs commented on Sep 11, 2026

    @spencerbeggs
    Contributor

    @tenkoverse I have a parallel use case in my utility repo's JSONC parser. My implementation is tailored to my (admittedly) harebrained needs (caching Twoslash renders), but the internals look similar. I added them to my utility package for basically the same reason.

  3. tenkoverse commented on Sep 16, 2026

    @tenkoverse
    Author

    fyi #8052 looks like the same general problem from the opposite direction.

  4. ArjunCodess commented on Sep 19, 2026

    @ArjunCodess

    hi, i'd like to work on this.

    right now field order in a struct changes both the json and the sha-256. i'd add a canonical representation that sorts object keys, then fingerprint that, so A and B compare equal. constraints still stay in the hash.

    should reorder-only be the v1 guarantee, and do you want Schema.equivalence on top of that or just the fingerprint helper?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions