Why patch-tile adapter, Ξ±=1 fixed, smooth Ξ±-ramp curriculum, single canonical trim β and what was tried and rejected. Companion to docs/inn_arch_notes.md + docs/arch_notes.md.
Carry an 896-bit data payload (ECC-encoded to 1024 b) in an RGB cover image. Training is fixed at 256Γ256 for compute budget reasons; the deployment target is real photos at native capture resolution (12 MP-class / 1024Β²+), and the patch-tile design is resolution-scaling by construction (Β§3).
| requirement | target | deployed (idea097/d001) |
|---|---|---|
| bit accuracy | β₯ 0.99 on WeChat chain @ 1024Γ1024 (worst real-platform pipeline at native capture scale); β₯ 0.95 floor across the full 14-attack Γ 7-size sweep. Crop / rotate / large resize / noise / blur = reject by design. | 0.992 on chain_wechat @ 1024Β² (β target); avg 0.995 / min 0.955 over 7-size sweep; 0.999 / 0.992 at 1024Β² alone |
| container fidelity vs cover | indistinguishable to human eyes β both high PSNR and high SSIM (PSNR alone misses structural artefacts that SSIM catches) | PSNR 37.4 dB Β· SSIM 0.940 (clean, 1024Β²) |
| inference | β€ 1 s per 12 MP image on a flagship phone NPU (~17 TFLOPS fp16) | ~25 ms est (12 MP fp16) β 40Γ under budget |
| model size | β€ TrustMark Q (21 M params, 85 MB) | 0.167 M params Β· 0.7 MB fp32 (130Γ under) |
"mild blur" was an earlier draft target. The current threat model is compression-only (see Β§6); blur is intentionally out of scope alongside crop / rotate / noise. The eval suite reflects this β no blur, no Gaussian noise, no geometric attacks.
Two halves of a coupling INN are invertible only if both halves of the output are kept. Steganography drops half (z) at the encoder; the verifier supplies zeros and the network learns to make the bits recoverable from the container alone.
EMBED (camera) EXTRACT (verifier)
ββββββββββββββββββββββββββββββ ββββββββββββββββββββββββββββββββββββ
bits ββ β PatchBitAdapter (fixed) β attack β PatchBitAdapter (fixed) β ββ bits'
host β β Haar DWT β ββββ β Haar DWT (same) β ββ container
β concat host_dwt β secret β β concat c_dwt β zeros (z=0) β
β DKiS INN β 8 or 16 fwd β β DKiS INN β 8 or 16 reverse β
β split β keep 1st half β β split β keep 2nd half β
β inverse Haar DWT β β β
βββββββββββββββ¬βββββββββββββββ ββββββββββββββββββββββββββββββββββββ
βΌ
container ββββ transmission / attacks ββββββββββ
Edges and texture (LH / HL / HH subbands) have low energy in natural images and survive JPEG well β ideal hosts for low-amplitude bit signal. LL is left mostly untouched so the cover's structure is preserved. Haar DWT is reversible by construction with zero learnable params.
Strict invertibility by construction: x1, x2 = INN_reverse(y1, y2) is exact arithmetic, no learning involved. The decoder is the encoder run backwards β no asymmetric learned encoder/decoder pair to fall out of sync.
DKiS originally zero-initializes each subnet's conv5 output layer, making the INN start as exact identity β f(x2) = g(x2) = h(x2) = 0, so the secret never reaches the host half and bit gradients can't escape the BCE = ln(2) fixed point. INNCodec.__init__ re-initializes all conv5 weights with kaiming_normal Γ 0.1 so residuals start small but non-zero. Load-bearing for the network to ever leave the random-prediction floor.
Bits β tensor by construction. The hardest design decision in the project, and the one that has changed most times.
This section is the v3-base design rationale β the choices that led to the deployed model. The deployed
carrier (idea097/d001) carries three small extensions on top of this base; see the
"Deployed extensions" sub-section below for what diverges, and
model Β§ 2 for the paper-exact formal spec of the deployed pipeline.
| step | operation (v3-base, design rationale) |
|---|---|
| 1. bipolar | bits β {0,1}^N β Β±1 |
| 2. grid pad | place first N of 1024 slots in a 32Γ32 grid; pad unused slots with 0 |
| 3. scramble | fixed sign-mask multiply (deterministic PRNG) |
| 4. tile | nearest-neighbor up-sample to DWT spatial dims (H/2, W/2) |
| 5. broadcast | repeat across 12 DWT channels β secret (B, 12, H/2, W/2) |
| β decode | mean-pool over patch + sign-mask invert + sign-threshold β bit logits |
The deployed carrier (idea097_design001) trains and serves at N = 1024 (full grid; secret_len=1024 in train_args.txt) β the inner BCH(127,99) + outer RS(128,100) ECC codeword fills every slot, no padding. The 896-bit MWIP data payload is wrapped by that ECC before hitting the adapter; from the adapter's view the channel is 1024 b. The N=896 path (use 896, ignore 128) is the earlier no-ECC pre-image; PatchBitAdapter still defaults to n_bits=896 in source, but train_args at load-time overrides it.
The v3-base adapter round-trip is mathematically perfect (validated on smoke test). On the v3-base path, bit recovery is bottlenecked solely by the INN's robustness to attacks, not by an asymmetric learned encoder/decoder pair. The deployed extensions below relax the strict "zero-parameter" framing for ~50 k extract-side params; see below.
| property | patch tile (v3-base) |
|---|---|
| Adapter params | 0 on the embed side; ~24 on the deployed extract side (the idea031 expert bank, see below) |
| Easy collapse via adapter | no (bipolar signal is fixed; bank is initialized to a meaningful prior and lightly regularized) |
Deep collapse via INN f/g/h | possible in principle but requires many params to align β much harder |
| Spatial structure of bits | uniform coverage by construction |
| Robustness mechanism | explicit β averaging over patch (repetition code) |
| JPEG alignment at 256Β² training | exact at 8Γ8 patches = JPEG blocks |
| Known tradeoff | INN must do all perceptual hiding β may yield lower PSNR than a learned encoder/decoder pair would |
Earlier drafts paired the INN with a learnable ~5.4 M-param Linear adapter; that path collapsed (weights β 0 under attack pressure) and was dropped. The deployed model has no such adapter β patch tile only.
The deployed carrier (autohide/.../idea097_design001/code/inn_model.py, verified against
train_args.txt: --num_blocks 8 --subnet_type light --secret_len 1024 --no-amp)
generalises three pieces of the v3-base picture above. The rationale is preserved; the implementation
is richer. The full formal spec is in model Β§ 2 β what follows is the
one-paragraph "why" for each extension.
| extension | v3-base picture | deployed behavior | why |
|---|---|---|---|
| 12 balanced permutations encode side |
nearest-neighbor tile, broadcast across 12 DWT channels β every channel carries the same bipolar copy | each DWT channel c β {0..11} gets its own deterministic balanced permutation
Οc that scatters slot values to disjoint pixel positions. Seed 0xD1FF5E β c β (hΒ·1000003 + w). |
bit redundancy now spreads across both space and DWT channel; per-channel decorrelation lets the coupling INN move secret energy into whichever subbands have local headroom rather than enforcing the same spatial pattern everywhere. |
Subband-weighted MoE decode (idea031)decode side, +~12 k params |
uniform mean across 12 channels | per-batch decode-weight vector wc = router(xΜ) Β· expert_bank with a learnable
(K=2, 12) bank initialised to a [4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1, 1] / sum
subband prior (LL Γ 4, LH/HL Γ 2, HH Γ 1) and a small router conv stack reading the container RGB. |
JPEG / WeChat low-pass crushes HH harder than LL. The fixed subband prior already encodes that; the MoE adds an attack-conditioned tilt (two experts) so the decode can lean further into LL when the attack signature looks lossy. |
DWT cover predictor (idea034)decode side, +~35 k params |
reverse INN runs on concat[c_dwt, 0] |
a small 3-block residual head estimates host_dwt_est from the container RGB; the
reverse INN sees c_dwt_aug = c_dwt + 0.5 Β· (c_dwt β host_dwt_est) instead of c_dwt. |
amplifies the residual that carries the bit signal before the INN tries to recover it. The
predictor is zero-init at the head, so step 0 is mathematically identical to v3-base
(host_dwt_est = 0 β c_dwt_aug = c_dwt); training only ever improves on the base. |
| crop | patch RGB | patch DWT | pixels/bit (RGB) |
|---|---|---|---|
| 256Γ256 | 8Γ8 | 4Γ4 | 64 |
| 512Γ512 | 16Γ16 | 8Γ8 | 256 |
| 1024Γ1024 | 32Γ32 | 16Γ16 | 1024 |
| 12 MP (β4032Γ3024 β 4032Γ3008 after trim) | ~126Γ94 | ~63Γ47 | ~12 000 |
Grid fixed at 32Γ32 = 1024 slots regardless of input resolution. The deployed carrier uses all 1024 slots (ECC codeword fills the channel; the 896-b data payload is wrapped by BCH + RS before the adapter sees it). Patch size scales with the input. Requires H, W divisible by 64 (so DWT dims % 32 == 0).
Ξ± scaling the patch tile. Codex flagged it as a hidden collapse path β under attack pressure the optimizer can drive Ξ± β 0, recreating v1's "embed nothing" failure mode with one parameter instead of 5.4 M. Dropped. Reintroducing Ξ± as a fixed schedule (ramp 0.1 β 1.0) is listed in Β§11.
Fixed pseudo-random Β±1 sign-mask applied to patch values before tiling. Deterministic from seed; verifier knows it. Zero learned params, breaks the bit-index-to-image-region coupling (otherwise bit #0 would always be top-left and an attacker could zero out that corner).
Pre-tile sign-mask is not spread-spectrum β it's a permutation/scrambler. Approaching SS-style localized-erasure robustness would require pseudo-random Β±1 within each patch instead of constant; that's ablation 6 in Β§11.
Replaces the 3-stage step function (clean β low β high) that collapsed runs 1 and 2 at the discrete AβB boundary at step ~5,800.
| step range | Ξ± | what happens |
|---|---|---|
| 0 β 5,000 | 0 | clean β attack module is skipped entirely |
| 5,000 β 15,000 | linear 0 β 1 | each kornia child op fires at p_max Β· Ξ±; encoder sees a continuously-increasing fraction of attacked batches |
| 15,000 β 30,000 | 1 | full target severity |
A single AttackLayer(severity="high") instance is shared throughout. NoiseCurriculum.attack(step) mutates each child's Bernoulli p in place. At step 5,800 (the historical collapse point) Ξ± is now ~0.08 instead of 100% β the encoder sees ~8% of batches with attacks rather than 100%.
relu(rms_floor β βcontainer β coverβ_rms) β outlaws the basin in the loss landscape rather than delaying the slide. Optional companion: bit-weight ramp Γ2β5 across the Ξ±-ramp window.
Runs 1 (56248029) and 2 (56250090) both collapsed at the same AβB step-function boundary at ~step 5,800 in 2026-04-25:
| run | job | stage A end (~4,600) | post-transition (~5,800) |
|---|---|---|---|
| 1 | 56248029 | bit_acc 1.0, PSNR 49 | bit_acc 0.5, PSNR 63, secret pathway killed (no diagnostic yet) |
| 2 | 56250090 | bit_acc 1.0, PSNR 49, secret_rec_rms 93 | bit_acc 0.5, PSNR 64, secret_rec_rms 0.019, secret_rec_err_rms 0.94 |
Run 2 confirmed via the new diagnostics that the recovered secret tensor was driven 5,000Γ toward zero in the very first 1,200 steps after attacks turned on β unambiguous "embed nothing" basin.
| op | Q range / scale | Bernoulli p |
|---|---|---|
resize (down-then-up) | scale ~ U(0.5, 1.0) | 0.5 |
jpeg1 | Q ~ U(40, 95) | 0.9 |
jpeg2 (chained) | Q ~ U(20, 40) | 0.3 |
Forward order: resize first, then JPEG β matches platform pipelines. Output spatial dims preserved by construction. Geometric (crop, rotate) and other photometric (blur, noise, color jitter, grayscale) attacks intentionally absent.
Logged per --log_every step. Both secret_rec_rms and secret_rec_err_rms were added 2026-04-25 to catch embed-nothing collapse the moment it starts.
| metric | healthy | collapse signal |
|---|---|---|
bit_acc, bit_acc_ema | β 1.0 | flat at 0.5 |
psnr(container, cover) | 30β35 dB | rises past ~50 dB = container β cover, secret discarded |
l1_container_cover | ~0.01β0.02 | drops toward 0 with bit_acc still 0.5 |
secret_rec_rms | large + stable through ramp (run 1 saw ~93 in clean phase) | drops toward 0 = INN killed the secret pathway |
secret_rec_err_rms | small relative to secret_rec_rms | rises toward secret_rms (~1.0) = recovered tensor uncorrelated |
alpha | matches curriculum schedule | discontinuities = curriculum bug |
img_loss, bit_loss | finite, decreasing | NaN |
Compression-only. The training and eval attack suites are limited to canonical-delivery transforms β resize + JPEG + JPEG chains. Geometric attacks (crop, rotate) and other photometric attacks (blur, noise, color) are out of scope by design.
Defaults flipped to (num_blocks=8, subnet_type="light") on 2026-04-28. The legacy 16/RDB baseline is still reachable; eval auto-detects the architecture from saved args, so 16/RDB checkpoints still load.
| backbone | blocks | subnet | params | FLOPs vs baseline | status |
|---|---|---|---|---|---|
| 16 / RDB | 16 | 5-conv dense (RDB) | 4.05 M | 1Γ | legacy baseline. collapses on Ξ±-ramp at Ξ±β0.16 |
| 8 / RDB | 8 | 5-conv dense | 2.03 M | 0.5Γ | ablation: blocks only |
| 16 / light | 16 | 2-conv flat (Light) | 0.334 M | ~0.08Γ | ablation: subnet only |
| 8 / light | 8 | 2-conv flat (Light) | 0.167 M | ~0.04Γ | deployed mobile target β survived Ξ±-ramp without modification |
LightConvBlock: Conv2d(inβ32, 3Γ3) β LeakyReLU β Conv2d(32βout, 3Γ3). No dense concat β flat input/output channel profile that NPUs pipeline well (Apple Neural Engine, Qualcomm Hexagon). Per-block FLOPs ~12Γ lower than RDB at the same in/out channels.
Combined with halving the block count: ~24Γ FLOP reduction, ~24Γ param reduction.
Hyperparameters used by the deployed run (idea097_design001, in sibling autohide repo): lr=4e-4, bit_weight=20.0, img_weight=1.0, --no-amp, single-resolution 256Β² training, 30,000 steps. Curriculum: stage A 0β5,000 clean, stage B 5,000β15,000 Bernoulli Ξ±-ramp, stage C 15,000β30,000 at Ξ±=1 (autohide defaults; unchanged). The imagehide tree's multi-res sampler (pixel-budget 1.0 M, DPED + CLIC + COCO/DIV2K) is the in-flight research direction (idea084/d001 β idea112/d003), not what the deployed checkpoint used.
Single canonical center-trim. Verifier rejects images that don't fit the trim convention rather than searching all 1024 shifts. Codex tightening β preserves the false-accept surface.
1. capture full-res RGB image (W Γ H), e.g., 4032 Γ 3024
2. center-trim to (W', H') where W', H' are floor-divisible by 32:
W' = (W // 32) * 32 β 4032 (already divisible)
H' = (H // 32) * 32 β 3008
trim symmetrically: drop (W β W')/2 from each of left+right, similar top+bottom
keep the trimmed strips set aside (unmodified)
3. apply INN embed at (W', H') with 32Γ32 grid β patches sized (W'/32, H'/32)
4. reattach the trimmed strips around the watermarked region
resulting image is original (W Γ H) with a watermarked W' Γ H' interior
the strips (β€ 31 px wide) are unwatermarked but pixel-identical to original
5. JPEG-compress, save
1. receive image of dimensions (W Γ H) 2. validate: W and H are within the declared canonical-delivery profile (natively-captured camera resolution; reject crops/resizes that change W or H) 3. center-trim to (W', H') using the same floor-divide-by-32 convention 4. apply INN extract at (W', H') β 1024 logits β take 896 β un-scramble β BCE > 0 5. proceed with H-mismatch check (Hamming < Ο) and Ed25519 verification
From the two-round codex review (2026-04-25) and the design's history. Listed so future iterations don't redo a settled rejection without new evidence.
The CNN baseline we left behind. TrustMark is payload-agnostic via two Linear layers (encoder input, decoder output); a payload sweep just resizes those.
| component | role |
|---|---|
Secret2Image.dense | Linear(secret_len β 16Β·16Β·3) β reshape β upsample β concat with cover β U-Net encoder. Payload-dependent. |
SecretDecoder.decoder.fc | final FC of the backbone (ResNet-18/50/DenseNet), resized to Linear(fc.in_features β secret_len). Payload-dependent. |
| everything else | U-Net conv backbone, ResNet50, discriminator, noise layer, loss β all payload-independent |
| payload | final bit_acc_ema | stage | note |
|---|---|---|---|
| 100 b | 0.981 | 2 | comfortable |
| 256 b | 0.913 | 2 | marginal |
| 512 b | 0.795 | 0 (stall) | cliff |
| 1024 b | 0.682 | 0 (stall) | random+ |
| 2048 b | 0.623 | 0 (stall) | random+ |
Secret2Image is the binder β for β₯ 1024 b we'd need to widen to 32Γ32Γ3 (one full sweep). This is the reason for the INN pivot.
The secret projects through a 16Γ16Γ3 = 768-dim spatial bottleneck before entering the U-Net. For 2000 bits that's 0.38 dims per bit. For 4000 bits, 0.19. Independent of how many parameters the linear layer has, this caps capacity. Options were:
Secret2Image output spatial map (16Γ16Γ3 β 32Γ32Γ3) β untested, another full sweepCodex flagged these as load-bearing β none of the design's robustness claims can be settled without them.
| # | ablation | question |
|---|---|---|
| 1 | Patch adapter under noise | Does the v3 patch actually hold under noise? The whole motivation. |
| 2 | Deep-collapse detection | Watch L1(container, cover) per Β§5. If it nosedives at any stage transition while bit_acc stays at 0.5, INN f/g/h are collapsing toward identity. Kill criterion to be set empirically. |
| 3 | Crop-size ablation | Train at one size, eval at {256, 384, 512, 768, 1024}. If 768/1024 fail clean, the receptive-field ceiling is real and deployment scope narrows. |
| 4 | JPEG phase sweep | All 64 phase shifts Γ q β {20, 30, 40, 50}. Measure actual misalignment penalty. |
| 5 | Trim-search vs single-canonical FA | "Single fixed center-trim" vs "try all 1024 trim shifts and accept if any decodes." Report FA delta. |
| 6 | PR Β±1 within patches vs constant | SS-like (PR Β±1 spreading code per patch) vs current (constant Β±1), matched energy. Test under localized erasure. |
| 7 | Patch grid size | 16 / 32 / 64 β capacity vs robustness tradeoff. 32 is default. |
| 8 | Mean-pool vs learned Wiener | A 1Γ1 conv pre-pool could weight-pool optimally if learned. Adds params; might recover lost SNR. |
| 9 | Per-channel sign mask | 12 different DWT-channel sign masks instead of broadcasting one mask. Cross-channel redundancy. |
| 10 | Fixed Ξ± schedule | Ramp 0.1 β 1.0 (vs current Ξ± = 1 throughout). Preserves "start gentle" without the learnable collapse path. |
| 11 | INN-as-adapter | Replace patch-tile adapter with a small 4-block coupling INN. Tied weights, ~200 k params. Probably overkill given the 0-param patch design works. |