GNM for Unity

Unity support for Google's GNM parametric human models: import the model data, bake it into compact runtime assets, evaluate identity, expression and pose in-engine.

Starts with GNM Head. Built to extend to body and perception as Google releases them.

Unaffiliated community port. Not endorsed by or associated with Google LLC. See NOTICE.

_{GNM head v3.0 evaluated in-engine, flat colours and no textures.}

Install

Unity 6000.0+. In Window > Package Manager > + > Install package from git URL:

https://github.com/fake-skate/gnm-unity.git

Or add to Packages/manifest.json directly:

"com.fakeskate.gnm": "https://github.com/fake-skate/gnm-unity.git"

Append #<tag-or-sha> to pin a revision. Without one, Unity tracks the default branch and will pick up changes on any resolve.

Then, before anything will render:

1. Window > GNM > Model Data > Download and Bake All. Fetches 53 MB from the pinned upstream commit, verifies its checksum, and bakes it into Assets/GNM/. One-time, a couple of minutes.
2. Window > GNM > Rebuild Sample Scene. Generates Assets/GnmSample/ with a GNM Character, per-component materials and a gaze target, and leaves it open.

The scene is generated rather than shipped. A .unity file is opaque YAML full of GUIDs, and the baked model it would have to point at is produced locally on each machine, so a committed scene would arrive with a broken reference. GnmCharacter resolves a baked model itself.

Neither the 53 MB source data nor the baked blob belongs in version control. The fetched source lives in GnmModelData/, beside Assets/; the bake writes one folder per model under Assets/GNM/. Ignore those, not Assets/GNM/ wholesale, because materials land in Assets/GNM/Materials/ and are ordinary project assets worth keeping:

GnmModelData/
Assets/GNM/head-*/
Assets/GNM/head-*.meta

Status

Early but working, and verified against the reference implementation. See below.

Component	Upstream	Here
Head (gnm/shape)	released	fetch, bake and shape evaluation working
Body	planned, no date	schema-ready
Perception (mesh/photo → params)	planned, no date	not started

Working end to end: install from git URL, one-click download and bake, then a GnmCharacter component driving identity, expression, head pose and gaze onto a Mesh with per-component submeshes.

Not yet: a SkinnedMeshRenderer path, the baked identity/expression preset table, and a GPU evaluation path.

Performance

A head costs 2.72 ms per frame in the worst case, about a sixth of a 60 fps budget, and that is with every parameter changing at once. What a frame actually pays depends on what moved. Measured on a Ryzen 5 5500, GNM head v3.0, 17,821 vertices:

What changed	Cost per frame
Nothing	0 ms
Head pose only	0.33 ms
Gaze target moved	0.35 ms
Expression, fixed identity	0.96 ms
Identity, fixed expression	1.85 ms
Everything at once	2.72 ms

These are alternatives, not additions: a frame pays one row, not a base cost plus an increment. Identity is the expensive input and it is normally set once at character creation, so a character that only talks and looks around never pays the bottom two rows.

Treat these as prototype figures. They are one head on one machine, evaluated on the CPU; there is no GPU path yet, and the cost is per character, so a crowd is a different problem from the one measured here.

All 253 identity and 383 expression components are kept, so no degree of freedom is lost against the reference model. Values are quantised to int16, costing a worst-case 0.001 mm on a head spanning 340 mm.

Profiler markers are published under Gnm., so filter on that in Window > Analysis > Profiler.

Watch graphics memory on your own target. The mesh is rewritten every frame, and Mesh.MarkDynamic() is the obvious thing to reach for. Under D3D12 it grew graphics memory without bound, roughly 1.5 MB per rebuild, until the head stopped rendering. It is deliberately not used and the reason is commented where someone would otherwise add it back, but a per-frame mesh upload path is worth watching on whatever backend you ship.

The mesh must also carry tangents. HDRP's Lit shader with tangent-space normal mapping builds its shading basis from them, and without a tangent stream the surface shades black rather than merely losing normal-map detail.

Coordinate conventions

GNM is right-handed, +Y up, +Z forward, with the character's left at +X. Unity is left-handed with left at -X, so:

• positions are mirrored in X (GnmMeshTopology.ToUnity)
• triangle winding is reversed to compensate for the mirror, otherwise the mesh is inside-out and invisible under backface culling
• source triangles are counter-clockwise with outward normals (verified: per-vertex normals accumulated from CCW faces agree with the model's own compute_vertex_normals on 100% of vertices)

Scale is metres already; the neutral head and neck span about 0.34 units.

UVs are stored per triangle corner, and 614 vertices carry different UVs depending on the triangle. Unity allows one UV per vertex, so those are duplicated: 17,821 model vertices become 18,437 render vertices. GnmMeshTopology.SourceIndex maps each render vertex back to its model vertex, which is what lets evaluated positions be scattered onto the split mesh.

Design

Everything is written against the schema, not against "head"

The GNM model file is a generic parametric-mesh container. gnm_head.npz holds a template mesh, a PCA identity basis, a PCA expression basis, joints with skinning weights, and a pose-corrective regressor, plus version / variant / mesh_component_names metadata. A body model is the same schema at different dimensions: more joints, larger template, likely no expression basis.

So the importer and the runtime evaluator are dimension-agnostic and treat optional blocks as optional. Runtime/Shape/ handles any GNM shape model; head-specific behaviour is configuration, not code. Google's own layout says the same thing: the package is gnm/shape, not gnm/head.

Model data is fetched, never committed

The upstream gnm_head.npz is 53 MB. It is not vendored. Editor tooling downloads it from a pinned commit (never main, since upstream is moving fast and a shipped plugin must not have bytes change underneath it), verifies a checksum, and caches it outside Assets/. A commit SHA rather than a tag because upstream has published neither tags nor releases.

Apache 2.0 permits redistribution, so this is an engineering choice rather than a licensing constraint. If upstream availability becomes a problem, mirror it.

Source data is an authoring input, not a shipped asset

The .npz never reaches a player build. The importer bakes it into a GnmModelAsset, a small ScriptableObject holding metadata and an array table, plus a .bytes blob holding the numeric data. The blob is separate because Unity writes ScriptableObjects as YAML under text serialization, which would turn an 82 MB basis into an unusable asset file and a very slow import. Blob entries are 16-byte aligned so they can be read back as NativeArray<T> views with no copy.

	Raw	Notes
Expression basis	82 MB (383 × 17821 × 3, f32)	needed at runtime
Identity basis	54 MB (253 × 17821 × 3, f32)	character creation only: bake to mesh, then unload
Everything else	~14 MB	topology, UVs, skinning, correctives
Total	150 MB	53 MB compressed on disk; 143 MiB baked, 42.8 MB after compaction

No component is dropped. The dense bake is lossless, and stays the reference the compact form is verified against. Compaction then removes entries that are exactly zero, which cannot move a vertex, and quantises what remains to int16 against a per-component scale, for a worst-case 0.001 mm on a head spanning 340 mm. Every identity and expression component survives both steps.

Truncating the basis itself is a further step, not taken. PCA components are variance-ordered, so keeping only the leading expression components at fp16 should reach roughly 8 MB, but the cut point is deliberately not guessed at: it will be chosen by measuring reconstruction error against the dense bake. The basis is shared across all characters, so per-character cost is only an identity vector plus an expression weight vector.

Semantic samplers are baked to presets, not run at runtime

Upstream ships two small pre-trained Keras models that turn descriptions into parameter vectors, IdentitySampler.sample_identity(gender, ethnicity) and ExpressionSampler.sample_expression(expression), because the raw 253 identity dimensions are meaningless statistical axes that no player can be handed directly.

Rather than run inference in-engine, sample them offline across the enum space and ship the resulting vectors as a preset table. Blending between presets is a lerp, so the character-creator sliders still work, and the presets can be art-directed, so bad draws get culled. No ML at runtime.

Enums available upstream: Gender (2), Ethnicity (4), Expression (20: surprise, disgust, suck, compress_face, stretch_face, happy, squint, platysma, blow, funneler, smile_wide, corners_down, pucker, wink_left, wink_right, mouth_left, mouth_right, lips_roll_in, snarl, tongue_center).

Head model facts

Vertices	17,821
Triangles	35,324
Mesh parts	skin, L/R eye, upper + lower teeth/gums, tongue
Joints	4: neck, head, left_eye, right_eye
Identity components	253 (170 head + 80 teeth + 3)
Expression components	383
Vertex groups	46 named regions

Gaze is skeletal, not blendshape. left_eye and right_eye are real joints with skinning weights, so gaze is two bone rotations. Drive it from a LookAt against the camera or a target.

Expression components are region-partitioned, not FACS-named. 100 × left_eye_region, 100 × right_eye_region, 1 × pupils, ~182 × mouth_nose_ears. No free semantic mapping, but the partition means the eye/brow region can be driven independently of the mouth.

The 46 named vertex groups (left_brow_region, nose_region, irises, pupils, chin_region, and 41 more) give masking for free.

Texturing

The package ships no textures and the generated GameObject uses flat colours, so it has no texture dependencies. Textures are authored per project. Four tools in Tools~/ work against the model's own UVs and vertex groups:

uv_layout.py	renders the UV layout, one image per component
make_eye_texture.py	generates a UV-correct eye from the pupils, irises and scleras groups
flatten_skin.py	high-passes a skin albedo, dropping low-frequency features and keeping grain
make_skin_features.py	paints lips, nostrils and cheek warmth from the model's vertex groups

Each component has its own 0 to 1 UV space rather than a shared atlas: the per-part areas sum to well over 100%, and the two eyes occupy identical coordinates. So every part needs its own texture, and each gets the full resolution of that texture.

Placement that has to line up with the geometry can be read from the model instead of eyeballed. The vertex groups are defined on the template, so anything derived from them stays correct for every identity.

GNM_PREVIEW_TEXTURES=<folder> loads <component>.png or albedo_<component>.png into the preview for checking. Components with no texture fall back to a flat colour, so a partial set still renders.

Layout

Runtime/
  Core/     model asset and array table types
  Shape/    parametric mesh models: PCA evaluation, LBS skinning, pose
            correctives, mesh build, GnmCharacter
Editor/
  Core/     pinned-commit fetcher, .npz reader, bake and compaction,
            self test, evaluator and gaze verification, render preview
  Shape/    GnmCharacter inspector
Tools~/     numpy reference oracles and the texturing tools
            (excluded from Unity by the ~ suffix)

Verifying

Arrays are addressed by upstream name and never mapped onto fixed fields, so the checks below apply unchanged to any GNM model, not just the head.

Confirming that data merely loads proves very little: endianness, offset, stride and alignment mistakes all yield arrays of exactly the right shape full of wrong numbers, which surfaces much later as a subtly deformed mesh. So the summaries are diffed against numpy.

# in Unity: Window > GNM > Self Test > Verify .npz Reader  (or Verify Baked Asset)
python Tools~/selftest_reference.py emit <model.npz> reference.json
python Tools~/selftest_reference.py compare reference.json <selftest.json>

Headless, which is how it is normally run:

GNM_SELFTEST_OUT=out.json Unity.exe -batchmode -nographics \
  -projectPath <proj> -executeMethod FakeSkate.Gnm.Editor.GnmSelfTest.RunBakeBatch

Endpoints, min and max must match bit-exactly. Sums carry a 1e-9 relative tolerance because C# accumulates sequentially in double while numpy sums pairwise in float64.

The evaluator is checked the same way, against upstream's own Python model rather than against numpy. This is the check that matters most: a wrong basis stride, a transposed einsum or a row/column-major mix-up all still produce a perfectly face-shaped face, so nothing about this can be validated by looking at it.

python Tools~/eval_reference.py params  --gnm <GNM checkout> --out params.bin
GNM_EVAL_PARAMS=params.bin GNM_EVAL_OUT=unity.bin Unity.exe -batchmode -nographics \
  -projectPath <proj> -executeMethod FakeSkate.Gnm.Editor.GnmEvalVerify.RunBatch
python Tools~/eval_reference.py compare --gnm <GNM checkout> --params params.bin --unity unity.bin

Parameters are generated once by the reference and read from disk by Unity, rather than derived from a formula in both languages. Otherwise a last-ulp difference between C# and Python sin would register as an evaluator mismatch that isn't one. They are drawn from a fixed seed rather than left neutral, since an all-zero parameter vector multiplies every basis component by zero and would hide exactly the indexing bugs this is meant to catch.

Current status on Unity 6000.4.10f1:

• all 23 arrays of gnm_head.npz match numpy, both read directly and round-tripped through a baked asset
• evaluated bind-pose vertices and joints match the reference Python model to 5.4e-07 absolute (1.1e-06 relative to the head's bounding box), which is float32 accumulation noise
• skinned vertices match to 4.2e-07 absolute (8.9e-07 relative)
• 4 gaze targets checked on both eyes aim to within 0.000°, on the correct side, with independent vergence

Conventions need a different check

None of the above can catch a coordinate-convention error: the reference has no notion of Unity's handedness, so a mirrored or inside-out head still compares to 5e-07. GnmRenderPreview covers what's left, writing a front / three-quarter / profile contact sheet:

GNM_PREVIEW_OUT=preview.png Unity.exe -batchmode -projectPath <proj> \
  -executeMethod FakeSkate.Gnm.Editor.GnmRenderPreview.RunBatch

Note this runs without -nographics, which would disable rendering.

The image establishes winding (an inside-out mesh does not shade as a solid) and forward axis. It deliberately does not establish handedness: a neutral face is near-symmetric, so a mirrored head is pixel-indistinguishable. That case is covered by an assertion instead: the left_eye vertex group's centroid must land at negative X in Unity space.

License

Apache 2.0. See LICENSE and NOTICE.