Mesh specification

[glyg]

As discussed in the feb. 2021 ngff community call, and following this image.sc thread

The idea is to follow PLY specification to store meshes in ome-zarr. A ply file is organised in:

• a header (translated in the .attrs of the zarr group)
• the points (a zarr.Array)
• for each element (e.g. faces) the list of vertices. Those are group by number of sides, with one zarr.Array per polygon.

There is a draft implementation here: https://github.com/centuri-engineering/ply-zarr

Some questions:

• I don't know how to deal with binary formats
• PLY allows for datatype specification at the column level, how to deal with that in zarr?

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/mesh-data-in-ome-zarr/44653/15

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/next-call-on-next-gen-bioimaging-data-tools-feb-23/48386/9

[joshmoore]

cc: @jfkotw @kephale who were also interested during the meeting. I defer on whether or not this issue covers all of "vector-based".

[glyg]

I'd argue for GEOjson for ROIs and points and such & keep meshes in their niche

[joshmoore]

@glyg, so this block from ply-zarr is the critical bit for discussion?

ply_header = {
    "format": "ascii 1.0",
    "comments": [f"created by ply_zarr v0.0.1, {datetime.now().isoformat()}",],
    "elements": {
        "vertex": {
            "size": 47,
            "properties": [
                ("double", "x"),
                ("double", "y"),
                ("double", "z")
            ]
        },
        "face": {
            "size": 105,
            "properties": [
                ("list", "uint8", "int32", "vertex_indices"),
            ]
        }
    }
}

[glyg]

Yes, this mirrors the specification for the PLY header, then it seems natural to store the faces in separate arrays according to their number of sides.

[glyg]

see a more concrete example of mixing meshes, images and labels here

I assume the xarray compatibility also applies here, I'll look into that next.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/ngff-status-update-may-2021/52918/1

[jburel]

cc @normanrz

[normanrz]

We recently implemented the mesh format from Neuroglancer in webKnossos: https://github.com/google/neuroglancer/blob/master/src/neuroglancer/datasource/precomputed/meshes.md

It's been great for our purposes:

• Supports billions of meshes
• Uses draco encoding, which is very storage-efficient
• Supports chunked mesh loading, which is important for very large objects
• Supports multi level-of-detail for progressive loading

I think that format would be a great candidate to be adopted by OME-NGFF.

[glyg]

@normanrz thanks for the input, those features indeed sound great (esp. multi-res!).

If I understand correctly though, only triangular meshes are supported? The other consumer / producer of meshes is the modeling community (i.e. physical biology), who would need more generic meshes, for example with polygonal (>3) 2D cells, polyhedral 3D cells, or even quadratic tetrahedra.

Would draco be able to handle that kind of data?
How would a zarr implementation work? Is it enough to "just" put the draco encoded in the store and add a dependency to be able to read / write them?

Also, maybe storing generic FEM meshes is out of scope for ome-ngff and triangles are enough.

[normanrz]

If I understand correctly though, only triangular meshes are supported? The other consumer / producer of meshes is the modeling community (i.e. physical biology), who would need more generic meshes, for example with polygonal (>3) 2D cells, polyhedral 3D cells, or even quadratic tetrahedra.

Yes, I think draco only supports triangular meshes (and point clouds). We could look into allowing other encodings in addition to draco.

How would a zarr implementation work? Is it enough to "just" put the draco encoded in the store and add a dependency to be able to read / write them?

That is a good question that we haven't fully figured out yet. We currently store all the data in a single binary file. The file consists of a) a directory structure (hash map) to locate the meshfile for a specified id within b) a long blob of mesh data. In b) each meshfile has a binary metadata header that describes the available chunks and level-of-details.
One implementation on top of zarr would be to store each meshfile as on chunk (e.g. in a 2D uint8 array). This would create a lot of chunk files and might create some issues, because the chunks will have different byte lengths.