Skip to content

A big pile of imporovements#10

Merged
tomba merged 44 commits intomasterfrom
test
Apr 27, 2026
Merged

A big pile of imporovements#10
tomba merged 44 commits intomasterfrom
test

Conversation

@tomba
Copy link
Copy Markdown
Owner

@tomba tomba commented Apr 23, 2026

Main things:

  • Full stride support
  • More formats supported
  • A big improvement to tests
  • Fixed RGB & YUV opencv conv issues

tomba added 30 commits April 23, 2026 13:33
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
libcamera uses its own names which do not always match pixutils' names.
And trying to match based on fourccs is challenging. So let's add a new
field, 'libcamera_name', so that we can explicitly map between libcamera
and pixutils formats.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
The 'packed' field was a bit too ambiguous.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Add a helper to get the bitspp for RAW formats.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Disable GC for the run, do a bit more warmup, use perf_counter.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Start fixing the multi-plane stride issues by fixing the public API.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
If the caller gives an integer as the stride, and the format is a
multi-plane format, use similar stride extrapolation as libcamera does.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Many of them were quite wrong...

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Convert a random buffer to BGR888 with each available backend (opencv,
numba, numpy) and compare the outputs pairwise. Catches bugs such as
channel swaps, wrong plane offsets, or off-by-one unpacking that locked
per-backend SHAs in the existing test cannot detect.

Comparison uses max-per-channel mean, 99th percentile, and max of the
per-pixel absolute difference. Per-channel (not aggregate) mean is the
key: a channel swap leaves one channel near zero while the other two
diverge by ~85 on random data, which stands out even when the aggregate
mean overlaps with legitimate algorithmic differences (e.g. different
demosaic methods). Tolerances are split by color class (RGB exact, YUV
a few LSB, RAW loose).

Test discovery probes each format at 32x32 with a shared oversized
buffer and emits test methods only for formats where >=2 backends
accept the conversion, keeping the reported skip count at zero.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Allocate the output once and do the channel swap into it directly. The
alpha-drop paths use a slice-copy instead of np.delete, which is
simpler and avoids the strided fancy-indexing. Results are now
naturally C-contiguous, and gives a nice increase in perf.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Shift the intermediate 16-bit array in place and then cast, instead of
allocating a shifted copy and then casting. Saves one full-size buffer
allocation in the RAW conversion path.
np.clip() allocates a new array by default; passing out= makes it
modify the dot-product result in place. Avoids one intermediate
float-array allocation per frame.
All three planes are overwritten immediately after allocation, so
np.zeros() needlessly zero-fills the buffer. np.empty() skips the
fill.
tomba added 14 commits April 23, 2026 13:33
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Camshark calls internal conversion functions with an integer stride,
which fails now. Add a hack to convert the stride-int to a stride-tuple.
Remove later.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
In most cases this should be no-op, but it makes sure we will handle a
non-contig and shaped arrays correctly too.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Make numpy/numba converters return None for unsupported formats so that
the top-level conv.py can manage unsupported formats in one place.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Catching any exception hides actual issues.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
The earlier check did the job, but it wasn't quite correct either.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
@stefanklug
Copy link
Copy Markdown
Collaborator

@tomba I skimmed through the changes and that looks all sane. I think we could take the risk of just merging it :-)

@tomba tomba merged commit 0decb81 into master Apr 27, 2026
6 checks passed
@tomba
Copy link
Copy Markdown
Owner Author

tomba commented Apr 27, 2026

@tomba I skimmed through the changes and that looks all sane. I think we could take the risk of just merging it :-)

Risk on!

Comment thread pixutils/conv/conv.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants