Show4DSTEM - Python widget for real-time 4D-STEM explorer and virtual detectors

[bobleesj]

What this PR does

A combination of react, anywidget, WebGPU to get real-time 4D-STEM explorer.

For developers:

Install Method 1. quantem.widget using conda

conda create -n qt-widget-env python=3.12 nodejs
conda activate qt-widget-env
pip install -e .
cd widget && npm install && npm run build
pip install -e .

Install Method 2. quantem.widget using uv with node already installed locally

uv sync --all-packages
cd widget && npm install
npm run build

API design:

from quantem.widget import Show4DSTEM
dataset = read_4dstem(
    '/home/bobleesj/datalamella_2_002_master.h5',
    file_type='arina'
)
widget = Show4DSTEM(dataset)
widget

Notebooks:

test_show4dstem_arina.ipynb

test_show4dstem_legacy_h5.ipynb

[bobleesj]

@dasol-yoon has agreed to test the API/features. Keeping it as a draft PR since folder clean-up is needed but it's a good starting point since the very basic features are done - virtual detectors, zoom, pan, color map, reset, zip, length, laster animation

[cophus]

Excellent work! I have several comments:
-I think the FFT panel it should be hidden by default (or a separate widget)
-rectangular scans don't work correctly
-we should add automatic center spot detection for the virtual detectors
-controls next to the appropriate panels, separate colorbars, separate reset buttons (and reset should just do the FOV / detectors, maybe not colormaps etc. too)
-need vmin and vmax color controls.
-in light mode, many of the cells "go dark" when activating the widget:

I'll make another comment with my suggested layout.

[cophus]

Suggested layout:

-Both sides have independent color controls, for cmap, vmin, vmax (and maybe linear/log/power scaling).
-Reset buttons reset view only.
-export button on both panels
-FFT panel either not shown by default or part of another widget
-Combine LAADF + HAADF into single ADF category (maybe a full ADF by default)
-auto-magic detector of BF center disk radius for virtual detectors

[bobleesj]

@cophus Thanks and superb feedback from the very first alpha tester!

Generous enough to even show mock designs as well.

Will do:

• FFT panel hidden by default (a new widget possible later)
• Fix rectangular scans
• Automatic center spot detection
• Separate reset buttons (and reset should just do the FOV / detectors, maybe not colormaps etc. too)
• vmin and vmax color controls.
• Check light/dark mode (configurfe)

Another layer of comments

• Both sides have independent color controls, for cmap, vmin, vmax (and maybe linear/log/power scaling).
• Combine LAADF + HAADF into single ADF category (maybe a full ADF by default)
• Suggested design layout

[bobleesj]

@cophus All done.

@dasol-yoon I will upload v2 tmr or the day after.

[bobleesj]

v2 updated

[gvarnavi]

Hey @bobleesj -- nicely done, very impressive!

Had a quick look on a large-ish Arina dataset (256,256,192,192) and here are my quick thoughts (video-real time):

compressed_demo.mov

• It takes a considerable amount of time to load.. I suspect this is a data transfer bottleneck? Are you using binary buffers and traitlets.Bytes?
• There should be an option for [hot-pixel filtering](def filter_hot_pixels() (very common for Arina datasets)
• I think the DP scale default should be log, or perhaps using quantile-clipping before the histogram otherwise the range is a hard to use
• Circular virtual imaging is also very slow..
• It's not obvious to me what the reset buttons ought to do, but I wasn't expecting it to atleast reset the position, possibly even the virtual detector type
• I think the FFT panel would be better added as a third column?
  • There also seems to be a bug with the boundaries? Perhaps apply a hanning window before the first?

[bobleesj]

@gvarnavi Thanks George for testing! 🙏

Ah, I should have checked with unbinned Arina data. Will fix:

george-v1 after ophus-v1

• Fix loading speed
• hot-pixel filtering
• Circular virtual imaging is slow
• FFT on third column (agree, since now the size layout is 1 to 1)
• FFT scale default log (I will see if I can do clipping thing)
• FFT check boundaries - hanning window

Regarding the reset button, for now, based on the following feedback (we can discuss of course)

Reset buttons reset view only. @cophus

[bobleesj]

@gvarnavi

I think it's working mostly with full ~9 Gb dataset. Will discuss more today's dev call

[gvarnavi]

@bobleesj this looks much better indeed! Will try and check on the gold dataset before the dev call

[bobleesj]

@gvarnavi indeed, much better with feedback. -

Testing on my macbook now - it can't handle 9 Gb... but for smaller datasets (48x48x192x192(, VI is real-time

[gvarnavi]

Hmm.. hot_pixel_filter seems to be wrong on the dataset I'm testing -- I get an empty dataset on load

I tried switching it off as follows but that didn't work either

import traitlets

class MyWidget(widget.Show4DSTEM):
    # Overrides the default value to False while keeping metadata
    hot_pixel_filter = traitlets.Bool(False).tag(sync=True)

What's the best way to override these defaults? Perhaps add them as kwargs to the Show4DSTEM class?

[bobleesj]

I will report back - loading full arina dataset to macbook/laptop

for overriding defaults - I will make it easy to toggle on and off sth like this:

# # Manual calibration (overrides dataset values)
# widget_config = Show4DSTEM(
#     dataset.array,
#     pixel_size=2.39,      # Å (real-space)
#     k_pixel_size=0.46,    # mrad (k-space)
#     center=(96, 96),
#     bf_radius=24,
#     log_scale=True,
# )
# widget_config

I think it's eaiser than subclassing a widget since i haven't done subclassing yet

[bobleesj]

@gvarnavi works on your dataset. I will test a bit more.

Tested across ~4 data types. I will wait for @dasol-yoon's feedback on her dataset type.