New example voltage_acq_int_clk_plot_wfm.py does not behave as expected.

[Emanuel-Lpz]

Bug Report

This shipping example works great for slow sample rates; however, at high sample rates (for instance, 1MS/s), it takes up to 30 secs to display the graph. The bottleneck seems to be the timestamp generation: get_timestamps creates a timestamp object for every single sample, converting each timestamp to timedelta and calculating offsets is expensive.

Repro or Code Sample

Test with the original voltage_acq_int_clk_plot_wfm.py example, at sample rates.

Expected Behavior

Bottlebeck performance with this new example.

Current Behavior

Up to 30 seconds before data is properly displayed at high sample rates.

Possible Solution

I made some changes to the code, mainly on line 13, integrating NumPy. The behavior is much better, but probably you have a better solution:

import matplotlib.pyplot as plot
import numpy as np
import nidaqmx
from nidaqmx.constants import AcquisitionType, READ_ALL_AVAILABLE

with nidaqmx.Task() as task:
    task.ai_channels.add_ai_voltage_chan(physical_channel= "USB-6453/ai0")
    sampling_rate = 100000
    task.timing.cfg_samp_clk_timing(rate=sampling_rate, sample_mode=AcquisitionType.FINITE, samps_per_chan=100000)
    task.start()
    waveform = task.read_waveform(READ_ALL_AVAILABLE)

    time_offsets = np.arange(waveform.sample_count) / sampling_rate
    plot.plot(time_offsets, waveform.scaled_data)
    plot.xlabel("Seconds")
    plot.ylabel(waveform.units)
    plot.title(waveform.channel_name)
    plot.grid(True)
    plot.show()   

Context

This is not critical, but it would benefit the example code collection.

Your Environment

• Operating system and version: Windows 11
• NI-DAQmx version: 25Q3
• nidaqmx-python version: 1.4.7
• Python 3.11

AB#3775416

[bkeryan]

Thank you for the feedback.

I agree that creating a list of timestamps does not have good performance for large buffer sizes. That's why the nitypes package has a DateTimeArray class which is backed by a NumPy array. However, the waveform timing object does not yet use it for get_timestamps().

The waveform timing conveys additional information that I would want the example to preserve:

• The DAQ device may not be able to sample at exactly 100000 S/s. If this happens, DAQmx coerces the sample rate to the closest supported rate. You can read back the coerced rate by getting task.timing.samp_clk_rate. The waveform timing also reflects the coerced rate.
• For multi-channel acquisitions on devices that multiplex all channels to a single ADC, each channel is acquired at a slightly different time. The waveform timestamps include this skew, or interchannel delay. That way, if you connect the same signal to multiple channels, it will line up correctly on the graph, even though each channel sampled the signal at a different time.

[Emanuel-Lpz]

@bkeryan thanks a lot for hearing my comments. You are right, I reviewed my proposal again, and it's not the same as the original code; it assumes that the timestamps are perfectly uniform which is not the case for scenarios such as the ones you listed. TBH I haven't figured out a better way to calculate timestamps and improve the code performance at high sample rates.

[bkeryan]

DAQ data uses regular timing, so your assumption that the timestamps are uniform is correct.

To use the coerced sample rate, try this (untested):

time_offsets = np.arange(waveform.sample_count) / waveform.sample_interval.total_seconds()

I think the interchannel delay would involve something like this (also untested):

min_start_time = min(w.timing.start_time for w in waveforms)
time_offset_per_channel = [(w.timing.start_time - min_start_time).total_seconds() for w in waveforms]

Then add time_offset_per_channel[i] to time_offsets when plotting waveforms[i].

[bkeryan]

I compared performance of voltage_acq_int_clk_plot_data.py vs. voltage_acq_int_clk_plot_wfm.py, using a real PCIe-6353.

Increase the sample rate and number of samples:

diff --git a/examples/analog_in/voltage_acq_int_clk_plot_data.py b/examples/analog_in/voltage_acq_int_clk_plot_data.py
index f495efe6..f575510f 100644
--- a/examples/analog_in/voltage_acq_int_clk_plot_data.py
+++ b/examples/analog_in/voltage_acq_int_clk_plot_data.py
@@ -12,11 +12,11 @@ from nidaqmx.constants import READ_ALL_AVAILABLE, AcquisitionType

 with nidaqmx.Task() as task:
     task.ai_channels.add_ai_voltage_chan("Dev1/ai0")
-    task.timing.cfg_samp_clk_timing(1000.0, sample_mode=AcquisitionType.FINITE, samps_per_chan=50)
+    task.timing.cfg_samp_clk_timing(1e6, sample_mode=AcquisitionType.FINITE, samps_per_chan=100_000)

     data = task.read(READ_ALL_AVAILABLE)

     plot.plot(data)
     plot.ylabel("Amplitude")
     plot.title("Waveform")
-    plot.show()
+    plot.show(block=False)
diff --git a/examples/analog_in/voltage_acq_int_clk_plot_wfm.py b/examples/analog_in/voltage_acq_int_clk_plot_wfm.py
index a87c6fc4..6d19c6bf 100644
--- a/examples/analog_in/voltage_acq_int_clk_plot_wfm.py
+++ b/examples/analog_in/voltage_acq_int_clk_plot_wfm.py
@@ -12,7 +12,7 @@ from nidaqmx.constants import READ_ALL_AVAILABLE, AcquisitionType

 with nidaqmx.Task() as task:
     task.ai_channels.add_ai_voltage_chan("Dev1/ai0")
-    task.timing.cfg_samp_clk_timing(1000.0, sample_mode=AcquisitionType.FINITE, samps_per_chan=50)
+    task.timing.cfg_samp_clk_timing(1e6, sample_mode=AcquisitionType.FINITE, samps_per_chan=100_000)

     waveform = task.read_waveform(READ_ALL_AVAILABLE)

@@ -24,4 +24,4 @@ with nidaqmx.Task() as task:
     plot.title(waveform.channel_name)
     plot.grid(True)

-    plot.show()
+    plot.show(block=False)

Measure execution time with PowerShell Measure-Command:

PS D:\dev\nidaqmx-python\examples\analog_in> Measure-Command { poetry run python voltage_acq_int_clk_plot_data.py }


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 2
Milliseconds      : 503
Ticks             : 25039161
TotalDays         : 2.89805104166667E-05
TotalHours        : 0.00069553225
TotalMinutes      : 0.041731935
TotalSeconds      : 2.5039161
TotalMilliseconds : 2503.9161



PS D:\dev\nidaqmx-python\examples\analog_in> Measure-Command { poetry run python voltage_acq_int_clk_plot_wfm.py }


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 33
Milliseconds      : 263
Ticks             : 332631298
TotalDays         : 0.000384989928240741
TotalHours        : 0.00923975827777778
TotalMinutes      : 0.554385496666667
TotalSeconds      : 33.2631298
TotalMilliseconds : 33263.1298

Profile with the cProfile module:

PS D:\dev\nidaqmx-python\examples\analog_in> poetry run python -m cProfile .\voltage_acq_int_clk_plot_data.py
         1512668 function calls (1488637 primitive calls) in 1.896 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       59    0.002    0.000    2.223    0.038 __init__.py:1(<module>)
    510/1    0.008    0.000    1.897    1.897 {built-in method builtins.exec}
        1    0.000    0.000    1.897    1.897 voltage_acq_int_clk_plot_data.py:1(<module>)
   548/13    0.006    0.000    1.456    0.112 <frozen importlib._bootstrap>:1165(_find_and_load)
   545/13    0.003    0.000    1.456    0.112 <frozen importlib._bootstrap>:1120(_find_and_load_unlocked)
   524/14    0.003    0.000    1.451    0.104 <frozen importlib._bootstrap>:666(_load_unlocked)
  1221/22    0.001    0.000    1.449    0.066 <frozen importlib._bootstrap>:233(_call_with_frames_removed)
   479/13    0.002    0.000    1.448    0.111 <frozen importlib._bootstrap_external>:934(exec_module)
   492/26    0.001    0.000    1.189    0.046 {built-in method builtins.__import__}
  468/107    0.002    0.000    1.078    0.010 <frozen importlib._bootstrap>:1207(_handle_fromlist)
        1    0.000    0.000    0.676    0.676 pyplot.py:1(<module>)
1715/1624    0.025    0.000    0.637    0.000 {built-in method builtins.__build_class__}
      106    0.001    0.000    0.402    0.004 artist.py:158(_update_set_signature_and_docstring)
      105    0.000    0.000    0.396    0.004 artist.py:125(__init_subclass__)
        1    0.000    0.000    0.353    0.353 figure.py:1(<module>)
      120    0.001    0.000    0.318    0.003 artist.py:1832(kwdoc)
      120    0.007    0.000    0.271    0.002 artist.py:1593(pprint_setters)
        1    0.000    0.000    0.268    0.268 image.py:1(<module>)
        1    0.000    0.000    0.251    0.251 backend_bases.py:1(<module>)
     4409    0.019    0.000    0.221    0.000 artist.py:1461(get_valid_values)
     7941    0.123    0.000    0.196    0.000 inspect.py:867(cleandoc)
        1    0.000    0.000    0.192    0.192 pyplot.py:3830(plot)
        3    0.000    0.000    0.182    0.061 pyplot.py:2782(gca)
      479    0.006    0.000    0.170    0.000 <frozen importlib._bootstrap_external>:1007(get_code)
        1    0.000    0.000    0.164    0.164 text.py:1(<module>)
...

PS D:\dev\nidaqmx-python\examples\analog_in> poetry run python -m cProfile .\voltage_acq_int_clk_plot_wfm.py
         193316611 function calls (187292454 primitive calls) in 122.647 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    510/1    0.008    0.000  122.648  122.648 {built-in method builtins.exec}
        1    0.035    0.035  122.648  122.648 voltage_acq_int_clk_plot_wfm.py:1(<module>)
   300011   11.090    0.000  116.220    0.000 _timedelta.py:83(__new__)
   100001    0.176    0.000   81.408    0.001 _regular.py:56(_generate_regular_timestamps)
   100001    1.058    0.000   81.216    0.001 _datetime.py:337(__add__)
 12000440    9.936    0.000   62.978    0.000 fractions.py:357(forward)
 23100847   28.146    0.000   41.068    0.000 fractions.py:62(__new__)
   100001    0.324    0.000   40.046    0.000 _timedelta.py:286(__add__)
        1    0.124    0.124   39.303   39.303 voltage_acq_int_clk_plot_wfm.py:20(<listcomp>)
   100002    0.537    0.000   38.971    0.000 _datetime.py:374(__sub__)
5100201/2700113    3.023    0.000   31.832    0.000 {built-in method builtins.round}
5100187/2700099    8.959    0.000   30.150    0.000 fractions.py:620(__round__)
  7200264   15.608    0.000   29.394    0.000 fractions.py:484(_mul)
  3600132    8.985    0.000   16.254    0.000 fractions.py:452(_add)
17507322/17507321    4.625    0.000    7.896    0.000 {built-in method builtins.isinstance}
5300251/4100207    1.500    0.000    7.825    0.000 {built-in method builtins.divmod}
  1200044    2.402    0.000    5.443    0.000 fractions.py:526(_divmod)
23506469/23503481    5.119    0.000    5.166    0.000 {built-in method __new__ of type object at 0x00007FF8DE31DF90}
  1200044    0.938    0.000    5.103    0.000 fractions.py:202(limit_denominator)
 25800948    4.365    0.000    4.365    0.000 fractions.py:261(denominator)
 21600792    4.294    0.000    4.294    0.000 {built-in method math.gcd}
 23100847    3.958    0.000    3.958    0.000 fractions.py:257(numerator)
  4801944    2.018    0.000    3.271    0.000 <frozen abc>:117(__instancecheck__)
       59    0.002    0.000    2.219    0.038 __init__.py:1(<module>)
   548/13    0.005    0.000    1.457    0.112 <frozen importlib._bootstrap>:1165(_find_and_load)
...

[bkeryan]

Updated waveform example:

diff --git a/examples/analog_in/voltage_acq_int_clk_plot_wfm.py b/examples/analog_in/voltage_acq_int_clk_plot_wfm.py
index a87c6fc4..9b7b5a26 100644
--- a/examples/analog_in/voltage_acq_int_clk_plot_wfm.py
+++ b/examples/analog_in/voltage_acq_int_clk_plot_wfm.py
@@ -6,22 +6,25 @@ Run 'pip install matplotlib' to install the matplotlib module.
 """

 import matplotlib.pyplot as plot
-
 import nidaqmx
+import numpy as np
 from nidaqmx.constants import READ_ALL_AVAILABLE, AcquisitionType

 with nidaqmx.Task() as task:
     task.ai_channels.add_ai_voltage_chan("Dev1/ai0")
-    task.timing.cfg_samp_clk_timing(1000.0, sample_mode=AcquisitionType.FINITE, samps_per_chan=50)
+    task.timing.cfg_samp_clk_timing(1e6, sample_mode=AcquisitionType.FINITE, samps_per_chan=100_000)

     waveform = task.read_waveform(READ_ALL_AVAILABLE)

-    timestamps = list(waveform.timing.get_timestamps(0, waveform.sample_count))
-    time_offsets = [(ts - timestamps[0]).total_seconds() for ts in timestamps]
+    time_offsets = np.linspace(
+        0,
+        waveform.sample_count * waveform.timing.sample_interval.total_seconds(),
+        waveform.sample_count,
+    )
     plot.plot(time_offsets, waveform.scaled_data)
     plot.xlabel("Seconds")
     plot.ylabel(waveform.units)
     plot.title(waveform.channel_name)
     plot.grid(True)

-    plot.show()
+    plot.show(block=False)

PS D:\dev\nidaqmx-python\examples\analog_in> Measure-Command { poetry run python voltage_acq_int_clk_plot_wfm.py }


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 2
Milliseconds      : 482
Ticks             : 24826685
TotalDays         : 2.87345891203704E-05
TotalHours        : 0.000689630138888889
TotalMinutes      : 0.0413778083333333
TotalSeconds      : 2.4826685
TotalMilliseconds : 2482.6685

PS D:\dev\nidaqmx-python\examples\analog_in> poetry run python -m cProfile .\voltage_acq_int_clk_plot_wfm.py
         1516193 function calls (1492068 primitive calls) in 1.912 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       59    0.002    0.000    2.219    0.038 __init__.py:1(<module>)
    510/1    0.008    0.000    1.913    1.913 {built-in method builtins.exec}
        1    0.000    0.000    1.913    1.913 voltage_acq_int_clk_plot_wfm.py:1(<module>)
   548/13    0.005    0.000    1.460    0.112 <frozen importlib._bootstrap>:1165(_find_and_load)
   545/13    0.003    0.000    1.459    0.112 <frozen importlib._bootstrap>:1120(_find_and_load_unlocked)
   524/14    0.003    0.000    1.454    0.104 <frozen importlib._bootstrap>:666(_load_unlocked)
  1221/22    0.001    0.000    1.452    0.066 <frozen importlib._bootstrap>:233(_call_with_frames_removed)
   479/13    0.002    0.000    1.452    0.112 <frozen importlib._bootstrap_external>:934(exec_module)
   492/26    0.001    0.000    1.192    0.046 {built-in method builtins.__import__}
  468/107    0.002    0.000    1.080    0.010 <frozen importlib._bootstrap>:1207(_handle_fromlist)
        1    0.000    0.000    0.679    0.679 pyplot.py:1(<module>)
1715/1624    0.025    0.000    0.641    0.000 {built-in method builtins.__build_class__}
      106    0.001    0.000    0.404    0.004 artist.py:158(_update_set_signature_and_docstring)
      105    0.001    0.000    0.398    0.004 artist.py:125(__init_subclass__)
        1    0.000    0.000    0.354    0.354 figure.py:1(<module>)
      120    0.001    0.000    0.317    0.003 artist.py:1832(kwdoc)
        1    0.000    0.000    0.270    0.270 image.py:1(<module>)
      120    0.007    0.000    0.269    0.002 artist.py:1593(pprint_setters)
        1    0.000    0.000    0.252    0.252 backend_bases.py:1(<module>)
     4409    0.019    0.000    0.220    0.000 artist.py:1461(get_valid_values)
     7941    0.124    0.000    0.197    0.000 inspect.py:867(cleandoc)
        1    0.000    0.000    0.192    0.192 pyplot.py:3830(plot)
        5    0.000    0.000    0.185    0.037 pyplot.py:2782(gca)
      479    0.006    0.000    0.169    0.000 <frozen importlib._bootstrap_external>:1007(get_code)
        1    0.000    0.000    0.163    0.163 text.py:1(<module>)

The updated example seems to have equivalent performance to voltage_acq_int_clk_plot_data.py.