Sending large and sending chunked MIDI messages fails

[probonopd]

I am trying to send a 4096 byte SysEx (DX7 bank dump).

However, it seems like I can't send MIDI messages that large.

Hence, I am trying to split them up into multiple parts ("chunks"), but then these don't get sent due to

24d6984

What is the rationale behind this?

[rsta2]

The plain MIDI data must be parsed to create USB MIDI events from it. The parser must start at a new MIDI event and must end, when a MIDI event is completed. The second condition is checked in this test. This means, you cannot send chunks of a MIDI SysEx message via this parser. It will not work.

There are two possible solutions:

1. You implement the parser by yourself. Because you know, that you want to create SysEx events, this will be easier, than to create a full MIDI parser. With the created USB MIDI events you call directly CUSBMIDIDevice::SendEventPackets().
2. I could add a chunk size parameter to CUSBMIDIDevice::SendPlainMIDI(), and generate multiple USB transfers for each chunk of USB MIDI data. This would be a synchronous call, which returns, when all events have been send out.

What do you prefer?

[probonopd]

Hi @rsta2, thanks for your feedback. I tried #1 but I am probably doing something wrong, because I always get usbmidigadgetendpoint.cpp(134) - - - assertion failed: !(nLength % CUSBMIDIDevice::EventPacketSize). I tried without and with chunking the SysEx message before handing it over to the sending code.

Here is the code in question:
https://github.com/probonopd/MiniDexed/pull/905/files

00:15:54.277 >1 - 192.168.0.61 midikeyboard - - - MIDI0: SysEx data length: [5]:
00:15:54.278 >1 - 192.168.0.61 midikeyboard - - - 0000:
00:15:54.278 >1 - 192.168.0.61 midikeyboard - - -  0xf0
00:15:54.278 >1 - 192.168.0.61 midikeyboard - - -  0x43
00:15:54.280 >1 - 192.168.0.61 midikeyboard - - -  0x20
00:15:54.280 >1 - 192.168.0.61 midikeyboard - - -  0x09
00:15:54.280 >1 - 192.168.0.61 midikeyboard - - -  0xf7
00:15:54.281 >1 - 192.168.0.61 midikeyboard - - -
00:15:54.281 >1 - 192.168.0.61 midikeyboard - - - MIDI-SYSEX: channel: 0, len: 5, TG: 0
00:15:54.281 >1 - 192.168.0.61 midikeyboard - - - SysEx bank dump request: device 0
00:15:54.281 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank
00:15:54.282 >1 - 192.168.0.61 minidexed - - - getSysExBankDump: called for TG=0
00:15:54.282 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: after getSysExBankDump
00:15:54.282 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: before chunking
00:15:54.282 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: after chunker creation
00:15:54.282 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: device found, starting chunk send loop
00:15:54.283 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: sending chunk 0, size=512
00:15:54.283 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: sending chunk 1, size=512
00:15:54.283 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: sending chunk 2, size=512
00:15:54.284 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: sending chunk 3, size=512
00:15:54.284 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: sending chunk 4, size=512
00:15:54.284 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: sending chunk 5, size=512
00:15:54.284 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: sending chunk 6, size=512
00:15:54.284 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: sending chunk 7, size=512
00:15:54.284 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: all chunks sent, total=8
00:15:54.284 >1 - 192.168.0.61 midikeyboard - - - Send SYSEX bank dump 0 to "umidi1" in 512-byte chunks
00:15:54.284 >1 - 192.168.0.61 midikeyboard - - - SendSystemExclusiveBank: exit
00:15:54.285 >1 - 192.168.0.61 usbmidigadgetendpoint.cpp(134) - - - stack[1] is 0x113E94
00:15:54.285 >1 - 192.168.0.61 usbmidigadgetendpoint.cpp(134) - - - stack[11] is 0xF7CC4
00:15:54.285 >1 - 192.168.0.61 usbmidigadgetendpoint.cpp(134) - - - assertion failed: !(nLength % CUSBMIDIDevice::EventPacketSize)

Can you spot what I am doing wrong?

On a semi-unrelated note, do you have an idea why this fails as of lately?

make[1]: Leaving directory '/home/runner/work/MiniDexed/MiniDexed/circle-stdlib/libs/circle/tools/bootloader'
make clean -C armstub
make[1]: Entering directory '/home/runner/work/MiniDexed/MiniDexed/circle-stdlib/libs/circle/boot/armstub'
rm -f *.o *.elf *.bin *.lst
make[1]: Leaving directory '/home/runner/work/MiniDexed/MiniDexed/circle-stdlib/libs/circle/boot/armstub'
Downloading LICENCE.broadcom ...
(...)
Downloading bcm2712-rpi-cm5l-cm5io.dtb ...
make: *** [Makefile:32: firmware] Error 8

Reference:

• Try to dump banks (does not work yet) probonopd/MiniDexed#905

[rsta2]

USB MIDI events are always 4 bytes long, so the handed over length must be a multiple of it. This is not the case here. You have to convert the plain MIDI events to USB MIDI events, which have the format, which is described in this document (section 4, pg. 16): https://usb.org/document-library/usb-midi-devices-10

[probonopd]

Isn't this what

https://github.com/probonopd/MiniDexed/pull/905/files#diff-1be02ea5e5d5cad1fea18f8a5272da320fb69ba78c7523382ee9e83c0e79802fR116-R157

is doing?

[rsta2]

Of course, if you say, it does, it does. But unfortunately I have currently not the time to check this in detail. What I can say is, that this assertion fails, because SendEventPackets() gets USB MIDI events data with a wrong length. The length must be a multiple of 4. The firmware download error is discussed in #562.

[probonopd]

So I think I would prefer if the whole chunking logic could be encapsulated in Circle, so that it would be possible to send large SysEx in the same way as small SysEx. Do you think you could do that?

[rsta2]

Yes, I will prepare something.

[rsta2]

There is a new parameter nChunkSize for SendPlainMIDI() now on the develop branch. With this USB MIDI can be send in chunks. The chunk size must be a multiple of 4.

[rsta2]

The update is included in Circle Step50.