Video: support take picture and record video mavlink messages

[rmackay9]

PR #295 added support for RPanion Server to take pictures and record videos to the SD Card. It would be a nice enhancement if these actions could be triggered from the autopilot (running ArduPilot)

The user focused AP Camera setup wiki is here but in short users set CAM1_TYPE = 5 (MAVLink) or 6 (MAVLinkCamV2). The latter is the more feature complete driver so this is preferred.

For 6/MAVLinkCamV2 driver, the issue is likely that AP is not receiving a message it needs to find the camera.

Some references that may help:

• AP dev wiki that briefly explains AP's camera mavlink message support including commands that can be copy-pasted into MAVProxy's console
• AP's simple camera mavlink driver (e.g. CAM1_TYPE = 5)
• AP's MAVLinkCamV2 driver (e.g. CAM1_TYPE = 6)

[stephendade]

I had a quick look through the MavlinkCamV2 driver. Most of those messages are already supported by Rpanion-server, but there appears to be some issues with the mavlink routing preventing the camera messages getting to ArduPilot.

Will investigate further in the coming weeks.

[ddd999]

Controlling the camera via Mavlink messages did work at one point but there were a lot of changes to the media recording additions before the PR was merged so something along the line must have broken it.

[rmackay9]

@ddd999,

I suppose this is not the best place to post this but in any case, I could not immediately get recording video to work.

[ddd999]

Found one of the problems, which is that some messages with IDs >255 were being sent using MavLink V1, which only supports message IDs up to 255. Possible fix for that here: ddd999@ddd1ce6. I was able to trigger a still capture via the "Trigger Camera" button in Mission Planner after making that change.

There's still something going wrong in video mode though, so I'll have to look into that further.

[ddd999]

I tried again and I can't reproduce the video mode error in the screenshot. My setup is:

• Rpanion running on a Pi Zero 2W, with my desktop PC's IP address set as a telemetry destination
• Flight controller board connected to the Pi via serial port
• Desktop PC running Mission Planner

In Mission Planner, when I select the camera device from the dropdown at the top right, clicking Do Action --> "Trigger_Camera" causes the Pi to take a still photo in Photo mode and either start or stop recording video in Video mode. Note, this is with the drone sitting on a desk, not flying.

I've tried setting up the simulator in Mission Planner (e.g., to simulate a waypoint/photo mission with Rpanion) and I must have something wrong with the MAVLink routing because in this setup, I can get telemetry from the sim into Rpanion, and Mission Planner sees the Companion Computer and Camera devices in the top-right connections dropdown, but MavLink Inspector doesn't show any messages coming from the CC or camera-- just the messages coming from the SITL autopilot, and from the MP GCS.

[rmackay9]

Hi @ddd999,

Thanks for investigating. BTW, I think using MP could cloud the issue because MP can directly send mavlink messages to Rpanion. I think it's better to send the take-picture command to the autopilot and have the autopilot then send the command to Rpanion.

[ddd999]

What would be the best way to test that? Send commands to the autopilot from MavProxy?

Isn't that still the opposite of what would happen when the AP is in Auto mode running a mission? Ie, wouldn't the camera commands come directly from the AP in that case?

[rmackay9]

Hi @ddd999,

I think MAVproxy is probably best and we have copy-paste-able camera commands on this dev wiki page. So for example to take a picture I would send IMAGE_START_CAPTURE.

Another approach would be to use MP's aux function screen to send the take-picture command. This is known to work and will only send to the autopilot.

Thanks again for this!

[ddd999]

Based on my understanding of that dev wiki page and this page, taking still images should work, at least in the most basic way, with the existing implementation.

Looks like we need to also process MAV_CMD_VIDEO_START_CAPTURE and MAV_CMD_VIDEO_STOP_CAPTURE messages for video mode to be (minimally) compliant with the spec.

@stephendade if you do have time to investigate the routing at some point that would be super helpful.

[ddd999]

I'm working on improving the MavLinkV2 message support over on this branch: https://github.com/ddd999/Rpanion-server/tree/mavcamera. However, the routing issue still eludes me.

[stephendade]

I've tried setting up the simulator in Mission Planner (e.g., to simulate a waypoint/photo mission with Rpanion) and I must have something wrong with the MAVLink routing because in this setup, I can get telemetry from the sim into Rpanion, and Mission Planner sees the Companion Computer and Camera devices in the top-right connections dropdown, but MavLink Inspector doesn't show any messages coming from the CC or camera-- just the messages coming from the SITL autopilot, and from the MP GCS.

I've got SITL working, using the "extra command line" of --serial1=udpclient:192.168.1.161:5000 in Mission Planner's SITL tab (replacing the IP address with the IP address of Rpanion-server). Then start Rpanion-server with "UDP Server" input, port 5000. Using this, Mission Planner can see all the heartbeats and I can trigger photos.