ROI tracking using the wrong body parts when using SLEAP CSV files

[yadav154-maker]

Hello,

I was attempting to analyze and visualize my ROI tracking when I kept on noticing that the wrong body parts were being tracked. When I clicked track nose the center would be used (first image):

and when I center the left ear would be tracked instead (second image):

I later noticed that when I uploaded the csv files to simba, some columns were removed/changed from the original files
This is what the original csv file from SLEAP looks like:

and this is what the input csv looks like in simba:

I think this is what is causing the issue. The body part names are in the same order as the columns in the SLEAP files. I had skipped all of the outlier corrections so that shouldn't have an impact on the ROI tracking. I don't know how to fix this issue or why exactly this is happening.

Thank you!

[sronilsson]

Hi @yadav154-maker, thanks for raising this!

One question, just so I understand properly:

When SimBA does the ROI analysis, it pulls the data from the project_folder\csv\outlier_corrected_movement_location directory of your SimBA project. Does the data from your last screenshot come from that folder?

When you are using one of SimBA's default body-part configurations, SimBA expects the body-parts from the pose-estimation data in a specific order, for example see the numbers next to the body-parts.

These body-part names, are applied in that order to the data when you run (or skip) outlier correction before the data saved in the project_folder\csv\outlier_corrected_movement_location` directory, and used for your ROI analysis.

Let me know hoe the data looks like oin that folder and I can advice you how to fix it if that is turning out to be the issue or if its something else.

[yadav154-maker]

Hi!

The screenshot is actually from the input csv but the files in the outlier_corrected_movement_location folder look exactly the same:

I used the user defined body-part configuration instead of the default ones since we had 10 body parts in the csv files and there wasn't a configuration that would fit that. The body parts used were:
nose
left_ear
right_ear
neck
left_forelimb
right_forelimb
center
tail_base
head
tail_tip

Here's what the tracking looks like when I visualized the roi features. I had clicked on nose for body part but once again it used the center. There seems to be no issue the overall tracking:

We actually don't need to all of the body parts (such as the tail_tip) for simba but we weren't able to drop body parts using the SLEAP csv files. I didn't run any movement or location corrections because we would run into issues where the tracking data would be missing and our tracking data is near perfect.

Thank you so much!

[sronilsson]

Hey @yadav154-maker!

Got it! A bit of background, and possible solution to get this fixed.

After you have selected a body-part configuration and created a SimBA project, the body-parts you listed in the graphical interface gets saved in a CSV file in your SimBA project at location \project_folder\logs\measures\pose_configs\bp_names\project_bp_names.csv. If you open that file, you'll see something like this (but with your 10 body-parts):

These body-part names, as listed, will be applied to any new pose-estimation data coming in.

Do you see any differences in order between how these body-parts, in the \project_folder\logs\measures\pose_configs\bp_names\project_bp_names.csv file, and the order of the original SLEAP tracking data, are listed?

Simon

[yadav154-maker]

One of the first things I did check was the bp_names/project bp names and the order of the body parts are the same. I decided to create a new project and it seems like it was an error with the columns:

[13:58:58] SIMBA COUNT ERROR: The SimBA project expects 27 data columns, but your SLEAP data file at C:/Users/Windows/Desktop/.sleap/csv_predictions\Rat_odor_CPP_Pilot2_AMAnimal1_Test1_Pretest 1_TestDate08-15-24_Converted.csv contains 30 columns.

If I am correct the error seems to be with the SLEAP csv files containing three extra columns (track, frame_idx, instance_score) and I have to remove the columns before uploading to simba. Is it possible to remove these columns through simba using the SLEAP csv files?

[sronilsson]

Hey @yadav154-maker ! It is more likely that you missed one of the body-parts when you created the project?

Thre missing columns could represent the data for one body-part (x, y, p). If you check the \project_folder\logs\measures\pose_configs\bp_names\project_bp_names.csv file of the SimBA project, are all the body-parts listed?

[sronilsson]

Next, is it possible, that the order of the body-part data is mixed up on the SLEAP end? Meaning, inside the SLEAP data, the first column with the "nose" header actually contains the center data?

I could help you check this, if needed, if you share the data file together with the video. But may be worth looking into.

[yadav154-maker]

Hi!

I did look into the project bp names and all. Everything seems to be correct it mainly seems like there was a problem within simba with this. Is it okay if I send you the original csv file and mp4 file to look at?

thank you so much!

[sronilsson]

Cheers @yadav154-maker - maybe it is the pose-estimation tool then... you can share it in a google drive link if you want? if you want to make it accessible to all or sronilsson@gmail.com I will take a look!

[sronilsson]

PS. Share the original CSV from SLEAP with the associated video.

[yadav154-maker]

Hi!

I just created a folder on google drive. Here's the link:

https://drive.google.com/drive/folders/1F4UbyjeIeAZ702BasZ2eBQK-B3pMtJw8?usp=drive_link

[sronilsson]

Thanks @yadav154-maker - I took a look, but the tracking did not align with the animal.

I realize that you sent me tracking for one video, and then the original video for another video.

It says AMAnimal1 and AMAnimal2 in the filenames also.

The visualizations show it tracks a different animal from the video:

[yadav154-maker]

Hi!

So sorry about that! I added in the right video now

[sronilsson]

got it, thanks @yadav154-maker !

[sronilsson]

Cheers again @yadav154-maker - much appreciated, you caught a bug!

If you i) update simba with pip install simba-uw-tf-dev --upgrade, and ii) import the data again, doe sit work?

There was a line that had sneaked in there, which double checks your header names when importing CSV sleap, but also started to sort the names alphabetically 🫤

But, it should be gone now, PLEASE LET ME KNOW HOW GOES!

[yadav154-maker]

It worked! Thank you so much!