DeepLabCut MultiMouse Data Demo

Open In Colab

DeepLabCut MultiMouse Data Demo#

alt text

DeepLabCut/DeepLabCut

Note: this Colab notebook was written to accompany the Nature Methods publication Multi-animal pose estimation, identification and tracking with DeepLabCut with the TensorFlow engine. To learn about DeepLabCut 3.0+ and the PyTorch engine, you can check out our other notebooks (such as COLAB_YOURDATA_maDLC_TrainNetwork_VideoAnalysis.ipynb).

This notebook illustrates how to use COLAB for a multi-animal DeepLabCut (maDLC) Demo 3 mouse project:#

To get started, please go to “Runtime” ->”change runtime type”->select “Python3”, and then select “GPU”#

As the COLAB environments were updated to CUDA 12.X and Python 3.11, we need to install DeepLabCut and TensorFlow in a distinct way to get TensorFlow to connect to the GPU.

# Install TensorFlow, tensorpack and tf_slim versions compatible with DeepLabCut
!pip install "tensorflow==2.12.1" "tensorpack>=0.11" "tf_slim>=1.1.0"

# Downgrade PyTorch to a version using CUDA 11.8 and cudnn 8
# This will also install the required CUDA libraries, for both PyTorch and TensorFlow
!pip install torch==2.3.1 torchvision --index-url https://download.pytorch.org/whl/cu118

# Install the correct (older) version of DeepLabCut
!pip install deeplabcut==2.3.11

# As described in https://www.tensorflow.org/install/pip#step-by-step_instructions, 
# create symbolic links to NVIDIA shared libraries:
!ln -svf /usr/local/lib/python3.11/dist-packages/nvidia/*/lib/*.so* /usr/local/lib/python3.11/dist-packages/tensorflow

Important - Restart the Runtime for the updated packages to be imported!#

PLEASE, click “restart runtime” from the output above before proceeding!

No information needs edited in the cells below, you can simply click run on each:

Download our Demo Project from our server:#

# Download our demo project:
import requests
from io import BytesIO
from zipfile import ZipFile

url_record = 'https://zenodo.org/api/records/7883589'
response = requests.get(url_record)
if response.status_code == 200:
    file = response.json()['files'][0]
    title = file['key']
    print(f"Downloading {title}...")
    with requests.get(file['links']['self'], stream=True) as r:
        with ZipFile(BytesIO(r.content)) as zf:
            zf.extractall(path='/content')
else:
    raise ValueError(f'The URL {url_record} could not be reached.')

Analyze a novel 3 mouse video with our maDLC DLCRNet, pretrained on 3 mice data (i.e., here you extract detections and association costs):#

import deeplabcut as dlc
import os

project_path = "/content/demo-me-2021-07-14"
config_path = os.path.join(project_path, "config.yaml")
video = os.path.join(project_path, "videos", "videocompressed1.mp4")

dlc.analyze_videos(config_path,[video], shuffle=0, videotype="mp4",auto_track=False )

Next, you compute the local, spatio-temporal grouping and track body part assemblies frame-by-frame:#

TRACK_METHOD = "ellipse"  # Could also be "box", but "ellipse" was found to be more robust on this dataset.

dlc.convert_detections2tracklets(
    config_path,
    [video],
    videotype='mp4',
    shuffle=0,
    track_method=TRACK_METHOD,
    ignore_bodyparts=["tail1", "tail2", "tailend"],  # Some body parts can optionally be ignored during tracking for better assembly (but they are used later)
)

Reconstruct full animal trajectories (tracks from tracklets):#

dlc.stitch_tracklets(
    config_path,
    [video],
    videotype='mp4',
    shuffle=0,
    track_method=TRACK_METHOD,
    n_tracks=3,
)

Create a pretty video output:#

#Filter the predictions to remove small jitter, if desired:
dlc.filterpredictions(config_path, 
                                 [video], 
                                 shuffle=0,
                                 videotype='mp4', 
                                 track_method = TRACK_METHOD)

dlc.create_labeled_video(
    config_path,
    [video],
    videotype='mp4',
    shuffle=0,
    color_by="individual",
    keypoints_only=False,
    draw_skeleton=True,
    filtered=True,
    track_method=TRACK_METHOD,
)

Now, on the left panel if you click the folder icon, you will see the project folder “demo-me..”; click on this and go into “videos” and you can find the “…_id_labeled.mp4” video, which you can double-click on to download and inspect!

Create Plots of your data:#

after running, you can look in “videos”, “plot-poses” to check out the trajectories! (sometimes you need to click the folder refresh icon to see it). Within the folder, for example, see plotmus1.png to vide the bodyparts over time vs. pixel position.

dlc.plot_trajectories(config_path, [video], shuffle=0,videotype='mp4', track_method=TRACK_METHOD)
Contents