DeepLabCut MultiMouse Data Demo

Open In Colab

DeepLabCut MultiMouse Data Demo#

alt text

DeepLabCut/DeepLabCut

This Colab notebook was written to accompany the Nature Methods publication

Lauer, J., Zhou, M., Ye, S., Menegas, W., Schneider, S., Nath, T., Rahman, M. M., Di Santo, V., Soberanes, D., Feng, G., Murthy, V. N., Lauder, G., Dulac, C., Mathis, M. W., & Mathis, A. (2022).
Multi-animal pose estimation, identification and tracking with DeepLabCut.
Nature Methods, 19(4), 496–504.
https://doi.org/10.1038/s41592-022-01443-0

Note: The paper used DeepLabCut version 2.2.1 for Python 3.8 with the TensorFlow engine. Since this Python version is no longer supported on Google Colab, this notebook uses a more recent version of DeepLabCut (namely, 3.0.x), which still supports TensorFlow. To learn about DeepLabCut 3.0+ and the PyTorch engine, you can check out our other notebooks (such as COLAB_YOURDATA_maDLC_TrainNetwork_VideoAnalysis.ipynb).

This notebook illustrates how to use COLAB for a multi-animal DeepLabCut (maDLC) Demo 3 mouse project:#

To get started, please go to “Runtime” ->”change runtime type”->select “Python3”, and then select “GPU”#

Note that DeepLabCut version 2.2.1 requires Python < 3.10 and no longer runs on Google Colab. You can still run it locally in a Python 3.8 environment. In Google Colab, use a more recent TensorFlow-enabled version of DeepLabCut than can be installed with Python 3.12 (e.g. DeepLabCut version 3.0, which supports both PyTorch and TensorFlow).

# Install DeepLabCut 3.0 (for Google Colab with Python 3.12)
!pip install --pre "deeplabcut[tf]<3.1"

Important - Restart the Runtime for the updated packages to be imported!#

PLEASE, click “restart runtime” from the output above before proceeding!

No information needs edited in the cells below, you can simply click run on each:

Download our Demo Project from our server:#

# Download our demo project:
from io import BytesIO
from pathlib import Path
from zipfile import ZipFile

import requests

url_record = "https://zenodo.org/api/records/7883589"
response = requests.get(url_record)
if response.status_code == 200:
    file = response.json()["files"][0]
    title = file["key"]
    print(f"Downloading {title}...")
    with requests.get(file["links"]["self"], stream=True) as r:
        with ZipFile(BytesIO(r.content)) as zf:
            zf.extractall(path="/content")
    # Fix missing metadata.yaml
    data_dir = Path("/content/demo-me-2021-07-14/training-datasets/iteration-0/UnaugmentedDataSet_demoJul14")
    data_dir.mkdir(parents=True, exist_ok=True)
    (data_dir / "metadata.yaml").write_text(
        "\nshuffles:\n  demoJul14-trainset95shuffle0:\n    train_fraction: 0.95\n    index: 0\n"
        "    split: 1\n    engine: TensorFlow\n"
    )
else:
    raise ValueError(f"The URL {url_record} could not be reached.")

Analyze a novel 3 mouse video with our maDLC DLCRNet, pretrained on 3 mice data (i.e., here you extract detections and association costs):#

import os

import deeplabcut as dlc

project_path = "/content/demo-me-2021-07-14"
config_path = os.path.join(project_path, "config.yaml")
video = os.path.join(project_path, "videos", "videocompressed1.mp4")

dlc.analyze_videos(
    config_path,
    [video],
    shuffle=0,
    videotype="mp4",
    auto_track=False,
    engine=dlc.Engine.TF,
)

Next, you compute the local, spatio-temporal grouping and track body part assemblies frame-by-frame:#

TRACK_METHOD = "ellipse"  # Could also be "box", but "ellipse" was found to be more robust on this dataset.

dlc.convert_detections2tracklets(
    config_path,
    [video],
    videotype="mp4",
    shuffle=0,
    track_method=TRACK_METHOD,
    ignore_bodyparts=[
        "tail1",
        "tail2",
        "tailend",
    ],  # Some body parts can optionally be ignored during tracking for better assembly (but they are used later)
)

Reconstruct full animal trajectories (tracks from tracklets):#

dlc.stitch_tracklets(
    config_path,
    [video],
    videotype="mp4",
    shuffle=0,
    track_method=TRACK_METHOD,
    n_tracks=3,
)

Create a pretty video output:#

# Filter the predictions to remove small jitter, if desired:
dlc.filterpredictions(config_path, [video], shuffle=0, videotype="mp4", track_method=TRACK_METHOD)

dlc.create_labeled_video(
    config_path,
    [video],
    videotype="mp4",
    shuffle=0,
    color_by="individual",
    keypoints_only=False,
    draw_skeleton=True,
    filtered=True,
    track_method=TRACK_METHOD,
)

Now, on the left panel if you click the folder icon, you will see the project folder “demo-me..”; click on this and go into “videos” and you can find the “…_id_labeled.mp4” video, which you can double-click on to download and inspect!

Create Plots of your data:#

after running, you can look in “videos”, “plot-poses” to check out the trajectories! (sometimes you need to click the folder refresh icon to see it). Within the folder, for example, see plotmus1.png to vide the bodyparts over time vs. pixel position.

dlc.plot_trajectories(config_path, [video], shuffle=0, videotype="mp4", track_method=TRACK_METHOD)
Contents