A development roadmap for DeepLabCut

:loudspeaker: :hourglass_flowing_sand: :construction:

General Enhancements:

• DeepLabCut PyTorch & Model Zoo –> DLC 3.0 🔥

• DLC-CookBook v0.1

• DLC BLog for releases and user-highlights

• New Docker containers into main repo / linked to Docker hub and repo(s)

• 3D >2 camera support –> better 3D in PyTorch version 🔥

General NN Improvements:

• EfficientNet backbones added (currently SOTA on ImageNet). https://openaccess.thecvf.com/content/WACV2021/html/Mathis_Pretraining_Boosts_Out-of-Domain_Robustness_for_Pose_Estimation_WACV_2021_paper.html DeepLabCut/DeepLabCut

• New multi-fusion multi-scale networks; DLCRNet_ms5

• BUCTD Integration, see ICCV 2023 paper at https://arxiv.org/abs/2306.07879

deeplabcut 2.2: multi-animal pose estimation and tracking with DeepLabCut

• alpha testing complete (early May 2020)

• beta release: 2.2.b5 on 5 / 22 / 20 :smile:

• beta release: 2.2b8 released 9/2020 :smile:

• beta release 2.2b9 (rolled into 2.1.9 –> candidate release, slotted for Oct 2020)

• 2.2rc1

• 2.2rc2

• 2.2rc3

• Manuscript Lauer et al 2021 https://www.biorxiv.org/content/10.1101/2021.04.30.442096v1

• full 2.2 stable release

real-time module with DEMO for how to set up on your camera system, integration with our [Camera Control Software](AdaptiveMotorControlLab/Camera_Control)

• Integration with Bonsai completed! See: bonsai-rx/deeplabcut

• Integration with Auto-pi-lot. See: https://auto-pi-lot.com/

• DeepLabCut-live! released Aug 5th, 2020: preprint & code: https://www.biorxiv.org/content/10.1101/2020.08.04.236422v1

• DeepLabCut-live! published in eLife

DeepLabCut Model Zoo: a collection of pretrained models for plug-in-play DLC and community crowd-sourcing.

• BETA release with 2.1.8b0: http://www.mousemotorlab.org/dlc-modelzoo

• full release with 2.1.8.1 http://www.mousemotorlab.org/dlc-modelzoo

• Manuscript forthcoming! –> see arXiv https://arxiv.org/abs/2203.07436

• new models added; horse, cheetah

• TopView_Mouse model

• Quadruped model

• contribution module

• PyTorch Model zoo code

DeepLabCut GUI and DeepLabCut-core:

• to make DLC more modular, we will move core functions to DeepLabCut/DeepLabCut-core

• DLC-core depreciated, and core is now simply pip install deeplabcut GUI is with pip install deeplabcut[gui]

• new GUI for DeepLabCut; due to extended issues with wxPython, we will be moving to release a napari plugin napari/napari

• New project management GUI

• tensorflow 2.2 support in DeepLabCut-core: DeepLabCut/DeepLabCut#601

• DeepLabCut-Core to be depreciated; TF2 will go into main repo.

• TF2 support while also maintaining TF1 support until 2022.

• Web-based GUI for labeling –> Colab training pipeline for users (full no-install DLC)