DeepLabCut 3.0 - Pytorch User Guide#
Using DeepLabCut 3.0#
DeepLabCut 3.0 keeps the same high-level API that you know, but has a full new PyTorch backend. Moreover, it is a rewrite that more developer friendly, more powerful, and built for modern deep learning-based computer vision applications.
NOTE🔥: We suggest that if you’re just starting with DeepLabCut you start with the PyTorch backend.
You will easily know which “engine” you are using by looking at the main config.yaml file, or top right corner in the GUI.
If you have DeepLabCut projects in TensorFlow, we’ve got you covered too: you can seamlessly switch to train your already labeled data
by simply switching the engine (and thereby also compare performance). In short, expect a boost 🔥.
In short, PyTorch models can be trained in any DeepLabCut project. If you have a project already made, simply add a new key to your
project config.yaml file specifying engine: pytorch. Then any new training dataset
that will be created will be a PyTorch model (see
Creating Shuffles and Model Configuration)
to learn more about training PyTorch models. To train Tensorflow models again, you can
set engine: tensorflow.
Installation#
During the alpha phase, you can use the yaml we provide, or create a new env.
If you are a beginner user, please see these docs!
If you are an advanced user, here is a quick start. “Install PyTorch”, then:
conda install -c conda-forge pytables==3.8.0
pip install git+https://github.com/DeepLabCut/DeepLabCut.git@pytorch_dlc#egg=deeplabcut[gui,modelzoo,wandb]
Using the GUI#
You can use the GUI to train DeepLabCut projects. However, you cannot switch between
PyTorch and Tensorflow models while using the GUI. If you have set your engine to
pytorch, then the GUI will only offer the creation of PyTorch shuffles.
You can create tensorflow shuffles and train them again by setting the
engine: tensorflow in the top right corner of the GUI.
Major changes#
From iterations to epochs#
Pytorch models in DeepLabCut 3.0 are trained for a set number of epochs, instead of a
maximum number of iterations. An epoch is a single pass through the training dataset,
which means your model has seen each training image exactly once.
So if you have 64 training images for your network, an epoch is 64 iterations with batch size 1 (or 32 iterations with batch size 2, 16 with batch size 4, etc.).
API#
Creating Shuffles and Model Configuration#
You can configure models using the pytorch_config.yaml file, as described
here. You can use the same methods to create new shuffles in
DeepLabCut 3.0 as you did for Tensorflow models (deeplabcut.create_training_dataset
and deeplabcut.create_training_model_comparison).
More information about the different PyTorch model architectures available in DeepLabCut is available here. You can see a list of supported architectures/variants by using:
from deeplabcut.pose_estimation_pytorch import available_models
print(available_models())
Development State and Road Map 🚧#
The table below describes the DeepLabCut API methods that have been implemented, as well as indications which options are not yet implemented, and which parameters are not valid for the DLC 3.0 API.
API Method |
Implemented |
Parameters not yet implemented |
Parameters invalid for pytorch |
|---|---|---|---|
|
🟢 |
|
|
|
🟢 |
||
|
🟢 |
|
|
|
🔴 |
|
|
|
🟢 |
|
|
|
🔴 |
||
|
🟠 |
the name has changed to |
|
|
🟢 |
|
|
|
🔴 |
||
|
🔴 |
||
|
🔴 |
||
|
🔴 |
||
|
🔴 |
||
|
🔴 |