deeplabcut.pose_estimation_tensorflow

Modules:

Name	Description
auxfun_models	DeepLabCut2.0 Toolbox (deeplabcut.org)
auxfun_multianimal	DeepLabCut2.0 Toolbox (deeplabcut.org)
auxiliaryfunctions	DeepLabCut2.0 Toolbox (deeplabcut.org)
backbones
config
core
datasets
export
inferenceutils
lib
modelzoo
nnets
predict
predict_multianimal
predict_videos
trackingutils
training
util	Adapted from DeeperCut by Eldar Insafutdinov:
visualize	Adapted from DeeperCut by Eldar Insafutdinov
visualizemaps

Functions:

Name	Description
AnalyzeVideo	Helper function for analyzing a video.
GetPoseDynamic	Non batch wise pose estimation for video cap by dynamically cropping around
GetPoseF	Batchwise prediction of pose.
GetPoseF_GTF	Batchwise prediction of pose.
GetPoseF_OV	Prediction of pose.
GetPoseS	Non batch wise pose estimation for video cap.
GetPoseS_GTF	Non batch wise pose estimation for video cap.
GetPosesofFrames	Batchwise prediction of pose for frame list in directory.
Plotting	Function used for plotting GT and predictions.
analyze_time_lapse_frames	Analyzed all images (of type = frametype) in a folder and stores the output in
analyze_videos	Makes prediction based on a trained network.
argmax_pose_predict	Combine scoremat and offsets to the final pose.
calculatepafdistancebounds	Returns distances along paf edges in train/test data
cfg_from_file	Load a config from file filename and merge it into the default options.
collect_video_paths	Collects video paths from a given set of data paths: directories, files, or a mix
convert_detections2tracklets	This should be called at the end of deeplabcut.analyze_videos for multianimal
evaluate_network	Evaluates the network.
extract_cnn_output	Extract locref + scmap from network.
extract_maps	Extracts the scoremap, locref, partaffinityfields (if available).
extract_save_all_maps	Extracts the scoremap, location refinement field and part affinity field prediction of the model. The maps
get_available_requested_snapshots	Intersects the requested snapshot names with the available snapshots.
get_snapshots_by_index	Assume available_snapshots is ordered in ascending order.
keypoint_error	Computes the RMSE error for each bodypart.
make_results_file	Makes result file in csv format and saves under evaluation_results directory.
pairwisedistances	Calculates the pairwise Euclidean distance metric over body parts vs.
renamed_parameter	Support a renamed keyword argument while warning callers to update.
return_evaluate_network_data	Returns the results for (previously evaluated) network.
return_train_network_path	Returns the training and test pose config file names as well as the folder where
stitch_tracklets	Stitch sparse tracklets into full tracks via a graph-based, minimum-cost flow
train_network	Trains the network with the labels in the training dataset.
visualize_locrefs	Plots a scoremap and the corresponding location refinement field on an image.
visualize_paf	Plots the PAF on top of the image.
visualize_scoremaps	Plots scoremaps as an image overlay.

AnalyzeVideo

AnalyzeVideo(
    video,
    DLCscorer,
    DLCscorerlegacy,
    trainFraction,
    cfg,
    dlc_cfg,
    sess,
    inputs,
    outputs,
    pdindex,
    save_as_csv,
    destfolder=None,
    TFGPUinference=True,
    dynamic=(False, 0.5, 10),
    use_openvino="CPU" if is_openvino_available else None,
)

Helper function for analyzing a video.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def AnalyzeVideo(
    video,
    DLCscorer,
    DLCscorerlegacy,
    trainFraction,
    cfg,
    dlc_cfg,
    sess,
    inputs,
    outputs,
    pdindex,
    save_as_csv,
    destfolder=None,
    TFGPUinference=True,
    dynamic=(False, 0.5, 10),
    use_openvino="CPU" if is_openvino_available else None,
):
    """Helper function for analyzing a video."""
    print("Starting to analyze % ", video)

    if destfolder is None:
        destfolder = str(Path(video).parents[0])
    auxiliaryfunctions.attempt_to_make_folder(destfolder)
    vname = Path(video).stem
    try:
        _ = auxiliaryfunctions.load_analyzed_data(destfolder, vname, DLCscorer)
    except FileNotFoundError as e:
        print("Loading ", video)
        cap = cv2.VideoCapture(video)
        if not cap.isOpened():
            raise OSError("Video could not be opened. Please check the file integrity.") from e
        # https://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html#videocapture-get
        fps = cap.get(cv2.CAP_PROP_FPS)
        nframes = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
        duration = nframes * 1.0 / fps
        size = (
            int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)),
            int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
        )
        ny, nx = size
        print(
            "Duration of video [s]: ",
            round(duration, 2),
            ", recorded with ",
            round(fps, 2),
            "fps!",
        )
        print(
            "Overall # of frames: ",
            nframes,
            " found with (before cropping) frame dimensions: ",
            nx,
            ny,
        )

        dynamic_analysis_state, detectiontreshold, margin = dynamic
        start = time.time()
        print("Starting to extract posture")
        if dynamic_analysis_state:
            PredictedData, nframes = GetPoseDynamic(
                cfg,
                dlc_cfg,
                sess,
                inputs,
                outputs,
                cap,
                nframes,
                detectiontreshold,
                margin,
            )
            # GetPoseF_GTF(cfg,dlc_cfg, sess, inputs, outputs,cap,nframes,int(dlc_cfg["batch_size"]))
        else:
            if int(dlc_cfg["batch_size"]) > 1:
                args = (
                    cfg,
                    dlc_cfg,
                    sess,
                    inputs,
                    outputs,
                    cap,
                    nframes,
                    int(dlc_cfg["batch_size"]),
                )
                if use_openvino:
                    PredictedData, nframes = GetPoseF_OV(*args)
                elif TFGPUinference:
                    PredictedData, nframes = GetPoseF_GTF(*args)
                else:
                    PredictedData, nframes = GetPoseF(*args)
            else:
                if TFGPUinference:
                    PredictedData, nframes = GetPoseS_GTF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes)
                else:
                    PredictedData, nframes = GetPoseS(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes)

        stop = time.time()
        if cfg["cropping"]:
            coords = [cfg["x1"], cfg["x2"], cfg["y1"], cfg["y2"]]
        else:
            coords = [0, nx, 0, ny]

        dictionary = {
            "start": start,
            "stop": stop,
            "run_duration": stop - start,
            "Scorer": DLCscorer,
            "DLC-model-config file": dlc_cfg,
            "fps": fps,
            "batch_size": dlc_cfg["batch_size"],
            "frame_dimensions": (ny, nx),
            "nframes": nframes,
            "iteration (active-learning)": cfg["iteration"],
            "training set fraction": trainFraction,
            "cropping": cfg["cropping"],
            "cropping_parameters": coords,
            # "gpu_info": device_lib.list_local_devices()
        }
        metadata = {"data": dictionary}

        print(f"Saving results in {destfolder}...")
        dataname = os.path.join(destfolder, vname + DLCscorer + ".h5")
        auxiliaryfunctions.save_data(
            PredictedData[:nframes, :],
            metadata,
            dataname,
            pdindex,
            range(nframes),
            save_as_csv,
        )
    return DLCscorer

GetPoseDynamic

GetPoseDynamic(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, detectiontreshold, margin)

Non batch wise pose estimation for video cap by dynamically cropping around previously detected parts.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPoseDynamic(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, detectiontreshold, margin):
    """Non batch wise pose estimation for video cap by dynamically cropping around
    previously detected parts."""
    if cfg["cropping"]:
        ny, nx = checkcropping(cfg, cap)
    else:
        ny, nx = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)), int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    x1, x2, y1, y2 = 0, nx, 0, ny
    detected = False
    # TODO: perform detection on resized image (For speed)

    PredictedData = np.zeros((nframes, 3 * len(dlc_cfg["all_joints_names"])))
    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))
    while cap.isOpened():
        if counter != 0 and counter % step == 0:
            pbar.update(step)

        ret, frame = cap.read()
        if ret:
            # print(counter,x1,x2,y1,y2,detected)
            originalframe = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            if cfg["cropping"]:
                frame = img_as_ubyte(originalframe[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])[y1:y2, x1:x2]
            else:
                frame = img_as_ubyte(originalframe[y1:y2, x1:x2])

            pose = predict.getpose(frame, dlc_cfg, sess, inputs, outputs).flatten()
            detection = np.any(pose[2::3] > detectiontreshold)  # is anything detected?
            if detection:
                pose[0::3], pose[1::3] = (
                    pose[0::3] + x1,
                    pose[1::3] + y1,
                )  # offset according to last bounding box
                x1, x2, y1, y2 = getboundingbox(
                    pose[0::3], pose[1::3], nx, ny, margin
                )  # coordinates for next iteration
                if not detected:
                    detected = True  # object detected
            else:
                if (
                    detected and (x1 + y1 + y2 - ny + x2 - nx) != 0
                ):  # was detected in last frame and dyn. cropping was performed >>
                    # but object lost in cropped variant >> re-run on full frame!
                    # print("looking again, lost!")
                    if cfg["cropping"]:
                        frame = img_as_ubyte(originalframe[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])
                    else:
                        frame = img_as_ubyte(originalframe)
                    pose = predict.getpose(frame, dlc_cfg, sess, inputs, outputs).flatten()  # no offset is necessary

                _x0, _y0 = x1, y1
                x1, x2, y1, y2 = 0, nx, 0, ny
                detected = False

            PredictedData[counter, :] = pose
        elif counter >= nframes:
            break
        counter += 1

    pbar.close()
    return PredictedData, nframes

GetPoseF

GetPoseF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize)

Batchwise prediction of pose.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPoseF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize):
    """Batchwise prediction of pose."""
    PredictedData = np.zeros((nframes, dlc_cfg["num_outputs"] * 3 * len(dlc_cfg["all_joints_names"])))
    batch_ind = 0  # keeps track of which image within a batch should be written to
    batch_num = 0  # keeps track of which batch you are at
    ny, nx = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)), int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    if cfg["cropping"]:
        ny, nx = checkcropping(cfg, cap)

    frames = np.empty((batchsize, ny, nx, 3), dtype="ubyte")  # this keeps all frames in a batch
    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))
    inds = []
    while cap.isOpened():
        if counter != 0 and counter % step == 0:
            pbar.update(step)
        ret, frame = cap.read()
        if ret:
            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            if cfg["cropping"]:
                frames[batch_ind] = img_as_ubyte(frame[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])
            else:
                frames[batch_ind] = img_as_ubyte(frame)
            inds.append(counter)
            if batch_ind == batchsize - 1:
                pose = predict.getposeNP(frames, dlc_cfg, sess, inputs, outputs)
                PredictedData[inds] = pose
                batch_ind = 0
                inds.clear()
                batch_num += 1
            else:
                batch_ind += 1
        elif counter >= nframes:
            if batch_ind > 0:
                pose = predict.getposeNP(
                    frames, dlc_cfg, sess, inputs, outputs
                )  # process the whole batch (some frames might be from previous batch!)
                PredictedData[inds[:batch_ind]] = pose[:batch_ind]
            break
        counter += 1

    pbar.close()
    return PredictedData, nframes

GetPoseF_GTF

GetPoseF_GTF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize)

Batchwise prediction of pose.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPoseF_GTF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize):
    """Batchwise prediction of pose."""
    PredictedData = np.zeros((nframes, 3 * len(dlc_cfg["all_joints_names"])))
    batch_ind = 0  # keeps track of which image within a batch should be written to
    batch_num = 0  # keeps track of which batch you are at
    ny = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    nx = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    if cfg["cropping"]:
        ny, nx = checkcropping(cfg, cap)

    # Flip x, y, confidence and reshape
    pose_tensor = predict.extract_GPUprediction(outputs, dlc_cfg)
    pose_tensor = tf.gather(pose_tensor, [1, 0, 2], axis=1)
    pose_tensor = tf.reshape(pose_tensor, (batchsize, -1))

    frames = np.empty((batchsize, ny, nx, 3), dtype="ubyte")
    pbar = tqdm(total=nframes)
    counter = -1
    inds = []
    while cap.isOpened() and counter < nframes - 1:
        ret, frame = cap.read()
        counter += 1
        if not ret:
            warnings.warn(f"Could not decode frame #{counter}.", stacklevel=2)
            continue

        if cfg["cropping"]:
            frame = img_as_ubyte(frame[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])
        else:
            frame = img_as_ubyte(frame)
        frames[batch_ind] = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        inds.append(counter)
        if batch_ind == batchsize - 1:
            pose = sess.run(pose_tensor, feed_dict={inputs: frames})
            PredictedData[inds] = pose
            batch_ind = 0
            batch_num += 1
            inds.clear()
            pbar.update(batchsize)
        else:
            batch_ind += 1

    if batch_ind > 0:
        pose = sess.run(pose_tensor, feed_dict={inputs: frames})
        PredictedData[inds[:batch_ind]] = pose[:batch_ind]
        pbar.update(batch_ind)

    pbar.close()
    return PredictedData, nframes

GetPoseF_OV

GetPoseF_OV(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize)

Prediction of pose.

Source code in deeplabcut/pose_estimation_tensorflow/core/openvino/session.py

def GetPoseF_OV(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize):
    """Prediction of pose."""
    PredictedData = np.zeros((nframes, 3 * len(dlc_cfg["all_joints_names"])))
    ny, nx = int(cap.get(4)), int(cap.get(3))
    if cfg["cropping"]:
        from ...predict_videos import checkcropping

        ny, nx = checkcropping(cfg, cap)

    sess._init_model(ny, nx)

    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))

    def completion_callback(request, inp_id):
        pose = next(iter(request.results.values()))

        pose[:, [0, 1, 2]] = pose[:, [1, 0, 2]]  # change order to have x,y,confidence
        pose = np.reshape(pose, (1, -1))  # bring into batchsize times x,y,conf etc.
        PredictedData[inp_id] = pose

    sess.infer_queue.set_callback(completion_callback)

    while cap.isOpened():
        if counter % step == 0:
            pbar.update(step)
        ret, frame = cap.read()
        if not ret:
            break

        # Prepare input data
        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        if cfg["cropping"]:
            frame = frame[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]]

        sess.infer_queue.start_async({sess.input_name: np.expand_dims(frame, axis=0)}, counter)

        counter += 1

    sess.infer_queue.wait_all()

    pbar.close()
    return PredictedData, nframes

GetPoseS

GetPoseS(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes)

Non batch wise pose estimation for video cap.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPoseS(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes):
    """Non batch wise pose estimation for video cap."""
    if cfg["cropping"]:
        ny, nx = checkcropping(cfg, cap)

    PredictedData = np.zeros((nframes, dlc_cfg["num_outputs"] * 3 * len(dlc_cfg["all_joints_names"])))
    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))
    while cap.isOpened():
        if counter != 0 and counter % step == 0:
            pbar.update(step)

        ret, frame = cap.read()
        if ret:
            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            if cfg["cropping"]:
                frame = img_as_ubyte(frame[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])
            else:
                frame = img_as_ubyte(frame)
            pose = predict.getpose(frame, dlc_cfg, sess, inputs, outputs)
            PredictedData[counter, :] = (
                pose.flatten()
            )  # NOTE: thereby cfg['all_joints_names'] should be same order as bodyparts!
        elif counter >= nframes:
            break
        counter += 1

    pbar.close()
    return PredictedData, nframes

GetPoseS_GTF

GetPoseS_GTF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes)

Non batch wise pose estimation for video cap.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPoseS_GTF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes):
    """Non batch wise pose estimation for video cap."""
    if cfg["cropping"]:
        ny, nx = checkcropping(cfg, cap)

    pose_tensor = predict.extract_GPUprediction(outputs, dlc_cfg)  # extract_output_tensor(outputs, dlc_cfg)
    PredictedData = np.zeros((nframes, 3 * len(dlc_cfg["all_joints_names"])))
    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))
    while cap.isOpened():
        if counter != 0 and counter % step == 0:
            pbar.update(step)

        ret, frame = cap.read()
        if ret:
            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            if cfg["cropping"]:
                frame = img_as_ubyte(frame[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])
            else:
                frame = img_as_ubyte(frame)

            pose = sess.run(
                pose_tensor,
                feed_dict={inputs: np.expand_dims(frame, axis=0).astype(float)},
            )
            pose[:, [0, 1, 2]] = pose[:, [1, 0, 2]]
            # pose = predict.getpose(frame, dlc_cfg, sess, inputs, outputs)
            PredictedData[counter, :] = (
                pose.flatten()
            )  # NOTE: thereby cfg['all_joints_names'] should be same order as bodyparts!
        elif counter >= nframes:
            break
        counter += 1

    pbar.close()
    return PredictedData, nframes

GetPosesofFrames

GetPosesofFrames(cfg, dlc_cfg, sess, inputs, outputs, directory, framelist, nframes, batchsize)

Batchwise prediction of pose for frame list in directory.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPosesofFrames(cfg, dlc_cfg, sess, inputs, outputs, directory, framelist, nframes, batchsize):
    """Batchwise prediction of pose for frame list in directory."""
    from deeplabcut.utils.auxfun_videos import imread

    print("Starting to extract posture")
    im = imread(os.path.join(directory, framelist[0]), mode="skimage")

    ny, nx, nc = np.shape(im)
    print(
        "Overall # of frames: ",
        nframes,
        " found with (before cropping) frame dimensions: ",
        nx,
        ny,
    )

    PredictedData = np.zeros((nframes, dlc_cfg["num_outputs"] * 3 * len(dlc_cfg["all_joints_names"])))
    batch_ind = 0  # keeps track of which image within a batch should be written to
    batch_num = 0  # keeps track of which batch you are at
    if cfg["cropping"]:
        print(
            "Cropping based on the x1 = {} x2 = {} y1 = {} y2 = {}. "
            "You can adjust the cropping coordinates in the config.yaml file.".format(
                cfg["x1"], cfg["x2"], cfg["y1"], cfg["y2"]
            )
        )
        nx, ny = cfg["x2"] - cfg["x1"], cfg["y2"] - cfg["y1"]
        if nx > 0 and ny > 0:
            pass
        else:
            raise Exception("Please check the order of cropping parameter!")
        if cfg["x1"] >= 0 and cfg["x2"] < int(np.shape(im)[1]) and cfg["y1"] >= 0 and cfg["y2"] < int(np.shape(im)[0]):
            pass  # good cropping box
        else:
            raise Exception("Please check the boundary of cropping!")

    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))

    if batchsize == 1:
        for counter, framename in enumerate(framelist):
            im = imread(os.path.join(directory, framename), mode="skimage")

            if counter != 0 and counter % step == 0:
                pbar.update(step)

            if cfg["cropping"]:
                frame = img_as_ubyte(im[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"], :])
            else:
                frame = img_as_ubyte(im)

            pose = predict.getpose(frame, dlc_cfg, sess, inputs, outputs)
            PredictedData[counter, :] = pose.flatten()
    else:
        frames = np.empty((batchsize, ny, nx, 3), dtype="ubyte")  # this keeps all the frames of a batch
        for counter, framename in enumerate(framelist):
            im = imread(os.path.join(directory, framename), mode="skimage")

            if counter != 0 and counter % step == 0:
                pbar.update(step)

            if cfg["cropping"]:
                frames[batch_ind] = img_as_ubyte(im[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"], :])
            else:
                frames[batch_ind] = img_as_ubyte(im)

            if batch_ind == batchsize - 1:
                pose = predict.getposeNP(frames, dlc_cfg, sess, inputs, outputs)
                PredictedData[batch_num * batchsize : (batch_num + 1) * batchsize, :] = pose
                batch_ind = 0
                batch_num += 1
            else:
                batch_ind += 1

        if batch_ind > 0:  # take care of the last frames (the batch that might have been processed)
            pose = predict.getposeNP(
                frames, dlc_cfg, sess, inputs, outputs
            )  # process the whole batch (some frames might be from previous batch!)
            PredictedData[batch_num * batchsize : batch_num * batchsize + batch_ind, :] = pose[:batch_ind, :]

    pbar.close()
    return PredictedData, nframes, nx, ny

Plotting

Plotting(cfg, comparisonbodyparts, DLCscorer, trainIndices, DataCombined, foldername)

Function used for plotting GT and predictions.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def Plotting(cfg, comparisonbodyparts, DLCscorer, trainIndices, DataCombined, foldername):
    """Function used for plotting GT and predictions."""
    from deeplabcut.utils import visualization

    colors = visualization.get_cmap(len(comparisonbodyparts), name=cfg["colormap"])
    NumFrames = np.size(DataCombined.index)
    fig, ax = visualization.create_minimal_figure()
    for ind in tqdm(np.arange(NumFrames)):
        ax = visualization.plot_and_save_labeled_frame(
            DataCombined,
            ind,
            trainIndices,
            cfg,
            colors,
            comparisonbodyparts,
            DLCscorer,
            foldername,
            fig,
            ax,
        )
        visualization.erase_artists(ax)

analyze_time_lapse_frames

analyze_time_lapse_frames(
    config, directory, frametype=".png", shuffle=1, trainingsetindex=0, gputouse=None, save_as_csv=False, modelprefix=""
)

Analyzed all images (of type = frametype) in a folder and stores the output in one file.

You can crop the frames (before analysis), by changing 'cropping'=True and setting 'x1','x2','y1','y2' in the config file.

Output: The labels are stored as MultiIndex Pandas Array, which contains the name of the network, body part name, (x, y) label position

in pixels, and the likelihood for each frame per body part. These arrays are stored in an efficient Hierarchical Data Format (HDF)

in the same directory, where the video is stored. However, if the flag save_as_csv is set to True, the data can also be exported in

comma-separated values format (.csv), which in turn can be imported in many programs, such as MATLAB, R, Prism, etc.

Parameters

config : string Full path of the config.yaml file as a string.

string

Full path to directory containing the frames that shall be analyzed

string, optional

Checks for the file extension of the frames. Only images with this extension are analyzed. The default is .png

int, optional

An integer specifying the shuffle index of the training dataset used for training the network. The default is 1.

int, optional

Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml).

gputouse: int, optional. Natural number indicating the number of your GPU (see number in nvidia-smi). If you do not have a GPU, set to None. See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries

bool, optional

Saves the predictions in a .csv file. The default is False; if provided it must be either True or False

Examples

If you want to analyze all frames in /analysis/project/timelapseexperiment1

deeplabcut.analyze_videos( '/analysis/project/reaching-task/config.yaml', '/analysis/project/timelapseexperiment1' )

---

Note: for test purposes one can extract all frames from a video with ffmpeg, e.g. ffmpeg -i testvideo.avi thumb%04d.png

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def analyze_time_lapse_frames(
    config,
    directory,
    frametype=".png",
    shuffle=1,
    trainingsetindex=0,
    gputouse=None,
    save_as_csv=False,
    modelprefix="",
):
    """Analyzed all images (of type = frametype) in a folder and stores the output in
    one file.

    You can crop the frames (before analysis),
    by changing 'cropping'=True and setting 'x1','x2','y1','y2' in the config file.

    Output:
    The labels are stored as MultiIndex Pandas Array,
    which contains the name of the network, body part name, (x, y) label position \n
    in pixels, and the likelihood for each frame per body part.
    These arrays are stored in an efficient Hierarchical Data Format (HDF) \n
    in the same directory, where the video is stored.
    However, if the flag save_as_csv is set to True, the data can also be exported in \n
    comma-separated values format (.csv), which in turn can be imported in many programs, such as MATLAB, R, Prism, etc.

    Parameters
    ----------
    config : string
        Full path of the config.yaml file as a string.

    directory: string
        Full path to directory containing the frames that shall be analyzed

    frametype: string, optional
        Checks for the file extension of the frames.
        Only images with this extension are analyzed. The default is ``.png``

    shuffle: int, optional
        An integer specifying the shuffle index of the training dataset used for training the network. The default is 1.

    trainingsetindex: int, optional
        Integer specifying which TrainingsetFraction to use.
        By default the first (note that TrainingFraction is a list in config.yaml).

    gputouse: int, optional. Natural number indicating the number of your GPU (see number in nvidia-smi).
    If you do not have a GPU, set to None.
    See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries

    save_as_csv: bool, optional
        Saves the predictions in a .csv file. The default is ``False``;
        if provided it must be either ``True`` or ``False``

    Examples
    --------
    If you want to analyze all frames in /analysis/project/timelapseexperiment1
    >>> deeplabcut.analyze_videos(
        '/analysis/project/reaching-task/config.yaml',
        '/analysis/project/timelapseexperiment1'
        )
    --------

    Note: for test purposes one can extract all frames from a video with ffmpeg,
    e.g. ffmpeg -i testvideo.avi thumb%04d.png
    """
    if "TF_CUDNN_USE_AUTOTUNE" in os.environ:
        del os.environ["TF_CUDNN_USE_AUTOTUNE"]  # was potentially set during training

    if gputouse is not None:  # gpu selection
        auxfun_models.set_visible_devices(gputouse)

    tf.compat.v1.reset_default_graph()
    start_path = os.getcwd()  # record cwd to return to this directory in the end

    cfg = auxiliaryfunctions.read_config(config)
    trainFraction = cfg["TrainingFraction"][trainingsetindex]
    modelfolder = os.path.join(
        cfg["project_path"],
        str(auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)),
    )
    path_test_config = Path(modelfolder) / "test" / "pose_cfg.yaml"
    try:
        dlc_cfg = load_config(str(path_test_config))
    except FileNotFoundError as e:
        raise FileNotFoundError(
            f"It seems the model for shuffle {shuffle} and trainFraction {trainFraction} does not exist."
        ) from e

    Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
        train_folder=Path(modelfolder) / "train",
    )

    if cfg["snapshotindex"] == "all":
        print(
            "Snapshotindex is set to 'all' in the config.yaml file. "
            "Running video analysis with all snapshots is very costly! "
            "Use the function 'evaluate_network' to choose the best the snapshot. "
            "For now, changing snapshot index to -1!"
        )
        snapshotindex = -1
    else:
        snapshotindex = cfg["snapshotindex"]

    print(f"Using {Snapshots[snapshotindex]}", "for model", modelfolder)

    ##################################################
    # Load and setup CNN part detector
    ##################################################

    # Check if data already was generated:
    dlc_cfg["init_weights"] = os.path.join(modelfolder, "train", Snapshots[snapshotindex])
    trainingsiterations = (dlc_cfg["init_weights"].split(os.sep)[-1]).split("-")[-1]

    # update batchsize (based on parameters in config.yaml)
    dlc_cfg["batch_size"] = cfg["batch_size"]

    # Name for scorer:
    DLCscorer, DLCscorerlegacy = auxiliaryfunctions.get_scorer_name(
        cfg,
        shuffle,
        trainFraction,
        trainingsiterations=trainingsiterations,
        modelprefix=modelprefix,
    )
    sess, inputs, outputs = predict.setup_pose_prediction(dlc_cfg)

    # update number of outputs and adjust pandas indices
    dlc_cfg["num_outputs"] = cfg.get("num_outputs", 1)

    xyz_labs_orig = ["x", "y", "likelihood"]
    suffix = [str(s + 1) for s in range(dlc_cfg["num_outputs"])]
    suffix[0] = ""  # first one has empty suffix for backwards compatibility
    xyz_labs = [x + s for s in suffix for x in xyz_labs_orig]

    pdindex = pd.MultiIndex.from_product(
        [[DLCscorer], dlc_cfg["all_joints_names"], xyz_labs],
        names=["scorer", "bodyparts", "coords"],
    )

    if gputouse is not None:  # gpu selectinon
        auxfun_models.set_visible_devices(gputouse)

    ##################################################
    # Loading the images
    ##################################################
    # checks if input is a directory
    if os.path.isdir(directory):
        """Analyzes all the frames in the directory."""
        print("Analyzing all frames in the directory: ", directory)
        os.chdir(directory)
        framelist = np.sort([fn for fn in os.listdir(os.curdir) if (frametype in fn)])
        vname = Path(directory).stem
        notanalyzed, dataname, DLCscorer = auxiliaryfunctions.check_if_not_analyzed(
            directory, vname, DLCscorer, DLCscorerlegacy, flag="framestack"
        )
        if notanalyzed:
            nframes = len(framelist)
            if nframes > 0:
                start = time.time()

                PredictedData, nframes, nx, ny = GetPosesofFrames(
                    cfg,
                    dlc_cfg,
                    sess,
                    inputs,
                    outputs,
                    directory,
                    framelist,
                    nframes,
                    dlc_cfg["batch_size"],
                )
                stop = time.time()

                if cfg["cropping"]:
                    coords = [cfg["x1"], cfg["x2"], cfg["y1"], cfg["y2"]]
                else:
                    coords = [0, nx, 0, ny]

                dictionary = {
                    "start": start,
                    "stop": stop,
                    "run_duration": stop - start,
                    "Scorer": DLCscorer,
                    "config file": dlc_cfg,
                    "batch_size": dlc_cfg["batch_size"],
                    "num_outputs": dlc_cfg["num_outputs"],
                    "frame_dimensions": (ny, nx),
                    "nframes": nframes,
                    "cropping": cfg["cropping"],
                    "cropping_parameters": coords,
                }
                metadata = {"data": dictionary}

                print(f"Saving results in {directory}...")

                auxiliaryfunctions.save_data(
                    PredictedData[:nframes, :],
                    metadata,
                    dataname,
                    pdindex,
                    framelist,
                    save_as_csv,
                )
                print("The folder was analyzed. Now your research can truly start!")
                print("If the tracking is not satisfactory for some frame, consider expanding the training set.")
            else:
                print("No frames were found. Consider changing the path or the frametype.")

    os.chdir(str(start_path))

analyze_videos

analyze_videos(
    config,
    videos,
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    gputouse=None,
    save_as_csv=False,
    in_random_order=True,
    destfolder=None,
    batchsize=None,
    cropping=None,
    TFGPUinference=True,
    dynamic=(False, 0.5, 10),
    modelprefix="",
    robust_nframes=False,
    allow_growth=False,
    use_shelve=False,
    auto_track=True,
    n_tracks=None,
    animal_names=None,
    calibrate=False,
    identity_only=False,
    use_openvino="CPU" if is_openvino_available else None,
)

Makes prediction based on a trained network.

The index of the trained network is specified by parameters in the config file (in particular the variable 'snapshotindex').

The labels are stored as MultiIndex Pandas Array, which contains the name of the network, body part name, (x, y) label position in pixels, and the likelihood for each frame per body part. These arrays are stored in an efficient Hierarchical Data Format (HDF) in the same directory where the video is stored. However, if the flag save_as_csv is set to True, the data can also be exported in comma-separated values format (.csv), which in turn can be imported in many programs, such as MATLAB, R, Prism, etc.

Parameters

config: str Full path of the config.yaml file.

list[str]

A list of strings containing the full paths to videos for analysis or a path to the directory, where all the videos with same extension are stored.

str | Sequence[str] | None, optional, default=None

Controls how videos are filtered, based on file extension. File paths and directory contents are treated differently: - None (default): file paths are accepted as-is; directories are scanned for files with a recognized video extension. - str or Sequence[str] (e.g. "mp4" or ["mp4", "avi"]): both file paths and directory contents are filtered by the given extension(s).

int, optional, default=1

An integer specifying the shuffle index of the training dataset used for training the network.

int, optional, default=0

Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml).

int or None, optional, default=None

Indicates the GPU to use (see number in nvidia-smi). If you do not have a GPU put None. See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries

bool, optional, default=False

Saves the predictions in a .csv file.

bool, optional (default=True)

Whether or not to analyze videos in a random order. This is only relevant when specifying a video directory in videos.

string or None, optional, default=None

Specifies the destination folder for analysis data. If None, the path of the video is used. Note that for subsequent analysis this folder also needs to be passed.

int or None, optional, default=None

Change batch size for inference; if given overwrites value in pose_cfg.yaml.

list or None, optional, default=None

List of cropping coordinates as [x1, x2, y1, y2]. Note that the same cropping parameters will then be used for all videos. If different video crops are desired, run analyze_videos on individual videos with the corresponding cropping coordinates.

bool, optional, default=True

Perform inference on GPU with TensorFlow code. Introduced in "Pretraining boosts out-of-domain robustness for pose estimation" by Alexander Mathis, Mert Yüksekgönül, Byron Rogers, Matthias Bethge, Mackenzie W. Mathis. Source: https://arxiv.org/abs/1909.11229

tuple(bool, float, int) triple containing (state, detectiontreshold, margin)

If the state is true, then dynamic cropping will be performed. That means that if an object is detected (i.e. any body part > detectiontreshold), then object boundaries are computed according to the smallest/largest x position and smallest/largest y position of all body parts. This window is expanded by the margin and from then on only the posture within this crop is analyzed (until the object is lost, i.e. <detectiontreshold). The current position is utilized for updating the crop window for the next frame (this is why the margin is important and should be set large enough given the movement of the animal).

str, optional, default=""

Directory containing the deeplabcut models to use when evaluating the network. By default, the models are assumed to exist in the project folder.

bool, optional, default=False

Evaluate a video's number of frames in a robust manner. This option is slower (as the whole video is read frame-by-frame), but does not rely on metadata, hence its robustness against file corruption.

bool, optional, default=False.

For some smaller GPUs the memory issues happen. If True, the memory allocator does not pre-allocate the entire specified GPU memory region, instead starting small and growing as needed. See issue: https://forum.image.sc/t/how-to-stop-running-out-of-vram/30551/2

bool, optional, default=False

By default, data are dumped in a pickle file at the end of the video analysis. Otherwise, data are written to disk on the fly using a "shelf"; i.e., a pickle-based, persistent, database-like object by default, resulting in constant memory footprint.

The following parameters are only relevant for multi-animal projects:

bool, optional, default=True

By default, tracking and stitching are automatically performed, producing the final h5 data file. This is equivalent to the behavior for single-animal projects.

If False, one must run convert_detections2tracklets and stitch_tracklets afterwards, in order to obtain the h5 file.

This function has 3 related sub-calls:

bool, optional, default=False

If True and animal identity was learned by the model, assembly and tracking rely exclusively on identity prediction.

bool, optional, default=False

If True, use training data to calibrate the animal assembly procedure. This improves its robustness to wrong body part links, but requires very little missing data.

int or None, optional, default=None

Number of tracks to reconstruct. By default, taken as the number of individuals defined in the config.yaml. Another number can be passed if the number of animals in the video is different from the number of animals the model was trained on.

list[str], optional

If you want the names given to individuals in the labeled data file, you can specify those names as a list here. If given and n_tracks is None, n_tracks will be set to len(animal_names). If n_tracks is not None, then it must be equal to len(animal_names). If it is not given, then animal_names will be loaded from the individuals in the project config.yaml file.

str, optional

Use "CPU" for inference if OpenVINO is available in the Python environment.

Returns

DLCScorer: str the scorer used to analyze the videos

Examples

Analyzing a single video on Windows

deeplabcut.analyze_videos( 'C:\myproject\reaching-task\config.yaml', ['C:\yourusername\rig-95\Videos\reachingvideo1.avi'], )

Analyzing a single video on Linux/MacOS

deeplabcut.analyze_videos( '/analysis/project/reaching-task/config.yaml', ['/analysis/project/videos/reachingvideo1.avi'], )

Analyze all videos of type avi in a folder

deeplabcut.analyze_videos( '/analysis/project/reaching-task/config.yaml', ['/analysis/project/videos'], video_extensions='.avi', )

Analyze multiple videos

deeplabcut.analyze_videos( '/analysis/project/reaching-task/config.yaml', [ '/analysis/project/videos/reachingvideo1.avi', '/analysis/project/videos/reachingvideo2.avi', ], )

Analyze multiple videos with shuffle=2

deeplabcut.analyze_videos( '/analysis/project/reaching-task/config.yaml', [ '/analysis/project/videos/reachingvideo1.avi', '/analysis/project/videos/reachingvideo2.avi', ], shuffle=2, )

Analyze multiple videos with shuffle=2, save results as an additional csv file

deeplabcut.analyze_videos( '/analysis/project/reaching-task/config.yaml', [ '/analysis/project/videos/reachingvideo1.avi', '/analysis/project/videos/reachingvideo2.avi', ], shuffle=2, save_as_csv=True, )

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

@renamed_parameter(old="videotype", new="video_extensions", since="3.0.0")
def analyze_videos(
    config,
    videos,
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    gputouse=None,
    save_as_csv=False,
    in_random_order=True,
    destfolder=None,
    batchsize=None,
    cropping=None,
    TFGPUinference=True,
    dynamic=(False, 0.5, 10),
    modelprefix="",
    robust_nframes=False,
    allow_growth=False,
    use_shelve=False,
    auto_track=True,
    n_tracks=None,
    animal_names=None,
    calibrate=False,
    identity_only=False,
    use_openvino="CPU" if is_openvino_available else None,
):
    """Makes prediction based on a trained network.

    The index of the trained network is specified by parameters in the config file
    (in particular the variable 'snapshotindex').

    The labels are stored as MultiIndex Pandas Array, which contains the name of
    the network, body part name, (x, y) label position in pixels, and the
    likelihood for each frame per body part. These arrays are stored in an
    efficient Hierarchical Data Format (HDF) in the same directory where the video
    is stored. However, if the flag save_as_csv is set to True, the data can also
    be exported in comma-separated values format (.csv), which in turn can be
    imported in many programs, such as MATLAB, R, Prism, etc.

    Parameters
    ----------
    config: str
        Full path of the config.yaml file.

    videos: list[str]
        A list of strings containing the full paths to videos for analysis or a path to
        the directory, where all the videos with same extension are stored.

    video_extensions : str | Sequence[str] | None, optional, default=None
        Controls how ``videos`` are filtered, based on file extension.
        File paths and directory contents are treated differently:
        - ``None`` (default): file paths are accepted as-is; directories are
          scanned for files with a recognized video extension.
        - ``str`` or ``Sequence[str]`` (e.g. ``"mp4"`` or ``["mp4", "avi"]``):
          both file paths and directory contents are filtered by the given
          extension(s).

    shuffle: int, optional, default=1
        An integer specifying the shuffle index of the training dataset used for
        training the network.

    trainingsetindex: int, optional, default=0
        Integer specifying which TrainingsetFraction to use.
        By default the first (note that TrainingFraction is a list in config.yaml).

    gputouse: int or None, optional, default=None
        Indicates the GPU to use (see number in ``nvidia-smi``). If you do not have a
        GPU put ``None``.
        See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries

    save_as_csv: bool, optional, default=False
        Saves the predictions in a .csv file.

    in_random_order: bool, optional (default=True)
        Whether or not to analyze videos in a random order.
        This is only relevant when specifying a video directory in `videos`.

    destfolder: string or None, optional, default=None
        Specifies the destination folder for analysis data. If ``None``, the path of
        the video is used. Note that for subsequent analysis this folder also needs to
        be passed.

    batchsize: int or None, optional, default=None
        Change batch size for inference; if given overwrites value in ``pose_cfg.yaml``.

    cropping: list or None, optional, default=None
        List of cropping coordinates as [x1, x2, y1, y2].
        Note that the same cropping parameters will then be used for all videos.
        If different video crops are desired, run ``analyze_videos`` on individual
        videos with the corresponding cropping coordinates.

    TFGPUinference: bool, optional, default=True
        Perform inference on GPU with TensorFlow code. Introduced in "Pretraining
        boosts out-of-domain robustness for pose estimation" by Alexander Mathis,
        Mert Yüksekgönül, Byron Rogers, Matthias Bethge, Mackenzie W. Mathis.
        Source: https://arxiv.org/abs/1909.11229

    dynamic: tuple(bool, float, int) triple containing (state, detectiontreshold, margin)
        If the state is true, then dynamic cropping will be performed. That means that if
        an object is detected (i.e. any body part > detectiontreshold),
        then object boundaries are computed according to
        the smallest/largest x position and smallest/largest y position of all body parts.
        This window is expanded by the margin and from then on only the posture within
        this crop is analyzed (until the object is lost, i.e. <detectiontreshold).
        The current position is utilized for updating the crop window for the next frame
        (this is why the margin is important and should be set large
        enough given the movement of the animal).

    modelprefix: str, optional, default=""
        Directory containing the deeplabcut models to use when evaluating the network.
        By default, the models are assumed to exist in the project folder.

    robust_nframes: bool, optional, default=False
        Evaluate a video's number of frames in a robust manner.
        This option is slower (as the whole video is read frame-by-frame),
        but does not rely on metadata, hence its robustness against file corruption.

    allow_growth: bool, optional, default=False.
        For some smaller GPUs the memory issues happen. If ``True``, the memory
        allocator does not pre-allocate the entire specified GPU memory region, instead
        starting small and growing as needed.
        See issue: https://forum.image.sc/t/how-to-stop-running-out-of-vram/30551/2

    use_shelve: bool, optional, default=False
        By default, data are dumped in a pickle file at the end of the video analysis.
        Otherwise, data are written to disk on the fly using a "shelf"; i.e., a
        pickle-based, persistent, database-like object by default, resulting in
        constant memory footprint.

    The following parameters are only relevant for multi-animal projects:

    auto_track: bool, optional, default=True
        By default, tracking and stitching are automatically performed, producing the
        final h5 data file. This is equivalent to the behavior for single-animal
        projects.

        If ``False``, one must run ``convert_detections2tracklets`` and
        ``stitch_tracklets`` afterwards, in order to obtain the h5 file.

    This function has 3 related sub-calls:

    identity_only: bool, optional, default=False
        If ``True`` and animal identity was learned by the model, assembly and tracking
        rely exclusively on identity prediction.

    calibrate: bool, optional, default=False
        If ``True``, use training data to calibrate the animal assembly procedure. This
        improves its robustness to wrong body part links, but requires very little
        missing data.

    n_tracks: int or None, optional, default=None
        Number of tracks to reconstruct. By default, taken as the number of individuals
        defined in the config.yaml. Another number can be passed if the number of
        animals in the video is different from the number of animals the model was
        trained on.

    animal_names: list[str], optional
        If you want the names given to individuals in the labeled data file, you can
        specify those names as a list here. If given and `n_tracks` is None, `n_tracks`
        will be set to `len(animal_names)`. If `n_tracks` is not None, then it must be
        equal to `len(animal_names)`. If it is not given, then `animal_names` will
        be loaded from the `individuals` in the project config.yaml file.

    use_openvino: str, optional
        Use "CPU" for inference if OpenVINO is available in the Python environment.

    Returns
    -------
    DLCScorer: str
        the scorer used to analyze the videos

    Examples
    --------

    Analyzing a single video on Windows

    >>> deeplabcut.analyze_videos(
            'C:\\myproject\\reaching-task\\config.yaml',
            ['C:\\yourusername\\rig-95\\Videos\\reachingvideo1.avi'],
        )

    Analyzing a single video on Linux/MacOS

    >>> deeplabcut.analyze_videos(
            '/analysis/project/reaching-task/config.yaml',
            ['/analysis/project/videos/reachingvideo1.avi'],
        )

    Analyze all videos of type ``avi`` in a folder

    >>> deeplabcut.analyze_videos(
            '/analysis/project/reaching-task/config.yaml',
            ['/analysis/project/videos'],
            video_extensions='.avi',
        )

    Analyze multiple videos

    >>> deeplabcut.analyze_videos(
            '/analysis/project/reaching-task/config.yaml',
            [
                '/analysis/project/videos/reachingvideo1.avi',
                '/analysis/project/videos/reachingvideo2.avi',
            ],
        )

    Analyze multiple videos with ``shuffle=2``

    >>> deeplabcut.analyze_videos(
            '/analysis/project/reaching-task/config.yaml',
            [
                '/analysis/project/videos/reachingvideo1.avi',
                '/analysis/project/videos/reachingvideo2.avi',
            ],
            shuffle=2,
        )

    Analyze multiple videos with ``shuffle=2``, save results as an additional csv file

    >>> deeplabcut.analyze_videos(
            '/analysis/project/reaching-task/config.yaml',
            [
                '/analysis/project/videos/reachingvideo1.avi',
                '/analysis/project/videos/reachingvideo2.avi',
            ],
            shuffle=2,
            save_as_csv=True,
        )
    """
    if "TF_CUDNN_USE_AUTOTUNE" in os.environ:
        del os.environ["TF_CUDNN_USE_AUTOTUNE"]  # was potentially set during training

    if gputouse is not None:  # gpu selection
        auxfun_models.set_visible_devices(gputouse)

    tf.compat.v1.reset_default_graph()
    start_path = os.getcwd()  # record cwd to return to this directory in the end

    cfg = auxiliaryfunctions.read_config(config)
    trainFraction = cfg["TrainingFraction"][trainingsetindex]
    iteration = cfg["iteration"]

    if cropping is not None:
        cfg["cropping"] = True
        cfg["x1"], cfg["x2"], cfg["y1"], cfg["y2"] = cropping
        print("Overwriting cropping parameters:", cropping)
        print("These are used for all videos, but won't be save to the cfg file.")

    modelfolder = os.path.join(
        cfg["project_path"],
        str(auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)),
    )
    path_test_config = Path(modelfolder) / "test" / "pose_cfg.yaml"
    try:
        dlc_cfg = load_config(str(path_test_config))
    except FileNotFoundError as e:
        raise FileNotFoundError(
            f"It seems the model for iteration {iteration} and shuffle "
            f"{shuffle} and trainFraction {trainFraction} does not exist."
        ) from e

    Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
        train_folder=Path(modelfolder) / "train",
    )

    if cfg["snapshotindex"] == "all":
        print(
            "Snapshotindex is set to 'all' in the config.yaml file."
            "Running video analysis with all snapshots is very costly! "
            "Use the function 'evaluate_network' to choose the best the snapshot. "
            "For now, changing snapshot index to -1!"
        )
        snapshotindex = -1
    else:
        snapshotindex = cfg["snapshotindex"]

    print(f"Using {Snapshots[snapshotindex]}", "for model", modelfolder)

    ##################################################
    # Load and setup CNN part detector
    ##################################################

    # Check if data already was generated:
    dlc_cfg["init_weights"] = os.path.join(modelfolder, "train", Snapshots[snapshotindex])
    trainingsiterations = (dlc_cfg["init_weights"].split(os.sep)[-1]).split("-")[-1]
    # Update number of output and batchsize
    dlc_cfg["num_outputs"] = cfg.get("num_outputs", dlc_cfg.get("num_outputs", 1))

    if batchsize is None:
        # update batchsize (based on parameters in config.yaml)
        dlc_cfg["batch_size"] = cfg["batch_size"]
    else:
        dlc_cfg["batch_size"] = batchsize
        cfg["batch_size"] = batchsize

    if "multi-animal" in dlc_cfg["dataset_type"]:
        dynamic = (False, 0.5, 10)  # setting dynamic mode to false
        TFGPUinference = False

    if dynamic[0]:  # state=true
        # (state,detectiontreshold,margin)=dynamic
        print("Starting analysis in dynamic cropping mode with parameters:", dynamic)
        dlc_cfg["num_outputs"] = 1
        TFGPUinference = False
        dlc_cfg["batch_size"] = 1
        print(
            "Switching batchsize to 1, num_outputs (per animal) to 1 "
            "and TFGPUinference to False (all these features are not supported in this mode)."
        )

    # Name for scorer:
    DLCscorer, DLCscorerlegacy = auxiliaryfunctions.get_scorer_name(
        cfg,
        shuffle,
        trainFraction,
        trainingsiterations=trainingsiterations,
        modelprefix=modelprefix,
    )
    if dlc_cfg["num_outputs"] > 1:
        if TFGPUinference:
            print(
                "Switching to numpy-based keypoint extraction code, "
                "as multiple point extraction is not supported by TF code currently."
            )
            TFGPUinference = False
        print("Extracting ", dlc_cfg["num_outputs"], "instances per bodypart")
        xyz_labs_orig = ["x", "y", "likelihood"]
        suffix = [str(s + 1) for s in range(dlc_cfg["num_outputs"])]
        suffix[0] = ""  # first one has empty suffix for backwards compatibility
        xyz_labs = [x + s for s in suffix for x in xyz_labs_orig]
    else:
        xyz_labs = ["x", "y", "likelihood"]

    if use_openvino:
        sess, inputs, outputs = predict.setup_openvino_pose_prediction(dlc_cfg, device=use_openvino)
    elif TFGPUinference:
        sess, inputs, outputs = predict.setup_GPUpose_prediction(dlc_cfg, allow_growth=allow_growth)
    else:
        sess, inputs, outputs = predict.setup_pose_prediction(dlc_cfg, allow_growth=allow_growth)

    pdindex = pd.MultiIndex.from_product(
        [[DLCscorer], dlc_cfg["all_joints_names"], xyz_labs],
        names=["scorer", "bodyparts", "coords"],
    )

    ##################################################
    # Looping over videos
    ##################################################
    Videos = collect_video_paths(videos, extensions=video_extensions, shuffle=in_random_order)
    if len(Videos) > 0:
        if "multi-animal" in dlc_cfg["dataset_type"]:
            from deeplabcut.pose_estimation_tensorflow.predict_multianimal import (
                AnalyzeMultiAnimalVideo,
            )

            for video in Videos:
                AnalyzeMultiAnimalVideo(
                    video,
                    DLCscorer,
                    trainFraction,
                    cfg,
                    dlc_cfg,
                    sess,
                    inputs,
                    outputs,
                    destfolder,
                    robust_nframes=robust_nframes,
                    use_shelve=use_shelve,
                )
                if auto_track:  # tracker type is taken from default in cfg
                    convert_detections2tracklets(
                        config,
                        [video],
                        video_extensions,
                        shuffle,
                        trainingsetindex,
                        destfolder=destfolder,
                        modelprefix=modelprefix,
                        calibrate=calibrate,
                        identity_only=identity_only,
                    )
                    stitch_tracklets(
                        config,
                        [video],
                        video_extensions,
                        shuffle,
                        trainingsetindex,
                        destfolder=destfolder,
                        n_tracks=n_tracks,
                        animal_names=animal_names,
                        modelprefix=modelprefix,
                        save_as_csv=save_as_csv,
                    )
        else:
            for video in Videos:
                DLCscorer = AnalyzeVideo(
                    video,
                    DLCscorer,
                    DLCscorerlegacy,
                    trainFraction,
                    cfg,
                    dlc_cfg,
                    sess,
                    inputs,
                    outputs,
                    pdindex,
                    save_as_csv,
                    destfolder,
                    TFGPUinference,
                    dynamic,
                    use_openvino,
                )

        os.chdir(str(start_path))
        if "multi-animal" in dlc_cfg["dataset_type"]:
            print(
                "The videos are analyzed. Time to assemble animals and track 'em... \n"
                " Call 'create_video_with_all_detections' to check multi-animal detection quality before tracking."
            )
            print(
                "If the tracking is not satisfactory for some videos, consider expanding the training set. "
                "You can use the function 'extract_outlier_frames' to extract a few representative outlier frames."
            )
        else:
            print(
                "The videos are analyzed. Now your research can truly start! \n "
                "You can create labeled videos with 'create_labeled_video'"
            )
            print(
                "If the tracking is not satisfactory for some videos, consider expanding the training set. "
                "You can use the function 'extract_outlier_frames' to extract a few representative outlier frames."
            )
        return DLCscorer  # note: this is either DLCscorer or DLCscorerlegacy depending on what was used!
    else:
        print("No video(s) were found. Please check your paths and/or video_extensions filter.")
        return DLCscorer

argmax_pose_predict

argmax_pose_predict(scmap, offmat, stride)

Combine scoremat and offsets to the final pose.

Source code in deeplabcut/pose_estimation_tensorflow/core/predict.py

def argmax_pose_predict(scmap, offmat, stride):
    """Combine scoremat and offsets to the final pose."""
    num_joints = scmap.shape[2]
    pose = []
    for joint_idx in range(num_joints):
        maxloc = np.unravel_index(np.argmax(scmap[:, :, joint_idx]), scmap[:, :, joint_idx].shape)
        offset = np.array(offmat[maxloc][joint_idx])[::-1]
        pos_f8 = np.array(maxloc).astype("float") * stride + 0.5 * stride + offset
        pose.append(np.hstack((pos_f8[::-1], [scmap[maxloc][joint_idx]])))
    return np.array(pose)

calculatepafdistancebounds

calculatepafdistancebounds(config, shuffle=0, trainingsetindex=0, modelprefix='', numdigits=0, onlytrain=False)

Returns distances along paf edges in train/test data

---

config : string Full path of the config.yaml file as a string.

integer

integers specifying shuffle index of the training dataset. The default is 0.

int, optional

Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). This variable can also be set to "all".

numdigits: number of digits to round for distances.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def calculatepafdistancebounds(config, shuffle=0, trainingsetindex=0, modelprefix="", numdigits=0, onlytrain=False):
    """
    Returns distances along paf edges in train/test data

    ----------
    config : string
        Full path of the config.yaml file as a string.

    shuffle: integer
        integers specifying shuffle index of the training dataset. The default is 0.

    trainingsetindex: int, optional
        Integer specifying which TrainingsetFraction to use.
        By default the first (note that TrainingFraction is a list in config.yaml). This
        variable can also be set to "all".

    numdigits: number of digits to round for distances.

    """
    import os

    from deeplabcut.pose_estimation_tensorflow.config import load_config
    from deeplabcut.utils import auxfun_multianimal, auxiliaryfunctions

    # Read file path for pose_config file. >> pass it on
    cfg = auxiliaryfunctions.read_config(config)

    if cfg["multianimalproject"]:
        (
            individuals,
            uniquebodyparts,
            multianimalbodyparts,
        ) = auxfun_multianimal.extractindividualsandbodyparts(cfg)

        # Loading human annotatated data
        trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg)
        trainFraction = cfg["TrainingFraction"][trainingsetindex]
        modelfolder = os.path.join(
            cfg["project_path"],
            str(auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)),
        )

        # Load meta data & annotations
        Data = pd.read_hdf(
            os.path.join(
                cfg["project_path"],
                str(trainingsetfolder),
                "CollectedData_" + cfg["scorer"] + ".h5",
            )
        )[cfg["scorer"]]

        path_train_config, path_test_config, _ = return_train_network_path(
            config=config,
            shuffle=shuffle,
            trainingsetindex=trainingsetindex,
            modelprefix=modelprefix,
        )
        train_pose_cfg = load_config(str(path_train_config))
        test_pose_cfg = load_config(str(path_test_config))

        _, trainIndices, _, _ = auxiliaryfunctions.load_metadata(
            Path(cfg["project_path"]) / train_pose_cfg["metadataset"]
        )

        # get the graph!
        partaffinityfield_graph = test_pose_cfg["partaffinityfield_graph"]
        jointnames = [test_pose_cfg["all_joints_names"][i] for i in range(len(test_pose_cfg["all_joints"]))]
        path_inferencebounds_config = Path(modelfolder) / "test" / "inferencebounds.yaml"
        inferenceboundscfg = {}
        for _pi, edge in enumerate(partaffinityfield_graph):
            j1, j2 = jointnames[edge[0]], jointnames[edge[1]]
            ds_within = []
            ds_across = []
            for ind in individuals:
                for ind2 in individuals:
                    if ind != "single" and ind2 != "single":
                        if (ind, j1, "x") in Data.keys() and (
                            ind2,
                            j2,
                            "y",
                        ) in Data.keys():
                            distances = (
                                np.sqrt(
                                    (Data[ind, j1, "x"] - Data[ind2, j2, "x"]) ** 2
                                    + (Data[ind, j1, "y"] - Data[ind2, j2, "y"]) ** 2
                                )
                                / test_pose_cfg["stride"]
                            )
                        else:
                            distances = None

                        if distances is not None:
                            if onlytrain:
                                distances = distances.iloc[trainIndices]
                            if ind == ind2:
                                ds_within.extend(distances.values.flatten())
                            else:
                                ds_across.extend(distances.values.flatten())

            edgeencoding = str(edge[0]) + "_" + str(edge[1])
            inferenceboundscfg[edgeencoding] = {}
            if len(ds_within) > 0:
                inferenceboundscfg[edgeencoding]["intra_max"] = str(round(np.nanmax(ds_within), numdigits))
                inferenceboundscfg[edgeencoding]["intra_min"] = str(round(np.nanmin(ds_within), numdigits))
            else:
                inferenceboundscfg[edgeencoding]["intra_max"] = str(
                    1e5
                )  # large number (larger than any image diameter)
                inferenceboundscfg[edgeencoding]["intra_min"] = str(0)

            # NOTE: the inter-animal distances are currently not used, but are interesting to compare to intra_*
            if len(ds_across) > 0:
                inferenceboundscfg[edgeencoding]["inter_max"] = str(round(np.nanmax(ds_across), numdigits))
                inferenceboundscfg[edgeencoding]["inter_min"] = str(round(np.nanmin(ds_across), numdigits))
            else:
                inferenceboundscfg[edgeencoding]["inter_max"] = str(
                    1e5
                )  # large number (larger than image diameters in typical experiments)
                inferenceboundscfg[edgeencoding]["inter_min"] = str(0)

        auxiliaryfunctions.write_plainconfig(str(path_inferencebounds_config), dict(inferenceboundscfg))
        return inferenceboundscfg
    else:
        print("You might as well bring owls to Athens.")
        return {}

cfg_from_file

cfg_from_file(filename)

Load a config from file filename and merge it into the default options.

Source code in deeplabcut/pose_estimation_tensorflow/config.py

def cfg_from_file(filename):
    """Load a config from file filename and merge it into the default options."""
    with open(filename) as f:
        yaml_cfg = yaml.load(f, Loader=yaml.SafeLoader)

    # Update the snapshot path to the corresponding path!
    trainpath = str(filename).split("pose_cfg.yaml")[0]
    yaml_cfg["snapshot_prefix"] = trainpath + "snapshot"
    # the default is: "./snapshot"

    # reloading defaults, as they can bleed over from a previous run otherwise
    import importlib

    from . import default_config

    importlib.reload(default_config)

    default_cfg = default_config.cfg
    _merge_a_into_b(yaml_cfg, default_cfg)

    logging.info("Config:\n" + pprint.pformat(default_cfg))
    return default_cfg  # updated

collect_video_paths

collect_video_paths(
    data_path: str | Path | list[str | Path],
    extensions: str | Sequence[str] | None = None,
    shuffle: bool = False,
    exclude_patterns: Sequence[str] = DEFAULT_EXCLUDE_PATTERNS,
) -> list[Path]

Collects video paths from a given set of data paths: directories, files, or a mix of both. Directories are scanned one level deep (non-recursively).

Files and directories are treated differently with respect to extension filtering: - File paths are accepted as-is when extensions is None; only filtered when extensions is explicitly set. - Directory contents are always filtered by extension: by SUPPORTED_VIDEOS when extensions is None, or by the given value(s) otherwise. - exclude_patterns are always applied to both files and directory contents.

Parameters:

Name	Type	Description	Default
data_path	str | Path | list[str | Path]	Path or list of paths to folders containing videos, or individual video files. Can be a mix of directories and files.	required
extensions	str | Sequence[str] | None	Controls extension filtering for collected video files. - None (default): file paths are accepted without extension filtering; directories are scanned for files with a recognized video extension. - str or Sequence[str] (e.g. "mp4" or ["mp4", "avi"]): both file paths and directory contents are filtered to only include files matching the given extension(s). - Empty str "" is treated as None (deprecated, keep for backwards compatibility).	None
shuffle	bool	Whether to shuffle the order of videos. If False, videos are returned in sorted order for deterministic behavior.	False
exclude_patterns	Sequence[str]	Patterns to exclude from the collection. Defaults to DEFAULT_EXCLUDE_PATTERNS. Set to [] to disable pattern exclusion.	DEFAULT_EXCLUDE_PATTERNS

Returns:

Type	Description
list[Path]	The paths of videos to analyze. Duplicate paths are removed.

Raises:

Type	Description
FileNotFoundError	If any path in data_path does not exist.
ValueError	If extensions is an empty sequence.

Source code in deeplabcut/utils/auxfun_videos.py

def collect_video_paths(
    data_path: str | Path | list[str | Path],
    extensions: str | Sequence[str] | None = None,
    shuffle: bool = False,
    exclude_patterns: Sequence[str] = DEFAULT_EXCLUDE_PATTERNS,
) -> list[Path]:
    """
    Collects video paths from a given set of data paths: directories, files, or a mix
    of both. Directories are scanned one level deep (non-recursively).

    Files and directories are treated differently with respect to extension filtering:
    - File paths are accepted as-is when ``extensions`` is ``None``; only filtered when
      ``extensions`` is explicitly set.
    - Directory contents are always filtered by extension: by ``SUPPORTED_VIDEOS`` when
      ``extensions`` is ``None``, or by the given value(s) otherwise.
    - ``exclude_patterns`` are always applied to both files and directory contents.

    Args:
        data_path: Path or list of paths to folders containing videos, or individual
            video files. Can be a mix of directories and files.
        extensions: Controls extension filtering for collected video files.
            - ``None`` (default): file paths are accepted without extension filtering;
              directories are scanned for files with a recognized video extension.
            - ``str`` or ``Sequence[str]`` (e.g. ``"mp4"`` or ``["mp4", "avi"]``):
              both file paths and directory contents are filtered to only include files
              matching the given extension(s).
            - Empty ``str`` ``""`` is treated as ``None`` (deprecated, keep for backwards
              compatibility).
        shuffle: Whether to shuffle the order of videos. If ``False``, videos are
            returned in sorted order for deterministic behavior.
        exclude_patterns: Patterns to exclude from the collection. Defaults to
            ``DEFAULT_EXCLUDE_PATTERNS``. Set to ``[]`` to disable pattern exclusion.

    Returns:
        The paths of videos to analyze. Duplicate paths are removed.

    Raises:
        FileNotFoundError: If any path in ``data_path`` does not exist.
        ValueError: If ``extensions`` is an empty sequence.
    """
    if isinstance(data_path, (str, Path)):
        data_path = [data_path]

    def _coerce_extensions(extensions: str | Sequence[str] | None) -> set[str] | None:
        """Coerce the extensions argument to a set of dot-prefixed suffixes, or None."""
        if extensions is None:
            return None

        if extensions in ["", ("",), [""], {""}]:
            warnings.warn(
                "Passing an empty string for filtering video type extensions is deprecated; pass None instead.",
                DLCDeprecationWarning,
                stacklevel=3,
            )
            return None

        if isinstance(extensions, str):
            return {f".{extensions.lstrip('.').lower()}"}

        if not isinstance(extensions, Sequence):
            raise TypeError(f"extensions must be a string, a sequence or None, got {type(extensions)}")

        if len(extensions) == 0:
            raise ValueError("Video type extensions filter needs to be a non-empty sequence.")
        return {f".{e.lstrip('.').lower()}" for e in extensions}

    explicit_suffixes = _coerce_extensions(extensions)
    implicit_suffixes = {f".{ext.lower()}" for ext in SUPPORTED_VIDEOS}

    videos: list[Path] = []
    for path in map(Path, data_path):
        if not path.exists():
            raise FileNotFoundError(f"Could not find: {path}. Check access rights.")

        if path.is_dir():
            # Discriminate videos from other files; skip excluded patterns (e.g. prior DLC outputs).
            allowed = explicit_suffixes if explicit_suffixes else implicit_suffixes
            videos.extend(
                f
                for f in path.iterdir()
                if f.is_file()
                and f.suffix.lower() in allowed
                and not any(f.match(pattern) for pattern in exclude_patterns)
            )
        elif path.is_file():
            # Accept all caller-supplied files; ONLY filter extensions if set. ALWAYS filter exclude patterns.
            if explicit_suffixes is None or path.suffix.lower() in explicit_suffixes:
                if not any(path.match(pattern) for pattern in exclude_patterns):
                    videos.append(path)

    # Resolve video paths and remove duplicates
    unique_videos = list(dict.fromkeys(v.resolve() for v in videos))
    if shuffle:
        random.shuffle(unique_videos)
    else:
        unique_videos.sort()

    if any(fn.suffix.lower().lstrip(".") not in SUPPORTED_VIDEOS for fn in unique_videos if fn.suffix):
        warnings.warn(
            f"Some videos have unsupported extensions: {unique_videos} \nSupported extensions are: {SUPPORTED_VIDEOS}",
            stacklevel=2,
        )
    return unique_videos

convert_detections2tracklets

convert_detections2tracklets(
    config,
    videos,
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    overwrite=False,
    destfolder=None,
    ignore_bodyparts=None,
    inferencecfg=None,
    modelprefix="",
    greedy=False,
    calibrate=False,
    window_size=0,
    identity_only=False,
    track_method="",
)

This should be called at the end of deeplabcut.analyze_videos for multianimal projects!

Parameters

config : string Full path of the config.yaml file as a string.

list

A list of strings containing the full paths to videos for analysis or a path to the directory, where all the videos with same extension are stored.

str | Sequence[str] | None, optional, default=None

Controls how videos are filtered, based on file extension. File paths and directory contents are treated differently: - None (default): file paths are accepted as-is; directories are scanned for files with a recognized video extension. - str or Sequence[str] (e.g. "mp4" or ["mp4", "avi"]): both file paths and directory contents are filtered by the given extension(s).

int, optional

An integer specifying the shuffle index of the training dataset used for training the network. The default is 1.

int, optional

Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml).

bool, optional.

Overwrite tracks file i.e. recompute tracks from full detections and overwrite.

string, optional

Specifies the destination folder for analysis data (default is the path of the video). Note that for subsequent analysis this folder also needs to be passed.

optional

List of body part names that should be ignored during tracking (advanced). By default, all the body parts are used.

Default is None.

Configuration file for inference (assembly of individuals). Ideally should be obtained from cross validation (during evaluation). By default the parameters are loaded from inference_cfg.yaml, but these get_level_values can be overwritten.

bool, optional (default=False)

If True, use training data to calibrate the animal assembly procedure. This improves its robustness to wrong body part links, but requires very little missing data.

int, optional (default=0)

Recurrent connections in the past window_size frames are prioritized during assembly. By default, no temporal coherence cost is added, and assembly is driven mainly by part affinity costs.

bool, optional (default=False)

If True and animal identity was learned by the model, assembly and tracking rely exclusively on identity prediction.

string, optional

Specifies the tracker used to generate the pose estimation data. For multiple animals, must be either 'box', 'skeleton', or 'ellipse' and will be taken from the config.yaml file if none is given.

Examples

If you want to convert detections to tracklets:

deeplabcut.convert_detections2tracklets( '/analysis/project/reaching-task/config.yaml', ['/analysis/project/video1.mp4'], video_extensions='.mp4' )

If you want to convert detections to tracklets based on box_tracker:

deeplabcut.convert_detections2tracklets( '/analysis/project/reaching-task/config.yaml', ['/analysis/project/video1.mp4'], video_extensions='.mp4', track_method='box' )

---

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

@renamed_parameter(old="videotype", new="video_extensions", since="3.0.0")
def convert_detections2tracklets(
    config,
    videos,
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    overwrite=False,
    destfolder=None,
    ignore_bodyparts=None,
    inferencecfg=None,
    modelprefix="",
    greedy=False,
    calibrate=False,
    window_size=0,
    identity_only=False,
    track_method="",
):
    """This should be called at the end of deeplabcut.analyze_videos for multianimal
    projects!

    Parameters
    ----------
    config : string
        Full path of the config.yaml file as a string.

    videos : list
        A list of strings containing the full paths to videos for analysis
        or a path to the directory, where all the videos with same extension are stored.

    video_extensions : str | Sequence[str] | None, optional, default=None
        Controls how ``videos`` are filtered, based on file extension.
        File paths and directory contents are treated differently:
        - ``None`` (default): file paths are accepted as-is; directories are
          scanned for files with a recognized video extension.
        - ``str`` or ``Sequence[str]`` (e.g. ``"mp4"`` or ``["mp4", "avi"]``):
          both file paths and directory contents are filtered by the given
          extension(s).

    shuffle: int, optional
        An integer specifying the shuffle index of the training dataset used for training the network.
        The default is 1.

    trainingsetindex: int, optional
        Integer specifying which TrainingsetFraction to use.
        By default the first (note that TrainingFraction is a list in config.yaml).

    overwrite: bool, optional.
        Overwrite tracks file i.e. recompute tracks from full detections and overwrite.

    destfolder: string, optional
        Specifies the destination folder for analysis data (default is the path of the video).
        Note that for subsequent analysis this folder also needs to be passed.

    ignore_bodyparts: optional
        List of body part names that should be ignored during tracking (advanced).
        By default, all the body parts are used.

    inferencecfg: Default is None.
        Configuration file for inference (assembly of individuals). Ideally
        should be obtained from cross validation (during evaluation). By default
        the parameters are loaded from inference_cfg.yaml, but these get_level_values
        can be overwritten.

    calibrate: bool, optional (default=False)
        If True, use training data to calibrate the animal assembly procedure.
        This improves its robustness to wrong body part links,
        but requires very little missing data.

    window_size: int, optional (default=0)
        Recurrent connections in the past `window_size` frames are
        prioritized during assembly. By default, no temporal coherence cost
        is added, and assembly is driven mainly by part affinity costs.

    identity_only: bool, optional (default=False)
        If True and animal identity was learned by the model,
        assembly and tracking rely exclusively on identity prediction.

    track_method: string, optional
         Specifies the tracker used to generate the pose estimation data.
         For multiple animals, must be either 'box', 'skeleton', or 'ellipse'
         and will be taken from the config.yaml file if none is given.


    Examples
    --------
    If you want to convert detections to tracklets:
    >>> deeplabcut.convert_detections2tracklets(
        '/analysis/project/reaching-task/config.yaml',
        ['/analysis/project/video1.mp4'],
        video_extensions='.mp4'
        )

    If you want to convert detections to tracklets based on box_tracker:
    >>> deeplabcut.convert_detections2tracklets(
        '/analysis/project/reaching-task/config.yaml',
        ['/analysis/project/video1.mp4'],
        video_extensions='.mp4',
        track_method='box'
        )

    --------
    """
    cfg = auxiliaryfunctions.read_config(config)
    track_method = auxfun_multianimal.get_track_method(cfg, track_method=track_method)

    if len(cfg["multianimalbodyparts"]) == 1 and track_method != "box":
        warnings.warn("Switching to `box` tracker for single point tracking...", stacklevel=2)
        track_method = "box"
        cfg["default_track_method"] = track_method
        auxiliaryfunctions.write_config(config, cfg)

    trainFraction = cfg["TrainingFraction"][trainingsetindex]
    start_path = os.getcwd()  # record cwd to return to this directory in the end

    # TODO: add cropping as in video analysis!
    # if cropping is not None:
    #    cfg['cropping']=True
    #    cfg['x1'],cfg['x2'],cfg['y1'],cfg['y2']=cropping
    #    print("Overwriting cropping parameters:", cropping)
    #    print("These are used for all videos, but won't be save to the cfg file.")

    modelfolder = os.path.join(
        cfg["project_path"],
        str(auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)),
    )
    path_test_config = Path(modelfolder) / "test" / "pose_cfg.yaml"
    try:
        dlc_cfg = load_config(str(path_test_config))
    except FileNotFoundError as e:
        raise FileNotFoundError(
            f"It seems the model for shuffle {shuffle} and trainFraction {trainFraction} does not exist."
        ) from e

    if "multi-animal" not in dlc_cfg["dataset_type"]:
        raise ValueError("This function is only required for multianimal projects!")

    path_inference_config = Path(modelfolder) / "test" / "inference_cfg.yaml"
    if inferencecfg is None:  # then load or initialize
        inferencecfg = auxfun_multianimal.read_inferencecfg(path_inference_config, cfg)
    else:
        auxfun_multianimal.check_inferencecfg_sanity(cfg, inferencecfg)

    if len(cfg["multianimalbodyparts"]) == 1 and track_method != "box":
        warnings.warn("Switching to `box` tracker for single point tracking...", stacklevel=2)
        track_method = "box"
        # Also ensure `boundingboxslack` is greater than zero, otherwise overlap
        # between trackers cannot be evaluated, resulting in empty tracklets.
        inferencecfg["boundingboxslack"] = max(inferencecfg["boundingboxslack"], 40)

    Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
        train_folder=Path(modelfolder) / "train",
    )

    if cfg["snapshotindex"] == "all":
        print(
            "Snapshotindex is set to 'all' in the config.yaml file. "
            "Running video analysis with all snapshots is very costly! "
            "Use the function 'evaluate_network' to choose the best the snapshot. "
            "For now, changing snapshot index to -1!"
        )
        snapshotindex = -1
    else:
        snapshotindex = cfg["snapshotindex"]

    print(f"Using {Snapshots[snapshotindex]}", "for model", modelfolder)
    dlc_cfg["init_weights"] = os.path.join(modelfolder, "train", Snapshots[snapshotindex])
    trainingsiterations = (dlc_cfg["init_weights"].split(os.sep)[-1]).split("-")[-1]

    # Name for scorer:
    DLCscorer, DLCscorerlegacy = auxiliaryfunctions.get_scorer_name(
        cfg,
        shuffle,
        trainFraction,
        trainingsiterations=trainingsiterations,
        modelprefix=modelprefix,
    )

    ##################################################
    # Looping over videos
    ##################################################
    Videos = collect_video_paths(videos, extensions=video_extensions)
    if len(Videos) > 0:
        for video in Videos:
            print("Processing... ", video)
            videofolder = str(Path(video).parents[0])
            if destfolder is None:
                destfolder = videofolder
            auxiliaryfunctions.attempt_to_make_folder(destfolder)
            vname = Path(video).stem
            dataname = os.path.join(destfolder, vname + DLCscorer + ".h5")
            data, metadata = auxfun_multianimal.LoadFullMultiAnimalData(dataname)
            if track_method == "ellipse":
                method = "el"
            elif track_method == "box":
                method = "bx"
            else:
                method = "sk"
            trackname = dataname.split(".h5")[0] + f"_{method}.pickle"
            # NOTE: If dataname line above is changed then line below is obsolete?
            # trackname = trackname.replace(videofolder, destfolder)
            if os.path.isfile(trackname) and not overwrite:  # TODO: check if metadata are identical (same parameters!)
                print("Tracklets already computed", trackname)
                print("Set overwrite = True to overwrite.")
            else:
                print("Analyzing", dataname)
                DLCscorer = metadata["data"]["Scorer"]
                all_jointnames = data["metadata"]["all_joints_names"]

                numjoints = len(all_jointnames)

                # TODO: adjust this for multi + unique bodyparts!
                # this is only for multianimal parts and uniquebodyparts as one (not one
                # uniquebodyparts guy tracked etc. )
                bodypartlabels = [bpt for i, bpt in enumerate(all_jointnames) for _ in range(3)]
                scorers = len(bodypartlabels) * [DLCscorer]
                xylvalue = int(len(bodypartlabels) / 3) * ["x", "y", "likelihood"]
                pdindex = pd.MultiIndex.from_arrays(
                    np.vstack([scorers, bodypartlabels, xylvalue]),
                    names=["scorer", "bodyparts", "coords"],
                )

                imnames = [fn for fn in data if fn != "metadata"]

                if track_method == "box":
                    mot_tracker = trackingutils.SORTBox(
                        inferencecfg["max_age"],
                        inferencecfg["min_hits"],
                        inferencecfg.get("iou_threshold", 0.3),
                    )
                elif track_method == "skeleton":
                    mot_tracker = trackingutils.SORTSkeleton(
                        numjoints,
                        inferencecfg["max_age"],
                        inferencecfg["min_hits"],
                        inferencecfg.get("oks_threshold", 0.5),
                    )
                else:
                    mot_tracker = trackingutils.SORTEllipse(
                        inferencecfg.get("max_age", 1),
                        inferencecfg.get("min_hits", 1),
                        inferencecfg.get("iou_threshold", 0.6),
                    )
                tracklets = {}
                multi_bpts = cfg["multianimalbodyparts"]
                assembly_builder = inferenceutils.Assembler(
                    data,
                    max_n_individuals=inferencecfg["topktoretain"],
                    n_multibodyparts=len(multi_bpts),
                    greedy=greedy,
                    pcutoff=inferencecfg.get("pcutoff", 0.1),
                    min_affinity=inferencecfg.get("pafthreshold", 0.05),
                    window_size=window_size,
                    identity_only=identity_only,
                    min_n_links=inferencecfg["minimalnumberofconnections"],
                )
                assemblies_filename = dataname.split(".h5")[0] + "_assemblies.pickle"
                if not os.path.exists(assemblies_filename) or overwrite:
                    if calibrate:
                        trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg)
                        train_data_file = os.path.join(
                            cfg["project_path"],
                            str(trainingsetfolder),
                            "CollectedData_" + cfg["scorer"] + ".h5",
                        )
                        assembly_builder.calibrate(train_data_file)
                    assembly_builder.assemble()
                    assembly_builder.to_pickle(assemblies_filename)
                else:
                    assembly_builder.from_pickle(assemblies_filename)
                    print(f"Loading assemblies from {assemblies_filename}")
                try:
                    data.close()
                except AttributeError:
                    pass

                if cfg["uniquebodyparts"]:  # Initialize storage of the 'single' individual track
                    tracklets["single"] = {}
                    _single = {}
                    for index, imname in enumerate(imnames):
                        single_detection = assembly_builder.unique.get(index)
                        if single_detection is None:
                            continue
                        imindex = int(re.findall(r"\d+", imname)[0])
                        _single[imindex] = single_detection
                    tracklets["single"].update(_single)

                if inferencecfg["topktoretain"] == 1:
                    tracklets[0] = {}
                    for index, imname in tqdm(enumerate(imnames)):
                        assemblies = assembly_builder.assemblies.get(index)
                        if assemblies is None:
                            continue
                        tracklets[0][imname] = assemblies[0].data
                else:
                    keep = set(multi_bpts).difference(ignore_bodyparts or [])
                    keep_inds = sorted(multi_bpts.index(bpt) for bpt in keep)
                    for index, imname in tqdm(enumerate(imnames)):
                        assemblies = assembly_builder.assemblies.get(index)
                        if assemblies is None:
                            continue
                        animals = np.stack([assembly_builder.data for assembly_builder in assemblies])
                        if not identity_only:
                            if track_method == "box":
                                xy = trackingutils.calc_bboxes_from_keypoints(
                                    animals[:, keep_inds],
                                    inferencecfg["boundingboxslack"],
                                )  # TODO: get cropping parameters and utilize!
                            else:
                                xy = animals[:, keep_inds, :2]
                            trackers = mot_tracker.track(xy)
                        else:
                            # Optimal identity assignment based on soft voting
                            mat = np.zeros((len(assemblies), inferencecfg["topktoretain"]))
                            for nrow, assembly in enumerate(assemblies):
                                for k, v in assembly.soft_identity.items():
                                    mat[nrow, k] = v
                            inds = linear_sum_assignment(mat, maximize=True)
                            trackers = np.c_[inds][:, ::-1]
                        trackingutils.fill_tracklets(tracklets, trackers, animals, imname)

                tracklets["header"] = pdindex
                with open(trackname, "wb") as f:
                    pickle.dump(tracklets, f, pickle.HIGHEST_PROTOCOL)

        os.chdir(str(start_path))

        print(
            "The tracklets were created (i.e., under the hood deeplabcut.convert_detections2tracklets was run). "
            "Now you can 'refine_tracklets' in the GUI, or run 'deeplabcut.stitch_tracklets'."
        )
    else:
        print("No video(s) found. Please check your path!")

evaluate_network

evaluate_network(
    config,
    Shuffles=None,
    trainingsetindex=0,
    plotting=False,
    show_errors=True,
    comparisonbodyparts="all",
    gputouse=None,
    rescale=False,
    modelprefix="",
    per_keypoint_evaluation: bool = False,
    snapshots_to_evaluate: list[str] = None,
)

Evaluates the network.

Evaluates the network based on the saved models at different stages of the training network. The evaluation results are stored in the .h5 and .csv file under the subdirectory 'evaluation_results'. Change the snapshotindex parameter in the config file to 'all' in order to evaluate all the saved models.

Parameters

config : string Full path of the config.yaml file.

list, optional, default=[1]

List of integers specifying the shuffle indices of the training dataset.

int or str, optional, default=0

Integer specifying which "TrainingsetFraction" to use. Note that "TrainingFraction" is a list in config.yaml. This variable can also be set to "all".

bool or str, optional, default=False

Plots the predictions on the train and test images. If provided it must be either True, False, "bodypart", or "individual". Setting to True defaults as "bodypart" for multi-animal projects.

bool, optional, default=True

Display train and test errors.

str or list, optional, default="all"

The average error will be computed for those body parts only. The provided list has to be a subset of the defined body parts.

int or None, optional, default=None

Indicates the GPU to use (see number in nvidia-smi). If you do not have a GPU put `None``. See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries

bool, optional, default=False

Evaluate the model at the 'global_scale' variable (as set in the pose_config.yaml file for a particular project). I.e. every image will be resized according to that scale and prediction will be compared to the resized ground truth. The error will be reported in pixels at rescaled to the original size. I.e. For a [200,200] pixel image evaluated at global_scale=.5, the predictions are calculated on [100,100] pixel images, compared to 1/2*ground truth and this error is then multiplied by 2!. The evaluation images are also shown for the original size!

str, optional, default=""

Directory containing the deeplabcut models to use when evaluating the network. By default, the models are assumed to exist in the project folder.

bool, default=False

Compute the train and test RMSE for each keypoint, and save the results to a {model_name}-keypoint-results.csv in the evaluation-results folder

List[str], optional, default=None

List of snapshot names to evaluate (e.g. ["snapshot-50000", "snapshot-75000", ...])

Returns

None

Examples

If you do not want to plot and evaluate with shuffle set to 1.

deeplabcut.evaluate_network( '/analysis/project/reaching-task/config.yaml', Shuffles=[1], )

If you want to plot and evaluate with shuffle set to 0 and 1.

deeplabcut.evaluate_network( '/analysis/project/reaching-task/config.yaml', Shuffles=[0, 1], plotting=True, )

If you want to plot assemblies for a maDLC project

deeplabcut.evaluate_network( '/analysis/project/reaching-task/config.yaml', Shuffles=[1], plotting="individual", )

Note: This defaults to standard plotting for single-animal projects.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def evaluate_network(
    config,
    Shuffles=None,
    trainingsetindex=0,
    plotting=False,
    show_errors=True,
    comparisonbodyparts="all",
    gputouse=None,
    rescale=False,
    modelprefix="",
    per_keypoint_evaluation: bool = False,
    snapshots_to_evaluate: list[str] = None,
):
    """Evaluates the network.

    Evaluates the network based on the saved models at different stages of the training
    network. The evaluation results are stored in the .h5 and .csv file under the
    subdirectory 'evaluation_results'. Change the snapshotindex parameter in the config
    file to 'all' in order to evaluate all the saved models.

    Parameters
    ----------
    config : string
        Full path of the config.yaml file.

    Shuffles: list, optional, default=[1]
        List of integers specifying the shuffle indices of the training dataset.

    trainingsetindex: int or str, optional, default=0
        Integer specifying which "TrainingsetFraction" to use.
        Note that "TrainingFraction" is a list in config.yaml. This variable can also
        be set to "all".

    plotting: bool or str, optional, default=False
        Plots the predictions on the train and test images.
        If provided it must be either ``True``, ``False``, ``"bodypart"``, or
        ``"individual"``. Setting to ``True`` defaults as ``"bodypart"`` for
        multi-animal projects.

    show_errors: bool, optional, default=True
        Display train and test errors.

    comparisonbodyparts: str or list, optional, default="all"
        The average error will be computed for those body parts only.
        The provided list has to be a subset of the defined body parts.

    gputouse: int or None, optional, default=None
        Indicates the GPU to use (see number in ``nvidia-smi``). If you do not have a
        GPU put `None``.
        See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries

    rescale: bool, optional, default=False
        Evaluate the model at the ``'global_scale'`` variable (as set in the
        ``pose_config.yaml`` file for a particular project). I.e. every image will be
        resized according to that scale and prediction will be compared to the resized
        ground truth. The error will be reported in pixels at rescaled to the
        *original* size. I.e. For a [200,200] pixel image evaluated at
        ``global_scale=.5``, the predictions are calculated on [100,100] pixel images,
        compared to 1/2*ground truth and this error is then multiplied by 2!.
        The evaluation images are also shown for the original size!

    modelprefix: str, optional, default=""
        Directory containing the deeplabcut models to use when evaluating the network.
        By default, the models are assumed to exist in the project folder.

    per_keypoint_evaluation: bool, default=False
        Compute the train and test RMSE for each keypoint, and save the results to
        a {model_name}-keypoint-results.csv in the evaluation-results folder

    snapshots_to_evaluate: List[str], optional, default=None
        List of snapshot names to evaluate (e.g. ["snapshot-50000", "snapshot-75000", ...])

    Returns
    -------
    None

    Examples
    --------
    If you do not want to plot and evaluate with shuffle set to 1.

    >>> deeplabcut.evaluate_network(
            '/analysis/project/reaching-task/config.yaml', Shuffles=[1],
        )

    If you want to plot and evaluate with shuffle set to 0 and 1.

    >>> deeplabcut.evaluate_network(
            '/analysis/project/reaching-task/config.yaml',
            Shuffles=[0, 1],
            plotting=True,
        )

    If you want to plot assemblies for a maDLC project

    >>> deeplabcut.evaluate_network(
            '/analysis/project/reaching-task/config.yaml',
            Shuffles=[1],
            plotting="individual",
        )

    Note: This defaults to standard plotting for single-animal projects.
    """
    if Shuffles is None:
        Shuffles = [1]
    if plotting not in (True, False, "bodypart", "individual"):
        raise ValueError(f"Unknown value for `plotting`={plotting}")

    import os

    start_path = os.getcwd()
    from deeplabcut.utils import auxiliaryfunctions

    cfg = auxiliaryfunctions.read_config(config)

    if cfg.get("multianimalproject", False):
        from .evaluate_multianimal import evaluate_multianimal_full

        # TODO: Make this code not so redundant!
        evaluate_multianimal_full(
            config=config,
            Shuffles=Shuffles,
            trainingsetindex=trainingsetindex,
            plotting=plotting,
            comparisonbodyparts=comparisonbodyparts,
            gputouse=gputouse,
            modelprefix=modelprefix,
            per_keypoint_evaluation=per_keypoint_evaluation,
            snapshots_to_evaluate=snapshots_to_evaluate,
        )
    else:
        import tensorflow as tf

        from deeplabcut.pose_estimation_tensorflow.config import load_config
        from deeplabcut.pose_estimation_tensorflow.core import predict
        from deeplabcut.pose_estimation_tensorflow.datasets.utils import data_to_input
        from deeplabcut.utils import auxiliaryfunctions, conversioncode
        from deeplabcut.utils.auxfun_videos import imread, imresize

        # If a string was passed in, auto-convert to True for backward compatibility
        plotting = bool(plotting)

        if "TF_CUDNN_USE_AUTOTUNE" in os.environ:
            del os.environ["TF_CUDNN_USE_AUTOTUNE"]  # was potentially set during training

        tf.compat.v1.reset_default_graph()
        os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"  #
        #    tf.logging.set_verbosity(tf.logging.WARN)

        start_path = os.getcwd()
        # Read file path for pose_config file. >> pass it on
        cfg = auxiliaryfunctions.read_config(config)
        if gputouse is not None:  # gpu selectinon
            os.environ["CUDA_VISIBLE_DEVICES"] = str(gputouse)

        if trainingsetindex == "all":
            TrainingFractions = cfg["TrainingFraction"]
        else:
            if 0 <= trainingsetindex < len(cfg["TrainingFraction"]):
                TrainingFractions = [cfg["TrainingFraction"][int(trainingsetindex)]]
            else:
                raise Exception(
                    "Please check the trainingsetindex! ",
                    trainingsetindex,
                    " should be an integer from 0 .. ",
                    int(len(cfg["TrainingFraction"]) - 1),
                )

        # Loading human annotatated data
        trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg)
        Data = pd.read_hdf(
            os.path.join(
                cfg["project_path"],
                str(trainingsetfolder),
                "CollectedData_" + cfg["scorer"] + ".h5",
            )
        )

        # Get list of body parts to evaluate network for
        comparisonbodyparts = auxiliaryfunctions.intersection_of_body_parts_and_ones_given_by_user(
            cfg, comparisonbodyparts
        )
        # Make folder for evaluation
        auxiliaryfunctions.attempt_to_make_folder(str(cfg["project_path"] + "/evaluation-results/"))
        for shuffle in Shuffles:
            for trainFraction in TrainingFractions:
                ##################################################
                # Load and setup CNN part detector
                ##################################################

                modelfolder_rel_path = auxiliaryfunctions.get_model_folder(
                    trainFraction, shuffle, cfg, modelprefix=modelprefix
                )
                modelfolder = Path(cfg["project_path"]) / modelfolder_rel_path

                # TODO: Unlike using create_training_dataset()
                # If create_training_model_comparison() is used there won't
                #  necessarily be training fractions for every shuffle which will raise the FileNotFoundError..
                #  Not sure if this should throw an exception or just be a warning...
                if not modelfolder.exists():
                    raise FileNotFoundError(
                        f"Model with shuffle {shuffle} and trainFraction {trainFraction} does not exist."
                    )

                if trainingsetindex == "all":
                    train_frac_idx = cfg["TrainingFraction"].index(trainFraction)
                else:
                    train_frac_idx = trainingsetindex

                path_train_config, path_test_config, _ = return_train_network_path(
                    config=config,
                    shuffle=shuffle,
                    trainingsetindex=train_frac_idx,
                    modelprefix=modelprefix,
                )

                test_pose_cfg = load_config(str(path_test_config))
                train_pose_cfg = load_config(str(path_train_config))
                # Load meta data
                _, trainIndices, testIndices, _ = auxiliaryfunctions.load_metadata(
                    Path(cfg["project_path"], train_pose_cfg["metadataset"])
                )

                # change batch size, if it was edited during analysis!
                test_pose_cfg["batch_size"] = 1  # in case this was edited for analysis.

                # Create folder structure to store results.
                evaluationfolder = os.path.join(
                    cfg["project_path"],
                    str(auxiliaryfunctions.get_evaluation_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)),
                )
                auxiliaryfunctions.attempt_to_make_folder(evaluationfolder, recursive=True)

                Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
                    train_folder=Path(modelfolder) / "train",
                )

                if snapshots_to_evaluate is not None:
                    snapshot_names = get_available_requested_snapshots(
                        requested_snapshots=snapshots_to_evaluate,
                        available_snapshots=Snapshots,
                    )
                else:
                    snapshot_names = get_snapshots_by_index(
                        idx=cfg["snapshotindex"],
                        available_snapshots=Snapshots,
                    )

                final_result = []

                ########################### RESCALING (to global scale)
                if rescale:
                    scale = test_pose_cfg["global_scale"]
                    Data = (
                        pd.read_hdf(
                            os.path.join(
                                cfg["project_path"],
                                str(trainingsetfolder),
                                "CollectedData_" + cfg["scorer"] + ".h5",
                            )
                        )
                        * scale
                    )
                else:
                    scale = 1

                conversioncode.guarantee_multiindex_rows(Data)
                ##################################################
                # Compute predictions over images
                ##################################################
                for snapshot_name in snapshot_names:
                    test_pose_cfg["init_weights"] = os.path.join(
                        str(modelfolder), "train", snapshot_name
                    )  # setting weights to corresponding snapshot.
                    training_iterations = int(snapshot_name.split("-")[-1])

                    # Name for deeplabcut net (based on its parameters)
                    DLCscorer, DLCscorerlegacy = auxiliaryfunctions.get_scorer_name(
                        cfg,
                        shuffle,
                        trainFraction,
                        trainingsiterations=training_iterations,
                        modelprefix=modelprefix,
                    )
                    print(
                        "Running ",
                        DLCscorer,
                        " with # of training iterations:",
                        training_iterations,
                    )
                    (
                        notanalyzed,
                        resultsfilename,
                        DLCscorer,
                    ) = auxiliaryfunctions.check_if_not_evaluated(
                        str(evaluationfolder),
                        DLCscorer,
                        DLCscorerlegacy,
                        snapshot_name,
                    )
                    if notanalyzed:
                        # Specifying state of model (snapshot / training state)
                        sess, inputs, outputs = predict.setup_pose_prediction(test_pose_cfg)
                        Numimages = len(Data.index)
                        PredicteData = np.zeros((Numimages, 3 * len(test_pose_cfg["all_joints_names"])))
                        print("Running evaluation ...")
                        for imageindex, imagename in tqdm(enumerate(Data.index)):
                            image = imread(
                                os.path.join(cfg["project_path"], *imagename),
                                mode="skimage",
                            )
                            if scale != 1:
                                image = imresize(image, scale)

                            image_batch = data_to_input(image)
                            # Compute prediction with the CNN
                            outputs_np = sess.run(outputs, feed_dict={inputs: image_batch})
                            scmap, locref = predict.extract_cnn_output(outputs_np, test_pose_cfg)

                            # Extract maximum scoring location from the heatmap, assume 1 person
                            pose = predict.argmax_pose_predict(scmap, locref, test_pose_cfg["stride"])
                            PredicteData[imageindex, :] = (
                                pose.flatten()
                            )  # NOTE: thereby     cfg_test['all_joints_names'] should be same order as bodyparts!

                        sess.close()  # closes the current tf session

                        index = pd.MultiIndex.from_product(
                            [
                                [DLCscorer],
                                test_pose_cfg["all_joints_names"],
                                ["x", "y", "likelihood"],
                            ],
                            names=["scorer", "bodyparts", "coords"],
                        )

                        # Saving results
                        DataMachine = pd.DataFrame(PredicteData, columns=index, index=Data.index)
                        DataMachine.to_hdf(resultsfilename, key="df_with_missing")

                        print(
                            "Analysis is done and the results are stored (see evaluation-results) for snapshot: ",
                            snapshot_name,
                        )
                        DataCombined = pd.concat([Data.T, DataMachine.T], axis=0, sort=False).T

                        RMSE, RMSEpcutoff = pairwisedistances(
                            DataCombined,
                            cfg["scorer"],
                            DLCscorer,
                            cfg["pcutoff"],
                            comparisonbodyparts,
                        )
                        testerror = np.nanmean(RMSE.iloc[testIndices].values.flatten())
                        trainerror = np.nanmean(RMSE.iloc[trainIndices].values.flatten())
                        testerrorpcutoff = np.nanmean(RMSEpcutoff.iloc[testIndices].values.flatten())
                        trainerrorpcutoff = np.nanmean(RMSEpcutoff.iloc[trainIndices].values.flatten())
                        results = [
                            training_iterations,
                            int(100 * trainFraction),
                            shuffle,
                            np.round(trainerror, 2),
                            np.round(testerror, 2),
                            cfg["pcutoff"],
                            np.round(trainerrorpcutoff, 2),
                            np.round(testerrorpcutoff, 2),
                        ]
                        final_result.append(results)

                        if per_keypoint_evaluation:
                            df_keypoint_error = keypoint_error(RMSE, RMSEpcutoff, trainIndices, testIndices)
                            kpt_filename = DLCscorer + "-keypoint-results.csv"
                            df_keypoint_error.to_csv(Path(evaluationfolder) / kpt_filename)

                        if show_errors:
                            print(
                                "Results for",
                                training_iterations,
                                " training iterations:",
                                int(100 * trainFraction),
                                shuffle,
                                "train error:",
                                np.round(trainerror, 2),
                                "pixels. Test error:",
                                np.round(testerror, 2),
                                " pixels.",
                            )
                            print(
                                "With pcutoff of",
                                cfg["pcutoff"],
                                " train error:",
                                np.round(trainerrorpcutoff, 2),
                                "pixels. Test error:",
                                np.round(testerrorpcutoff, 2),
                                "pixels",
                            )
                            if scale != 1:
                                print(
                                    "The predictions have been calculated for"
                                    f" rescaled images (and rescaled ground truth). Scale: {scale}"
                                )
                            print(
                                "Thereby, the errors are given by the average distances "
                                "between the labels by DLC and the scorer."
                            )

                        if plotting:
                            print("Plotting...")
                            foldername = os.path.join(
                                str(evaluationfolder),
                                "LabeledImages_" + DLCscorer + "_" + snapshot_name,
                            )
                            auxiliaryfunctions.attempt_to_make_folder(foldername)
                            Plotting(
                                cfg,
                                comparisonbodyparts,
                                DLCscorer,
                                trainIndices,
                                DataCombined * 1.0 / scale,
                                foldername,
                            )  # Rescaling coordinates to have figure in original size!

                        tf.compat.v1.reset_default_graph()
                        # print(final_result)
                    else:
                        DataMachine = pd.read_hdf(resultsfilename)
                        conversioncode.guarantee_multiindex_rows(DataMachine)
                        if plotting:
                            DataCombined = pd.concat([Data.T, DataMachine.T], axis=0, sort=False).T
                            foldername = os.path.join(
                                str(evaluationfolder),
                                "LabeledImages_" + DLCscorer + "_" + snapshot_name,
                            )
                            if not os.path.exists(foldername):
                                print(
                                    "Plotting..."
                                    "(warning, scale might be inconsistent in comparison "
                                    "to when data was analyzed; i.e. if you used rescale)"
                                )
                                auxiliaryfunctions.attempt_to_make_folder(foldername)
                                Plotting(
                                    cfg,
                                    comparisonbodyparts,
                                    DLCscorer,
                                    trainIndices,
                                    DataCombined * 1.0 / scale,
                                    foldername,
                                )
                            else:
                                print("Plots already exist for this snapshot... Skipping to the next one.")

                if len(final_result) > 0:  # Only append if results were calculated
                    make_results_file(final_result, evaluationfolder, DLCscorer)
                    print(
                        "The network is evaluated and the results are stored in the subdirectory 'evaluation_results'."
                    )
                    print(
                        "Please check the results, then choose the best model (snapshot) for prediction. "
                        "You can update the config.yaml file with the appropriate index for the 'snapshotindex'.\n"
                        "Use the function 'analyze_video' to make predictions on new videos."
                    )
                    print(
                        "Otherwise, consider adding more labeled-data and retraining the network "
                        "(see DeepLabCut workflow Fig 2, Nath 2019)"
                    )

    # returning to initial folder
    os.chdir(str(start_path))

extract_cnn_output

extract_cnn_output(outputs_np, cfg)

Extract locref + scmap from network.

Source code in deeplabcut/pose_estimation_tensorflow/core/predict.py

def extract_cnn_output(outputs_np, cfg):
    """Extract locref + scmap from network."""
    scmap = outputs_np[0]
    scmap = np.squeeze(scmap)
    locref = None
    if cfg["location_refinement"]:
        locref = np.squeeze(outputs_np[1])
        shape = locref.shape
        locref = np.reshape(locref, (shape[0], shape[1], -1, 2))
        locref *= cfg["locref_stdev"]
    if len(scmap.shape) == 2:  # for single body part!
        scmap = np.expand_dims(scmap, axis=2)
    return scmap, locref

extract_maps

extract_maps(config, shuffle=0, trainingsetindex=0, gputouse=None, rescale=False, Indices=None, modelprefix='')

Extracts the scoremap, locref, partaffinityfields (if available).

Returns a dictionary indexed by: trainingsetfraction, snapshotindex, and imageindex for those keys, each item contains: (image,scmap,locref,paf,bpt names,partaffinity graph, imagename, True/False if this image was in trainingset)

---

config : string Full path of the config.yaml file as a string.

integer

integers specifying shuffle index of the training dataset. The default is 0.

int, optional

Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). This variable can also be set to "all".

bool, default False

Evaluate the model at the 'global_scale' variable (as set in the test/pose_config.yaml file for a particular project). I.e. every image will be resized according to that scale and prediction will be compared to the resized ground truth. The error will be reported in pixels at rescaled to the original size. I.e. For a [200,200] pixel image evaluated at global_scale=.5, the predictions are calculated on [100,100] pixel images, compared to 1/2*ground truth and this error is then multiplied by 2!. The evaluation images are also shown for the original size!

Examples

If you want to extract the data for image 0 and 103 (of the training set) for model trained with shuffle 0.

deeplabcut.extract_maps(configfile,0,Indices=[0,103])

Source code in deeplabcut/pose_estimation_tensorflow/visualizemaps.py

def extract_maps(
    config,
    shuffle=0,
    trainingsetindex=0,
    gputouse=None,
    rescale=False,
    Indices=None,
    modelprefix="",
):
    """Extracts the scoremap, locref, partaffinityfields (if available).

    Returns a dictionary indexed by: trainingsetfraction, snapshotindex, and imageindex
    for those keys, each item contains: (image,scmap,locref,paf,bpt names,partaffinity graph,
    imagename, True/False if this image was in trainingset)
    ----------
    config : string
        Full path of the config.yaml file as a string.

    shuffle: integer
        integers specifying shuffle index of the training dataset. The default is 0.

    trainingsetindex: int, optional
        Integer specifying which TrainingsetFraction to use. By default the first
        (note that TrainingFraction is a list in config.yaml).
        This variable can also be set to "all".

    rescale: bool, default False
        Evaluate the model at the 'global_scale' variable
        (as set in the test/pose_config.yaml file for a particular project).
        I.e. every image will be resized according to that scale
        and prediction will be compared to the resized ground truth.
        The error will be reported in pixels at rescaled to the *original* size.
        I.e. For a [200,200] pixel image evaluated at global_scale=.5, the predictions are calculated
        on [100,100] pixel images, compared to 1/2*ground truth and this error is then multiplied by 2!.
        The evaluation images are also shown for the original size!

    Examples
    --------
    If you want to extract the data for image 0 and 103 (of the training set) for model trained with shuffle 0.
    >>> deeplabcut.extract_maps(configfile,0,Indices=[0,103])
    """
    from pathlib import Path

    import numpy as np
    import pandas as pd
    import tensorflow as tf
    from tqdm import tqdm

    from deeplabcut.pose_estimation_tensorflow.config import load_config
    from deeplabcut.pose_estimation_tensorflow.core import (
        predict,
    )
    from deeplabcut.pose_estimation_tensorflow.core import (
        predict_multianimal as predictma,
    )
    from deeplabcut.pose_estimation_tensorflow.datasets.utils import data_to_input
    from deeplabcut.utils import auxiliaryfunctions
    from deeplabcut.utils.auxfun_videos import imread, imresize

    tf.compat.v1.reset_default_graph()
    os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"  #
    #    tf.logging.set_verbosity(tf.logging.WARN)

    start_path = os.getcwd()
    # Read file path for pose_config file. >> pass it on
    cfg = auxiliaryfunctions.read_config(config)

    if gputouse is not None:  # gpu selectinon
        os.environ["CUDA_VISIBLE_DEVICES"] = str(gputouse)

    if trainingsetindex == "all":
        TrainingFractions = cfg["TrainingFraction"]
    else:
        if trainingsetindex < len(cfg["TrainingFraction"]) and trainingsetindex >= 0:
            TrainingFractions = [cfg["TrainingFraction"][int(trainingsetindex)]]
        else:
            raise Exception(
                "Please check the trainingsetindex! ",
                trainingsetindex,
                " should be an integer from 0 .. ",
                int(len(cfg["TrainingFraction"]) - 1),
            )

    # Loading human annotatated data
    trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg)
    Data = pd.read_hdf(
        os.path.join(
            cfg["project_path"],
            str(trainingsetfolder),
            "CollectedData_" + cfg["scorer"] + ".h5",
        )
    )

    # Make folder for evaluation
    auxiliaryfunctions.attempt_to_make_folder(str(cfg["project_path"] + "/evaluation-results/"))

    Maps = {}
    for trainFraction in TrainingFractions:
        Maps[trainFraction] = {}
        ##################################################
        # Load and setup CNN part detector
        ##################################################
        datafn, metadatafn = auxiliaryfunctions.get_data_and_metadata_filenames(
            trainingsetfolder, trainFraction, shuffle, cfg
        )

        modelfolder = os.path.join(
            cfg["project_path"],
            str(auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)),
        )
        path_test_config = Path(modelfolder) / "test" / "pose_cfg.yaml"
        # Load meta data
        (
            data,
            trainIndices,
            testIndices,
            trainFraction,
        ) = auxiliaryfunctions.load_metadata(os.path.join(cfg["project_path"], metadatafn))
        try:
            dlc_cfg = load_config(str(path_test_config))
        except FileNotFoundError as e:
            raise FileNotFoundError(
                f"It seems the model for shuffle {shuffle} and trainFraction {trainFraction} does not exist."
            ) from e

        # change batch size, if it was edited during analysis!
        dlc_cfg["batch_size"] = 1  # in case this was edited for analysis.

        # Create folder structure to store results.
        evaluationfolder = os.path.join(
            cfg["project_path"],
            str(auxiliaryfunctions.get_evaluation_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)),
        )
        auxiliaryfunctions.attempt_to_make_folder(evaluationfolder, recursive=True)

        Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
            train_folder=Path(modelfolder) / "train",
        )

        if cfg["snapshotindex"] == -1:
            snapindices = [-1]
        elif cfg["snapshotindex"] == "all":
            snapindices = range(len(Snapshots))
        elif cfg["snapshotindex"] < len(Snapshots):
            snapindices = [cfg["snapshotindex"]]
        else:
            print("Invalid choice, only -1 (last), any integer up to last, or all (as string)!")

        ########################### RESCALING (to global scale)
        scale = dlc_cfg["global_scale"] if rescale else 1
        Data *= scale

        bptnames = [dlc_cfg["all_joints_names"][i] for i in range(len(dlc_cfg["all_joints"]))]

        for snapindex in snapindices:
            dlc_cfg["init_weights"] = os.path.join(
                str(modelfolder), "train", Snapshots[snapindex]
            )  # setting weights to corresponding snapshot.
            (dlc_cfg["init_weights"].split(os.sep)[-1]).split("-")[
                -1
            ]  # read how many training siterations that corresponds to.

            # Name for deeplabcut net (based on its parameters)
            # DLCscorer,DLCscorerlegacy =
            # auxiliaryfunctions.GetScorerName(cfg,shuffle,trainFraction,trainingsiterations)
            # notanalyzed, resultsfilename,
            # DLCscorer=auxiliaryfunctions.CheckifNotEvaluated(str(evaluationfolder),
            # DLCscorer,DLCscorerlegacy,Snapshots[snapindex])
            # print("Extracting maps for ", DLCscorer, " with # of trainingiterations:", trainingsiterations)
            # if notanalyzed: #this only applies to ask if h5 exists...

            # Specifying state of model (snapshot / training state)
            sess, inputs, outputs = predict.setup_pose_prediction(dlc_cfg)
            Numimages = len(Data.index)
            np.zeros((Numimages, 3 * len(dlc_cfg["all_joints_names"])))
            print("Analyzing data...")
            if Indices is None:
                Indices = enumerate(Data.index)
            else:
                Ind = [Data.index[j] for j in Indices]
                Indices = enumerate(Ind)

            DATA = {}
            for imageindex, imagename in tqdm(Indices):
                image = imread(os.path.join(cfg["project_path"], *imagename), mode="skimage")

                if scale != 1:
                    image = imresize(image, scale)

                image_batch = data_to_input(image)

                # Compute prediction with the CNN
                outputs_np = sess.run(outputs, feed_dict={inputs: image_batch})

                if cfg.get("multianimalproject", False):
                    scmap, locref, paf = predictma.extract_cnn_output(outputs_np, dlc_cfg)
                    pagraph = dlc_cfg["partaffinityfield_graph"]
                else:
                    scmap, locref = predict.extract_cnn_output(outputs_np, dlc_cfg)
                    paf = None
                    pagraph = []
                peaks = outputs_np[-1]

                if imageindex in testIndices:
                    trainingfram = False
                else:
                    trainingfram = True

                DATA[imageindex] = [
                    image,
                    scmap,
                    locref,
                    paf,
                    peaks,
                    bptnames,
                    pagraph,
                    imagename,
                    trainingfram,
                ]
            Maps[trainFraction][Snapshots[snapindex]] = DATA
    os.chdir(str(start_path))
    return Maps

extract_save_all_maps

extract_save_all_maps(
    config,
    shuffle=1,
    trainingsetindex=0,
    comparisonbodyparts="all",
    extract_paf=True,
    all_paf_in_one=True,
    gputouse=None,
    rescale=False,
    Indices=None,
    modelprefix="",
    dest_folder=None,
)

Extracts the scoremap, location refinement field and part affinity field prediction of the model. The maps will be rescaled to the size of the input image and stored in the corresponding model folder in /evaluation-results.

---

config : string Full path of the config.yaml file as a string.

integer

integers specifying shuffle index of the training dataset. The default is 1.

int, optional

Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). This variable can also be set to "all".

list of bodyparts, Default is "all".

The average error will be computed for those body parts only (Has to be a subset of the body parts).

bool

Extract part affinity fields by default. Note that turning it off will make the function much faster.

bool

By default, all part affinity fields are displayed on a single frame. If false, individual fields are shown on separate frames.

default None

For which images shall the scmap/locref and paf be computed? Give a list of images

int, optional (default=None)

Number of plots per row in grid plots. By default, calculated to approximate a squared grid of plots

Examples

Calculated maps for images 0, 1 and 33.

deeplabcut.extract_save_all_maps('/analysis/project/reaching-task/config.yaml', shuffle=1,Indices=[0,1,33])

Source code in deeplabcut/pose_estimation_tensorflow/visualizemaps.py

def extract_save_all_maps(
    config,
    shuffle=1,
    trainingsetindex=0,
    comparisonbodyparts="all",
    extract_paf=True,
    all_paf_in_one=True,
    gputouse=None,
    rescale=False,
    Indices=None,
    modelprefix="",
    dest_folder=None,
):
    """
    Extracts the scoremap, location refinement field and part affinity field prediction of the model. The maps
    will be rescaled to the size of the input image and stored in the corresponding model folder in /evaluation-results.

    ----------
    config : string
        Full path of the config.yaml file as a string.

    shuffle: integer
        integers specifying shuffle index of the training dataset. The default is 1.

    trainingsetindex: int, optional
        Integer specifying which TrainingsetFraction to use.
        By default the first (note that TrainingFraction is a list in config.yaml).
        This variable can also be set to "all".

    comparisonbodyparts: list of bodyparts, Default is "all".
        The average error will be computed for those body parts only (Has to be a subset of the body parts).

    extract_paf : bool
        Extract part affinity fields by default.
        Note that turning it off will make the function much faster.

    all_paf_in_one : bool
        By default, all part affinity fields are displayed on a single frame.
        If false, individual fields are shown on separate frames.

    Indices: default None
        For which images shall the scmap/locref and paf be computed? Give a list of images

    nplots_per_row: int, optional (default=None)
        Number of plots per row in grid plots. By default, calculated to approximate a squared grid of plots

    Examples
    --------
    Calculated maps for images 0, 1 and 33.
    >>> deeplabcut.extract_save_all_maps('/analysis/project/reaching-task/config.yaml', shuffle=1,Indices=[0,1,33])

    """

    from tqdm import tqdm

    from deeplabcut.utils.auxiliaryfunctions import (
        attempt_to_make_folder,
        get_evaluation_folder,
        intersection_of_body_parts_and_ones_given_by_user,
        read_config,
    )

    cfg = read_config(config)
    data = extract_maps(config, shuffle, trainingsetindex, gputouse, rescale, Indices, modelprefix)

    comparisonbodyparts = intersection_of_body_parts_and_ones_given_by_user(cfg, comparisonbodyparts)

    print("Saving plots...")
    for frac, values in data.items():
        if not dest_folder:
            dest_folder = os.path.join(
                cfg["project_path"],
                str(get_evaluation_folder(frac, shuffle, cfg, modelprefix=modelprefix)),
                "maps",
            )
        attempt_to_make_folder(dest_folder)
        filepath = "{imname}_{map}_{label}_{shuffle}_{frac}_{snap}.png"
        dest_path = os.path.join(dest_folder, filepath)
        for snap, maps in values.items():
            for imagenr in tqdm(maps):
                (
                    image,
                    scmap,
                    locref,
                    paf,
                    peaks,
                    bptnames,
                    pafgraph,
                    impath,
                    trainingframe,
                ) = maps[imagenr]
                if not extract_paf:
                    paf = None
                label = "train" if trainingframe else "test"
                imname = impath[-1]
                scmap, (locref_x, locref_y), paf = resize_all_maps(image, scmap, locref, paf)
                to_plot = [i for i, bpt in enumerate(bptnames) if bpt in comparisonbodyparts]
                list_of_inds = []
                for n, edge in enumerate(pafgraph):
                    if any(ind in to_plot for ind in edge):
                        list_of_inds.append([(2 * n, 2 * n + 1), (bptnames[edge[0]], bptnames[edge[1]])])
                if len(to_plot) > 1:
                    map_ = scmap[:, :, to_plot].sum(axis=2)
                    locref_x_ = locref_x[:, :, to_plot].sum(axis=2)
                    locref_y_ = locref_y[:, :, to_plot].sum(axis=2)
                elif len(to_plot) == 1 and len(bptnames) > 1:
                    map_ = scmap[:, :, to_plot]
                    locref_x_ = locref_x[:, :, to_plot]
                    locref_y_ = locref_y[:, :, to_plot]
                else:
                    map_ = scmap[..., 0]
                    locref_x_ = locref_x[..., 0]
                    locref_y_ = locref_y[..., 0]
                fig1, _ = visualize_scoremaps(image, map_)
                temp = dest_path.format(
                    imname=imname,
                    map="scmap",
                    label=label,
                    shuffle=shuffle,
                    frac=frac,
                    snap=snap,
                )
                fig1.savefig(temp)

                fig2, _ = visualize_locrefs(image, map_, locref_x_, locref_y_)
                temp = dest_path.format(
                    imname=imname,
                    map="locref",
                    label=label,
                    shuffle=shuffle,
                    frac=frac,
                    snap=snap,
                )
                fig2.savefig(temp)

                if paf is not None:
                    if not all_paf_in_one:
                        for inds, names in list_of_inds:
                            fig3, _ = visualize_paf(image, paf[:, :, [inds]])
                            temp = dest_path.format(
                                imname=imname,
                                map=f"paf_{'_'.join(names)}",
                                label=label,
                                shuffle=shuffle,
                                frac=frac,
                                snap=snap,
                            )
                            fig3.savefig(temp)
                    else:
                        inds = [elem[0] for elem in list_of_inds]
                        n_inds = len(inds)
                        cmap = plt.cm.get_cmap(cfg["colormap"], n_inds)
                        colors = cmap(range(n_inds))
                        fig3, _ = visualize_paf(image, paf[:, :, inds], colors=colors)
                        temp = dest_path.format(
                            imname=imname,
                            map="paf",
                            label=label,
                            shuffle=shuffle,
                            frac=frac,
                            snap=snap,
                        )
                        fig3.savefig(temp)
                plt.close("all")

get_available_requested_snapshots

get_available_requested_snapshots(requested_snapshots: list[str], available_snapshots: list[str]) -> list[str]

Intersects the requested snapshot names with the available snapshots.

Returns: snapshot names

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def get_available_requested_snapshots(
    requested_snapshots: list[str],
    available_snapshots: list[str],
) -> list[str]:
    """Intersects the requested snapshot names with the available snapshots.

    Returns: snapshot names
    """
    snapshot_names = []
    missing_snapshots = []
    for snap in requested_snapshots:
        if snap in available_snapshots:
            snapshot_names.append(snap)
        else:
            missing_snapshots.append(snap)

    if len(snapshot_names) == 0:
        raise ValueError(f"None of the requested snapshots were found: \n{missing_snapshots}")
    elif len(missing_snapshots) > 0:
        print(f"The following requested snapshots were not found and will be skipped:\n{missing_snapshots}")

    return snapshot_names

get_snapshots_by_index

get_snapshots_by_index(idx: int | str, available_snapshots: list[str]) -> list[str]

Assume available_snapshots is ordered in ascending order.

Returns snapshot names.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def get_snapshots_by_index(
    idx: int | str,
    available_snapshots: list[str],
) -> list[str]:
    """Assume available_snapshots is ordered in ascending order.

    Returns snapshot names.
    """
    if isinstance(idx, int) and -len(available_snapshots) <= idx < len(available_snapshots):
        return [available_snapshots[idx]]
    elif idx == "all":
        return available_snapshots

    raise IndexError(
        f"Invalid index: {idx}. The index should be an int less than the number of "
        f"available snapshots, negative indexing is supported. The keyword 'all' "
        f"is also a valid option."
    )

keypoint_error

keypoint_error(
    df_error: DataFrame, df_error_p_cutoff: DataFrame, train_indices: list[int], test_indices: list[int]
) -> pd.DataFrame

Computes the RMSE error for each bodypart.

The error dataframes can be in single animal format (non-hierarchical columns, one column for each bodypart) or multi-animal format (hierarchical columns with 3 levels: "scorer", "individuals", "bodyparts").

Parameters:

Name	Type	Description	Default
df_error	DataFrame	dataframe containing the RMSE error for each image, individual and bodypart	required
df_error_p_cutoff	DataFrame	dataframe containing the RMSE error with p-cutoff for each image, individual and bodypart	required
train_indices	list[int]	the indices of rows in the dataframe that are in the train set	required
test_indices	list[int]	the indices of rows in the dataframe that are in the test set	required

Returns:

Type	Description
DataFrame	A dataframe containing 4 rows (train and test error, with and without p-cutoff) and one column for each bodypart.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def keypoint_error(
    df_error: pd.DataFrame,
    df_error_p_cutoff: pd.DataFrame,
    train_indices: list[int],
    test_indices: list[int],
) -> pd.DataFrame:
    """Computes the RMSE error for each bodypart.

    The error dataframes can be in single animal format (non-hierarchical columns, one
    column for each bodypart) or multi-animal format (hierarchical columns with 3
    levels: "scorer", "individuals", "bodyparts").

    Args:
        df_error: dataframe containing the RMSE error for each image, individual and
            bodypart
        df_error_p_cutoff: dataframe containing the RMSE error with p-cutoff for each
            image, individual and bodypart
        train_indices: the indices of rows in the dataframe that are in the train set
        test_indices: the indices of rows in the dataframe that are in the test set

    Returns:
        A dataframe containing 4 rows (train and test error, with and without p-cutoff)
        and one column for each bodypart.
    """
    df_error = df_error.copy()
    df_error_p_cutoff = df_error_p_cutoff.copy()

    error_rows = []
    for row_name, df in [
        ("Train error (px)", df_error.iloc[train_indices, :]),
        ("Test error (px)", df_error.iloc[test_indices, :]),
        ("Train error (px) with p-cutoff", df_error_p_cutoff.iloc[train_indices, :]),
        ("Test error (px) with p-cutoff", df_error_p_cutoff.iloc[test_indices, :]),
    ]:
        df_flat = df.copy()
        if isinstance(df.columns, pd.MultiIndex):
            # MA projects have column indices "scorer", "individuals" and "bodyparts"
            # Drop the scorer level, and put individuals in rows
            df_flat = df.droplevel("scorer", axis=1).stack(level="individuals").copy()

        bodypart_error = df_flat.mean()
        bodypart_error["Error Type"] = row_name
        error_rows.append(bodypart_error)

    # The error rows are series; stack in axis 1 and pivot to get DF
    keypoint_error_df = pd.concat(error_rows, axis=1)
    return keypoint_error_df.T.set_index("Error Type")

make_results_file

make_results_file(final_result, evaluationfolder, DLCscorer)

Makes result file in csv format and saves under evaluation_results directory.

If the file exists (typically, when the network has already been evaluated), newer results are appended to it.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def make_results_file(final_result, evaluationfolder, DLCscorer):
    """Makes result file in csv format and saves under evaluation_results directory.

    If the file exists (typically, when the network has already been evaluated), newer
    results are appended to it.
    """
    col_names = [
        "Training iterations:",
        "%Training dataset",
        "Shuffle number",
        " Train error(px)",
        " Test error(px)",
        "p-cutoff used",
        "Train error with p-cutoff",
        "Test error with p-cutoff",
    ]
    df = pd.DataFrame(final_result, columns=col_names)
    output_path = os.path.join(str(evaluationfolder), DLCscorer + "-results.csv")
    if os.path.exists(output_path):
        temp = pd.read_csv(output_path, index_col=0)
        df = pd.concat((temp, df)).reset_index(drop=True)

    df.to_csv(output_path)

    ## Also storing one "large" table with results:
    # note: evaluationfolder.parents[0] to get common folder above all shuffle evaluations.
    df = pd.DataFrame(final_result, columns=col_names)
    output_path = os.path.join(str(Path(evaluationfolder).parents[0]), "CombinedEvaluation-results.csv")
    if os.path.exists(output_path):
        temp = pd.read_csv(output_path, index_col=0)
        df = pd.concat((temp, df)).reset_index(drop=True)

    df.to_csv(output_path)

pairwisedistances

pairwisedistances(DataCombined, scorer1, scorer2, pcutoff=-1, bodyparts=None)

Calculates the pairwise Euclidean distance metric over body parts vs.

images

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def pairwisedistances(DataCombined, scorer1, scorer2, pcutoff=-1, bodyparts=None):
    """Calculates the pairwise Euclidean distance metric over body parts vs.

    images
    """
    mask = DataCombined[scorer2].xs("likelihood", level=1, axis=1) >= pcutoff
    if bodyparts is None:
        Pointwisesquareddistance = (DataCombined[scorer1] - DataCombined[scorer2]) ** 2
        RMSE = np.sqrt(
            Pointwisesquareddistance.xs("x", level=1, axis=1) + Pointwisesquareddistance.xs("y", level=1, axis=1)
        )  # Euclidean distance (proportional to RMSE)
        return RMSE, RMSE[mask]
    else:
        Pointwisesquareddistance = (DataCombined[scorer1][bodyparts] - DataCombined[scorer2][bodyparts]) ** 2
        RMSE = np.sqrt(
            Pointwisesquareddistance.xs("x", level=1, axis=1) + Pointwisesquareddistance.xs("y", level=1, axis=1)
        )  # Euclidean distance (proportional to RMSE)
        return RMSE, RMSE[mask]

renamed_parameter

renamed_parameter(*, old: str, new: str, since: str | None = None) -> Callable[[Callable[P, R]], Callable[P, R]]

Support a renamed keyword argument while warning callers to update.

Parameters:

Name	Type	Description	Default
old	str	The old parameter name that callers may still pass.	required
new	str	The current parameter name the function actually accepts.	required
since	str | None	Version when the rename happened.	None

Rules

• new must be the name used in the function signature and all internal call-sites. old must not appear in the signature.
• Do not chain renames. If A was renamed to B and B is later renamed to C, replace the A→B decorator with A→C directly rather than stacking a second decorator. Example: @renamed_parameter(old="A", new="C", since="12.4.0") @renamed_parameter(old="B", new="C", since="13.0.0") def func(*, C: int): print(f"C={C}")
• Multiple independent renames on the same function (e.g. batchsize→batch_size and videotype→video_extensions) are fine as long as they do not form a chain.
• This decorator only intercepts keyword arguments. Positional arguments are passed through unchanged; renaming a parameter that callers commonly pass positionally will not be caught.

Source code in deeplabcut/utils/deprecation.py

def renamed_parameter(
    *,
    old: str,
    new: str,
    since: str | None = None,
) -> Callable[[Callable[P, R]], Callable[P, R]]:
    """Support a renamed keyword argument while warning callers to update.

    Args:
        old: The old parameter name that callers may still pass.
        new: The current parameter name the function actually accepts.
        since: Version when the rename happened.

    Rules:
        - ``new`` must be the name used in the function signature and all
          internal call-sites.  ``old`` must **not** appear in the signature.
        - Do **not** chain renames.  If ``A`` was renamed to ``B`` and ``B``
          is later renamed to ``C``, replace the ``A→B`` decorator with
          ``A→C`` directly rather than stacking a second decorator.
            Example:
                @renamed_parameter(old="A", new="C", since="12.4.0")
                @renamed_parameter(old="B", new="C", since="13.0.0")
                def func(*, C: int):
                    print(f"C={C}")
        - Multiple independent renames on the same function (e.g.
          ``batchsize→batch_size`` *and* ``videotype→video_extensions``) are fine
          as long as they do not form a chain.
        - This decorator only intercepts **keyword** arguments.  Positional
          arguments are passed through unchanged; renaming a parameter that
          callers commonly pass positionally will not be caught.
    """

    def decorator(fn: Callable[P, R]) -> Callable[P, R]:
        sig = inspect.signature(fn)

        # Guard: disallow chaining renames (A→B stacked on top of B→C).
        existing = getattr(fn, "__deprecated_params__", ())
        for prev in existing:
            if prev.old_parameter == new:
                raise ValueError(
                    f"@renamed_parameter: chaining renames is not allowed. "
                    f"'{old}' → '{new}' would chain with the existing "
                    f"'{prev.old_parameter}' → '{prev.new_parameter}' rename "
                    f"on {fn.__qualname__}. "
                    f"Use '{old}' → '{prev.new_parameter}' directly instead."
                )

        # Guard: 'new' must actually exist in the function's signature.
        if new not in sig.parameters:
            raise ValueError(
                f"@renamed_parameter: '{new}' is not a parameter of "
                f"{fn.__qualname__}. "
                f"Available parameters: {list(sig.parameters)}"
            )

        # Guard: 'old' must NOT exist in the signature.
        if old in sig.parameters:
            raise ValueError(
                f"@renamed_parameter: '{old}' is still a parameter of "
                f"{fn.__qualname__}. Use either old name or new name: '{new}'."
            )

        info = DeprecationInfo(
            kind="parameter",
            target=fn.__qualname__,
            since=since,
            old_parameter=old,
            new_parameter=new,
        )
        message = info.format_message()

        @functools.wraps(fn)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
            if old in kwargs:
                if new in kwargs:
                    raise TypeError(f"{fn.__qualname__} received both '{old}' and '{new}'. Use only '{new}'.")
                warnings.warn(message, DLCDeprecationWarning, stacklevel=2)
                kwargs[new] = kwargs.pop(old)
            return fn(*args, **kwargs)

        wrapper.__deprecated_params__ = (*existing, info)
        return wrapper

    return decorator

return_evaluate_network_data

return_evaluate_network_data(
    config,
    shuffle=0,
    trainingsetindex=0,
    comparisonbodyparts="all",
    Snapindex=None,
    rescale=False,
    fulldata=False,
    show_errors=True,
    modelprefix="",
    returnjustfns=True,
)

Returns the results for (previously evaluated) network. deeplabcut.evaluate_network(..) Returns list of (per model): [trainingsiterations,tr ainfraction,shuffle,trainerror,testerror,pcutoff,trainerrorpcutoff,testerrorpcutoff, Snapshots[snapindex],scale,net_type]

If fulldata=True, also returns (the complete annotation and prediction array) Returns list of: (DataMachine, Data, data, trainIndices, testIndices, trainFraction, DLCscorer,comparisonbodyparts, cfg, Snapshots[snapindex])

---

config : string Full path of the config.yaml file as a string.

integer

integers specifying shuffle index of the training dataset. The default is 0.

int, optional

Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). This variable can also be set to "all".

list of bodyparts, Default is "all".

The average error will be computed for those body parts only (Has to be a subset of the body parts).

bool, default False

Evaluate the model at the 'global_scale' variable (as set in the test/pose_config.yaml file for a particular project). I.e. every image will be resized according to that scale and prediction will be compared to the resized ground truth. The error will be reported in pixels at rescaled to the original size. I.e. For a [200,200] pixel image evaluated at global_scale=.5, the predictions are calculated on [100,100] pixel images, compared to 1/2*ground truth and this error is then multiplied by 2!. The evaluation images are also shown for the original size!

Examples

If you do not want to plot

deeplabcut._evaluate_network_data('/analysis/project/reaching-task/config.yaml', shuffle=[1])

---

If you want to plot

deeplabcut.evaluate_network('/analysis/project/reaching-task/config.yaml',shuffle=[1],plotting=True)

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def return_evaluate_network_data(
    config,
    shuffle=0,
    trainingsetindex=0,
    comparisonbodyparts="all",
    Snapindex=None,
    rescale=False,
    fulldata=False,
    show_errors=True,
    modelprefix="",
    returnjustfns=True,
):
    """Returns the results for (previously evaluated) network.
    deeplabcut.evaluate_network(..) Returns list of (per model): [trainingsiterations,tr
    ainfraction,shuffle,trainerror,testerror,pcutoff,trainerrorpcutoff,testerrorpcutoff,
    Snapshots[snapindex],scale,net_type]

    If fulldata=True, also returns (the complete annotation and prediction array)
    Returns list of:
    (DataMachine, Data, data, trainIndices, testIndices, trainFraction,
    DLCscorer,comparisonbodyparts, cfg, Snapshots[snapindex])
    ----------
    config : string
        Full path of the config.yaml file as a string.

    shuffle: integer
        integers specifying shuffle index of the training dataset. The default is 0.

    trainingsetindex: int, optional
        Integer specifying which TrainingsetFraction to use.
        By default the first (note that TrainingFraction is a list in config.yaml).
        This variable can also be set to "all".

    comparisonbodyparts: list of bodyparts, Default is "all".
        The average error will be computed for those body parts only (Has to be a subset of the body parts).

    rescale: bool, default False
        Evaluate the model at the 'global_scale' variable
        (as set in the test/pose_config.yaml file for a particular project).
        I.e. every image will be resized according to that scale and
        prediction will be compared to the resized ground truth. The error will be reported
        in pixels at rescaled to the *original* size.
        I.e. For a [200,200] pixel image evaluated at global_scale=.5, the predictions are calculated
        on [100,100] pixel images, compared to 1/2*ground truth and this error is then multiplied by 2!.
        The evaluation images are also shown for the original size!

    Examples
    --------
    If you do not want to plot
    >>> deeplabcut._evaluate_network_data('/analysis/project/reaching-task/config.yaml', shuffle=[1])
    --------
    If you want to plot
    >>> deeplabcut.evaluate_network('/analysis/project/reaching-task/config.yaml',shuffle=[1],plotting=True)
    """

    import os

    from deeplabcut.pose_estimation_tensorflow.config import load_config
    from deeplabcut.utils import auxiliaryfunctions

    start_path = os.getcwd()
    # Read file path for pose_config file. >> pass it on
    cfg = auxiliaryfunctions.read_config(config)

    # Loading human annotatated data
    trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg)
    # Data=pd.read_hdf(
    # os.path.join(
    # cfg["project_path"],
    # str(trainingsetfolder
    # ),'CollectedData_' + cfg["scorer"] + '.h5'),'df_with_missing'
    # )

    # Get list of body parts to evaluate network for
    comparisonbodyparts = auxiliaryfunctions.intersection_of_body_parts_and_ones_given_by_user(cfg, comparisonbodyparts)
    ##################################################
    # Load data...
    ##################################################
    trainFraction = cfg["TrainingFraction"][trainingsetindex]
    modelfolder = os.path.join(
        cfg["project_path"],
        str(auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)),
    )
    path_train_config, path_test_config, _ = return_train_network_path(
        config=config,
        shuffle=shuffle,
        trainingsetindex=trainingsetindex,
        modelprefix=modelprefix,
    )

    try:
        test_pose_cfg = load_config(str(path_test_config))
    except FileNotFoundError as e:
        raise FileNotFoundError(
            f"It seems the model for shuffle {shuffle} and trainFraction {trainFraction} does not exist."
        ) from e

    train_pose_cfg = load_config(str(path_train_config))
    # Load meta data
    data, trainIndices, testIndices, _ = auxiliaryfunctions.load_metadata(
        Path(cfg["project_path"]) / train_pose_cfg["metadataset"],
    )

    ########################### RESCALING (to global scale)
    if rescale:
        scale = test_pose_cfg["global_scale"]
        print("Rescaling Data to ", scale)
        Data = (
            pd.read_hdf(
                os.path.join(
                    cfg["project_path"],
                    str(trainingsetfolder),
                    "CollectedData_" + cfg["scorer"] + ".h5",
                )
            )
            * scale
        )
    else:
        scale = 1
        Data = pd.read_hdf(
            os.path.join(
                cfg["project_path"],
                str(trainingsetfolder),
                "CollectedData_" + cfg["scorer"] + ".h5",
            )
        )

    evaluationfolder = os.path.join(
        cfg["project_path"],
        str(auxiliaryfunctions.get_evaluation_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)),
    )

    Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
        train_folder=Path(modelfolder) / "train",
    )

    if Snapindex is None:
        Snapindex = cfg["snapshotindex"]

    snapshot_names = get_snapshots_by_index(
        idx=Snapindex,
        available_snapshots=Snapshots,
    )

    DATA = []
    results = []
    resultsfns = []
    for snapshot_name in snapshot_names:
        test_pose_cfg["init_weights"] = os.path.join(
            str(modelfolder), "train", snapshot_name
        )  # setting weights to corresponding snapshot.
        trainingsiterations = (test_pose_cfg["init_weights"].split(os.sep)[-1]).split("-")[
            -1
        ]  # read how many training siterations that corresponds to.

        # name for deeplabcut net (based on its parameters)
        DLCscorer, DLCscorerlegacy = auxiliaryfunctions.get_scorer_name(
            cfg, shuffle, trainFraction, trainingsiterations, modelprefix=modelprefix
        )
        if not returnjustfns:
            print(
                "Retrieving ",
                DLCscorer,
                " with # of trainingiterations:",
                trainingsiterations,
            )

        (
            notanalyzed,
            resultsfilename,
            DLCscorer,
        ) = auxiliaryfunctions.check_if_not_evaluated(str(evaluationfolder), DLCscorer, DLCscorerlegacy, snapshot_name)
        # resultsfilename=os.path.join(str(evaluationfolder),DLCscorer + '-' +
        # str(Snapshots[snapindex])+  '.h5') # + '-' + str(snapshot)+  ' #'-' +
        # Snapshots[snapindex]+  '.h5')
        print(resultsfilename)
        resultsfns.append(resultsfilename)
        if not returnjustfns:
            if not notanalyzed and os.path.isfile(resultsfilename):  # data exists..
                DataMachine = pd.read_hdf(resultsfilename)
                DataCombined = pd.concat([Data.T, DataMachine.T], axis=0).T
                RMSE, RMSEpcutoff = pairwisedistances(
                    DataCombined,
                    cfg["scorer"],
                    DLCscorer,
                    cfg["pcutoff"],
                    comparisonbodyparts,
                )

                testerror = np.nanmean(RMSE.iloc[testIndices].values.flatten())
                trainerror = np.nanmean(RMSE.iloc[trainIndices].values.flatten())
                testerrorpcutoff = np.nanmean(RMSEpcutoff.iloc[testIndices].values.flatten())
                trainerrorpcutoff = np.nanmean(RMSEpcutoff.iloc[trainIndices].values.flatten())
                if show_errors:
                    print(
                        "Results for",
                        trainingsiterations,
                        " training iterations:",
                        int(100 * trainFraction),
                        shuffle,
                        "train error:",
                        np.round(trainerror, 2),
                        "pixels. Test error:",
                        np.round(testerror, 2),
                        " pixels.",
                    )
                    print(
                        "With pcutoff of",
                        cfg["pcutoff"],
                        " train error:",
                        np.round(trainerrorpcutoff, 2),
                        "pixels. Test error:",
                        np.round(testerrorpcutoff, 2),
                        "pixels",
                    )
                    print("Snapshot", snapshot_name)

                r = [
                    trainingsiterations,
                    int(100 * trainFraction),
                    shuffle,
                    np.round(trainerror, 2),
                    np.round(testerror, 2),
                    cfg["pcutoff"],
                    np.round(trainerrorpcutoff, 2),
                    np.round(testerrorpcutoff, 2),
                    snapshot_name,
                    scale,
                    test_pose_cfg["net_type"],
                ]
                results.append(r)
            else:
                print("Model not trained/evaluated!")
            if fulldata:
                DATA.append(
                    [
                        DataMachine,
                        Data,
                        data,
                        trainIndices,
                        testIndices,
                        trainFraction,
                        DLCscorer,
                        comparisonbodyparts,
                        cfg,
                        evaluationfolder,
                        snapshot_name,
                    ]
                )

    os.chdir(start_path)
    if returnjustfns:
        return resultsfns
    else:
        if fulldata:
            return DATA, results
        else:
            return results

return_train_network_path

return_train_network_path(config, shuffle=1, trainingsetindex=0, modelprefix='')

Returns the training and test pose config file names as well as the folder where the snapshot is.

Parameters

config : string Full path of the config.yaml file as a string.

int

Integer value specifying the shuffle index to select for training.

int, optional

Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml).

Returns the triple: trainposeconfigfile, testposeconfigfile, snapshotfolder

Source code in deeplabcut/pose_estimation_tensorflow/training.py

def return_train_network_path(config, shuffle=1, trainingsetindex=0, modelprefix=""):
    """Returns the training and test pose config file names as well as the folder where
    the snapshot is.

    Parameters
    ----------
    config : string
        Full path of the config.yaml file as a string.

    shuffle: int
        Integer value specifying the shuffle index to select for training.

    trainingsetindex: int, optional
        Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list
        in config.yaml).

    Returns the triple: trainposeconfigfile, testposeconfigfile, snapshotfolder
    """
    from deeplabcut.utils import auxiliaryfunctions

    cfg = auxiliaryfunctions.read_config(config)
    modelfoldername = auxiliaryfunctions.get_model_folder(
        cfg["TrainingFraction"][trainingsetindex], shuffle, cfg, modelprefix=modelprefix
    )
    trainposeconfigfile = Path(os.path.join(cfg["project_path"], str(modelfoldername), "train", "pose_cfg.yaml"))
    testposeconfigfile = Path(os.path.join(cfg["project_path"], str(modelfoldername), "test", "pose_cfg.yaml"))
    snapshotfolder = Path(os.path.join(cfg["project_path"], str(modelfoldername), "train"))

    return trainposeconfigfile, testposeconfigfile, snapshotfolder

stitch_tracklets

stitch_tracklets(
    config_path,
    videos,
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    n_tracks=None,
    animal_names: list[str] | None = None,
    min_length=10,
    split_tracklets=True,
    prestitch_residuals=True,
    max_gap=None,
    weight_func=None,
    destfolder=None,
    modelprefix="",
    track_method="",
    output_name="",
    transformer_checkpoint="",
    save_as_csv=False,
    **kwargs
)

Stitch sparse tracklets into full tracks via a graph-based, minimum-cost flow optimization problem.

Parameters

config_path : str Path to the main project config.yaml file.

list

A list of strings containing the full paths to videos for analysis or a path to the directory, where all the videos with same extension are stored.

str | Sequence[str] | None, optional, default=None

Controls how videos are filtered, based on file extension. File paths and directory contents are treated differently: - None (default): file paths are accepted as-is; directories are scanned for files with a recognized video extension. - str or Sequence[str] (e.g. "mp4" or ["mp4", "avi"]): both file paths and directory contents are filtered by the given extension(s).

int, optional

An integer specifying the shuffle index of the training dataset used for training the network. The default is 1.

int, optional

Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml).

int, optional

Number of tracks to reconstruct. By default, taken as the number of individuals defined in the config.yaml. Another number can be passed if the number of animals in the video is different from the number of animals the model was trained on.

list, optional

If you want the names given to individuals in the labeled data file, you can specify those names as a list here. If given and n_tracks is None, n_tracks will be set to len(animal_names). If n_tracks is not None, then it must be equal to len(animal_names). If it is not given, then animal_names will be loaded from the individuals in the project config.yaml file.

int, optional

Tracklets less than min_length frames of length are considered to be residuals; i.e., they do not participate in building the graph and finding the solution to the optimization problem, but are rather added last after "almost-complete" tracks are formed. The higher the value, the lesser the computational cost, but the higher the chance of discarding relatively long and reliable tracklets that are essential to solving the stitching task. Default is 10, and must be 3 at least.

bool, optional

By default, tracklets whose time indices are not consecutive integers are split in shorter tracklets whose time continuity is guaranteed. This is for example very powerful to get rid of tracking errors (e.g., identity switches) which are often signaled by a missing time frame at the moment they occur. Note though that for long occlusions where tracker re-identification capability can be trusted, setting split_tracklets to False is preferable.

bool, optional

Residuals will by default be grouped together according to their temporal proximity prior to being added back to the tracks. This is done to improve robustness and simultaneously reduce complexity.

int, optional

Maximal temporal gap to allow between a pair of tracklets. This is automatically determined by the TrackletStitcher by default.

callable, optional

Function accepting two tracklets as arguments and returning a scalar that must be inversely proportional to the likelihood that the tracklets belong to the same track; i.e., the higher the confidence that the tracklets should be stitched together, the lower the returned value.

string, optional

Specifies the destination folder for analysis data (default is the path of the video). Note that for subsequent analysis this folder also needs to be passed.

string, optional

Specifies the tracker used to generate the pose estimation data. For multiple animals, must be either 'box', 'skeleton', or 'ellipse' and will be taken from the config.yaml file if none is given.

str, optional

Name of the output h5 file. By default, tracks are automatically stored into the same directory as the pickle file and with its name.

bool, optional

Whether to write the tracks to a CSV file too (False by default).

additional arguments.

For torch-based shuffles, can be used to specify: - snapshot_index - detector_snapshot_index

Returns

A TrackletStitcher object

Source code in deeplabcut/refine_training_dataset/stitch.py

@renamed_parameter(old="videotype", new="video_extensions", since="3.0.0")
def stitch_tracklets(
    config_path,
    videos,
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    n_tracks=None,
    animal_names: list[str] | None = None,
    min_length=10,
    split_tracklets=True,
    prestitch_residuals=True,
    max_gap=None,
    weight_func=None,
    destfolder=None,
    modelprefix="",
    track_method="",
    output_name="",
    transformer_checkpoint="",
    save_as_csv=False,
    **kwargs,
):
    """Stitch sparse tracklets into full tracks via a graph-based, minimum-cost flow
    optimization problem.

    Parameters
    ----------
    config_path : str
        Path to the main project config.yaml file.

    videos : list
        A list of strings containing the full paths to videos for analysis or a path to the directory, where all the
        videos with same extension are stored.

    video_extensions : str | Sequence[str] | None, optional, default=None
        Controls how ``videos`` are filtered, based on file extension.
        File paths and directory contents are treated differently:
        - ``None`` (default): file paths are accepted as-is; directories are
          scanned for files with a recognized video extension.
        - ``str`` or ``Sequence[str]`` (e.g. ``"mp4"`` or ``["mp4", "avi"]``):
          both file paths and directory contents are filtered by the given
          extension(s).

    shuffle: int, optional
        An integer specifying the shuffle index of the training dataset used for training the network. The default is 1.

    trainingsetindex: int, optional
        Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list
        in config.yaml).

    n_tracks : int, optional
        Number of tracks to reconstruct. By default, taken as the number
        of individuals defined in the config.yaml. Another number can be
        passed if the number of animals in the video is different from
        the number of animals the model was trained on.

    animal_names: list, optional
        If you want the names given to individuals in the labeled data file, you can
        specify those names as a list here. If given and `n_tracks` is None, `n_tracks`
        will be set to `len(animal_names)`. If `n_tracks` is not None, then it must be
        equal to `len(animal_names)`. If it is not given, then `animal_names` will
        be loaded from the `individuals` in the project config.yaml file.

    min_length : int, optional
        Tracklets less than `min_length` frames of length
        are considered to be residuals; i.e., they do not participate
        in building the graph and finding the solution to the
        optimization problem, but are rather added last after
        "almost-complete" tracks are formed. The higher the value,
        the lesser the computational cost, but the higher the chance of
        discarding relatively long and reliable tracklets that are
        essential to solving the stitching task.
        Default is 10, and must be 3 at least.

    split_tracklets : bool, optional
        By default, tracklets whose time indices are not consecutive integers
        are split in shorter tracklets whose time continuity is guaranteed.
        This is for example very powerful to get rid of tracking errors
        (e.g., identity switches) which are often signaled by a missing
        time frame at the moment they occur. Note though that for long
        occlusions where tracker re-identification capability can be trusted,
        setting `split_tracklets` to False is preferable.

    prestitch_residuals : bool, optional
        Residuals will by default be grouped together according to their
        temporal proximity prior to being added back to the tracks.
        This is done to improve robustness and simultaneously reduce complexity.

    max_gap : int, optional
        Maximal temporal gap to allow between a pair of tracklets.
        This is automatically determined by the TrackletStitcher by default.

    weight_func : callable, optional
        Function accepting two tracklets as arguments and returning a scalar
        that must be inversely proportional to the likelihood that the tracklets
        belong to the same track; i.e., the higher the confidence that the
        tracklets should be stitched together, the lower the returned value.

    destfolder: string, optional
        Specifies the destination folder for analysis data (default is the path of the
        video). Note that for subsequent analysis this folder also needs to be passed.

    track_method: string, optional
         Specifies the tracker used to generate the pose estimation data.
         For multiple animals, must be either 'box', 'skeleton', or 'ellipse'
         and will be taken from the config.yaml file if none is given.

    output_name : str, optional
        Name of the output h5 file.
        By default, tracks are automatically stored into the same directory
        as the pickle file and with its name.

    save_as_csv: bool, optional
        Whether to write the tracks to a CSV file too (False by default).

    kwargs: additional arguments.
        For torch-based shuffles, can be used to specify:
            - snapshot_index
            - detector_snapshot_index

    Returns
    -------
    A TrackletStitcher object
    """
    vids = collect_video_paths(videos, extensions=video_extensions)
    if not vids:
        print("No video(s) found. Please check your path!")
        return

    cfg = auxiliaryfunctions.read_config(config_path)
    track_method = auxfun_multianimal.get_track_method(cfg, track_method=track_method)
    if track_method == "ctd":
        raise ValueError(
            "CTD tracking occurs directly during video analysis. No need to call "
            "`stitch_tracklets` with `track_method=='ctd'`."
        )

    if animal_names is None:
        animal_names = cfg["individuals"]
    elif n_tracks is not None and n_tracks != len(animal_names):
        raise ValueError(
            "When setting both `n_tracks` and `animal_names`, `n_tracks` must be equal "
            f"to len(animal_names)`. Found `n_tracks`={n_tracks} and `animal_names`="
            f"{animal_names} of length {len(animal_names)}.`"
        )

    if n_tracks is None:
        n_tracks = len(animal_names)

    DLCscorer, _ = deeplabcut.utils.auxiliaryfunctions.get_scorer_name(
        cfg,
        shuffle,
        cfg["TrainingFraction"][trainingsetindex],
        modelprefix=modelprefix,
        **kwargs,
    )

    if transformer_checkpoint:
        from deeplabcut.pose_tracking_pytorch import inference

        dlctrans = inference.DLCTrans(checkpoint=transformer_checkpoint)

    def trans_weight_func(tracklet1, tracklet2, nframe, feature_dict):
        zfill_width = int(np.ceil(np.log10(nframe)))
        if tracklet1 < tracklet2:
            ind_img1 = tracklet1.inds[-1]
            coord1 = tracklet1.data[-1][:, :2]
            ind_img2 = tracklet2.inds[0]
            coord2 = tracklet2.data[0][:, :2]
        else:
            ind_img2 = tracklet2.inds[-1]
            ind_img1 = tracklet1.inds[0]
            coord2 = tracklet2.data[-1][:, :2]
            coord1 = tracklet1.data[0][:, :2]
        t1 = (coord1, ind_img1)
        t2 = (coord2, ind_img2)

        dist = dlctrans(t1, t2, zfill_width, feature_dict)
        dist = (dist + 1) / 2

        return -dist

    base_weight_func = weight_func
    for video in vids:
        print("Processing... ", video)
        nframe = len(VideoWriter(video))
        videofolder = str(Path(video).parents[0])
        dest = destfolder or videofolder
        deeplabcut.utils.auxiliaryfunctions.attempt_to_make_folder(dest)
        vname = Path(video).stem

        feature_dict_path = os.path.join(dest, vname + DLCscorer + "_bpt_features.pickle")
        # should only exist one
        if transformer_checkpoint:
            import dbm

            try:
                feature_dict = shelve.open(feature_dict_path, flag="r")
            except dbm.error as err:
                raise FileNotFoundError(f"{feature_dict_path} does not exist. Did you run transformer_reID()?") from err

        dataname = os.path.join(dest, vname + DLCscorer + ".h5")

        method = TRACK_METHODS[track_method]
        pickle_file = dataname.split(".h5")[0] + f"{method}.pickle"
        try:
            stitcher = TrackletStitcher.from_pickle(
                pickle_file, n_tracks, min_length, split_tracklets, prestitch_residuals
            )
            current_weight_func = base_weight_func
            with_id = any(tracklet.identity != -1 for tracklet in stitcher)
            if with_id and weight_func is None:
                # Add in identity weighing before building the graph
                def current_weight_func(t1, t2, stitcher=stitcher):
                    w = 0.01 if t1.identity == t2.identity else 1
                    return w * stitcher.calculate_edge_weight(t1, t2)

            if transformer_checkpoint:
                stitcher.build_graph(
                    max_gap=max_gap,
                    weight_func=partial(trans_weight_func, nframe=nframe, feature_dict=feature_dict),
                )
            else:
                stitcher.build_graph(max_gap=max_gap, weight_func=current_weight_func)

            stitcher.stitch()
            if transformer_checkpoint:
                stitcher.write_tracks(
                    output_name=output_name,
                    animal_names=animal_names,
                    suffix="tr",
                    save_as_csv=save_as_csv,
                )
            else:
                stitcher.write_tracks(
                    output_name=output_name,
                    animal_names=animal_names,
                    suffix="",
                    save_as_csv=save_as_csv,
                )
        except FileNotFoundError as e:
            print(e, "\nSkipping...")

train_network

train_network(
    config,
    shuffle=1,
    trainingsetindex=0,
    max_snapshots_to_keep=5,
    displayiters=None,
    saveiters=None,
    maxiters=None,
    allow_growth=True,
    gputouse=None,
    autotune=False,
    keepdeconvweights=True,
    modelprefix="",
    superanimal_name="",
    superanimal_transfer_learning=False,
)

Trains the network with the labels in the training dataset.

Parameters
----------
config : string
    Full path of the config.yaml file as a string.

shuffle: int, optional, default=1
    Integer value specifying the shuffle index to select for training.

trainingsetindex: int, optional, default=0
    Integer specifying which TrainingsetFraction to use.
    Note that TrainingFraction is a list in config.yaml.

max_snapshots_to_keep: int or None
    Sets how many snapshots are kept, i.e. states of the trained network. Every
    saving iteration many times a snapshot is stored, however only the last
    ``max_snapshots_to_keep`` many are kept! If you change this to None, then all
    are kept.
    See: https://github.com/DeepLabCut/DeepLabCut/issues/8#issuecomment-387404835

displayiters: optional, default=None
    This variable is actually set in ``pose_config.yaml``. However, you can
    overwrite it with this hack. Don't use this regularly, just if you are too lazy
    to dig out the ``pose_config.yaml`` file for the corresponding project. If
    ``None``, the value from there is used, otherwise it is overwritten!

saveiters: optional, default=None
    This variable is actually set in ``pose_config.yaml``. However, you can
    overwrite it with this hack. Don't use this regularly, just if you are too lazy
    to dig out the ``pose_config.yaml`` file for the corresponding project.
    If ``None``, the value from there is used, otherwise it is overwritten!

maxiters: optional, default=None
    This variable is actually set in ``pose_config.yaml``. However, you can
    overwrite it with this hack. Don't use this regularly, just if you are too lazy
    to dig out the ``pose_config.yaml`` file for the corresponding project.
    If ``None``, the value from there is used, otherwise it is overwritten!

allow_growth: bool, optional, default=True.
    For some smaller GPUs the memory issues happen. If ``True``, the memory
    allocator does not pre-allocate the entire specified GPU memory region, instead
    starting small and growing as needed.
    See issue: https://forum.image.sc/t/how-to-stop-running-out-of-vram/30551/2

gputouse: optional, default=None
    Natural number indicating the number of your GPU (see number in nvidia-smi).
    If you do not have a GPU put None.
    See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries

autotune: bool, optional, default=False
    Property of TensorFlow, somehow faster if ``False``
    (as Eldar found out, see https://github.com/tensorflow/tensorflow/issues/13317).

keepdeconvweights: bool, optional, default=True
    Also restores the weights of the deconvolution layers (and the backbone) when
    training from a snapshot. Note that if you change the number of bodyparts, you
    need to set this to false for re-training.

modelprefix: str, optional, default=""
    Directory containing the deeplabcut models to use when evaluating the network.
    By default, the models are assumed to exist in the project folder.

superanimal_name: str, optional, default =""
    Specified if transfer learning with superanimal is desired

superanimal_transfer_learning: bool, optional, default = False.
    If set true, the training is transfer learning (new decoding layer). If set false,

and superanimal_name is True, then the training is fine-tuning (reusing the decoding layer)

Returns
-------
None

Examples
--------
To train the network for first shuffle of the training dataset

>>> deeplabcut.train_network('/analysis/project/reaching-task/config.yaml')

To train the network for second shuffle of the training dataset

>>> deeplabcut.train_network(
        '/analysis/project/reaching-task/config.yaml',
        shuffle=2,
        keepdeconvweights=True,
    )

Source code in deeplabcut/pose_estimation_tensorflow/training.py

def train_network(
    config,
    shuffle=1,
    trainingsetindex=0,
    max_snapshots_to_keep=5,
    displayiters=None,
    saveiters=None,
    maxiters=None,
    allow_growth=True,
    gputouse=None,
    autotune=False,
    keepdeconvweights=True,
    modelprefix="",
    superanimal_name="",
    superanimal_transfer_learning=False,
):
    """Trains the network with the labels in the training dataset.

        Parameters
        ----------
        config : string
            Full path of the config.yaml file as a string.

        shuffle: int, optional, default=1
            Integer value specifying the shuffle index to select for training.

        trainingsetindex: int, optional, default=0
            Integer specifying which TrainingsetFraction to use.
            Note that TrainingFraction is a list in config.yaml.

        max_snapshots_to_keep: int or None
            Sets how many snapshots are kept, i.e. states of the trained network. Every
            saving iteration many times a snapshot is stored, however only the last
            ``max_snapshots_to_keep`` many are kept! If you change this to None, then all
            are kept.
            See: https://github.com/DeepLabCut/DeepLabCut/issues/8#issuecomment-387404835

        displayiters: optional, default=None
            This variable is actually set in ``pose_config.yaml``. However, you can
            overwrite it with this hack. Don't use this regularly, just if you are too lazy
            to dig out the ``pose_config.yaml`` file for the corresponding project. If
            ``None``, the value from there is used, otherwise it is overwritten!

        saveiters: optional, default=None
            This variable is actually set in ``pose_config.yaml``. However, you can
            overwrite it with this hack. Don't use this regularly, just if you are too lazy
            to dig out the ``pose_config.yaml`` file for the corresponding project.
            If ``None``, the value from there is used, otherwise it is overwritten!

        maxiters: optional, default=None
            This variable is actually set in ``pose_config.yaml``. However, you can
            overwrite it with this hack. Don't use this regularly, just if you are too lazy
            to dig out the ``pose_config.yaml`` file for the corresponding project.
            If ``None``, the value from there is used, otherwise it is overwritten!

        allow_growth: bool, optional, default=True.
            For some smaller GPUs the memory issues happen. If ``True``, the memory
            allocator does not pre-allocate the entire specified GPU memory region, instead
            starting small and growing as needed.
            See issue: https://forum.image.sc/t/how-to-stop-running-out-of-vram/30551/2

        gputouse: optional, default=None
            Natural number indicating the number of your GPU (see number in nvidia-smi).
            If you do not have a GPU put None.
            See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries

        autotune: bool, optional, default=False
            Property of TensorFlow, somehow faster if ``False``
            (as Eldar found out, see https://github.com/tensorflow/tensorflow/issues/13317).

        keepdeconvweights: bool, optional, default=True
            Also restores the weights of the deconvolution layers (and the backbone) when
            training from a snapshot. Note that if you change the number of bodyparts, you
            need to set this to false for re-training.

        modelprefix: str, optional, default=""
            Directory containing the deeplabcut models to use when evaluating the network.
            By default, the models are assumed to exist in the project folder.

        superanimal_name: str, optional, default =""
            Specified if transfer learning with superanimal is desired

        superanimal_transfer_learning: bool, optional, default = False.
            If set true, the training is transfer learning (new decoding layer). If set false,
    and superanimal_name is True, then the training is fine-tuning (reusing the decoding layer)

        Returns
        -------
        None

        Examples
        --------
        To train the network for first shuffle of the training dataset

        >>> deeplabcut.train_network('/analysis/project/reaching-task/config.yaml')

        To train the network for second shuffle of the training dataset

        >>> deeplabcut.train_network(
                '/analysis/project/reaching-task/config.yaml',
                shuffle=2,
                keepdeconvweights=True,
            )
    """
    if allow_growth:
        os.environ["TF_FORCE_GPU_ALLOW_GROWTH"] = "true"

    # reload logger.
    import importlib
    import logging

    import tensorflow as tf

    importlib.reload(logging)
    logging.shutdown()

    from deeplabcut.utils import auxiliaryfunctions

    tf.compat.v1.reset_default_graph()
    start_path = os.getcwd()

    # Read file path for pose_config file. >> pass it on
    cfg = auxiliaryfunctions.read_config(config)
    modelfoldername = auxiliaryfunctions.get_model_folder(
        cfg["TrainingFraction"][trainingsetindex], shuffle, cfg, modelprefix=modelprefix
    )
    poseconfigfile = Path(os.path.join(cfg["project_path"], str(modelfoldername), "train", "pose_cfg.yaml"))
    if not poseconfigfile.is_file():
        print("The training datafile ", poseconfigfile, " is not present.")
        print("Probably, the training dataset for this specific shuffle index was not created.")
        print(
            "Try with a different shuffle/trainingsetfraction or use function 'create_training_dataset' to create a new"
            "trainingdataset with this shuffle index."
        )
    else:
        # Set environment variables
        if autotune is not False:  # see: https://github.com/tensorflow/tensorflow/issues/13317
            os.environ["TF_CUDNN_USE_AUTOTUNE"] = "0"
        if gputouse is not None:
            os.environ["CUDA_VISIBLE_DEVICES"] = str(gputouse)
    try:
        cfg_dlc = auxiliaryfunctions.read_plainconfig(poseconfigfile)

        if superanimal_name != "":
            import glob

            from dlclibrary.dlcmodelzoo.modelzoo_download import (
                MODELOPTIONS,
                download_huggingface_model,
            )

            from deeplabcut.modelzoo.utils import parse_available_supermodels

            dlc_root_path = auxiliaryfunctions.get_deeplabcut_path()
            parse_available_supermodels()
            weight_folder = str(
                Path(dlc_root_path)
                / "pose_estimation_tensorflow"
                / "models"
                / "pretrained"
                / (superanimal_name + "_weights")
            )

            if superanimal_name in MODELOPTIONS:
                if not os.path.exists(weight_folder):
                    download_huggingface_model(superanimal_name, weight_folder)
                else:
                    print(f"{weight_folder} exists, using the downloaded weights")
            else:
                print(
                    f"{superanimal_name} not available. Available ones are: ",
                    MODELOPTIONS,
                )

            snapshots = glob.glob(os.path.join(weight_folder, "snapshot-*.index"))
            init_weights = os.path.abspath(snapshots[0]).replace(".index", "")

            from deeplabcut.pose_estimation_tensorflow.core.train_multianimal import (
                train,
            )

            print("Selecting multi-animal trainer")
            train(
                str(poseconfigfile),
                displayiters,
                saveiters,
                maxiters,
                max_to_keep=max_snapshots_to_keep,
                keepdeconvweights=keepdeconvweights,
                allow_growth=allow_growth,
                init_weights=init_weights,
                remove_head=(True if superanimal_name != "" and superanimal_transfer_learning else False),
            )  # pass on path and file name for pose_cfg.yaml!

        elif "multi-animal" in cfg_dlc["dataset_type"]:
            from deeplabcut.pose_estimation_tensorflow.core.train_multianimal import (
                train,
            )

            print("Selecting multi-animal trainer")
            train(
                str(poseconfigfile),
                displayiters,
                saveiters,
                maxiters,
                max_to_keep=max_snapshots_to_keep,
                keepdeconvweights=keepdeconvweights,
                allow_growth=allow_growth,
            )  # pass on path and file name for pose_cfg.yaml!
        else:
            from deeplabcut.pose_estimation_tensorflow.core.train import train

            print("Selecting single-animal trainer")
            train(
                str(poseconfigfile),
                displayiters,
                saveiters,
                maxiters,
                max_to_keep=max_snapshots_to_keep,
                keepdeconvweights=keepdeconvweights,
                allow_growth=allow_growth,
            )  # pass on path and file name for pose_cfg.yaml!

    except BaseException as e:
        raise e
    finally:
        os.chdir(str(start_path))
    print(
        "The network is now trained and ready to evaluate. Use the function 'evaluate_network' to evaluate the network."
    )

visualize_locrefs

visualize_locrefs(
    image: ndarray, scmap: ndarray, locref_x: ndarray, locref_y: ndarray, step: int = 5, zoom_width: int = 0
) -> tuple[plt.Figure, plt.Axes]

Plots a scoremap and the corresponding location refinement field on an image.

Parameters:

Name	Type	Description	Default
image	ndarray	An image as a numpy array of shape (h, w, channels)	required
scmap	ndarray	A scoremap of shape (h, w)	required
locref_x	ndarray	The x-coordinate of the location refinement field, of shape (h, w)	required
locref_y	ndarray	The y-coordinate of the location refinement field, of shape (h, w)	required
step	int	The step with which to plot the location refinement field.	5
zoom_width	int	The zoom width with which to plot the scoremaps.	0

Returns:

Type	Description
tuple[Figure, Axes]	The figure and axis on which the image scoremap and locref field were plot.

Source code in deeplabcut/core/visualization.py

def visualize_locrefs(
    image: np.ndarray,
    scmap: np.ndarray,
    locref_x: np.ndarray,
    locref_y: np.ndarray,
    step: int = 5,
    zoom_width: int = 0,
) -> tuple[plt.Figure, plt.Axes]:
    """Plots a scoremap and the corresponding location refinement field on an image.

    Args:
        image: An image as a numpy array of shape (h, w, channels)
        scmap: A scoremap of shape (h, w)
        locref_x: The x-coordinate of the location refinement field, of shape (h, w)
        locref_y: The y-coordinate of the location refinement field, of shape (h, w)
        step: The step with which to plot the location refinement field.
        zoom_width: The zoom width with which to plot the scoremaps.

    Returns:
        The figure and axis on which the image scoremap and locref field were plot.
    """
    fig, ax = visualize_scoremaps(image, scmap)
    X, Y = np.meshgrid(np.arange(locref_x.shape[1]), np.arange(locref_x.shape[0]))
    M = np.zeros(locref_x.shape, dtype=bool)
    M[scmap < 0.5] = True
    U = np.ma.masked_array(locref_x, mask=M)
    V = np.ma.masked_array(locref_y, mask=M)
    ax.quiver(
        X[::step, ::step],
        Y[::step, ::step],
        U[::step, ::step],
        V[::step, ::step],
        color="r",
        units="x",
        scale_units="xy",
        scale=1,
        angles="xy",
    )
    if zoom_width > 0:
        maxloc = np.unravel_index(np.argmax(scmap), scmap.shape)
        ax.set_xlim(maxloc[1] - zoom_width, maxloc[1] + zoom_width)
        ax.set_ylim(maxloc[0] + zoom_width, maxloc[0] - zoom_width)
    return fig, ax

visualize_paf

visualize_paf(image: ndarray, paf: ndarray, step: int = 5, colors: list | None = None) -> tuple[plt.Figure, plt.Axes]

Plots the PAF on top of the image.

Parameters:

Name	Type	Description	Default
image	ndarray	Shape (height, width, channels). The image on which the model was run.	required
paf	ndarray	Shape (height, width, 2 * len(paf_graph)). The PAF output by the model.	required
step	int	The step with which to plot the scoremaps.	5
colors	list | None	The colormap to use.	None

Returns:

Type	Description
tuple[Figure, Axes]	The figure and axis on which the image PAF was plot.

Source code in deeplabcut/core/visualization.py

def visualize_paf(
    image: np.ndarray,
    paf: np.ndarray,
    step: int = 5,
    colors: list | None = None,
) -> tuple[plt.Figure, plt.Axes]:
    """Plots the PAF on top of the image.

    Args:
        image: Shape (height, width, channels). The image on which the model was run.
        paf: Shape (height, width, 2 * len(paf_graph)). The PAF output by the model.
        step: The step with which to plot the scoremaps.
        colors: The colormap to use.

    Returns:
        The figure and axis on which the image PAF was plot.
    """
    ny, nx = np.shape(image)[:2]
    fig, ax = form_figure(nx, ny)
    ax.imshow(image)
    n_fields = paf.shape[2]
    if colors is None:
        colors = ["r"] * n_fields
    for n in range(n_fields):
        U = paf[:, :, n, 0]
        V = paf[:, :, n, 1]
        X, Y = np.meshgrid(np.arange(U.shape[1]), np.arange(U.shape[0]))
        M = np.zeros(U.shape, dtype=bool)
        M[U**2 + V**2 < 0.5 * 0.5**2] = True
        U = np.ma.masked_array(U, mask=M)
        V = np.ma.masked_array(V, mask=M)
        ax.quiver(
            X[::step, ::step],
            Y[::step, ::step],
            U[::step, ::step],
            V[::step, ::step],
            scale=50,
            headaxislength=4,
            alpha=1,
            width=0.002,
            color=colors[n],
            angles="xy",
        )
    return fig, ax

visualize_scoremaps

visualize_scoremaps(image: ndarray, scmap: ndarray) -> tuple[plt.Figure, plt.Axes]

Plots scoremaps as an image overlay.

Parameters:

Name	Type	Description	Default
image	ndarray	An image as a numpy array of shape (h, w, channels)	required
scmap	ndarray	A scoremap of shape (h, w)	required

Returns:

Type	Description
tuple[Figure, Axes]	The figure and axis on which the image scoremap was plot.

Source code in deeplabcut/core/visualization.py

def visualize_scoremaps(
    image: np.ndarray,
    scmap: np.ndarray,
) -> tuple[plt.Figure, plt.Axes]:
    """Plots scoremaps as an image overlay.

    Args:
        image: An image as a numpy array of shape (h, w, channels)
        scmap: A scoremap of shape (h, w)

    Returns:
        The figure and axis on which the image scoremap was plot.
    """
    ny, nx = np.shape(image)[:2]
    fig, ax = form_figure(nx, ny)
    ax.imshow(image)
    ax.imshow(scmap, alpha=0.5)
    return fig, ax