deeplabcut.pose_estimation_tensorflow

Modules:

Name	Description
auxfun_models	DeepLabCut2.0 Toolbox (deeplabcut.org)
auxfun_multianimal	DeepLabCut2.0 Toolbox (deeplabcut.org)
auxiliaryfunctions	DeepLabCut2.0 Toolbox (deeplabcut.org)
backbones
config
core
datasets
export
inferenceutils
lib
modelzoo
nnets
predict
predict_multianimal
predict_videos
trackingutils
training
util	Adapted from DeeperCut by Eldar Insafutdinov:
visualize	Adapted from DeeperCut by Eldar Insafutdinov
visualizemaps

Functions:

Name	Description
AnalyzeVideo	Helper function for analyzing a video.
GetPoseDynamic	Non batch wise pose estimation for video cap by dynamically cropping around
GetPoseF	Batchwise prediction of pose.
GetPoseF_GTF	Batchwise prediction of pose.
GetPoseF_OV	Prediction of pose.
GetPoseS	Non batch wise pose estimation for video cap.
GetPoseS_GTF	Non batch wise pose estimation for video cap.
GetPosesofFrames	Batchwise prediction of pose for frame list in directory.
Plotting	Function used for plotting GT and predictions.
analyze_time_lapse_frames	Analyze all images (of type frametype) in a folder and store the output in
analyze_videos	Makes prediction based on a trained network.
argmax_pose_predict	Combine scoremat and offsets to the final pose.
calculatepafdistancebounds	Returns distances along paf edges in train/test data.
cfg_from_file	Load a config from file filename and merge it into the default options.
collect_video_paths	Collects video paths from a given set of data paths: directories, files, or a mix
convert_detections2tracklets	This should be called at the end of deeplabcut.analyze_videos for multianimal
evaluate_network	Evaluates the network.
extract_cnn_output	Extract locref + scmap from network.
extract_maps	Extracts the scoremap, locref, partaffinityfields (if available).
extract_save_all_maps	Extract scoremap, location refinement field and part affinity field predictions.
get_available_requested_snapshots	Intersects the requested snapshot names with the available snapshots.
get_snapshots_by_index	Assume available_snapshots is ordered in ascending order.
keypoint_error	Computes the RMSE error for each bodypart.
make_results_file	Makes result file in csv format and saves under evaluation_results directory.
pairwisedistances	Calculates the pairwise Euclidean distance metric over body parts vs.
renamed_parameter	Support a renamed keyword argument while warning callers to update.
return_evaluate_network_data	Returns the results for (previously evaluated) network.
return_train_network_path	Returns the training and test pose config file names as well as the folder where
stitch_tracklets	Stitch sparse tracklets into full tracks via a graph-based, minimum-cost flow
train_network	Trains the network with the labels in the training dataset.
visualize_locrefs	Plots a scoremap and the corresponding location refinement field on an image.
visualize_paf	Plots the PAF on top of the image.
visualize_scoremaps	Plots scoremaps as an image overlay.

AnalyzeVideo

AnalyzeVideo(
    video,
    DLCscorer,
    DLCscorerlegacy,
    trainFraction,
    cfg,
    dlc_cfg,
    sess,
    inputs,
    outputs,
    pdindex,
    save_as_csv,
    destfolder=None,
    TFGPUinference=True,
    dynamic=(False, 0.5, 10),
    use_openvino="CPU" if is_openvino_available else None,
)

Helper function for analyzing a video.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def AnalyzeVideo(
    video,
    DLCscorer,
    DLCscorerlegacy,
    trainFraction,
    cfg,
    dlc_cfg,
    sess,
    inputs,
    outputs,
    pdindex,
    save_as_csv,
    destfolder=None,
    TFGPUinference=True,
    dynamic=(False, 0.5, 10),
    use_openvino="CPU" if is_openvino_available else None,
):
    """Helper function for analyzing a video."""
    print("Starting to analyze % ", video)

    if destfolder is None:
        destfolder = str(Path(video).parents[0])
    auxiliaryfunctions.attempt_to_make_folder(destfolder)
    vname = Path(video).stem
    try:
        _ = auxiliaryfunctions.load_analyzed_data(destfolder, vname, DLCscorer)
    except FileNotFoundError as e:
        print("Loading ", video)
        cap = cv2.VideoCapture(video)
        if not cap.isOpened():
            raise OSError("Video could not be opened. Please check the file integrity.") from e
        # https://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html#videocapture-get
        fps = cap.get(cv2.CAP_PROP_FPS)
        nframes = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
        duration = nframes * 1.0 / fps
        size = (
            int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)),
            int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
        )
        ny, nx = size
        print(
            "Duration of video [s]: ",
            round(duration, 2),
            ", recorded with ",
            round(fps, 2),
            "fps!",
        )
        print(
            "Overall # of frames: ",
            nframes,
            " found with (before cropping) frame dimensions: ",
            nx,
            ny,
        )

        dynamic_analysis_state, detectiontreshold, margin = dynamic
        start = time.time()
        print("Starting to extract posture")
        if dynamic_analysis_state:
            PredictedData, nframes = GetPoseDynamic(
                cfg,
                dlc_cfg,
                sess,
                inputs,
                outputs,
                cap,
                nframes,
                detectiontreshold,
                margin,
            )
            # GetPoseF_GTF(cfg,dlc_cfg, sess, inputs, outputs,cap,nframes,int(dlc_cfg["batch_size"]))
        else:
            if int(dlc_cfg["batch_size"]) > 1:
                args = (
                    cfg,
                    dlc_cfg,
                    sess,
                    inputs,
                    outputs,
                    cap,
                    nframes,
                    int(dlc_cfg["batch_size"]),
                )
                if use_openvino:
                    PredictedData, nframes = GetPoseF_OV(*args)
                elif TFGPUinference:
                    PredictedData, nframes = GetPoseF_GTF(*args)
                else:
                    PredictedData, nframes = GetPoseF(*args)
            else:
                if TFGPUinference:
                    PredictedData, nframes = GetPoseS_GTF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes)
                else:
                    PredictedData, nframes = GetPoseS(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes)

        stop = time.time()
        if cfg["cropping"]:
            coords = [cfg["x1"], cfg["x2"], cfg["y1"], cfg["y2"]]
        else:
            coords = [0, nx, 0, ny]

        dictionary = {
            "start": start,
            "stop": stop,
            "run_duration": stop - start,
            "Scorer": DLCscorer,
            "DLC-model-config file": dlc_cfg,
            "fps": fps,
            "batch_size": dlc_cfg["batch_size"],
            "frame_dimensions": (ny, nx),
            "nframes": nframes,
            "iteration (active-learning)": cfg["iteration"],
            "training set fraction": trainFraction,
            "cropping": cfg["cropping"],
            "cropping_parameters": coords,
            # "gpu_info": device_lib.list_local_devices()
        }
        metadata = {"data": dictionary}

        print(f"Saving results in {destfolder}...")
        dataname = Path(destfolder) / (vname + DLCscorer + ".h5")
        auxiliaryfunctions.save_data(
            PredictedData[:nframes, :],
            metadata,
            dataname,
            pdindex,
            range(nframes),
            save_as_csv,
        )
    return DLCscorer

GetPoseDynamic

GetPoseDynamic(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, detectiontreshold, margin)

Non batch wise pose estimation for video cap by dynamically cropping around previously detected parts.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPoseDynamic(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, detectiontreshold, margin):
    """Non batch wise pose estimation for video cap by dynamically cropping around
    previously detected parts.
    """
    if cfg["cropping"]:
        ny, nx = checkcropping(cfg, cap)
    else:
        ny, nx = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)), int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    x1, x2, y1, y2 = 0, nx, 0, ny
    detected = False
    # TODO: perform detection on resized image (For speed)

    PredictedData = np.zeros((nframes, 3 * len(dlc_cfg["all_joints_names"])))
    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))
    while cap.isOpened():
        if counter != 0 and counter % step == 0:
            pbar.update(step)

        ret, frame = cap.read()
        if ret:
            # print(counter,x1,x2,y1,y2,detected)
            originalframe = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            if cfg["cropping"]:
                frame = img_as_ubyte(originalframe[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])[y1:y2, x1:x2]
            else:
                frame = img_as_ubyte(originalframe[y1:y2, x1:x2])

            pose = predict.getpose(frame, dlc_cfg, sess, inputs, outputs).flatten()
            detection = np.any(pose[2::3] > detectiontreshold)  # is anything detected?
            if detection:
                pose[0::3], pose[1::3] = (
                    pose[0::3] + x1,
                    pose[1::3] + y1,
                )  # offset according to last bounding box
                x1, x2, y1, y2 = getboundingbox(
                    pose[0::3], pose[1::3], nx, ny, margin
                )  # coordinates for next iteration
                if not detected:
                    detected = True  # object detected
            else:
                if (
                    detected and (x1 + y1 + y2 - ny + x2 - nx) != 0
                ):  # was detected in last frame and dyn. cropping was performed >>
                    # but object lost in cropped variant >> re-run on full frame!
                    # print("looking again, lost!")
                    if cfg["cropping"]:
                        frame = img_as_ubyte(originalframe[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])
                    else:
                        frame = img_as_ubyte(originalframe)
                    pose = predict.getpose(frame, dlc_cfg, sess, inputs, outputs).flatten()  # no offset is necessary

                _x0, _y0 = x1, y1
                x1, x2, y1, y2 = 0, nx, 0, ny
                detected = False

            PredictedData[counter, :] = pose
        elif counter >= nframes:
            break
        counter += 1

    pbar.close()
    return PredictedData, nframes

GetPoseF

GetPoseF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize)

Batchwise prediction of pose.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPoseF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize):
    """Batchwise prediction of pose."""
    PredictedData = np.zeros((nframes, dlc_cfg["num_outputs"] * 3 * len(dlc_cfg["all_joints_names"])))
    batch_ind = 0  # keeps track of which image within a batch should be written to
    batch_num = 0  # keeps track of which batch you are at
    ny, nx = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)), int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    if cfg["cropping"]:
        ny, nx = checkcropping(cfg, cap)

    frames = np.empty((batchsize, ny, nx, 3), dtype="ubyte")  # this keeps all frames in a batch
    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))
    inds = []
    while cap.isOpened():
        if counter != 0 and counter % step == 0:
            pbar.update(step)
        ret, frame = cap.read()
        if ret:
            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            if cfg["cropping"]:
                frames[batch_ind] = img_as_ubyte(frame[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])
            else:
                frames[batch_ind] = img_as_ubyte(frame)
            inds.append(counter)
            if batch_ind == batchsize - 1:
                pose = predict.getposeNP(frames, dlc_cfg, sess, inputs, outputs)
                PredictedData[inds] = pose
                batch_ind = 0
                inds.clear()
                batch_num += 1
            else:
                batch_ind += 1
        elif counter >= nframes:
            if batch_ind > 0:
                pose = predict.getposeNP(
                    frames, dlc_cfg, sess, inputs, outputs
                )  # process the whole batch (some frames might be from previous batch!)
                PredictedData[inds[:batch_ind]] = pose[:batch_ind]
            break
        counter += 1

    pbar.close()
    return PredictedData, nframes

GetPoseF_GTF

GetPoseF_GTF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize)

Batchwise prediction of pose.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPoseF_GTF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize):
    """Batchwise prediction of pose."""
    PredictedData = np.zeros((nframes, 3 * len(dlc_cfg["all_joints_names"])))
    batch_ind = 0  # keeps track of which image within a batch should be written to
    batch_num = 0  # keeps track of which batch you are at
    ny = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    nx = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    if cfg["cropping"]:
        ny, nx = checkcropping(cfg, cap)

    # Flip x, y, confidence and reshape
    pose_tensor = predict.extract_GPUprediction(outputs, dlc_cfg)
    pose_tensor = tf.gather(pose_tensor, [1, 0, 2], axis=1)
    pose_tensor = tf.reshape(pose_tensor, (batchsize, -1))

    frames = np.empty((batchsize, ny, nx, 3), dtype="ubyte")
    pbar = tqdm(total=nframes)
    counter = -1
    inds = []
    while cap.isOpened() and counter < nframes - 1:
        ret, frame = cap.read()
        counter += 1
        if not ret:
            warnings.warn(f"Could not decode frame #{counter}.", stacklevel=2)
            continue

        if cfg["cropping"]:
            frame = img_as_ubyte(frame[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])
        else:
            frame = img_as_ubyte(frame)
        frames[batch_ind] = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        inds.append(counter)
        if batch_ind == batchsize - 1:
            pose = sess.run(pose_tensor, feed_dict={inputs: frames})
            PredictedData[inds] = pose
            batch_ind = 0
            batch_num += 1
            inds.clear()
            pbar.update(batchsize)
        else:
            batch_ind += 1

    if batch_ind > 0:
        pose = sess.run(pose_tensor, feed_dict={inputs: frames})
        PredictedData[inds[:batch_ind]] = pose[:batch_ind]
        pbar.update(batch_ind)

    pbar.close()
    return PredictedData, nframes

GetPoseF_OV

GetPoseF_OV(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize)

Prediction of pose.

Source code in deeplabcut/pose_estimation_tensorflow/core/openvino/session.py

def GetPoseF_OV(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, batchsize):
    """Prediction of pose."""
    PredictedData = np.zeros((nframes, 3 * len(dlc_cfg["all_joints_names"])))
    ny, nx = int(cap.get(4)), int(cap.get(3))
    if cfg["cropping"]:
        from ...predict_videos import checkcropping

        ny, nx = checkcropping(cfg, cap)

    sess._init_model(ny, nx)

    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))

    def completion_callback(request, inp_id):
        pose = next(iter(request.results.values()))

        pose[:, [0, 1, 2]] = pose[:, [1, 0, 2]]  # change order to have x,y,confidence
        pose = np.reshape(pose, (1, -1))  # bring into batchsize times x,y,conf etc.
        PredictedData[inp_id] = pose

    sess.infer_queue.set_callback(completion_callback)

    while cap.isOpened():
        if counter % step == 0:
            pbar.update(step)
        ret, frame = cap.read()
        if not ret:
            break

        # Prepare input data
        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        if cfg["cropping"]:
            frame = frame[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]]

        sess.infer_queue.start_async({sess.input_name: np.expand_dims(frame, axis=0)}, counter)

        counter += 1

    sess.infer_queue.wait_all()

    pbar.close()
    return PredictedData, nframes

GetPoseS

GetPoseS(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes)

Non batch wise pose estimation for video cap.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPoseS(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes):
    """Non batch wise pose estimation for video cap."""
    if cfg["cropping"]:
        ny, nx = checkcropping(cfg, cap)

    PredictedData = np.zeros((nframes, dlc_cfg["num_outputs"] * 3 * len(dlc_cfg["all_joints_names"])))
    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))
    while cap.isOpened():
        if counter != 0 and counter % step == 0:
            pbar.update(step)

        ret, frame = cap.read()
        if ret:
            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            if cfg["cropping"]:
                frame = img_as_ubyte(frame[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])
            else:
                frame = img_as_ubyte(frame)
            pose = predict.getpose(frame, dlc_cfg, sess, inputs, outputs)
            PredictedData[counter, :] = (
                pose.flatten()
            )  # NOTE: thereby cfg['all_joints_names'] should be same order as bodyparts!
        elif counter >= nframes:
            break
        counter += 1

    pbar.close()
    return PredictedData, nframes

GetPoseS_GTF

GetPoseS_GTF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes)

Non batch wise pose estimation for video cap.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPoseS_GTF(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes):
    """Non batch wise pose estimation for video cap."""
    if cfg["cropping"]:
        ny, nx = checkcropping(cfg, cap)

    pose_tensor = predict.extract_GPUprediction(outputs, dlc_cfg)  # extract_output_tensor(outputs, dlc_cfg)
    PredictedData = np.zeros((nframes, 3 * len(dlc_cfg["all_joints_names"])))
    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))
    while cap.isOpened():
        if counter != 0 and counter % step == 0:
            pbar.update(step)

        ret, frame = cap.read()
        if ret:
            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            if cfg["cropping"]:
                frame = img_as_ubyte(frame[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"]])
            else:
                frame = img_as_ubyte(frame)

            pose = sess.run(
                pose_tensor,
                feed_dict={inputs: np.expand_dims(frame, axis=0).astype(float)},
            )
            pose[:, [0, 1, 2]] = pose[:, [1, 0, 2]]
            # pose = predict.getpose(frame, dlc_cfg, sess, inputs, outputs)
            PredictedData[counter, :] = (
                pose.flatten()
            )  # NOTE: thereby cfg['all_joints_names'] should be same order as bodyparts!
        elif counter >= nframes:
            break
        counter += 1

    pbar.close()
    return PredictedData, nframes

GetPosesofFrames

GetPosesofFrames(cfg, dlc_cfg, sess, inputs, outputs, directory, framelist, nframes, batchsize)

Batchwise prediction of pose for frame list in directory.

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def GetPosesofFrames(cfg, dlc_cfg, sess, inputs, outputs, directory, framelist, nframes, batchsize):
    """Batchwise prediction of pose for frame list in directory."""
    from deeplabcut.utils.auxfun_videos import imread

    print("Starting to extract posture")
    im = imread(Path(directory) / framelist[0], mode="skimage")

    ny, nx, nc = np.shape(im)
    print(
        "Overall # of frames: ",
        nframes,
        " found with (before cropping) frame dimensions: ",
        nx,
        ny,
    )

    PredictedData = np.zeros((nframes, dlc_cfg["num_outputs"] * 3 * len(dlc_cfg["all_joints_names"])))
    batch_ind = 0  # keeps track of which image within a batch should be written to
    batch_num = 0  # keeps track of which batch you are at
    if cfg["cropping"]:
        print(
            "Cropping based on the x1 = {} x2 = {} y1 = {} y2 = {}. "
            "You can adjust the cropping coordinates in the config.yaml file.".format(
                cfg["x1"], cfg["x2"], cfg["y1"], cfg["y2"]
            )
        )
        nx, ny = cfg["x2"] - cfg["x1"], cfg["y2"] - cfg["y1"]
        if nx > 0 and ny > 0:
            pass
        else:
            raise Exception("Please check the order of cropping parameter!")
        if cfg["x1"] >= 0 and cfg["x2"] < int(np.shape(im)[1]) and cfg["y1"] >= 0 and cfg["y2"] < int(np.shape(im)[0]):
            pass  # good cropping box
        else:
            raise Exception("Please check the boundary of cropping!")

    pbar = tqdm(total=nframes)
    counter = 0
    step = max(10, int(nframes / 100))

    if batchsize == 1:
        for counter, framename in enumerate(framelist):
            im = imread(Path(directory) / framename, mode="skimage")

            if counter != 0 and counter % step == 0:
                pbar.update(step)

            if cfg["cropping"]:
                frame = img_as_ubyte(im[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"], :])
            else:
                frame = img_as_ubyte(im)

            pose = predict.getpose(frame, dlc_cfg, sess, inputs, outputs)
            PredictedData[counter, :] = pose.flatten()
    else:
        frames = np.empty((batchsize, ny, nx, 3), dtype="ubyte")  # this keeps all the frames of a batch
        for counter, framename in enumerate(framelist):
            im = imread(Path(directory) / framename, mode="skimage")

            if counter != 0 and counter % step == 0:
                pbar.update(step)

            if cfg["cropping"]:
                frames[batch_ind] = img_as_ubyte(im[cfg["y1"] : cfg["y2"], cfg["x1"] : cfg["x2"], :])
            else:
                frames[batch_ind] = img_as_ubyte(im)

            if batch_ind == batchsize - 1:
                pose = predict.getposeNP(frames, dlc_cfg, sess, inputs, outputs)
                PredictedData[batch_num * batchsize : (batch_num + 1) * batchsize, :] = pose
                batch_ind = 0
                batch_num += 1
            else:
                batch_ind += 1

        if batch_ind > 0:  # take care of the last frames (the batch that might have been processed)
            pose = predict.getposeNP(
                frames, dlc_cfg, sess, inputs, outputs
            )  # process the whole batch (some frames might be from previous batch!)
            PredictedData[batch_num * batchsize : batch_num * batchsize + batch_ind, :] = pose[:batch_ind, :]

    pbar.close()
    return PredictedData, nframes, nx, ny

Plotting

Plotting(cfg, comparisonbodyparts, DLCscorer, trainIndices, DataCombined, foldername)

Function used for plotting GT and predictions.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def Plotting(cfg, comparisonbodyparts, DLCscorer, trainIndices, DataCombined, foldername):
    """Function used for plotting GT and predictions."""
    from deeplabcut.utils import visualization

    colors = visualization.get_cmap(len(comparisonbodyparts), name=cfg["colormap"])
    NumFrames = np.size(DataCombined.index)
    fig, ax = visualization.create_minimal_figure()
    for ind in tqdm(np.arange(NumFrames)):
        ax = visualization.plot_and_save_labeled_frame(
            DataCombined,
            ind,
            trainIndices,
            cfg,
            colors,
            comparisonbodyparts,
            DLCscorer,
            foldername,
            fig,
            ax,
        )
        visualization.erase_artists(ax)

analyze_time_lapse_frames

analyze_time_lapse_frames(
    config, directory, frametype=".png", shuffle=1, trainingsetindex=0, gputouse=None, save_as_csv=False, modelprefix=""
)

Analyze all images (of type frametype) in a folder and store the output in one file.

You can crop the frames (before analysis), by changing 'cropping'=True and setting 'x1','x2','y1','y2' in the config file.

Output labels are stored as a MultiIndex Pandas DataFrame containing the network name, body part name, (x, y) label position in pixels, and likelihood for each frame per body part. These arrays are stored in HDF format in the same directory as the images. If save_as_csv is True, data can also be exported as CSV.

Parameters:

Name	Type	Description	Default
config	string	Full path of the config.yaml file as a string.	required
directory	string	Full path to directory containing the frames that shall be analyzed.	required
frametype	string	Checks for the file extension of the frames. Only images with this extension are analyzed. Defaults to .png.	'.png'
shuffle	int	Shuffle index of the training dataset used for training the network. Defaults to 1.	1
trainingsetindex	int	Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). Defaults to 0.	0
gputouse	int	Natural number indicating the number of your GPU (see number in nvidia-smi). If you do not have a GPU, set to None. See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries	None
save_as_csv	bool	Saves the predictions in a .csv file. Defaults to False.	False
modelprefix	str	Directory containing the deeplabcut models to use. Defaults to "".	''

Examples:

If you want to analyze all frames in /analysis/project/timelapseexperiment1:

deeplabcut.analyze_images(
    "/analysis/project/reaching-task/config.yaml",
    "/analysis/project/timelapseexperiment1",
)

Note

For test purposes one can extract all frames from a video with ffmpeg, e.g. ffmpeg -i testvideo.avi thumb%04d.png

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

def analyze_time_lapse_frames(
    config,
    directory,
    frametype=".png",
    shuffle=1,
    trainingsetindex=0,
    gputouse=None,
    save_as_csv=False,
    modelprefix="",
):
    """Analyze all images (of type ``frametype``) in a folder and store the output in
    one file.

    You can crop the frames (before analysis),
    by changing 'cropping'=True and setting 'x1','x2','y1','y2' in the config file.

    Output labels are stored as a MultiIndex Pandas DataFrame containing the network
    name, body part name, (x, y) label position in pixels, and likelihood for each
    frame per body part. These arrays are stored in HDF format in the same directory
    as the images. If ``save_as_csv`` is True, data can also be exported as CSV.

    Args:
        config (string): Full path of the config.yaml file as a string.
        directory (string): Full path to directory containing the frames that shall be analyzed.
        frametype (string, optional): Checks for the file extension of the frames.
            Only images with this extension are analyzed. Defaults to ``.png``.
        shuffle (int, optional): Shuffle index of the training dataset used for training the network. Defaults to 1.
        trainingsetindex (int, optional): Integer specifying which TrainingsetFraction to use.
            By default the first (note that TrainingFraction is a list in config.yaml).
            Defaults to 0.
        gputouse (int, optional): Natural number indicating the number of your GPU (see number in nvidia-smi).
            If you do not have a GPU, set to None.
            See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries
        save_as_csv (bool, optional): Saves the predictions in a .csv file. Defaults to False.
        modelprefix (str, optional): Directory containing the deeplabcut models to use.
            Defaults to "".

    Examples:
        If you want to analyze all frames in /analysis/project/timelapseexperiment1:

            deeplabcut.analyze_images(
                "/analysis/project/reaching-task/config.yaml",
                "/analysis/project/timelapseexperiment1",
            )

    Note:
        For test purposes one can extract all frames from a video with ffmpeg,
        e.g. ffmpeg -i testvideo.avi thumb%04d.png
    """
    if "TF_CUDNN_USE_AUTOTUNE" in os.environ:
        del os.environ["TF_CUDNN_USE_AUTOTUNE"]  # was potentially set during training

    if gputouse is not None:  # gpu selection
        auxfun_models.set_visible_devices(gputouse)

    tf.compat.v1.reset_default_graph()
    start_path = Path.cwd()  # record cwd to return to this directory in the end

    cfg = auxiliaryfunctions.read_config(config)
    trainFraction = cfg["TrainingFraction"][trainingsetindex]
    modelfolder = Path(cfg["project_path"]) / str(
        auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)
    )
    path_test_config = Path(modelfolder) / "test" / "pose_cfg.yaml"
    try:
        dlc_cfg = load_config(str(path_test_config))
    except FileNotFoundError as e:
        raise FileNotFoundError(
            f"It seems the model for shuffle {shuffle} and trainFraction {trainFraction} does not exist."
        ) from e

    Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
        train_folder=Path(modelfolder) / "train",
    )

    if cfg["snapshotindex"] == "all":
        print(
            "Snapshotindex is set to 'all' in the config.yaml file. "
            "Running video analysis with all snapshots is very costly! "
            "Use the function 'evaluate_network' to choose the best the snapshot. "
            "For now, changing snapshot index to -1!"
        )
        snapshotindex = -1
    else:
        snapshotindex = cfg["snapshotindex"]

    print(f"Using {Snapshots[snapshotindex]}", "for model", modelfolder)

    ##################################################
    # Load and setup CNN part detector
    ##################################################

    # Check if data already was generated:
    dlc_cfg["init_weights"] = str(Path(modelfolder) / "train" / Snapshots[snapshotindex])
    trainingsiterations = Path(dlc_cfg["init_weights"]).name.split("-")[-1]

    # update batchsize (based on parameters in config.yaml)
    dlc_cfg["batch_size"] = cfg["batch_size"]

    # Name for scorer:
    DLCscorer, DLCscorerlegacy = auxiliaryfunctions.get_scorer_name(
        cfg,
        shuffle,
        trainFraction,
        trainingsiterations=trainingsiterations,
        modelprefix=modelprefix,
    )
    sess, inputs, outputs = predict.setup_pose_prediction(dlc_cfg)

    # update number of outputs and adjust pandas indices
    dlc_cfg["num_outputs"] = cfg.get("num_outputs", 1)

    xyz_labs_orig = ["x", "y", "likelihood"]
    suffix = [str(s + 1) for s in range(dlc_cfg["num_outputs"])]
    suffix[0] = ""  # first one has empty suffix for backwards compatibility
    xyz_labs = [x + s for s in suffix for x in xyz_labs_orig]

    pdindex = pd.MultiIndex.from_product(
        [[DLCscorer], dlc_cfg["all_joints_names"], xyz_labs],
        names=["scorer", "bodyparts", "coords"],
    )

    if gputouse is not None:  # gpu selectinon
        auxfun_models.set_visible_devices(gputouse)

    ##################################################
    # Loading the images
    ##################################################
    # checks if input is a directory
    if Path(directory).is_dir():
        """Analyzes all the frames in the directory."""
        print("Analyzing all frames in the directory: ", directory)
        os.chdir(directory)
        framelist = np.sort([fn.name for fn in Path(directory).iterdir() if (frametype in fn.name)])
        vname = Path(directory).stem
        notanalyzed, dataname, DLCscorer = auxiliaryfunctions.check_if_not_analyzed(
            directory, vname, DLCscorer, DLCscorerlegacy, flag="framestack"
        )
        if notanalyzed:
            nframes = len(framelist)
            if nframes > 0:
                start = time.time()

                PredictedData, nframes, nx, ny = GetPosesofFrames(
                    cfg,
                    dlc_cfg,
                    sess,
                    inputs,
                    outputs,
                    directory,
                    framelist,
                    nframes,
                    dlc_cfg["batch_size"],
                )
                stop = time.time()

                if cfg["cropping"]:
                    coords = [cfg["x1"], cfg["x2"], cfg["y1"], cfg["y2"]]
                else:
                    coords = [0, nx, 0, ny]

                dictionary = {
                    "start": start,
                    "stop": stop,
                    "run_duration": stop - start,
                    "Scorer": DLCscorer,
                    "config file": dlc_cfg,
                    "batch_size": dlc_cfg["batch_size"],
                    "num_outputs": dlc_cfg["num_outputs"],
                    "frame_dimensions": (ny, nx),
                    "nframes": nframes,
                    "cropping": cfg["cropping"],
                    "cropping_parameters": coords,
                }
                metadata = {"data": dictionary}

                print(f"Saving results in {directory}...")

                auxiliaryfunctions.save_data(
                    PredictedData[:nframes, :],
                    metadata,
                    dataname,
                    pdindex,
                    framelist,
                    save_as_csv,
                )
                print("The folder was analyzed. Now your research can truly start!")
                print("If the tracking is not satisfactory for some frame, consider expanding the training set.")
            else:
                print("No frames were found. Consider changing the path or the frametype.")

    os.chdir(str(start_path))

analyze_videos

analyze_videos(
    config,
    videos,
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    gputouse=None,
    save_as_csv=False,
    in_random_order=True,
    destfolder=None,
    batchsize=None,
    cropping=None,
    TFGPUinference=True,
    dynamic=(False, 0.5, 10),
    modelprefix="",
    robust_nframes=False,
    allow_growth=False,
    use_shelve=False,
    auto_track=True,
    n_tracks=None,
    animal_names=None,
    calibrate=False,
    identity_only=False,
    use_openvino="CPU" if is_openvino_available else None,
)

Makes prediction based on a trained network.

The index of the trained network is specified by parameters in the config file (in particular the variable 'snapshotindex').

The labels are stored as MultiIndex Pandas Array, which contains the name of the network, body part name, (x, y) label position in pixels, and the likelihood for each frame per body part. These arrays are stored in an efficient Hierarchical Data Format (HDF) in the same directory where the video is stored. However, if the flag save_as_csv is set to True, the data can also be exported in comma-separated values format (.csv), which in turn can be imported in many programs, such as MATLAB, R, Prism, etc.

Parameters:

Name	Type	Description	Default
config	str	Full path of the config.yaml file.	required
videos	list[str]	A list of strings containing the full paths to videos for analysis or a path to the directory, where all the videos with same extension are stored.	required
video_extensions	str | Sequence[str] | None	Controls how videos are filtered, based on file extension. File paths and directory contents are treated differently: - None (default): file paths are accepted as-is; directories are scanned for files with a recognized video extension. - str or Sequence[str] (e.g. "mp4" or ["mp4", "avi"]): both file paths and directory contents are filtered by the given extension(s). Defaults to None.	None
shuffle	int	Shuffle index of the training dataset used for training the network. Defaults to 1.	1
trainingsetindex	int	Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). Defaults to 0.	0
gputouse	int or None	Indicates the GPU to use (see number in nvidia-smi). If you do not have a GPU put None. See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries. Defaults to None.	None
save_as_csv	bool	Saves the predictions in a .csv file. Defaults to False.	False
in_random_order	bool	Whether or not to analyze videos in a random order. This is only relevant when specifying a video directory in videos. Defaults to True.	True
destfolder	string or None	Destination folder for analysis data. If None, uses the video path. Pass this folder for subsequent analysis too. Defaults to None.	None
batchsize	int or None	Batch size for inference; overwrites pose_cfg.yaml if set. Defaults to None.	None
cropping	list or None	List of cropping coordinates as [x1, x2, y1, y2]. Note that the same cropping parameters will then be used for all videos. If different video crops are desired, run analyze_videos on individual videos with the corresponding cropping coordinates. Defaults to None.	None
TFGPUinference	bool	Perform inference on GPU with TensorFlow code. Introduced in "Pretraining boosts out-of-domain robustness for pose estimation" by Alexander Mathis, Mert Yüksekgönül, Byron Rogers, Matthias Bethge, Mackenzie W. Mathis. Source: https://arxiv.org/abs/1909.11229. Defaults to True.	True
dynamic	tuple[bool, float, int]	Triple containing (state, detection_threshold, margin). If state is True, dynamic cropping is performed: when any body part exceeds detection_threshold, object boundaries are computed from min/max x/y positions, expanded by margin, and only the posture within this crop is analyzed until the object is lost. Defaults to (False, 0.5, 10).	(False, 0.5, 10)
modelprefix	str	Directory containing the deeplabcut models to use when evaluating the network. By default, the models are assumed to exist in the project folder. Defaults to "".	''
robust_nframes	bool	Evaluate a video's number of frames in a robust manner. This option is slower (as the whole video is read frame-by-frame), but does not rely on metadata, hence its robustness against file corruption. Defaults to False.	False
allow_growth	bool	For some smaller GPUs the memory issues happen. If True, the memory allocator does not pre-allocate the entire specified GPU memory region, instead starting small and growing as needed. See issue: https://forum.image.sc/t/how-to-stop-running-out-of-vram/30551/2. Defaults to False.	False
use_shelve	bool	By default, data are dumped in a pickle file at the end of the video analysis. Otherwise, data are written to disk on the fly using a "shelf"; i.e., a pickle-based, persistent, database-like object by default, resulting in constant memory footprint. Defaults to False.	False
auto_track	bool	For multi-animal projects, automatically perform tracking and stitching to produce the final h5 file. If False, run convert_detections2tracklets and stitch_tracklets afterwards. Defaults to True.	True
identity_only	bool	If True and animal identity was learned by the model, assembly and tracking rely exclusively on identity prediction. Defaults to False.	False
calibrate	bool	If True, use training data to calibrate the animal assembly procedure. Defaults to False.	False
n_tracks	int or None	Number of tracks to reconstruct. By default from config.yaml. Pass another value if animal count differs from training. Defaults to None.	None
animal_names	list[str]	If you want the names given to individuals in the labeled data file, you can specify those names as a list here. If given and n_tracks is None, n_tracks will be set to len(animal_names). If n_tracks is not None, then it must be equal to len(animal_names). If it is not given, then animal_names will be loaded from the individuals in the project config.yaml file.	None
use_openvino	str	Use "CPU" for inference if OpenVINO is available in the Python environment. Defaults to "CPU" when OpenVINO is available, otherwise None.	'CPU' if is_openvino_available else None

Returns:

Name	Type	Description
str		DLCScorer; the scorer used to analyze the videos.

Examples:

Analyzing a single video on Windows:

deeplabcut.analyze_videos(
    'C:\myproject\reaching-task\config.yaml',
    ['C:\yourusername\rig-95\Videos\reachingvideo1.avi'],
)

Analyzing a single video on Linux/MacOS:

deeplabcut.analyze_videos(
    '/analysis/project/reaching-task/config.yaml',
    ['/analysis/project/videos/reachingvideo1.avi'],
)

Analyze all videos of type avi in a folder:

deeplabcut.analyze_videos(
    '/analysis/project/reaching-task/config.yaml',
    ['/analysis/project/videos'],
    video_extensions='.avi',
)

Analyze multiple videos:

deeplabcut.analyze_videos(
    '/analysis/project/reaching-task/config.yaml',
    [
        '/analysis/project/videos/reachingvideo1.avi',
        '/analysis/project/videos/reachingvideo2.avi',
    ],
)

Analyze multiple videos with shuffle=2:

deeplabcut.analyze_videos(
    '/analysis/project/reaching-task/config.yaml',
    [
        '/analysis/project/videos/reachingvideo1.avi',
        '/analysis/project/videos/reachingvideo2.avi',
    ],
    shuffle=2,
)

Analyze multiple videos with shuffle=2, save results as an additional csv file:

deeplabcut.analyze_videos(
    '/analysis/project/reaching-task/config.yaml',
    [
        '/analysis/project/videos/reachingvideo1.avi',
        '/analysis/project/videos/reachingvideo2.avi',
    ],
    shuffle=2,
    save_as_csv=True,
)

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

@renamed_parameter(old="videotype", new="video_extensions", since="3.0.0")
def analyze_videos(
    config,
    videos,
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    gputouse=None,
    save_as_csv=False,
    in_random_order=True,
    destfolder=None,
    batchsize=None,
    cropping=None,
    TFGPUinference=True,
    dynamic=(False, 0.5, 10),
    modelprefix="",
    robust_nframes=False,
    allow_growth=False,
    use_shelve=False,
    auto_track=True,
    n_tracks=None,
    animal_names=None,
    calibrate=False,
    identity_only=False,
    use_openvino="CPU" if is_openvino_available else None,
):
    """Makes prediction based on a trained network.

    The index of the trained network is specified by parameters in the config file
    (in particular the variable 'snapshotindex').

    The labels are stored as MultiIndex Pandas Array, which contains the name of
    the network, body part name, (x, y) label position in pixels, and the
    likelihood for each frame per body part. These arrays are stored in an
    efficient Hierarchical Data Format (HDF) in the same directory where the video
    is stored. However, if the flag save_as_csv is set to True, the data can also
    be exported in comma-separated values format (.csv), which in turn can be
    imported in many programs, such as MATLAB, R, Prism, etc.

    Args:
        config (str): Full path of the config.yaml file.
        videos (list[str]): A list of strings containing the full paths to videos for analysis or a path to
            the directory, where all the videos with same extension are stored.
        video_extensions (str | Sequence[str] | None, optional): Controls how ``videos`` are
            filtered, based on file extension. File paths and directory contents are
            treated differently:
            - ``None`` (default): file paths are accepted as-is; directories are
              scanned for files with a recognized video extension.
            - ``str`` or ``Sequence[str]`` (e.g. ``"mp4"`` or ``["mp4", "avi"]``):
              both file paths and directory contents are filtered by the given
              extension(s). Defaults to None.
        shuffle (int, optional): Shuffle index of the training dataset used for
            training the network. Defaults to 1.
        trainingsetindex (int, optional): Integer specifying which TrainingsetFraction to use.
            By default the first (note that TrainingFraction is a list in config.yaml). Defaults to 0.
        gputouse (int or None, optional): Indicates the GPU to use (see number in ``nvidia-smi``). If you do not have a
            GPU put ``None``.
            See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries. Defaults to None.
        save_as_csv (bool, optional): Saves the predictions in a .csv file. Defaults to False.
        in_random_order (bool, optional): Whether or not to analyze videos in a random order.
            This is only relevant when specifying a video directory in `videos`. Defaults to True.
        destfolder (string or None, optional): Destination folder for analysis data. If ``None``, uses the
            video path. Pass this folder for subsequent analysis too. Defaults to None.
        batchsize (int or None, optional): Batch size for inference; overwrites ``pose_cfg.yaml`` if set.
            Defaults to None.
        cropping (list or None, optional): List of cropping coordinates as [x1, x2, y1, y2].
            Note that the same cropping parameters will then be used for all videos.
            If different video crops are desired, run ``analyze_videos`` on individual
            videos with the corresponding cropping coordinates. Defaults to None.
        TFGPUinference (bool, optional): Perform inference on GPU with TensorFlow code. Introduced in "Pretraining
            boosts out-of-domain robustness for pose estimation" by Alexander Mathis,
            Mert Yüksekgönül, Byron Rogers, Matthias Bethge, Mackenzie W. Mathis.
            Source: https://arxiv.org/abs/1909.11229. Defaults to True.
        dynamic (tuple[bool, float, int], optional): Triple containing (state,
            detection_threshold, margin). If state is True, dynamic cropping is
            performed: when any body part exceeds detection_threshold, object boundaries
            are computed from min/max x/y positions, expanded by margin, and only the
            posture within this crop is analyzed until the object is lost. Defaults to
            (False, 0.5, 10).
        modelprefix (str, optional): Directory containing the deeplabcut models to use when evaluating the network.
            By default, the models are assumed to exist in the project folder. Defaults to "".
        robust_nframes (bool, optional): Evaluate a video's number of frames in a robust manner.
            This option is slower (as the whole video is read frame-by-frame),
            but does not rely on metadata, hence its robustness against file corruption. Defaults to False.
        allow_growth (bool, optional): For some smaller GPUs the memory issues happen. If ``True``, the memory
            allocator does not pre-allocate the entire specified GPU memory region, instead
            starting small and growing as needed.
            See issue: https://forum.image.sc/t/how-to-stop-running-out-of-vram/30551/2. Defaults to False.
        use_shelve (bool, optional): By default, data are dumped in a pickle file at the end of the video analysis.
            Otherwise, data are written to disk on the fly using a "shelf"; i.e., a
            pickle-based, persistent, database-like object by default, resulting in
            constant memory footprint. Defaults to False.
        auto_track (bool, optional): For multi-animal projects, automatically perform
            tracking and stitching to produce the final h5 file. If False, run
            ``convert_detections2tracklets`` and ``stitch_tracklets`` afterwards.
            Defaults to True.
        identity_only (bool, optional): If True and animal identity was learned by the
            model, assembly and tracking rely exclusively on identity prediction.
            Defaults to False.
        calibrate (bool, optional): If True, use training data to calibrate the animal
            assembly procedure. Defaults to False.
        n_tracks (int or None, optional): Number of tracks to reconstruct. By default from config.yaml.
            Pass another value if animal count differs from training. Defaults to None.
        animal_names (list[str], optional): If you want the names given to individuals in the labeled data file, you can
            specify those names as a list here. If given and `n_tracks` is None, `n_tracks`
            will be set to `len(animal_names)`. If `n_tracks` is not None, then it must be
            equal to `len(animal_names)`. If it is not given, then `animal_names` will
            be loaded from the `individuals` in the project config.yaml file.
        use_openvino (str, optional): Use "CPU" for inference if OpenVINO is available in the Python environment.
            Defaults to "CPU" when OpenVINO is available, otherwise None.

    Returns:
        str: DLCScorer; the scorer used to analyze the videos.

    Examples:
        Analyzing a single video on Windows:

            deeplabcut.analyze_videos(
                'C:\\myproject\\reaching-task\\config.yaml',
                ['C:\\yourusername\\rig-95\\Videos\\reachingvideo1.avi'],
            )

        Analyzing a single video on Linux/MacOS:

            deeplabcut.analyze_videos(
                '/analysis/project/reaching-task/config.yaml',
                ['/analysis/project/videos/reachingvideo1.avi'],
            )

        Analyze all videos of type ``avi`` in a folder:

            deeplabcut.analyze_videos(
                '/analysis/project/reaching-task/config.yaml',
                ['/analysis/project/videos'],
                video_extensions='.avi',
            )

        Analyze multiple videos:

            deeplabcut.analyze_videos(
                '/analysis/project/reaching-task/config.yaml',
                [
                    '/analysis/project/videos/reachingvideo1.avi',
                    '/analysis/project/videos/reachingvideo2.avi',
                ],
            )

        Analyze multiple videos with ``shuffle=2``:

            deeplabcut.analyze_videos(
                '/analysis/project/reaching-task/config.yaml',
                [
                    '/analysis/project/videos/reachingvideo1.avi',
                    '/analysis/project/videos/reachingvideo2.avi',
                ],
                shuffle=2,
            )

        Analyze multiple videos with ``shuffle=2``, save results as an additional csv file:

            deeplabcut.analyze_videos(
                '/analysis/project/reaching-task/config.yaml',
                [
                    '/analysis/project/videos/reachingvideo1.avi',
                    '/analysis/project/videos/reachingvideo2.avi',
                ],
                shuffle=2,
                save_as_csv=True,
            )
    """
    if "TF_CUDNN_USE_AUTOTUNE" in os.environ:
        del os.environ["TF_CUDNN_USE_AUTOTUNE"]  # was potentially set during training

    if gputouse is not None:  # gpu selection
        auxfun_models.set_visible_devices(gputouse)

    tf.compat.v1.reset_default_graph()
    start_path = Path.cwd()  # record cwd to return to this directory in the end

    cfg = auxiliaryfunctions.read_config(config)
    trainFraction = cfg["TrainingFraction"][trainingsetindex]
    iteration = cfg["iteration"]

    if cropping is not None:
        cfg["cropping"] = True
        cfg["x1"], cfg["x2"], cfg["y1"], cfg["y2"] = cropping
        print("Overwriting cropping parameters:", cropping)
        print("These are used for all videos, but won't be save to the cfg file.")

    modelfolder = Path(cfg["project_path"]) / str(
        auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)
    )
    path_test_config = Path(modelfolder) / "test" / "pose_cfg.yaml"
    try:
        dlc_cfg = load_config(str(path_test_config))
    except FileNotFoundError as e:
        raise FileNotFoundError(
            f"It seems the model for iteration {iteration} and shuffle "
            f"{shuffle} and trainFraction {trainFraction} does not exist."
        ) from e

    Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
        train_folder=Path(modelfolder) / "train",
    )

    if cfg["snapshotindex"] == "all":
        print(
            "Snapshotindex is set to 'all' in the config.yaml file."
            "Running video analysis with all snapshots is very costly! "
            "Use the function 'evaluate_network' to choose the best the snapshot. "
            "For now, changing snapshot index to -1!"
        )
        snapshotindex = -1
    else:
        snapshotindex = cfg["snapshotindex"]

    print(f"Using {Snapshots[snapshotindex]}", "for model", modelfolder)

    ##################################################
    # Load and setup CNN part detector
    ##################################################

    # Check if data already was generated:
    dlc_cfg["init_weights"] = str(Path(modelfolder) / "train" / Snapshots[snapshotindex])
    trainingsiterations = Path(dlc_cfg["init_weights"]).name.split("-")[-1]
    # Update number of output and batchsize
    dlc_cfg["num_outputs"] = cfg.get("num_outputs", dlc_cfg.get("num_outputs", 1))

    if batchsize is None:
        # update batchsize (based on parameters in config.yaml)
        dlc_cfg["batch_size"] = cfg["batch_size"]
    else:
        dlc_cfg["batch_size"] = batchsize
        cfg["batch_size"] = batchsize

    if "multi-animal" in dlc_cfg["dataset_type"]:
        dynamic = (False, 0.5, 10)  # setting dynamic mode to false
        TFGPUinference = False

    if dynamic[0]:  # state=true
        # (state,detectiontreshold,margin)=dynamic
        print("Starting analysis in dynamic cropping mode with parameters:", dynamic)
        dlc_cfg["num_outputs"] = 1
        TFGPUinference = False
        dlc_cfg["batch_size"] = 1
        print(
            "Switching batchsize to 1, num_outputs (per animal) to 1 "
            "and TFGPUinference to False (all these features are not supported in this mode)."
        )

    # Name for scorer:
    DLCscorer, DLCscorerlegacy = auxiliaryfunctions.get_scorer_name(
        cfg,
        shuffle,
        trainFraction,
        trainingsiterations=trainingsiterations,
        modelprefix=modelprefix,
    )
    if dlc_cfg["num_outputs"] > 1:
        if TFGPUinference:
            print(
                "Switching to numpy-based keypoint extraction code, "
                "as multiple point extraction is not supported by TF code currently."
            )
            TFGPUinference = False
        print("Extracting ", dlc_cfg["num_outputs"], "instances per bodypart")
        xyz_labs_orig = ["x", "y", "likelihood"]
        suffix = [str(s + 1) for s in range(dlc_cfg["num_outputs"])]
        suffix[0] = ""  # first one has empty suffix for backwards compatibility
        xyz_labs = [x + s for s in suffix for x in xyz_labs_orig]
    else:
        xyz_labs = ["x", "y", "likelihood"]

    if use_openvino:
        sess, inputs, outputs = predict.setup_openvino_pose_prediction(dlc_cfg, device=use_openvino)
    elif TFGPUinference:
        sess, inputs, outputs = predict.setup_GPUpose_prediction(dlc_cfg, allow_growth=allow_growth)
    else:
        sess, inputs, outputs = predict.setup_pose_prediction(dlc_cfg, allow_growth=allow_growth)

    pdindex = pd.MultiIndex.from_product(
        [[DLCscorer], dlc_cfg["all_joints_names"], xyz_labs],
        names=["scorer", "bodyparts", "coords"],
    )

    ##################################################
    # Looping over videos
    ##################################################
    Videos = collect_video_paths(videos, extensions=video_extensions, shuffle=in_random_order)
    if len(Videos) > 0:
        if "multi-animal" in dlc_cfg["dataset_type"]:
            from deeplabcut.pose_estimation_tensorflow.predict_multianimal import (
                AnalyzeMultiAnimalVideo,
            )

            for video in Videos:
                AnalyzeMultiAnimalVideo(
                    video,
                    DLCscorer,
                    trainFraction,
                    cfg,
                    dlc_cfg,
                    sess,
                    inputs,
                    outputs,
                    destfolder,
                    robust_nframes=robust_nframes,
                    use_shelve=use_shelve,
                )
                if auto_track:  # tracker type is taken from default in cfg
                    convert_detections2tracklets(
                        config,
                        [video],
                        video_extensions,
                        shuffle,
                        trainingsetindex,
                        destfolder=destfolder,
                        modelprefix=modelprefix,
                        calibrate=calibrate,
                        identity_only=identity_only,
                    )
                    stitch_tracklets(
                        config,
                        [video],
                        video_extensions,
                        shuffle,
                        trainingsetindex,
                        destfolder=destfolder,
                        n_tracks=n_tracks,
                        animal_names=animal_names,
                        modelprefix=modelprefix,
                        save_as_csv=save_as_csv,
                    )
        else:
            for video in Videos:
                DLCscorer = AnalyzeVideo(
                    video,
                    DLCscorer,
                    DLCscorerlegacy,
                    trainFraction,
                    cfg,
                    dlc_cfg,
                    sess,
                    inputs,
                    outputs,
                    pdindex,
                    save_as_csv,
                    destfolder,
                    TFGPUinference,
                    dynamic,
                    use_openvino,
                )

        os.chdir(str(start_path))
        if "multi-animal" in dlc_cfg["dataset_type"]:
            print(
                "The videos are analyzed. Time to assemble animals and track 'em... \n"
                " Call 'create_video_with_all_detections' to check multi-animal detection quality before tracking."
            )
            print(
                "If the tracking is not satisfactory for some videos, consider expanding the training set. "
                "You can use the function 'extract_outlier_frames' to extract a few representative outlier frames."
            )
        else:
            print(
                "The videos are analyzed. Now your research can truly start! \n "
                "You can create labeled videos with 'create_labeled_video'"
            )
            print(
                "If the tracking is not satisfactory for some videos, consider expanding the training set. "
                "You can use the function 'extract_outlier_frames' to extract a few representative outlier frames."
            )
        return DLCscorer  # note: this is either DLCscorer or DLCscorerlegacy depending on what was used!
    else:
        print("No video(s) were found. Please check your paths and/or video_extensions filter.")
        return DLCscorer

argmax_pose_predict

argmax_pose_predict(scmap, offmat, stride)

Combine scoremat and offsets to the final pose.

Source code in deeplabcut/pose_estimation_tensorflow/core/predict.py

def argmax_pose_predict(scmap, offmat, stride):
    """Combine scoremat and offsets to the final pose."""
    num_joints = scmap.shape[2]
    pose = []
    for joint_idx in range(num_joints):
        maxloc = np.unravel_index(np.argmax(scmap[:, :, joint_idx]), scmap[:, :, joint_idx].shape)
        offset = np.array(offmat[maxloc][joint_idx])[::-1]
        pos_f8 = np.array(maxloc).astype("float") * stride + 0.5 * stride + offset
        pose.append(np.hstack((pos_f8[::-1], [scmap[maxloc][joint_idx]])))
    return np.array(pose)

calculatepafdistancebounds

calculatepafdistancebounds(config, shuffle=0, trainingsetindex=0, modelprefix='', numdigits=0, onlytrain=False)

Returns distances along paf edges in train/test data.

Parameters:

Name	Type	Description	Default
config	string	Full path of the config.yaml file as a string.	required
shuffle	int	Integer specifying shuffle index of the training dataset. Defaults to 0.	0
trainingsetindex	int	Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). This variable can also be set to "all".	0
numdigits	int	Number of digits to round for distances.	0

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def calculatepafdistancebounds(config, shuffle=0, trainingsetindex=0, modelprefix="", numdigits=0, onlytrain=False):
    """Returns distances along paf edges in train/test data.

    Args:
        config (string): Full path of the config.yaml file as a string.
        shuffle (int): Integer specifying shuffle index of the training dataset.
            Defaults to 0.
        trainingsetindex (int, optional): Integer specifying which TrainingsetFraction
            to use. By default the first (note that TrainingFraction is a list in
            config.yaml). This variable can also be set to "all".
        numdigits (int): Number of digits to round for distances.
    """

    from deeplabcut.pose_estimation_tensorflow.config import load_config
    from deeplabcut.utils import auxfun_multianimal, auxiliaryfunctions

    # Read file path for pose_config file. >> pass it on
    cfg = auxiliaryfunctions.read_config(config)

    if cfg["multianimalproject"]:
        (
            individuals,
            uniquebodyparts,
            multianimalbodyparts,
        ) = auxfun_multianimal.extractindividualsandbodyparts(cfg)

        # Loading human annotatated data
        trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg)
        trainFraction = cfg["TrainingFraction"][trainingsetindex]
        modelfolder = Path(cfg["project_path"]) / str(
            auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)
        )

        # Load meta data & annotations
        Data = pd.read_hdf(
            Path(cfg["project_path"]) / str(trainingsetfolder) / ("CollectedData_" + cfg["scorer"] + ".h5")
        )[cfg["scorer"]]

        path_train_config, path_test_config, _ = return_train_network_path(
            config=config,
            shuffle=shuffle,
            trainingsetindex=trainingsetindex,
            modelprefix=modelprefix,
        )
        train_pose_cfg = load_config(str(path_train_config))
        test_pose_cfg = load_config(str(path_test_config))

        _, trainIndices, _, _ = auxiliaryfunctions.load_metadata(
            Path(cfg["project_path"]) / train_pose_cfg["metadataset"]
        )

        # get the graph!
        partaffinityfield_graph = test_pose_cfg["partaffinityfield_graph"]
        jointnames = [test_pose_cfg["all_joints_names"][i] for i in range(len(test_pose_cfg["all_joints"]))]
        path_inferencebounds_config = Path(modelfolder) / "test" / "inferencebounds.yaml"
        inferenceboundscfg = {}
        for _pi, edge in enumerate(partaffinityfield_graph):
            j1, j2 = jointnames[edge[0]], jointnames[edge[1]]
            ds_within = []
            ds_across = []
            for ind in individuals:
                for ind2 in individuals:
                    if ind != "single" and ind2 != "single":
                        if (ind, j1, "x") in Data.keys() and (
                            ind2,
                            j2,
                            "y",
                        ) in Data.keys():
                            distances = (
                                np.sqrt(
                                    (Data[ind, j1, "x"] - Data[ind2, j2, "x"]) ** 2
                                    + (Data[ind, j1, "y"] - Data[ind2, j2, "y"]) ** 2
                                )
                                / test_pose_cfg["stride"]
                            )
                        else:
                            distances = None

                        if distances is not None:
                            if onlytrain:
                                distances = distances.iloc[trainIndices]
                            if ind == ind2:
                                ds_within.extend(distances.values.flatten())
                            else:
                                ds_across.extend(distances.values.flatten())

            edgeencoding = str(edge[0]) + "_" + str(edge[1])
            inferenceboundscfg[edgeencoding] = {}
            if len(ds_within) > 0:
                inferenceboundscfg[edgeencoding]["intra_max"] = str(round(np.nanmax(ds_within), numdigits))
                inferenceboundscfg[edgeencoding]["intra_min"] = str(round(np.nanmin(ds_within), numdigits))
            else:
                inferenceboundscfg[edgeencoding]["intra_max"] = str(
                    1e5
                )  # large number (larger than any image diameter)
                inferenceboundscfg[edgeencoding]["intra_min"] = str(0)

            # NOTE: the inter-animal distances are currently not used, but are interesting to compare to intra_*
            if len(ds_across) > 0:
                inferenceboundscfg[edgeencoding]["inter_max"] = str(round(np.nanmax(ds_across), numdigits))
                inferenceboundscfg[edgeencoding]["inter_min"] = str(round(np.nanmin(ds_across), numdigits))
            else:
                inferenceboundscfg[edgeencoding]["inter_max"] = str(
                    1e5
                )  # large number (larger than image diameters in typical experiments)
                inferenceboundscfg[edgeencoding]["inter_min"] = str(0)

        auxiliaryfunctions.write_plainconfig(str(path_inferencebounds_config), dict(inferenceboundscfg))
        return inferenceboundscfg
    else:
        print("You might as well bring owls to Athens.")
        return {}

cfg_from_file

cfg_from_file(filename)

Load a config from file filename and merge it into the default options.

Source code in deeplabcut/pose_estimation_tensorflow/config.py

def cfg_from_file(filename):
    """Load a config from file filename and merge it into the default options."""
    with Path(filename).open() as f:
        yaml_cfg = yaml.load(f, Loader=yaml.SafeLoader)

    # Update the snapshot path to the corresponding path!
    trainpath = str(filename).split("pose_cfg.yaml")[0]
    yaml_cfg["snapshot_prefix"] = trainpath + "snapshot"
    # the default is: "./snapshot"

    # reloading defaults, as they can bleed over from a previous run otherwise
    import importlib

    from . import default_config

    importlib.reload(default_config)

    default_cfg = default_config.cfg
    _merge_a_into_b(yaml_cfg, default_cfg)

    logging.info("Config:\n" + pprint.pformat(default_cfg))
    return default_cfg  # updated

collect_video_paths

collect_video_paths(
    data_path: str | Path | list[str | Path],
    extensions: str | Sequence[str] | None = None,
    shuffle: bool = False,
    exclude_patterns: Sequence[str] = DEFAULT_EXCLUDE_PATTERNS,
) -> list[Path]

Collects video paths from a given set of data paths: directories, files, or a mix of both. Directories are scanned one level deep (non-recursively).

Files and directories are treated differently with respect to extension filtering: - File paths are accepted as-is when extensions is None; only filtered when extensions is explicitly set. - Directory contents are always filtered by extension: by SUPPORTED_VIDEOS when extensions is None, or by the given value(s) otherwise. - exclude_patterns are always applied to both files and directory contents.

Parameters:

Name	Type	Description	Default
data_path	str | Path | list[str | Path]	Path or list of paths to folders containing videos, or individual video files. Can be a mix of directories and files.	required
extensions	str | Sequence[str] | None	Controls extension filtering for collected video files. - None (default): file paths are accepted without extension filtering; directories are scanned for files with a recognized video extension. - str or Sequence[str] (e.g. "mp4" or ["mp4", "avi"]): both file paths and directory contents are filtered to only include files matching the given extension(s). - Empty str "" is treated as None (deprecated, keep for backwards compatibility).	None
shuffle	bool	Whether to shuffle the order of videos. If False, videos are returned in sorted order for deterministic behavior.	False
exclude_patterns	Sequence[str]	Patterns to exclude from the collection. Defaults to DEFAULT_EXCLUDE_PATTERNS. Set to [] to disable pattern exclusion.	DEFAULT_EXCLUDE_PATTERNS

Returns:

Type	Description
list[Path]	The paths of videos to analyze. Duplicate paths are removed.

Raises:

Type	Description
FileNotFoundError	If any path in data_path does not exist.
ValueError	If extensions is an empty sequence.

Source code in deeplabcut/utils/auxfun_videos.py

def collect_video_paths(
    data_path: str | Path | list[str | Path],
    extensions: str | Sequence[str] | None = None,
    shuffle: bool = False,
    exclude_patterns: Sequence[str] = DEFAULT_EXCLUDE_PATTERNS,
) -> list[Path]:
    """
    Collects video paths from a given set of data paths: directories, files, or a mix
    of both. Directories are scanned one level deep (non-recursively).

    Files and directories are treated differently with respect to extension filtering:
    - File paths are accepted as-is when ``extensions`` is ``None``; only filtered when
      ``extensions`` is explicitly set.
    - Directory contents are always filtered by extension: by ``SUPPORTED_VIDEOS`` when
      ``extensions`` is ``None``, or by the given value(s) otherwise.
    - ``exclude_patterns`` are always applied to both files and directory contents.

    Args:
        data_path: Path or list of paths to folders containing videos, or individual
            video files. Can be a mix of directories and files.
        extensions: Controls extension filtering for collected video files.
            - ``None`` (default): file paths are accepted without extension filtering;
              directories are scanned for files with a recognized video extension.
            - ``str`` or ``Sequence[str]`` (e.g. ``"mp4"`` or ``["mp4", "avi"]``):
              both file paths and directory contents are filtered to only include files
              matching the given extension(s).
            - Empty ``str`` ``""`` is treated as ``None`` (deprecated, keep for backwards
              compatibility).
        shuffle: Whether to shuffle the order of videos. If ``False``, videos are
            returned in sorted order for deterministic behavior.
        exclude_patterns: Patterns to exclude from the collection. Defaults to
            ``DEFAULT_EXCLUDE_PATTERNS``. Set to ``[]`` to disable pattern exclusion.

    Returns:
        The paths of videos to analyze. Duplicate paths are removed.

    Raises:
        FileNotFoundError: If any path in ``data_path`` does not exist.
        ValueError: If ``extensions`` is an empty sequence.
    """
    if isinstance(data_path, (str, Path)):
        data_path = [data_path]

    def _coerce_extensions(extensions: str | Sequence[str] | None) -> set[str] | None:
        """Coerce the extensions argument to a set of dot-prefixed suffixes, or None."""
        if extensions is None:
            return None

        if extensions in ["", ("",), [""], {""}]:
            warnings.warn(
                "Passing an empty string for filtering video type extensions is deprecated; pass None instead.",
                DLCDeprecationWarning,
                stacklevel=3,
            )
            return None

        if isinstance(extensions, str):
            return {f".{extensions.lstrip('.').lower()}"}

        if not isinstance(extensions, Sequence):
            raise TypeError(f"extensions must be a string, a sequence or None, got {type(extensions)}")

        if len(extensions) == 0:
            raise ValueError("Video type extensions filter needs to be a non-empty sequence.")
        return {f".{e.lstrip('.').lower()}" for e in extensions}

    explicit_suffixes = _coerce_extensions(extensions)
    implicit_suffixes = {f".{ext.lower()}" for ext in SUPPORTED_VIDEOS}

    videos: list[Path] = []
    for path in map(Path, data_path):
        if not path.exists():
            raise FileNotFoundError(f"Could not find: {path}. Check access rights.")

        if path.is_dir():
            # Discriminate videos from other files; skip excluded patterns (e.g. prior DLC outputs).
            allowed = explicit_suffixes if explicit_suffixes else implicit_suffixes
            videos.extend(
                f
                for f in path.iterdir()
                if f.is_file()
                and f.suffix.lower() in allowed
                and not any(f.match(pattern) for pattern in exclude_patterns)
            )
        elif path.is_file():
            # Accept all caller-supplied files; ONLY filter extensions if set. ALWAYS filter exclude patterns.
            if explicit_suffixes is None or path.suffix.lower() in explicit_suffixes:
                if not any(path.match(pattern) for pattern in exclude_patterns):
                    videos.append(path)

    # Resolve video paths and remove duplicates
    unique_videos = list(dict.fromkeys(v.absolute() for v in videos))
    if shuffle:
        random.shuffle(unique_videos)
    else:
        unique_videos.sort()

    if any(fn.suffix.lower().lstrip(".") not in SUPPORTED_VIDEOS for fn in unique_videos if fn.suffix):
        warnings.warn(
            f"Some videos have unsupported extensions: {unique_videos} \nSupported extensions are: {SUPPORTED_VIDEOS}",
            stacklevel=2,
        )
    return unique_videos

convert_detections2tracklets

convert_detections2tracklets(
    config,
    videos,
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    overwrite=False,
    destfolder=None,
    ignore_bodyparts=None,
    inferencecfg=None,
    modelprefix="",
    greedy=False,
    calibrate=False,
    window_size=0,
    identity_only=False,
    track_method="",
)

This should be called at the end of deeplabcut.analyze_videos for multianimal projects!

Parameters:

Name	Type	Description	Default
config	string	Full path of the config.yaml file as a string.	required
videos	list	A list of strings containing the full paths to videos for analysis or a path to the directory, where all the videos with same extension are stored.	required
video_extensions	str | Sequence[str] | None	Controls how videos are filtered, based on file extension. File paths and directory contents are treated differently: - None (default): file paths are accepted as-is; directories are scanned for files with a recognized video extension. - str or Sequence[str] (e.g. "mp4" or ["mp4", "avi"]): both file paths and directory contents are filtered by the given extension(s). Defaults to None.	None
shuffle	int	Shuffle index of the training dataset used for training the network. Defaults to 1.	1
trainingsetindex	int	Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). Defaults to 0.	0
overwrite	bool	Overwrite tracks file; recompute tracks from full detections. Defaults to False.	False
destfolder	string	Destination folder for analysis data (default is the path of the video). Note that for subsequent analysis this folder also needs to be passed.	None
ignore_bodyparts	list	List of body part names to ignore during tracking. By default, all body parts are used. Defaults to None.	None
inferencecfg	dict	Configuration for inference (assembly of individuals). Ideally obtained from cross validation. By default loaded from inference_cfg.yaml. Defaults to None.	None
modelprefix	str	Directory containing the deeplabcut models to use. Defaults to "".	''
greedy	bool	Use greedy assembly instead of default method. Defaults to False.	False
calibrate	bool	If True, use training data to calibrate the animal assembly procedure. This improves its robustness to wrong body part links, but requires very little missing data. Defaults to False.	False
window_size	int	Recurrent connections in the past window_size frames are prioritized during assembly. By default, no temporal coherence cost is added, and assembly is driven mainly by part affinity costs. Defaults to 0.	0
identity_only	bool	If True and animal identity was learned by the model, assembly and tracking rely exclusively on identity prediction. Defaults to False.	False
track_method	string	Specifies the tracker used to generate the pose estimation data. For multiple animals, must be either 'box', 'skeleton', or 'ellipse' and will be taken from the config.yaml file if none is given. Defaults to "".	''

Examples:

If you want to convert detections to tracklets:

deeplabcut.convert_detections2tracklets(
    '/analysis/project/reaching-task/config.yaml',
    ['/analysis/project/video1.mp4'],
    video_extensions='.mp4',
)

If you want to convert detections to tracklets based on box_tracker:

deeplabcut.convert_detections2tracklets(
    '/analysis/project/reaching-task/config.yaml',
    ['/analysis/project/video1.mp4'],
    video_extensions='.mp4',
    track_method='box',
)

Source code in deeplabcut/pose_estimation_tensorflow/predict_videos.py

@renamed_parameter(old="videotype", new="video_extensions", since="3.0.0")
def convert_detections2tracklets(
    config,
    videos,
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    overwrite=False,
    destfolder=None,
    ignore_bodyparts=None,
    inferencecfg=None,
    modelprefix="",
    greedy=False,
    calibrate=False,
    window_size=0,
    identity_only=False,
    track_method="",
):
    """This should be called at the end of deeplabcut.analyze_videos for multianimal
    projects!

    Args:
        config (string): Full path of the config.yaml file as a string.
        videos (list): A list of strings containing the full paths to videos for analysis
            or a path to the directory, where all the videos with same extension are stored.
        video_extensions (str | Sequence[str] | None, optional): Controls how ``videos`` are
            filtered, based on file extension. File paths and directory contents are
            treated differently:
            - ``None`` (default): file paths are accepted as-is; directories are
              scanned for files with a recognized video extension.
            - ``str`` or ``Sequence[str]`` (e.g. ``"mp4"`` or ``["mp4", "avi"]``):
              both file paths and directory contents are filtered by the given
              extension(s). Defaults to None.
        shuffle (int, optional): Shuffle index of the training dataset used for training the network.
            Defaults to 1.
        trainingsetindex (int, optional): Integer specifying which TrainingsetFraction to use.
            By default the first (note that TrainingFraction is a list in config.yaml).
            Defaults to 0.
        overwrite (bool, optional): Overwrite tracks file; recompute tracks from full
            detections. Defaults to False.
        destfolder (string, optional): Destination folder for analysis data (default is the path of the video).
            Note that for subsequent analysis this folder also needs to be passed.
        ignore_bodyparts (list, optional): List of body part names to ignore during tracking.
            By default, all body parts are used. Defaults to None.
        inferencecfg (dict, optional): Configuration for inference (assembly of individuals).
            Ideally obtained from cross validation. By default loaded from inference_cfg.yaml.
            Defaults to None.
        modelprefix (str, optional): Directory containing the deeplabcut models to use.
            Defaults to "".
        greedy (bool, optional): Use greedy assembly instead of default method.
            Defaults to False.
        calibrate (bool, optional): If True, use training data to calibrate the animal assembly procedure.
            This improves its robustness to wrong body part links,
            but requires very little missing data. Defaults to False.
        window_size (int, optional): Recurrent connections in the past `window_size` frames are
            prioritized during assembly. By default, no temporal coherence cost
            is added, and assembly is driven mainly by part affinity costs. Defaults to 0.
        identity_only (bool, optional): If True and animal identity was learned by the model,
            assembly and tracking rely exclusively on identity prediction. Defaults to False.
        track_method (string, optional): Specifies the tracker used to generate the pose estimation data.
            For multiple animals, must be either 'box', 'skeleton', or 'ellipse'
            and will be taken from the config.yaml file if none is given. Defaults to "".

    Examples:
        If you want to convert detections to tracklets:

            deeplabcut.convert_detections2tracklets(
                '/analysis/project/reaching-task/config.yaml',
                ['/analysis/project/video1.mp4'],
                video_extensions='.mp4',
            )

        If you want to convert detections to tracklets based on box_tracker:

            deeplabcut.convert_detections2tracklets(
                '/analysis/project/reaching-task/config.yaml',
                ['/analysis/project/video1.mp4'],
                video_extensions='.mp4',
                track_method='box',
            )
    """
    cfg = auxiliaryfunctions.read_config(config)
    track_method = auxfun_multianimal.get_track_method(cfg, track_method=track_method)

    if len(cfg["multianimalbodyparts"]) == 1 and track_method != "box":
        warnings.warn("Switching to `box` tracker for single point tracking...", stacklevel=2)
        track_method = "box"
        cfg["default_track_method"] = track_method
        auxiliaryfunctions.write_config(config, cfg)

    trainFraction = cfg["TrainingFraction"][trainingsetindex]
    start_path = Path.cwd()  # record cwd to return to this directory in the end

    # TODO: add cropping as in video analysis!
    # if cropping is not None:
    #    cfg['cropping']=True
    #    cfg['x1'],cfg['x2'],cfg['y1'],cfg['y2']=cropping
    #    print("Overwriting cropping parameters:", cropping)
    #    print("These are used for all videos, but won't be save to the cfg file.")

    modelfolder = Path(cfg["project_path"]) / str(
        auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)
    )
    path_test_config = Path(modelfolder) / "test" / "pose_cfg.yaml"
    try:
        dlc_cfg = load_config(str(path_test_config))
    except FileNotFoundError as e:
        raise FileNotFoundError(
            f"It seems the model for shuffle {shuffle} and trainFraction {trainFraction} does not exist."
        ) from e

    if "multi-animal" not in dlc_cfg["dataset_type"]:
        raise ValueError("This function is only required for multianimal projects!")

    path_inference_config = Path(modelfolder) / "test" / "inference_cfg.yaml"
    if inferencecfg is None:  # then load or initialize
        inferencecfg = auxfun_multianimal.read_inferencecfg(path_inference_config, cfg)
    else:
        auxfun_multianimal.check_inferencecfg_sanity(cfg, inferencecfg)

    if len(cfg["multianimalbodyparts"]) == 1 and track_method != "box":
        warnings.warn("Switching to `box` tracker for single point tracking...", stacklevel=2)
        track_method = "box"
        # Also ensure `boundingboxslack` is greater than zero, otherwise overlap
        # between trackers cannot be evaluated, resulting in empty tracklets.
        inferencecfg["boundingboxslack"] = max(inferencecfg["boundingboxslack"], 40)

    Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
        train_folder=Path(modelfolder) / "train",
    )

    if cfg["snapshotindex"] == "all":
        print(
            "Snapshotindex is set to 'all' in the config.yaml file. "
            "Running video analysis with all snapshots is very costly! "
            "Use the function 'evaluate_network' to choose the best the snapshot. "
            "For now, changing snapshot index to -1!"
        )
        snapshotindex = -1
    else:
        snapshotindex = cfg["snapshotindex"]

    print(f"Using {Snapshots[snapshotindex]}", "for model", modelfolder)
    dlc_cfg["init_weights"] = str(Path(modelfolder) / "train" / Snapshots[snapshotindex])
    trainingsiterations = Path(dlc_cfg["init_weights"]).name.split("-")[-1]

    # Name for scorer:
    DLCscorer, DLCscorerlegacy = auxiliaryfunctions.get_scorer_name(
        cfg,
        shuffle,
        trainFraction,
        trainingsiterations=trainingsiterations,
        modelprefix=modelprefix,
    )

    ##################################################
    # Looping over videos
    ##################################################
    Videos = collect_video_paths(videos, extensions=video_extensions)
    if len(Videos) > 0:
        for video in Videos:
            print("Processing... ", video)
            videofolder = str(Path(video).parents[0])
            if destfolder is None:
                destfolder = videofolder
            auxiliaryfunctions.attempt_to_make_folder(destfolder)
            vname = Path(video).stem
            dataname = Path(destfolder) / (vname + DLCscorer + ".h5")
            data, metadata = auxfun_multianimal.LoadFullMultiAnimalData(dataname)
            if track_method == "ellipse":
                method = "el"
            elif track_method == "box":
                method = "bx"
            else:
                method = "sk"
            trackname = dataname.with_name(f"{dataname.stem}_{method}.pickle")
            # NOTE: If dataname line above is changed then line below is obsolete?
            # trackname = trackname.replace(videofolder, destfolder)
            if Path(trackname).is_file() and not overwrite:  # TODO: check if metadata are identical (same parameters!)
                print("Tracklets already computed", trackname)
                print("Set overwrite = True to overwrite.")
            else:
                print("Analyzing", dataname)
                DLCscorer = metadata["data"]["Scorer"]
                all_jointnames = data["metadata"]["all_joints_names"]

                numjoints = len(all_jointnames)

                # TODO: adjust this for multi + unique bodyparts!
                # this is only for multianimal parts and uniquebodyparts as one (not one
                # uniquebodyparts guy tracked etc. )
                bodypartlabels = [bpt for i, bpt in enumerate(all_jointnames) for _ in range(3)]
                scorers = len(bodypartlabels) * [DLCscorer]
                xylvalue = int(len(bodypartlabels) / 3) * ["x", "y", "likelihood"]
                pdindex = pd.MultiIndex.from_arrays(
                    np.vstack([scorers, bodypartlabels, xylvalue]),
                    names=["scorer", "bodyparts", "coords"],
                )

                imnames = [fn for fn in data if fn != "metadata"]

                if track_method == "box":
                    mot_tracker = trackingutils.SORTBox(
                        inferencecfg["max_age"],
                        inferencecfg["min_hits"],
                        inferencecfg.get("iou_threshold", 0.3),
                    )
                elif track_method == "skeleton":
                    mot_tracker = trackingutils.SORTSkeleton(
                        numjoints,
                        inferencecfg["max_age"],
                        inferencecfg["min_hits"],
                        inferencecfg.get("oks_threshold", 0.5),
                    )
                else:
                    mot_tracker = trackingutils.SORTEllipse(
                        inferencecfg.get("max_age", 1),
                        inferencecfg.get("min_hits", 1),
                        inferencecfg.get("iou_threshold", 0.6),
                    )
                tracklets = {}
                multi_bpts = cfg["multianimalbodyparts"]
                assembly_builder = inferenceutils.Assembler(
                    data,
                    max_n_individuals=inferencecfg["topktoretain"],
                    n_multibodyparts=len(multi_bpts),
                    greedy=greedy,
                    pcutoff=inferencecfg.get("pcutoff", 0.1),
                    min_affinity=inferencecfg.get("pafthreshold", 0.05),
                    window_size=window_size,
                    identity_only=identity_only,
                    min_n_links=inferencecfg["minimalnumberofconnections"],
                )
                assemblies_filename = dataname.with_name(dataname.stem + "_assemblies.pickle")
                if not Path(assemblies_filename).exists() or overwrite:
                    if calibrate:
                        trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg)
                        train_data_file = (
                            Path(cfg["project_path"])
                            / str(trainingsetfolder)
                            / ("CollectedData_" + cfg["scorer"] + ".h5")
                        )
                        assembly_builder.calibrate(train_data_file)
                    assembly_builder.assemble()
                    assembly_builder.to_pickle(assemblies_filename)
                else:
                    assembly_builder.from_pickle(assemblies_filename)
                    print(f"Loading assemblies from {assemblies_filename}")
                try:
                    data.close()
                except AttributeError:
                    pass

                if cfg["uniquebodyparts"]:  # Initialize storage of the 'single' individual track
                    tracklets["single"] = {}
                    _single = {}
                    for index, imname in enumerate(imnames):
                        single_detection = assembly_builder.unique.get(index)
                        if single_detection is None:
                            continue
                        imindex = int(re.findall(r"\d+", imname)[0])
                        _single[imindex] = single_detection
                    tracklets["single"].update(_single)

                if inferencecfg["topktoretain"] == 1:
                    tracklets[0] = {}
                    for index, imname in tqdm(enumerate(imnames)):
                        assemblies = assembly_builder.assemblies.get(index)
                        if assemblies is None:
                            continue
                        tracklets[0][imname] = assemblies[0].data
                else:
                    keep = set(multi_bpts).difference(ignore_bodyparts or [])
                    keep_inds = sorted(multi_bpts.index(bpt) for bpt in keep)
                    for index, imname in tqdm(enumerate(imnames)):
                        assemblies = assembly_builder.assemblies.get(index)
                        if assemblies is None:
                            continue
                        animals = np.stack([assembly_builder.data for assembly_builder in assemblies])
                        if not identity_only:
                            if track_method == "box":
                                xy = trackingutils.calc_bboxes_from_keypoints(
                                    animals[:, keep_inds],
                                    inferencecfg["boundingboxslack"],
                                )  # TODO: get cropping parameters and utilize!
                            else:
                                xy = animals[:, keep_inds, :2]
                            trackers = mot_tracker.track(xy)
                        else:
                            # Optimal identity assignment based on soft voting
                            mat = np.zeros((len(assemblies), inferencecfg["topktoretain"]))
                            for nrow, assembly in enumerate(assemblies):
                                for k, v in assembly.soft_identity.items():
                                    mat[nrow, k] = v
                            inds = linear_sum_assignment(mat, maximize=True)
                            trackers = np.c_[inds][:, ::-1]
                        trackingutils.fill_tracklets(tracklets, trackers, animals, imname)

                tracklets["header"] = pdindex
                with Path(trackname).open("wb") as f:
                    pickle.dump(tracklets, f, pickle.HIGHEST_PROTOCOL)

        os.chdir(str(start_path))

        print(
            "The tracklets were created (i.e., under the hood deeplabcut.convert_detections2tracklets was run). "
            "Now you can 'refine_tracklets' in the GUI, or run 'deeplabcut.stitch_tracklets'."
        )
    else:
        print("No video(s) found. Please check your path!")

evaluate_network

evaluate_network(
    config,
    Shuffles=None,
    trainingsetindex=0,
    plotting=False,
    show_errors=True,
    comparisonbodyparts="all",
    gputouse=None,
    rescale=False,
    modelprefix="",
    per_keypoint_evaluation: bool = False,
    snapshots_to_evaluate: list[str] = None,
)

Evaluates the network.

Evaluates the network based on the saved models at different stages of the training network. The evaluation results are stored in the .h5 and .csv file under the subdirectory 'evaluation_results'. Change the snapshotindex parameter in the config file to 'all' in order to evaluate all the saved models.

Parameters:

Name	Type	Description	Default
config	string	Full path of the config.yaml file.	required
shuffles	list	List of integers specifying the shuffle indices of the training dataset. Defaults to [1].	required
trainingsetindex	int or str	Integer specifying which "TrainingsetFraction" to use. Note that "TrainingFraction" is a list in config.yaml. This variable can also be set to "all". Defaults to 0.	0
plotting	bool or str	Plots the predictions on the train and test images. If provided it must be either True, False, "bodypart", or "individual". Setting to True defaults as "bodypart" for multi-animal projects. Defaults to False.	False
show_errors	bool	Display train and test errors. Defaults to True.	True
comparisonbodyparts	str or list	The average error will be computed for those body parts only. The provided list has to be a subset of the defined body parts. Defaults to "all".	'all'
gputouse	int or None	Indicates the GPU to use (see number in nvidia-smi). If you do not have a GPU put `None``. See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries Defaults to None.	None
rescale	bool	Evaluate the model at the 'global_scale' variable (as set in the pose_config.yaml file for a particular project). I.e. every image will be resized according to that scale and prediction will be compared to the resized ground truth. The error will be reported in pixels at rescaled to the original size. I.e. For a [200,200] pixel image evaluated at global_scale=.5, the predictions are calculated on [100,100] pixel images, compared to 1/2*ground truth and this error is then multiplied by 2!. The evaluation images are also shown for the original size! Defaults to False.	False
modelprefix	str	Directory containing the deeplabcut models to use when evaluating the network. By default, the models are assumed to exist in the project folder. Defaults to "".	''
per_keypoint_evaluation	bool	Compute the train and test RMSE for each keypoint, and save the results to a {model_name}-keypoint-results.csv in the evaluation-results folder. Defaults to False.	False
snapshots_to_evaluate	list[str]	List of snapshot names to evaluate (e.g. ["snapshot-50000", "snapshot-75000", ...]). Defaults to None.	None

Returns:

Type	Description
	None

Examples:

If you do not want to plot and evaluate with shuffle set to 1:

deeplabcut.evaluate_network(
    "/analysis/project/reaching-task/config.yaml",
    Shuffles=[1],
)

If you want to plot and evaluate with shuffle set to 0 and 1:

deeplabcut.evaluate_network(
    "/analysis/project/reaching-task/config.yaml",
    Shuffles=[0, 1],
    plotting=True,
)

If you want to plot assemblies for a maDLC project:

deeplabcut.evaluate_network(
    "/analysis/project/reaching-task/config.yaml",
    Shuffles=[1],
    plotting="individual",
)

Note: This defaults to standard plotting for single-animal projects.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def evaluate_network(
    config,
    Shuffles=None,
    trainingsetindex=0,
    plotting=False,
    show_errors=True,
    comparisonbodyparts="all",
    gputouse=None,
    rescale=False,
    modelprefix="",
    per_keypoint_evaluation: bool = False,
    snapshots_to_evaluate: list[str] = None,
):
    """Evaluates the network.

    Evaluates the network based on the saved models at different stages of the training
    network. The evaluation results are stored in the .h5 and .csv file under the
    subdirectory 'evaluation_results'. Change the snapshotindex parameter in the config
    file to 'all' in order to evaluate all the saved models.

    Args:
        config (string): Full path of the config.yaml file.
        shuffles (list, optional): List of integers specifying the shuffle indices of
            the training dataset. Defaults to [1].
        trainingsetindex (int or str, optional): Integer specifying which
            "TrainingsetFraction" to use. Note that "TrainingFraction" is a list in
            config.yaml. This variable can also be set to "all". Defaults to 0.
        plotting (bool or str, optional): Plots the predictions on the train and test
            images. If provided it must be either ``True``, ``False``, ``"bodypart"``,
            or ``"individual"``. Setting to ``True`` defaults as ``"bodypart"`` for
            multi-animal projects. Defaults to False.
        show_errors (bool, optional): Display train and test errors. Defaults to True.
        comparisonbodyparts (str or list, optional): The average error will be computed
            for those body parts only. The provided list has to be a subset of the
            defined body parts. Defaults to "all".
        gputouse (int or None, optional): Indicates the GPU to use (see number in
            ``nvidia-smi``). If you do not have a GPU put `None``. See:
            https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries
            Defaults to None.
        rescale (bool, optional): Evaluate the model at the ``'global_scale'`` variable
            (as set in the ``pose_config.yaml`` file for a particular project). I.e.
            every image will be resized according to that scale and prediction will be
            compared to the resized ground truth. The error will be reported in pixels at
            rescaled to the *original* size. I.e. For a [200,200] pixel image evaluated
            at ``global_scale=.5``, the predictions are calculated on [100,100] pixel
            images, compared to 1/2*ground truth and this error is then multiplied by
            2!. The evaluation images are also shown for the original size! Defaults to
            False.
        modelprefix (str, optional): Directory containing the deeplabcut models to use
            when evaluating the network. By default, the models are assumed to exist in
            the project folder. Defaults to "".
        per_keypoint_evaluation (bool, optional): Compute the train and test RMSE for
            each keypoint, and save the results to a {model_name}-keypoint-results.csv in
            the evaluation-results folder. Defaults to False.
        snapshots_to_evaluate (list[str], optional): List of snapshot names to evaluate
            (e.g. ["snapshot-50000", "snapshot-75000", ...]). Defaults to None.

    Returns:
        None

    Examples:
        If you do not want to plot and evaluate with shuffle set to 1:

            deeplabcut.evaluate_network(
                "/analysis/project/reaching-task/config.yaml",
                Shuffles=[1],
            )

        If you want to plot and evaluate with shuffle set to 0 and 1:

            deeplabcut.evaluate_network(
                "/analysis/project/reaching-task/config.yaml",
                Shuffles=[0, 1],
                plotting=True,
            )

        If you want to plot assemblies for a maDLC project:

            deeplabcut.evaluate_network(
                "/analysis/project/reaching-task/config.yaml",
                Shuffles=[1],
                plotting="individual",
            )

        Note: This defaults to standard plotting for single-animal projects.
    """
    if Shuffles is None:
        Shuffles = [1]
    if plotting not in (True, False, "bodypart", "individual"):
        raise ValueError(f"Unknown value for `plotting`={plotting}")

    start_path = Path.cwd()
    from deeplabcut.utils import auxiliaryfunctions

    cfg = auxiliaryfunctions.read_config(config)

    if cfg.get("multianimalproject", False):
        from .evaluate_multianimal import evaluate_multianimal_full

        # TODO: Make this code not so redundant!
        evaluate_multianimal_full(
            config=config,
            Shuffles=Shuffles,
            trainingsetindex=trainingsetindex,
            plotting=plotting,
            comparisonbodyparts=comparisonbodyparts,
            gputouse=gputouse,
            modelprefix=modelprefix,
            per_keypoint_evaluation=per_keypoint_evaluation,
            snapshots_to_evaluate=snapshots_to_evaluate,
        )
    else:
        import tensorflow as tf

        from deeplabcut.pose_estimation_tensorflow.config import load_config
        from deeplabcut.pose_estimation_tensorflow.core import predict
        from deeplabcut.pose_estimation_tensorflow.datasets.utils import data_to_input
        from deeplabcut.utils import auxiliaryfunctions, conversioncode
        from deeplabcut.utils.auxfun_videos import imread, imresize

        # If a string was passed in, auto-convert to True for backward compatibility
        plotting = bool(plotting)

        if "TF_CUDNN_USE_AUTOTUNE" in os.environ:
            del os.environ["TF_CUDNN_USE_AUTOTUNE"]  # was potentially set during training

        tf.compat.v1.reset_default_graph()
        os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"  #
        #    tf.logging.set_verbosity(tf.logging.WARN)

        start_path = Path.cwd()
        # Read file path for pose_config file. >> pass it on
        cfg = auxiliaryfunctions.read_config(config)
        if gputouse is not None:  # gpu selectinon
            os.environ["CUDA_VISIBLE_DEVICES"] = str(gputouse)

        if trainingsetindex == "all":
            TrainingFractions = cfg["TrainingFraction"]
        else:
            if 0 <= trainingsetindex < len(cfg["TrainingFraction"]):
                TrainingFractions = [cfg["TrainingFraction"][int(trainingsetindex)]]
            else:
                raise Exception(
                    "Please check the trainingsetindex! ",
                    trainingsetindex,
                    " should be an integer from 0 .. ",
                    int(len(cfg["TrainingFraction"]) - 1),
                )

        # Loading human annotatated data
        trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg)
        Data = pd.read_hdf(
            Path(cfg["project_path"]) / str(trainingsetfolder) / ("CollectedData_" + cfg["scorer"] + ".h5")
        )

        # Get list of body parts to evaluate network for
        comparisonbodyparts = auxiliaryfunctions.intersection_of_body_parts_and_ones_given_by_user(
            cfg, comparisonbodyparts
        )
        # Make folder for evaluation
        auxiliaryfunctions.attempt_to_make_folder(Path(cfg["project_path"]) / "evaluation-results")
        for shuffle in Shuffles:
            for trainFraction in TrainingFractions:
                ##################################################
                # Load and setup CNN part detector
                ##################################################

                modelfolder_rel_path = auxiliaryfunctions.get_model_folder(
                    trainFraction, shuffle, cfg, modelprefix=modelprefix
                )
                modelfolder = Path(cfg["project_path"]) / modelfolder_rel_path

                # TODO: Unlike using create_training_dataset()
                # If create_training_model_comparison() is used there won't
                #  necessarily be training fractions for every shuffle which will raise the FileNotFoundError..
                #  Not sure if this should throw an exception or just be a warning...
                if not modelfolder.exists():
                    raise FileNotFoundError(
                        f"Model with shuffle {shuffle} and trainFraction {trainFraction} does not exist."
                    )

                if trainingsetindex == "all":
                    train_frac_idx = cfg["TrainingFraction"].index(trainFraction)
                else:
                    train_frac_idx = trainingsetindex

                path_train_config, path_test_config, _ = return_train_network_path(
                    config=config,
                    shuffle=shuffle,
                    trainingsetindex=train_frac_idx,
                    modelprefix=modelprefix,
                )

                test_pose_cfg = load_config(str(path_test_config))
                train_pose_cfg = load_config(str(path_train_config))
                # Load meta data
                _, trainIndices, testIndices, _ = auxiliaryfunctions.load_metadata(
                    Path(cfg["project_path"], train_pose_cfg["metadataset"])
                )

                # change batch size, if it was edited during analysis!
                test_pose_cfg["batch_size"] = 1  # in case this was edited for analysis.

                # Create folder structure to store results.
                evaluationfolder = Path(cfg["project_path"]) / str(
                    auxiliaryfunctions.get_evaluation_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)
                )
                auxiliaryfunctions.attempt_to_make_folder(evaluationfolder, recursive=True)

                Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
                    train_folder=Path(modelfolder) / "train",
                )

                if snapshots_to_evaluate is not None:
                    snapshot_names = get_available_requested_snapshots(
                        requested_snapshots=snapshots_to_evaluate,
                        available_snapshots=Snapshots,
                    )
                else:
                    snapshot_names = get_snapshots_by_index(
                        idx=cfg["snapshotindex"],
                        available_snapshots=Snapshots,
                    )

                final_result = []

                ########################### RESCALING (to global scale)
                if rescale:
                    scale = test_pose_cfg["global_scale"]
                    Data = (
                        pd.read_hdf(
                            Path(cfg["project_path"])
                            / str(trainingsetfolder)
                            / ("CollectedData_" + cfg["scorer"] + ".h5")
                        )
                        * scale
                    )
                else:
                    scale = 1

                conversioncode.guarantee_multiindex_rows(Data)
                ##################################################
                # Compute predictions over images
                ##################################################
                for snapshot_name in snapshot_names:
                    test_pose_cfg["init_weights"] = str(
                        Path(modelfolder) / "train" / snapshot_name
                    )  # setting weights to corresponding snapshot.
                    training_iterations = int(snapshot_name.split("-")[-1])

                    # Name for deeplabcut net (based on its parameters)
                    DLCscorer, DLCscorerlegacy = auxiliaryfunctions.get_scorer_name(
                        cfg,
                        shuffle,
                        trainFraction,
                        trainingsiterations=training_iterations,
                        modelprefix=modelprefix,
                    )
                    print(
                        "Running ",
                        DLCscorer,
                        " with # of training iterations:",
                        training_iterations,
                    )
                    (
                        notanalyzed,
                        resultsfilename,
                        DLCscorer,
                    ) = auxiliaryfunctions.check_if_not_evaluated(
                        str(evaluationfolder),
                        DLCscorer,
                        DLCscorerlegacy,
                        snapshot_name,
                    )
                    if notanalyzed:
                        # Specifying state of model (snapshot / training state)
                        sess, inputs, outputs = predict.setup_pose_prediction(test_pose_cfg)
                        Numimages = len(Data.index)
                        PredicteData = np.zeros((Numimages, 3 * len(test_pose_cfg["all_joints_names"])))
                        print("Running evaluation ...")
                        for imageindex, imagename in tqdm(enumerate(Data.index)):
                            image = imread(
                                Path(cfg["project_path"]).joinpath(*imagename),
                                mode="skimage",
                            )
                            if scale != 1:
                                image = imresize(image, scale)

                            image_batch = data_to_input(image)
                            # Compute prediction with the CNN
                            outputs_np = sess.run(outputs, feed_dict={inputs: image_batch})
                            scmap, locref = predict.extract_cnn_output(outputs_np, test_pose_cfg)

                            # Extract maximum scoring location from the heatmap, assume 1 person
                            pose = predict.argmax_pose_predict(scmap, locref, test_pose_cfg["stride"])
                            PredicteData[imageindex, :] = (
                                pose.flatten()
                            )  # NOTE: thereby     cfg_test['all_joints_names'] should be same order as bodyparts!

                        sess.close()  # closes the current tf session

                        index = pd.MultiIndex.from_product(
                            [
                                [DLCscorer],
                                test_pose_cfg["all_joints_names"],
                                ["x", "y", "likelihood"],
                            ],
                            names=["scorer", "bodyparts", "coords"],
                        )

                        # Saving results
                        DataMachine = pd.DataFrame(PredicteData, columns=index, index=Data.index)
                        DataMachine.to_hdf(resultsfilename, key="df_with_missing")

                        print(
                            "Analysis is done and the results are stored (see evaluation-results) for snapshot: ",
                            snapshot_name,
                        )
                        DataCombined = pd.concat([Data.T, DataMachine.T], axis=0, sort=False).T

                        RMSE, RMSEpcutoff = pairwisedistances(
                            DataCombined,
                            cfg["scorer"],
                            DLCscorer,
                            cfg["pcutoff"],
                            comparisonbodyparts,
                        )
                        testerror = np.nanmean(RMSE.iloc[testIndices].values.flatten())
                        trainerror = np.nanmean(RMSE.iloc[trainIndices].values.flatten())
                        testerrorpcutoff = np.nanmean(RMSEpcutoff.iloc[testIndices].values.flatten())
                        trainerrorpcutoff = np.nanmean(RMSEpcutoff.iloc[trainIndices].values.flatten())
                        results = [
                            training_iterations,
                            int(100 * trainFraction),
                            shuffle,
                            np.round(trainerror, 2),
                            np.round(testerror, 2),
                            cfg["pcutoff"],
                            np.round(trainerrorpcutoff, 2),
                            np.round(testerrorpcutoff, 2),
                        ]
                        final_result.append(results)

                        if per_keypoint_evaluation:
                            df_keypoint_error = keypoint_error(RMSE, RMSEpcutoff, trainIndices, testIndices)
                            kpt_filename = DLCscorer + "-keypoint-results.csv"
                            df_keypoint_error.to_csv(Path(evaluationfolder) / kpt_filename)

                        if show_errors:
                            print(
                                "Results for",
                                training_iterations,
                                " training iterations:",
                                int(100 * trainFraction),
                                shuffle,
                                "train error:",
                                np.round(trainerror, 2),
                                "pixels. Test error:",
                                np.round(testerror, 2),
                                " pixels.",
                            )
                            print(
                                "With pcutoff of",
                                cfg["pcutoff"],
                                " train error:",
                                np.round(trainerrorpcutoff, 2),
                                "pixels. Test error:",
                                np.round(testerrorpcutoff, 2),
                                "pixels",
                            )
                            if scale != 1:
                                print(
                                    "The predictions have been calculated for"
                                    f" rescaled images (and rescaled ground truth). Scale: {scale}"
                                )
                            print(
                                "Thereby, the errors are given by the average distances "
                                "between the labels by DLC and the scorer."
                            )

                        if plotting:
                            print("Plotting...")
                            foldername = Path(evaluationfolder) / ("LabeledImages_" + DLCscorer + "_" + snapshot_name)
                            auxiliaryfunctions.attempt_to_make_folder(foldername)
                            Plotting(
                                cfg,
                                comparisonbodyparts,
                                DLCscorer,
                                trainIndices,
                                DataCombined * 1.0 / scale,
                                foldername,
                            )  # Rescaling coordinates to have figure in original size!

                        tf.compat.v1.reset_default_graph()
                        # print(final_result)
                    else:
                        DataMachine = pd.read_hdf(resultsfilename)
                        conversioncode.guarantee_multiindex_rows(DataMachine)
                        if plotting:
                            DataCombined = pd.concat([Data.T, DataMachine.T], axis=0, sort=False).T
                            foldername = Path(evaluationfolder) / ("LabeledImages_" + DLCscorer + "_" + snapshot_name)
                            if not Path(foldername).exists():
                                print(
                                    "Plotting..."
                                    "(warning, scale might be inconsistent in comparison "
                                    "to when data was analyzed; i.e. if you used rescale)"
                                )
                                auxiliaryfunctions.attempt_to_make_folder(foldername)
                                Plotting(
                                    cfg,
                                    comparisonbodyparts,
                                    DLCscorer,
                                    trainIndices,
                                    DataCombined * 1.0 / scale,
                                    foldername,
                                )
                            else:
                                print("Plots already exist for this snapshot... Skipping to the next one.")

                if len(final_result) > 0:  # Only append if results were calculated
                    make_results_file(final_result, evaluationfolder, DLCscorer)
                    print(
                        "The network is evaluated and the results are stored in the subdirectory 'evaluation_results'."
                    )
                    print(
                        "Please check the results, then choose the best model (snapshot) for prediction. "
                        "You can update the config.yaml file with the appropriate index for the 'snapshotindex'.\n"
                        "Use the function 'analyze_video' to make predictions on new videos."
                    )
                    print(
                        "Otherwise, consider adding more labeled-data and retraining the network "
                        "(see DeepLabCut workflow Fig 2, Nath 2019)"
                    )

    # returning to initial folder
    os.chdir(str(start_path))

extract_cnn_output

extract_cnn_output(outputs_np, cfg)

Extract locref + scmap from network.

Source code in deeplabcut/pose_estimation_tensorflow/core/predict.py

def extract_cnn_output(outputs_np, cfg):
    """Extract locref + scmap from network."""
    scmap = outputs_np[0]
    scmap = np.squeeze(scmap)
    locref = None
    if cfg["location_refinement"]:
        locref = np.squeeze(outputs_np[1])
        shape = locref.shape
        locref = np.reshape(locref, (shape[0], shape[1], -1, 2))
        locref *= cfg["locref_stdev"]
    if len(scmap.shape) == 2:  # for single body part!
        scmap = np.expand_dims(scmap, axis=2)
    return scmap, locref

extract_maps

extract_maps(config, shuffle=0, trainingsetindex=0, gputouse=None, rescale=False, Indices=None, modelprefix='')

Extracts the scoremap, locref, partaffinityfields (if available).

Parameters:

Name	Type	Description	Default
config	string	Full path of the config.yaml file as a string.	required
shuffle	integer	Integer specifying shuffle index of the training dataset. Defaults to 0.	0
trainingsetindex	int	Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). This variable can also be set to "all". Defaults to 0.	0
gputouse	int	GPU index (see nvidia-smi). Use None if no GPU. Defaults to None.	None
rescale	bool	Evaluate the model at the 'global_scale' variable (as set in the test/pose_config.yaml file for a particular project). I.e. every image will be resized according to that scale and prediction will be compared to the resized ground truth. The error will be reported in pixels at rescaled to the original size. I.e. For a [200,200] pixel image evaluated at global_scale=.5, the predictions are calculated on [100,100] pixel images, compared to 1/2*ground truth and this error is then multiplied by 2!. The evaluation images are also shown for the original size! Defaults to False.	False
Indices	list	Image indices for which to extract maps. Defaults to None.	None
modelprefix	str	Directory containing the deeplabcut models to use. Defaults to "".	''

Returns:

Name	Type	Description
dict		Dictionary indexed by trainingsetfraction, snapshotindex, and imageindex; each item contains (image, scmap, locref, paf, bpt names, partaffinity graph, imagename, True/False if this image was in trainingset).

Examples:

If you want to extract the data for image 0 and 103 (of the training set) for model trained with shuffle 0:

deeplabcut.extract_maps(configfile, 0, Indices=[0, 103])

Source code in deeplabcut/pose_estimation_tensorflow/visualizemaps.py

def extract_maps(
    config,
    shuffle=0,
    trainingsetindex=0,
    gputouse=None,
    rescale=False,
    Indices=None,
    modelprefix="",
):
    """Extracts the scoremap, locref, partaffinityfields (if available).

    Args:
        config (string): Full path of the config.yaml file as a string.
        shuffle (integer): Integer specifying shuffle index of the training dataset. Defaults to 0.
        trainingsetindex (int, optional): Integer specifying which TrainingsetFraction to use. By default the first
            (note that TrainingFraction is a list in config.yaml).
            This variable can also be set to "all". Defaults to 0.
        gputouse (int, optional): GPU index (see nvidia-smi). Use None if no GPU.
            Defaults to None.
        rescale (bool, optional): Evaluate the model at the 'global_scale' variable
            (as set in the test/pose_config.yaml file for a particular project).
            I.e. every image will be resized according to that scale
            and prediction will be compared to the resized ground truth.
            The error will be reported in pixels at rescaled to the *original* size.
            I.e. For a [200,200] pixel image evaluated at global_scale=.5, the predictions are calculated
            on [100,100] pixel images, compared to 1/2*ground truth and this error is then multiplied by 2!.
            The evaluation images are also shown for the original size! Defaults to False.
        Indices (list, optional): Image indices for which to extract maps. Defaults to None.
        modelprefix (str, optional): Directory containing the deeplabcut models to use.
            Defaults to "".

    Returns:
        dict: Dictionary indexed by trainingsetfraction, snapshotindex, and imageindex;
            each item contains (image, scmap, locref, paf, bpt names, partaffinity graph,
            imagename, True/False if this image was in trainingset).

    Examples:
        If you want to extract the data for image 0 and 103 (of the training set) for model trained with shuffle 0:

            deeplabcut.extract_maps(configfile, 0, Indices=[0, 103])
    """

    import numpy as np
    import pandas as pd
    import tensorflow as tf
    from tqdm import tqdm

    from deeplabcut.pose_estimation_tensorflow.config import load_config
    from deeplabcut.pose_estimation_tensorflow.core import (
        predict,
    )
    from deeplabcut.pose_estimation_tensorflow.core import (
        predict_multianimal as predictma,
    )
    from deeplabcut.pose_estimation_tensorflow.datasets.utils import data_to_input
    from deeplabcut.utils import auxiliaryfunctions
    from deeplabcut.utils.auxfun_videos import imread, imresize

    tf.compat.v1.reset_default_graph()
    os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"  #
    #    tf.logging.set_verbosity(tf.logging.WARN)

    start_path = Path.cwd()
    # Read file path for pose_config file. >> pass it on
    cfg = auxiliaryfunctions.read_config(config)

    if gputouse is not None:  # gpu selectinon
        os.environ["CUDA_VISIBLE_DEVICES"] = str(gputouse)

    if trainingsetindex == "all":
        TrainingFractions = cfg["TrainingFraction"]
    else:
        if trainingsetindex < len(cfg["TrainingFraction"]) and trainingsetindex >= 0:
            TrainingFractions = [cfg["TrainingFraction"][int(trainingsetindex)]]
        else:
            raise Exception(
                "Please check the trainingsetindex! ",
                trainingsetindex,
                " should be an integer from 0 .. ",
                int(len(cfg["TrainingFraction"]) - 1),
            )

    # Loading human annotatated data
    trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg)
    Data = pd.read_hdf(Path(cfg["project_path"]) / str(trainingsetfolder) / ("CollectedData_" + cfg["scorer"] + ".h5"))

    # Make folder for evaluation
    auxiliaryfunctions.attempt_to_make_folder(Path(cfg["project_path"]) / "evaluation-results")

    Maps = {}
    for trainFraction in TrainingFractions:
        Maps[trainFraction] = {}
        ##################################################
        # Load and setup CNN part detector
        ##################################################
        datafn, metadatafn = auxiliaryfunctions.get_data_and_metadata_filenames(
            trainingsetfolder, trainFraction, shuffle, cfg
        )

        modelfolder = Path(cfg["project_path"]) / str(
            auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)
        )
        path_test_config = Path(modelfolder) / "test" / "pose_cfg.yaml"
        # Load meta data
        (
            data,
            trainIndices,
            testIndices,
            trainFraction,
        ) = auxiliaryfunctions.load_metadata(Path(cfg["project_path"]) / metadatafn)
        try:
            dlc_cfg = load_config(str(path_test_config))
        except FileNotFoundError as e:
            raise FileNotFoundError(
                f"It seems the model for shuffle {shuffle} and trainFraction {trainFraction} does not exist."
            ) from e

        # change batch size, if it was edited during analysis!
        dlc_cfg["batch_size"] = 1  # in case this was edited for analysis.

        # Create folder structure to store results.
        evaluationfolder = Path(cfg["project_path"]) / str(
            auxiliaryfunctions.get_evaluation_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)
        )
        auxiliaryfunctions.attempt_to_make_folder(evaluationfolder, recursive=True)

        Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
            train_folder=Path(modelfolder) / "train",
        )

        if cfg["snapshotindex"] == -1:
            snapindices = [-1]
        elif cfg["snapshotindex"] == "all":
            snapindices = range(len(Snapshots))
        elif cfg["snapshotindex"] < len(Snapshots):
            snapindices = [cfg["snapshotindex"]]
        else:
            print("Invalid choice, only -1 (last), any integer up to last, or all (as string)!")

        ########################### RESCALING (to global scale)
        scale = dlc_cfg["global_scale"] if rescale else 1
        Data *= scale

        bptnames = [dlc_cfg["all_joints_names"][i] for i in range(len(dlc_cfg["all_joints"]))]

        for snapindex in snapindices:
            dlc_cfg["init_weights"] = str(
                Path(modelfolder) / "train" / Snapshots[snapindex]
            )  # setting weights to corresponding snapshot.
            Path(dlc_cfg["init_weights"]).name.split("-")[-1]  # read how many training siterations that corresponds to.

            # Name for deeplabcut net (based on its parameters)
            # DLCscorer,DLCscorerlegacy =
            # auxiliaryfunctions.GetScorerName(cfg,shuffle,trainFraction,trainingsiterations)
            # notanalyzed, resultsfilename,
            # DLCscorer=auxiliaryfunctions.CheckifNotEvaluated(str(evaluationfolder),
            # DLCscorer,DLCscorerlegacy,Snapshots[snapindex])
            # print("Extracting maps for ", DLCscorer, " with # of trainingiterations:", trainingsiterations)
            # if notanalyzed: #this only applies to ask if h5 exists...

            # Specifying state of model (snapshot / training state)
            sess, inputs, outputs = predict.setup_pose_prediction(dlc_cfg)
            Numimages = len(Data.index)
            np.zeros((Numimages, 3 * len(dlc_cfg["all_joints_names"])))
            print("Analyzing data...")
            if Indices is None:
                Indices = enumerate(Data.index)
            else:
                Ind = [Data.index[j] for j in Indices]
                Indices = enumerate(Ind)

            DATA = {}
            for imageindex, imagename in tqdm(Indices):
                image = imread(Path(cfg["project_path"]).joinpath(*imagename), mode="skimage")

                if scale != 1:
                    image = imresize(image, scale)

                image_batch = data_to_input(image)

                # Compute prediction with the CNN
                outputs_np = sess.run(outputs, feed_dict={inputs: image_batch})

                if cfg.get("multianimalproject", False):
                    scmap, locref, paf = predictma.extract_cnn_output(outputs_np, dlc_cfg)
                    pagraph = dlc_cfg["partaffinityfield_graph"]
                else:
                    scmap, locref = predict.extract_cnn_output(outputs_np, dlc_cfg)
                    paf = None
                    pagraph = []
                peaks = outputs_np[-1]

                if imageindex in testIndices:
                    trainingfram = False
                else:
                    trainingfram = True

                DATA[imageindex] = [
                    image,
                    scmap,
                    locref,
                    paf,
                    peaks,
                    bptnames,
                    pagraph,
                    imagename,
                    trainingfram,
                ]
            Maps[trainFraction][Snapshots[snapindex]] = DATA
    os.chdir(str(start_path))
    return Maps

extract_save_all_maps

extract_save_all_maps(
    config,
    shuffle=1,
    trainingsetindex=0,
    comparisonbodyparts="all",
    extract_paf=True,
    all_paf_in_one=True,
    gputouse=None,
    rescale=False,
    Indices=None,
    modelprefix="",
    dest_folder=None,
)

Extract scoremap, location refinement field and part affinity field predictions.

Maps are rescaled to the size of the input image and stored in the corresponding model folder in /evaluation-results.

Parameters:

Name	Type	Description	Default
config	string	Full path of the config.yaml file as a string.	required
shuffle	integer	Integer specifying shuffle index of the training dataset. Defaults to 1.	1
trainingsetindex	int	Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). This variable can also be set to "all". Defaults to 0.	0
comparisonbodyparts	list of bodyparts	Average error for those body parts only (subset of all body parts). Defaults to "all".	'all'
extract_paf	bool	Extract part affinity fields by default. Note that turning it off will make the function much faster. Defaults to True.	True
all_paf_in_one	bool	By default, all part affinity fields are displayed on a single frame. If false, individual fields are shown on separate frames. Defaults to True.	True
gputouse	int	GPU index (see nvidia-smi). Use None if no GPU. Defaults to None.	None
rescale	bool	Evaluate at global_scale from pose_config.yaml. Defaults to False.	False
Indices	list	Image indices for which to compute scmap/locref/paf. Defaults to None.	None
modelprefix	str	Directory containing the deeplabcut models to use. Defaults to "".	''
dest_folder	string	Destination folder for saved maps. Defaults to None.	None

Examples:

Calculated maps for images 0, 1 and 33.

deeplabcut.extract_save_all_maps(
    "/analysis/project/reaching-task/config.yaml", shuffle=1, Indices=[0, 1, 33]
)

Source code in deeplabcut/pose_estimation_tensorflow/visualizemaps.py

def extract_save_all_maps(
    config,
    shuffle=1,
    trainingsetindex=0,
    comparisonbodyparts="all",
    extract_paf=True,
    all_paf_in_one=True,
    gputouse=None,
    rescale=False,
    Indices=None,
    modelprefix="",
    dest_folder=None,
):
    """Extract scoremap, location refinement field and part affinity field predictions.

    Maps are rescaled to the size of the input image and stored in the corresponding
    model folder in /evaluation-results.

    Args:
        config (string): Full path of the config.yaml file as a string.
        shuffle (integer): Integer specifying shuffle index of the training dataset. Defaults to 1.
        trainingsetindex (int, optional): Integer specifying which TrainingsetFraction to use.
            By default the first (note that TrainingFraction is a list in config.yaml).
            This variable can also be set to "all". Defaults to 0.
        comparisonbodyparts (list of bodyparts): Average error for those body parts only
            (subset of all body parts). Defaults to "all".
        extract_paf (bool, optional): Extract part affinity fields by default.
            Note that turning it off will make the function much faster. Defaults to True.
        all_paf_in_one (bool, optional): By default, all part affinity fields are displayed on a single frame.
            If false, individual fields are shown on separate frames. Defaults to True.
        gputouse (int, optional): GPU index (see nvidia-smi). Use None if no GPU.
            Defaults to None.
        rescale (bool, optional): Evaluate at global_scale from pose_config.yaml.
            Defaults to False.
        Indices (list, optional): Image indices for which to compute scmap/locref/paf.
            Defaults to None.
        modelprefix (str, optional): Directory containing the deeplabcut models to use.
            Defaults to "".
        dest_folder (string, optional): Destination folder for saved maps. Defaults to None.

    Examples:
        Calculated maps for images 0, 1 and 33.

            deeplabcut.extract_save_all_maps(
                "/analysis/project/reaching-task/config.yaml", shuffle=1, Indices=[0, 1, 33]
            )

    """
    from tqdm import tqdm

    from deeplabcut.utils.auxiliaryfunctions import (
        attempt_to_make_folder,
        get_evaluation_folder,
        intersection_of_body_parts_and_ones_given_by_user,
        read_config,
    )

    cfg = read_config(config)
    data = extract_maps(config, shuffle, trainingsetindex, gputouse, rescale, Indices, modelprefix)

    comparisonbodyparts = intersection_of_body_parts_and_ones_given_by_user(cfg, comparisonbodyparts)

    print("Saving plots...")
    for frac, values in data.items():
        if not dest_folder:
            dest_folder = (
                Path(cfg["project_path"])
                / str(get_evaluation_folder(frac, shuffle, cfg, modelprefix=modelprefix))
                / "maps"
            )
        attempt_to_make_folder(dest_folder)
        filepath = "{imname}_{map}_{label}_{shuffle}_{frac}_{snap}.png"
        dest_path = str(Path(dest_folder) / filepath)
        for snap, maps in values.items():
            for imagenr in tqdm(maps):
                (
                    image,
                    scmap,
                    locref,
                    paf,
                    peaks,
                    bptnames,
                    pafgraph,
                    impath,
                    trainingframe,
                ) = maps[imagenr]
                if not extract_paf:
                    paf = None
                label = "train" if trainingframe else "test"
                imname = impath[-1]
                scmap, (locref_x, locref_y), paf = resize_all_maps(image, scmap, locref, paf)
                to_plot = [i for i, bpt in enumerate(bptnames) if bpt in comparisonbodyparts]
                list_of_inds = []
                for n, edge in enumerate(pafgraph):
                    if any(ind in to_plot for ind in edge):
                        list_of_inds.append([(2 * n, 2 * n + 1), (bptnames[edge[0]], bptnames[edge[1]])])
                if len(to_plot) > 1:
                    map_ = scmap[:, :, to_plot].sum(axis=2)
                    locref_x_ = locref_x[:, :, to_plot].sum(axis=2)
                    locref_y_ = locref_y[:, :, to_plot].sum(axis=2)
                elif len(to_plot) == 1 and len(bptnames) > 1:
                    map_ = scmap[:, :, to_plot]
                    locref_x_ = locref_x[:, :, to_plot]
                    locref_y_ = locref_y[:, :, to_plot]
                else:
                    map_ = scmap[..., 0]
                    locref_x_ = locref_x[..., 0]
                    locref_y_ = locref_y[..., 0]
                fig1, _ = visualize_scoremaps(image, map_)
                temp = dest_path.format(
                    imname=imname,
                    map="scmap",
                    label=label,
                    shuffle=shuffle,
                    frac=frac,
                    snap=snap,
                )
                fig1.savefig(temp)

                fig2, _ = visualize_locrefs(image, map_, locref_x_, locref_y_)
                temp = dest_path.format(
                    imname=imname,
                    map="locref",
                    label=label,
                    shuffle=shuffle,
                    frac=frac,
                    snap=snap,
                )
                fig2.savefig(temp)

                if paf is not None:
                    if not all_paf_in_one:
                        for inds, names in list_of_inds:
                            fig3, _ = visualize_paf(image, paf[:, :, [inds]])
                            temp = dest_path.format(
                                imname=imname,
                                map=f"paf_{'_'.join(names)}",
                                label=label,
                                shuffle=shuffle,
                                frac=frac,
                                snap=snap,
                            )
                            fig3.savefig(temp)
                    else:
                        inds = [elem[0] for elem in list_of_inds]
                        n_inds = len(inds)
                        cmap = plt.cm.get_cmap(cfg["colormap"], n_inds)
                        colors = cmap(range(n_inds))
                        fig3, _ = visualize_paf(image, paf[:, :, inds], colors=colors)
                        temp = dest_path.format(
                            imname=imname,
                            map="paf",
                            label=label,
                            shuffle=shuffle,
                            frac=frac,
                            snap=snap,
                        )
                        fig3.savefig(temp)
                plt.close("all")

get_available_requested_snapshots

get_available_requested_snapshots(requested_snapshots: list[str], available_snapshots: list[str]) -> list[str]

Intersects the requested snapshot names with the available snapshots.

Returns:

Type	Description
list[str]	list[str]: Snapshot names.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def get_available_requested_snapshots(
    requested_snapshots: list[str],
    available_snapshots: list[str],
) -> list[str]:
    """Intersects the requested snapshot names with the available snapshots.

    Returns:
        list[str]: Snapshot names.
    """
    snapshot_names = []
    missing_snapshots = []
    for snap in requested_snapshots:
        if snap in available_snapshots:
            snapshot_names.append(snap)
        else:
            missing_snapshots.append(snap)

    if len(snapshot_names) == 0:
        raise ValueError(f"None of the requested snapshots were found: \n{missing_snapshots}")
    elif len(missing_snapshots) > 0:
        print(f"The following requested snapshots were not found and will be skipped:\n{missing_snapshots}")

    return snapshot_names

get_snapshots_by_index

get_snapshots_by_index(idx: int | str, available_snapshots: list[str]) -> list[str]

Assume available_snapshots is ordered in ascending order.

Returns snapshot names.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def get_snapshots_by_index(
    idx: int | str,
    available_snapshots: list[str],
) -> list[str]:
    """Assume available_snapshots is ordered in ascending order.

    Returns snapshot names.
    """
    if isinstance(idx, int) and -len(available_snapshots) <= idx < len(available_snapshots):
        return [available_snapshots[idx]]
    elif idx == "all":
        return available_snapshots

    raise IndexError(
        f"Invalid index: {idx}. The index should be an int less than the number of "
        f"available snapshots, negative indexing is supported. The keyword 'all' "
        f"is also a valid option."
    )

keypoint_error

keypoint_error(
    df_error: DataFrame, df_error_p_cutoff: DataFrame, train_indices: list[int], test_indices: list[int]
) -> pd.DataFrame

Computes the RMSE error for each bodypart.

The error dataframes can be in single animal format (non-hierarchical columns, one column for each bodypart) or multi-animal format (hierarchical columns with 3 levels: "scorer", "individuals", "bodyparts").

Parameters:

Name	Type	Description	Default
df_error	DataFrame	dataframe containing the RMSE error for each image, individual and bodypart	required
df_error_p_cutoff	DataFrame	dataframe containing the RMSE error with p-cutoff for each image, individual and bodypart	required
train_indices	list[int]	the indices of rows in the dataframe that are in the train set	required
test_indices	list[int]	the indices of rows in the dataframe that are in the test set	required

Returns:

Type	Description
DataFrame	A dataframe containing 4 rows (train and test error, with and without p-cutoff) and one column for each bodypart.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def keypoint_error(
    df_error: pd.DataFrame,
    df_error_p_cutoff: pd.DataFrame,
    train_indices: list[int],
    test_indices: list[int],
) -> pd.DataFrame:
    """Computes the RMSE error for each bodypart.

    The error dataframes can be in single animal format (non-hierarchical columns, one
    column for each bodypart) or multi-animal format (hierarchical columns with 3
    levels: "scorer", "individuals", "bodyparts").

    Args:
        df_error: dataframe containing the RMSE error for each image, individual and
            bodypart
        df_error_p_cutoff: dataframe containing the RMSE error with p-cutoff for each
            image, individual and bodypart
        train_indices: the indices of rows in the dataframe that are in the train set
        test_indices: the indices of rows in the dataframe that are in the test set

    Returns:
        A dataframe containing 4 rows (train and test error, with and without p-cutoff)
        and one column for each bodypart.
    """
    df_error = df_error.copy()
    df_error_p_cutoff = df_error_p_cutoff.copy()

    error_rows = []
    for row_name, df in [
        ("Train error (px)", df_error.iloc[train_indices, :]),
        ("Test error (px)", df_error.iloc[test_indices, :]),
        ("Train error (px) with p-cutoff", df_error_p_cutoff.iloc[train_indices, :]),
        ("Test error (px) with p-cutoff", df_error_p_cutoff.iloc[test_indices, :]),
    ]:
        df_flat = df.copy()
        if isinstance(df.columns, pd.MultiIndex):
            # MA projects have column indices "scorer", "individuals" and "bodyparts"
            # Drop the scorer level, and put individuals in rows
            df_flat = df.droplevel("scorer", axis=1).stack(level="individuals").copy()

        bodypart_error = df_flat.mean()
        bodypart_error["Error Type"] = row_name
        error_rows.append(bodypart_error)

    # The error rows are series; stack in axis 1 and pivot to get DF
    keypoint_error_df = pd.concat(error_rows, axis=1)
    return keypoint_error_df.T.set_index("Error Type")

make_results_file

make_results_file(final_result, evaluationfolder, DLCscorer)

Makes result file in csv format and saves under evaluation_results directory.

If the file exists (typically, when the network has already been evaluated), newer results are appended to it.

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def make_results_file(final_result, evaluationfolder, DLCscorer):
    """Makes result file in csv format and saves under evaluation_results directory.

    If the file exists (typically, when the network has already been evaluated), newer
    results are appended to it.
    """
    col_names = [
        "Training iterations:",
        "%Training dataset",
        "Shuffle number",
        " Train error(px)",
        " Test error(px)",
        "p-cutoff used",
        "Train error with p-cutoff",
        "Test error with p-cutoff",
    ]
    df = pd.DataFrame(final_result, columns=col_names)
    output_path = Path(evaluationfolder) / (DLCscorer + "-results.csv")
    if Path(output_path).exists():
        temp = pd.read_csv(output_path, index_col=0)
        df = pd.concat((temp, df)).reset_index(drop=True)

    df.to_csv(output_path)

    ## Also storing one "large" table with results:
    # note: evaluationfolder.parents[0] to get common folder above all shuffle evaluations.
    df = pd.DataFrame(final_result, columns=col_names)
    output_path = Path(evaluationfolder).parents[0] / "CombinedEvaluation-results.csv"
    if Path(output_path).exists():
        temp = pd.read_csv(output_path, index_col=0)
        df = pd.concat((temp, df)).reset_index(drop=True)

    df.to_csv(output_path)

pairwisedistances

pairwisedistances(DataCombined, scorer1, scorer2, pcutoff=-1, bodyparts=None)

Calculates the pairwise Euclidean distance metric over body parts vs.

images

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def pairwisedistances(DataCombined, scorer1, scorer2, pcutoff=-1, bodyparts=None):
    """Calculates the pairwise Euclidean distance metric over body parts vs.

    images
    """
    mask = DataCombined[scorer2].xs("likelihood", level=1, axis=1) >= pcutoff
    if bodyparts is None:
        Pointwisesquareddistance = (DataCombined[scorer1] - DataCombined[scorer2]) ** 2
        RMSE = np.sqrt(
            Pointwisesquareddistance.xs("x", level=1, axis=1) + Pointwisesquareddistance.xs("y", level=1, axis=1)
        )  # Euclidean distance (proportional to RMSE)
        return RMSE, RMSE[mask]
    else:
        Pointwisesquareddistance = (DataCombined[scorer1][bodyparts] - DataCombined[scorer2][bodyparts]) ** 2
        RMSE = np.sqrt(
            Pointwisesquareddistance.xs("x", level=1, axis=1) + Pointwisesquareddistance.xs("y", level=1, axis=1)
        )  # Euclidean distance (proportional to RMSE)
        return RMSE, RMSE[mask]

renamed_parameter

renamed_parameter(*, old: str, new: str, since: str | None = None) -> Callable[[Callable[P, R]], Callable[P, R]]

Support a renamed keyword argument while warning callers to update.

Parameters:

Name	Type	Description	Default
old	str	The old parameter name that callers may still pass.	required
new	str	The current parameter name the function actually accepts.	required
since	str | None	Version when the rename happened.	None

Rules

• new must be the name used in the function signature and all internal call-sites. old must not appear in the signature.
• Do not chain renames. If A was renamed to B and B is later renamed to C, replace the A→B decorator with A→C directly rather than stacking a second decorator. Example: @renamed_parameter(old="A", new="C", since="12.4.0") @renamed_parameter(old="B", new="C", since="13.0.0") def func(*, C: int): print(f"C={C}")
• Multiple independent renames on the same function (e.g. batchsize→batch_size and videotype→video_extensions) are fine as long as they do not form a chain.
• This decorator only intercepts keyword arguments. Positional arguments are passed through unchanged; renaming a parameter that callers commonly pass positionally will not be caught.

Source code in deeplabcut/core/deprecation.py

def renamed_parameter(
    *,
    old: str,
    new: str,
    since: str | None = None,
) -> Callable[[Callable[P, R]], Callable[P, R]]:
    """Support a renamed keyword argument while warning callers to update.

    Args:
        old: The old parameter name that callers may still pass.
        new: The current parameter name the function actually accepts.
        since: Version when the rename happened.

    Rules:
        - ``new`` must be the name used in the function signature and all
          internal call-sites.  ``old`` must **not** appear in the signature.
        - Do **not** chain renames.  If ``A`` was renamed to ``B`` and ``B``
          is later renamed to ``C``, replace the ``A→B`` decorator with
          ``A→C`` directly rather than stacking a second decorator.
            Example:
                @renamed_parameter(old="A", new="C", since="12.4.0")
                @renamed_parameter(old="B", new="C", since="13.0.0")
                def func(*, C: int):
                    print(f"C={C}")
        - Multiple independent renames on the same function (e.g.
          ``batchsize→batch_size`` *and* ``videotype→video_extensions``) are fine
          as long as they do not form a chain.
        - This decorator only intercepts **keyword** arguments.  Positional
          arguments are passed through unchanged; renaming a parameter that
          callers commonly pass positionally will not be caught.
    """

    def decorator(fn: Callable[P, R]) -> Callable[P, R]:
        sig = inspect.signature(fn)

        # Guard: disallow chaining renames (A→B stacked on top of B→C).
        existing = getattr(fn, "__deprecated_params__", ())
        for prev in existing:
            if prev.old_parameter == new:
                raise ValueError(
                    f"@renamed_parameter: chaining renames is not allowed. "
                    f"'{old}' → '{new}' would chain with the existing "
                    f"'{prev.old_parameter}' → '{prev.new_parameter}' rename "
                    f"on {fn.__qualname__}. "
                    f"Use '{old}' → '{prev.new_parameter}' directly instead."
                )

        # Guard: 'new' must actually exist in the function's signature.
        if new not in sig.parameters:
            raise ValueError(
                f"@renamed_parameter: '{new}' is not a parameter of "
                f"{fn.__qualname__}. "
                f"Available parameters: {list(sig.parameters)}"
            )

        # Guard: 'old' must NOT exist in the signature.
        if old in sig.parameters:
            raise ValueError(
                f"@renamed_parameter: '{old}' is still a parameter of "
                f"{fn.__qualname__}. Use either old name or new name: '{new}'."
            )

        info = DeprecationInfo(
            kind="parameter",
            target=fn.__qualname__,
            since=since,
            old_parameter=old,
            new_parameter=new,
        )
        message = info.format_message()

        @functools.wraps(fn)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
            if old in kwargs:
                if new in kwargs:
                    raise TypeError(f"{fn.__qualname__} received both '{old}' and '{new}'. Use only '{new}'.")
                warnings.warn(message, DLCDeprecationWarning, stacklevel=2)
                kwargs[new] = kwargs.pop(old)
            return fn(*args, **kwargs)

        wrapper.__deprecated_params__ = (*existing, info)
        return wrapper

    return decorator

return_evaluate_network_data

return_evaluate_network_data(
    config,
    shuffle=0,
    trainingsetindex=0,
    comparisonbodyparts="all",
    Snapindex=None,
    rescale=False,
    fulldata=False,
    show_errors=True,
    modelprefix="",
    returnjustfns=True,
)

Returns the results for (previously evaluated) network.

deeplabcut.evaluate_network(..) Returns list of (per model): [trainingsiterations,tr ainfraction,shuffle,trainerror,testerror,pcutoff,trainerrorpcutoff,testerrorpcutoff, Snapshots[snapindex],scale,net_type]

If fulldata=True, also returns (the complete annotation and prediction array) Returns list of: (DataMachine, Data, data, trainIndices, testIndices, trainFraction, DLCscorer,comparisonbodyparts, cfg, Snapshots[snapindex])

Parameters:

Name	Type	Description	Default
config	string	Full path of the config.yaml file as a string.	required
shuffle	int	Integer specifying shuffle index of the training dataset. Defaults to 0.	0
trainingsetindex	int	Integer specifying which TrainingsetFraction to use. By default the first (note that TrainingFraction is a list in config.yaml). This variable can also be set to "all".	0
comparisonbodyparts	list of bodyparts	The average error will be computed for those body parts only (Has to be a subset of the body parts). Defaults to "all".	'all'
rescale	bool	Evaluate the model at the 'global_scale' variable (as set in the test/pose_config.yaml file for a particular project). I.e. every image will be resized according to that scale and prediction will be compared to the resized ground truth. The error will be reported in pixels at rescaled to the original size. I.e. For a [200,200] pixel image evaluated at global_scale=.5, the predictions are calculated on [100,100] pixel images, compared to 1/2*ground truth and this error is then multiplied by 2!. The evaluation images are also shown for the original size! Defaults to False.	False

Examples:

If you do not want to plot:

deeplabcut._evaluate_network_data(
    "/analysis/project/reaching-task/config.yaml",
    shuffle=[1],
)

If you want to plot:

deeplabcut.evaluate_network(
    "/analysis/project/reaching-task/config.yaml",
    shuffle=[1],
    plotting=True,
)

Source code in deeplabcut/pose_estimation_tensorflow/core/evaluate.py

def return_evaluate_network_data(
    config,
    shuffle=0,
    trainingsetindex=0,
    comparisonbodyparts="all",
    Snapindex=None,
    rescale=False,
    fulldata=False,
    show_errors=True,
    modelprefix="",
    returnjustfns=True,
):
    """Returns the results for (previously evaluated) network.

    deeplabcut.evaluate_network(..) Returns list of (per model): [trainingsiterations,tr
    ainfraction,shuffle,trainerror,testerror,pcutoff,trainerrorpcutoff,testerrorpcutoff,
    Snapshots[snapindex],scale,net_type]

    If fulldata=True, also returns (the complete annotation and prediction array)
    Returns list of:
    (DataMachine, Data, data, trainIndices, testIndices, trainFraction,
    DLCscorer,comparisonbodyparts, cfg, Snapshots[snapindex])

    Args:
        config (string): Full path of the config.yaml file as a string.
        shuffle (int): Integer specifying shuffle index of the training dataset.
            Defaults to 0.
        trainingsetindex (int, optional): Integer specifying which TrainingsetFraction
            to use. By default the first (note that TrainingFraction is a list in
            config.yaml). This variable can also be set to "all".
        comparisonbodyparts (list of bodyparts): The average error will be computed for
            those body parts only (Has to be a subset of the body parts). Defaults to
            "all".
        rescale (bool): Evaluate the model at the 'global_scale' variable (as set in the
            test/pose_config.yaml file for a particular project). I.e. every image will
            be resized according to that scale and prediction will be compared to the
            resized ground truth. The error will be reported in pixels at rescaled to
            the *original* size. I.e. For a [200,200] pixel image evaluated at
            global_scale=.5, the predictions are calculated on [100,100] pixel images,
            compared to 1/2*ground truth and this error is then multiplied by 2!. The
            evaluation images are also shown for the original size! Defaults to False.

    Examples:
        If you do not want to plot:

            deeplabcut._evaluate_network_data(
                "/analysis/project/reaching-task/config.yaml",
                shuffle=[1],
            )

        If you want to plot:

            deeplabcut.evaluate_network(
                "/analysis/project/reaching-task/config.yaml",
                shuffle=[1],
                plotting=True,
            )
    """
    import os

    from deeplabcut.pose_estimation_tensorflow.config import load_config
    from deeplabcut.utils import auxiliaryfunctions

    start_path = Path.cwd()
    # Read file path for pose_config file. >> pass it on
    cfg = auxiliaryfunctions.read_config(config)

    # Loading human annotatated data
    trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg)
    # Data=pd.read_hdf(
    # Path(cfg["project_path"]) / str(trainingsetfolder) / ('CollectedData_' + cfg["scorer"] + '.h5'),
    # 'df_with_missing'
    # )

    # Get list of body parts to evaluate network for
    comparisonbodyparts = auxiliaryfunctions.intersection_of_body_parts_and_ones_given_by_user(cfg, comparisonbodyparts)
    ##################################################
    # Load data...
    ##################################################
    trainFraction = cfg["TrainingFraction"][trainingsetindex]
    modelfolder = Path(cfg["project_path"]) / str(
        auxiliaryfunctions.get_model_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)
    )
    path_train_config, path_test_config, _ = return_train_network_path(
        config=config,
        shuffle=shuffle,
        trainingsetindex=trainingsetindex,
        modelprefix=modelprefix,
    )

    try:
        test_pose_cfg = load_config(str(path_test_config))
    except FileNotFoundError as e:
        raise FileNotFoundError(
            f"It seems the model for shuffle {shuffle} and trainFraction {trainFraction} does not exist."
        ) from e

    train_pose_cfg = load_config(str(path_train_config))
    # Load meta data
    data, trainIndices, testIndices, _ = auxiliaryfunctions.load_metadata(
        Path(cfg["project_path"]) / train_pose_cfg["metadataset"],
    )

    ########################### RESCALING (to global scale)
    if rescale:
        scale = test_pose_cfg["global_scale"]
        print("Rescaling Data to ", scale)
        Data = (
            pd.read_hdf(Path(cfg["project_path"]) / str(trainingsetfolder) / ("CollectedData_" + cfg["scorer"] + ".h5"))
            * scale
        )
    else:
        scale = 1
        Data = pd.read_hdf(
            Path(cfg["project_path"]) / str(trainingsetfolder) / ("CollectedData_" + cfg["scorer"] + ".h5")
        )

    evaluationfolder = Path(cfg["project_path"]) / str(
        auxiliaryfunctions.get_evaluation_folder(trainFraction, shuffle, cfg, modelprefix=modelprefix)
    )

    Snapshots = auxiliaryfunctions.get_snapshots_from_folder(
        train_folder=Path(modelfolder) / "train",
    )

    if Snapindex is None:
        Snapindex = cfg["snapshotindex"]

    snapshot_names = get_snapshots_by_index(
        idx=Snapindex,
        available_snapshots=Snapshots,
    )

    DATA = []
    results = []
    resultsfns = []
    for snapshot_name in snapshot_names:
        test_pose_cfg["init_weights"] = str(
            Path(modelfolder) / "train" / snapshot_name
        )  # setting weights to corresponding snapshot.
        trainingsiterations = Path(test_pose_cfg["init_weights"]).name.split("-")[
            -1
        ]  # read how many training siterations that corresponds to.

        # name for deeplabcut net (based on its parameters)
        DLCscorer, DLCscorerlegacy = auxiliaryfunctions.get_scorer_name(
            cfg, shuffle, trainFraction, trainingsiterations, modelprefix=modelprefix
        )
        if not returnjustfns:
            print(
                "Retrieving ",
                DLCscorer,
                " with # of trainingiterations:",
                trainingsiterations,
            )

        (
            notanalyzed,
            resultsfilename,
            DLCscorer,
        ) = auxiliaryfunctions.check_if_not_evaluated(str(evaluationfolder), DLCscorer, DLCscorerlegacy, snapshot_name)
        # resultsfilename=os.path.join(str(evaluationfolder),DLCscorer + '-' +
        # str(Snapshots[snapindex])+  '.h5') # + '-' + str(snapshot)+  ' #'-' +
        # Snapshots[snapindex]+  '.h5')
        print(resultsfilename)
        resultsfns.append(resultsfilename)
        if not returnjustfns:
            if not notanalyzed and Path(resultsfilename).is_file():  # data exists..
                DataMachine = pd.read_hdf(resultsfilename)
                DataCombined = pd.concat([Data.T, DataMachine.T], axis=0).T
                RMSE, RMSEpcutoff = pairwisedistances(
                    DataCombined,
                    cfg["scorer"],
                    DLCscorer,
                    cfg["pcutoff"],
                    comparisonbodyparts,
                )

                testerror = np.nanmean(RMSE.iloc[testIndices].values.flatten())
                trainerror = np.nanmean(RMSE.iloc[trainIndices].values.flatten())
                testerrorpcutoff = np.nanmean(RMSEpcutoff.iloc[testIndices].values.flatten())
                trainerrorpcutoff = np.nanmean(RMSEpcutoff.iloc[trainIndices].values.flatten())
                if show_errors:
                    print(
                        "Results for",
                        trainingsiterations,
                        " training iterations:",
                        int(100 * trainFraction),
                        shuffle,
                        "train error:",
                        np.round(trainerror, 2),
                        "pixels. Test error:",
                        np.round(testerror, 2),
                        " pixels.",
                    )
                    print(
                        "With pcutoff of",
                        cfg["pcutoff"],
                        " train error:",
                        np.round(trainerrorpcutoff, 2),
                        "pixels. Test error:",
                        np.round(testerrorpcutoff, 2),
                        "pixels",
                    )
                    print("Snapshot", snapshot_name)

                r = [
                    trainingsiterations,
                    int(100 * trainFraction),
                    shuffle,
                    np.round(trainerror, 2),
                    np.round(testerror, 2),
                    cfg["pcutoff"],
                    np.round(trainerrorpcutoff, 2),
                    np.round(testerrorpcutoff, 2),
                    snapshot_name,
                    scale,
                    test_pose_cfg["net_type"],
                ]
                results.append(r)
            else:
                print("Model not trained/evaluated!")
            if fulldata:
                DATA.append(
                    [
                        DataMachine,
                        Data,
                        data,
                        trainIndices,
                        testIndices,
                        trainFraction,
                        DLCscorer,
                        comparisonbodyparts,
                        cfg,
                        evaluationfolder,
                        snapshot_name,
                    ]
                )

    os.chdir(start_path)
    if returnjustfns:
        return resultsfns
    else:
        if fulldata:
            return DATA, results
        else:
            return results

return_train_network_path

return_train_network_path(config, shuffle=1, trainingsetindex=0, modelprefix='')

Returns the training and test pose config file names as well as the folder where the snapshot is.

Parameters:

Name	Type	Description	Default
config	string	Full path of the config.yaml file as a string.	required
shuffle	int	Integer value specifying the shuffle index to select for training.	1
trainingsetindex	int	Which TrainingsetFraction to use. By default the first (TrainingFraction is a list in config.yaml). Defaults to 0.	0

Returns:

Name	Type	Description
tuple		trainposeconfigfile, testposeconfigfile, snapshotfolder.

Source code in deeplabcut/pose_estimation_tensorflow/training.py

def return_train_network_path(config, shuffle=1, trainingsetindex=0, modelprefix=""):
    """Returns the training and test pose config file names as well as the folder where
    the snapshot is.

    Args:
        config (string): Full path of the config.yaml file as a string.
        shuffle (int): Integer value specifying the shuffle index to select for training.
        trainingsetindex (int, optional): Which TrainingsetFraction to use. By default the first
            (TrainingFraction is a list in config.yaml). Defaults to 0.

    Returns:
        tuple: trainposeconfigfile, testposeconfigfile, snapshotfolder.
    """
    from deeplabcut.utils import auxiliaryfunctions

    cfg = auxiliaryfunctions.read_config(config)
    modelfoldername = auxiliaryfunctions.get_model_folder(
        cfg["TrainingFraction"][trainingsetindex], shuffle, cfg, modelprefix=modelprefix
    )
    trainposeconfigfile = Path(cfg["project_path"]) / str(modelfoldername) / "train" / "pose_cfg.yaml"
    testposeconfigfile = Path(cfg["project_path"]) / str(modelfoldername) / "test" / "pose_cfg.yaml"
    snapshotfolder = Path(cfg["project_path"]) / str(modelfoldername) / "train"

    return trainposeconfigfile, testposeconfigfile, snapshotfolder

stitch_tracklets

stitch_tracklets(
    config_path: str | Path,
    videos: list[str | Path],
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    n_tracks=None,
    animal_names: list[str] | None = None,
    min_length=10,
    split_tracklets=True,
    prestitch_residuals=True,
    max_gap=None,
    weight_func=None,
    destfolder=None,
    modelprefix="",
    track_method="",
    output_name="",
    transformer_checkpoint="",
    save_as_csv=False,
    **kwargs
)

Stitch sparse tracklets into full tracks via a graph-based, minimum-cost flow optimization problem.

Parameters:

Name	Type	Description	Default
config_path	str | Path	Path to the main project config.yaml file.	required
videos	list[str | Path]	Full paths to videos for analysis, or a directory where all videos with the same extension are stored.	required
video_extensions	str | Sequence[str] | None	Controls how videos are filtered, based on file extension. File paths and directory contents are treated differently: - None (default): file paths are accepted as-is; directories are scanned for files with a recognized video extension. - str or Sequence[str] (e.g. "mp4" or ["mp4", "avi"]): both file paths and directory contents are filtered by the given extension(s). Defaults to None.	None
shuffle	int	Shuffle index of the training dataset used for training the network. Defaults to 1.	1
trainingsetindex	int	Which TrainingsetFraction to use. By default the first (TrainingFraction is a list in config.yaml).	0
n_tracks	int	Number of tracks to reconstruct. By default, taken as the number of individuals defined in the config.yaml. Another number can be passed if the number of animals in the video is different from the number of animals the model was trained on.	None
animal_names	list	If you want the names given to individuals in the labeled data file, you can specify those names as a list here. If given and n_tracks is None, n_tracks will be set to len(animal_names). If n_tracks is not None, then it must be equal to len(animal_names). If it is not given, then animal_names will be loaded from the individuals in the project config.yaml file.	None
min_length	int	Tracklets less than min_length frames of length are considered to be residuals; i.e., they do not participate in building the graph and finding the solution to the optimization problem, but are rather added last after "almost-complete" tracks are formed. The higher the value, the lesser the computational cost, but the higher the chance of discarding relatively long and reliable tracklets that are essential to solving the stitching task. Default is 10, and must be 3 at least.	10
split_tracklets	bool	By default, tracklets whose time indices are not consecutive integers are split in shorter tracklets whose time continuity is guaranteed. This is for example very powerful to get rid of tracking errors (e.g., identity switches) which are often signaled by a missing time frame at the moment they occur. Note though that for long occlusions where tracker re-identification capability can be trusted, setting split_tracklets to False is preferable.	True
prestitch_residuals	bool	Residuals will by default be grouped together according to their temporal proximity prior to being added back to the tracks. This is done to improve robustness and simultaneously reduce complexity.	True
max_gap	int	Maximal temporal gap to allow between a pair of tracklets. This is automatically determined by the TrackletStitcher by default.	None
weight_func	callable	Function accepting two tracklets as arguments and returning a scalar that must be inversely proportional to the likelihood that the tracklets belong to the same track; i.e., the higher the confidence that the tracklets should be stitched together, the lower the returned value.	None
destfolder	string	Destination folder for analysis data (default is the path of the video). Note that for subsequent analysis this folder also needs to be passed.	None
track_method	string	Tracker used to generate the pose estimation data. For multiple animals, must be either 'box', 'skeleton', or 'ellipse' and will be taken from the config.yaml file if none is given.	''
output_name	str	Name of the output h5 file. By default, tracks are automatically stored into the same directory as the pickle file and with its name.	''
transformer_checkpoint	str	Path to transformer checkpoint for re-ID stitching. Defaults to "".	''
save_as_csv	bool	Whether to write the tracks to a CSV file too (False by default).	False
kwargs	dict	Additional arguments. For torch-based shuffles, can be used to specify: - snapshot_index - detector_snapshot_index	{}

Returns:

Name	Type	Description
TrackletStitcher		A TrackletStitcher object.

Source code in deeplabcut/refine_training_dataset/stitch.py

@renamed_parameter(old="videotype", new="video_extensions", since="3.0.0")
def stitch_tracklets(
    config_path: str | Path,
    videos: list[str | Path],
    video_extensions: str | Sequence[str] | None = None,
    shuffle=1,
    trainingsetindex=0,
    n_tracks=None,
    animal_names: list[str] | None = None,
    min_length=10,
    split_tracklets=True,
    prestitch_residuals=True,
    max_gap=None,
    weight_func=None,
    destfolder=None,
    modelprefix="",
    track_method="",
    output_name="",
    transformer_checkpoint="",
    save_as_csv=False,
    **kwargs,
):
    """Stitch sparse tracklets into full tracks via a graph-based, minimum-cost flow
    optimization problem.

    Args:
        config_path (str | Path): Path to the main project config.yaml file.
        videos (list[str | Path]): Full paths to videos for analysis, or a directory where all videos
            with the same extension are stored.
        video_extensions (str | Sequence[str] | None, optional): Controls how ``videos`` are
            filtered, based on file extension. File paths and directory contents are
            treated differently:
            - ``None`` (default): file paths are accepted as-is; directories are
              scanned for files with a recognized video extension.
            - ``str`` or ``Sequence[str]`` (e.g. ``"mp4"`` or ``["mp4", "avi"]``):
              both file paths and directory contents are filtered by the given
              extension(s). Defaults to None.
        shuffle (int, optional): Shuffle index of the training dataset used for training the network. Defaults to 1.
        trainingsetindex (int, optional): Which TrainingsetFraction to use. By default the
            first (TrainingFraction is a list in config.yaml).
        n_tracks (int, optional): Number of tracks to reconstruct. By default, taken as the number
            of individuals defined in the config.yaml. Another number can be
            passed if the number of animals in the video is different from
            the number of animals the model was trained on.
        animal_names (list, optional): If you want the names given to individuals in the labeled data file, you can
            specify those names as a list here. If given and `n_tracks` is None, `n_tracks`
            will be set to `len(animal_names)`. If `n_tracks` is not None, then it must be
            equal to `len(animal_names)`. If it is not given, then `animal_names` will
            be loaded from the `individuals` in the project config.yaml file.
        min_length (int, optional): Tracklets less than `min_length` frames of length
            are considered to be residuals; i.e., they do not participate
            in building the graph and finding the solution to the
            optimization problem, but are rather added last after
            "almost-complete" tracks are formed. The higher the value,
            the lesser the computational cost, but the higher the chance of
            discarding relatively long and reliable tracklets that are
            essential to solving the stitching task.
            Default is 10, and must be 3 at least.
        split_tracklets (bool, optional): By default, tracklets whose time indices are not consecutive integers
            are split in shorter tracklets whose time continuity is guaranteed.
            This is for example very powerful to get rid of tracking errors
            (e.g., identity switches) which are often signaled by a missing
            time frame at the moment they occur. Note though that for long
            occlusions where tracker re-identification capability can be trusted,
            setting `split_tracklets` to False is preferable.
        prestitch_residuals (bool, optional): Residuals will by default be grouped together according to their
            temporal proximity prior to being added back to the tracks.
            This is done to improve robustness and simultaneously reduce complexity.
        max_gap (int, optional): Maximal temporal gap to allow between a pair of tracklets.
            This is automatically determined by the TrackletStitcher by default.
        weight_func (callable, optional): Function accepting two tracklets as arguments and returning a scalar
            that must be inversely proportional to the likelihood that the tracklets
            belong to the same track; i.e., the higher the confidence that the
            tracklets should be stitched together, the lower the returned value.
        destfolder (string, optional): Destination folder for analysis data (default is the path of the
            video). Note that for subsequent analysis this folder also needs to be passed.
        track_method (string, optional): Tracker used to generate the pose estimation data.
            For multiple animals, must be either 'box', 'skeleton', or 'ellipse'
            and will be taken from the config.yaml file if none is given.
        output_name (str, optional): Name of the output h5 file.
            By default, tracks are automatically stored into the same directory
            as the pickle file and with its name.
        transformer_checkpoint (str, optional): Path to transformer checkpoint for re-ID
            stitching. Defaults to "".
        save_as_csv (bool, optional): Whether to write the tracks to a CSV file too (False by default).
        kwargs (dict, optional): Additional arguments.
            For torch-based shuffles, can be used to specify:
                - snapshot_index
                - detector_snapshot_index

    Returns:
        TrackletStitcher: A TrackletStitcher object.
    """
    vids = collect_video_paths(videos, extensions=video_extensions)
    if not vids:
        print("No video(s) found. Please check your path!")
        return

    cfg = auxiliaryfunctions.read_config(config_path)
    track_method = auxfun_multianimal.get_track_method(cfg, track_method=track_method)
    if track_method == "ctd":
        raise ValueError(
            "CTD tracking occurs directly during video analysis. No need to call "
            "`stitch_tracklets` with `track_method=='ctd'`."
        )

    if animal_names is None:
        animal_names = cfg["individuals"]
    elif n_tracks is not None and n_tracks != len(animal_names):
        raise ValueError(
            "When setting both `n_tracks` and `animal_names`, `n_tracks` must be equal "
            f"to len(animal_names)`. Found `n_tracks`={n_tracks} and `animal_names`="
            f"{animal_names} of length {len(animal_names)}.`"
        )

    if n_tracks is None:
        n_tracks = len(animal_names)

    DLCscorer, _ = deeplabcut.utils.auxiliaryfunctions.get_scorer_name(
        cfg,
        shuffle,
        cfg["TrainingFraction"][trainingsetindex],
        modelprefix=modelprefix,
        **kwargs,
    )

    if transformer_checkpoint:
        from deeplabcut.pose_tracking_pytorch import inference

        dlctrans = inference.DLCTrans(checkpoint=transformer_checkpoint)

    def trans_weight_func(tracklet1, tracklet2, nframe, feature_dict):
        zfill_width = int(np.ceil(np.log10(nframe)))
        if tracklet1 < tracklet2:
            ind_img1 = tracklet1.inds[-1]
            coord1 = tracklet1.data[-1][:, :2]
            ind_img2 = tracklet2.inds[0]
            coord2 = tracklet2.data[0][:, :2]
        else:
            ind_img2 = tracklet2.inds[-1]
            ind_img1 = tracklet1.inds[0]
            coord2 = tracklet2.data[-1][:, :2]
            coord1 = tracklet1.data[0][:, :2]
        t1 = (coord1, ind_img1)
        t2 = (coord2, ind_img2)

        dist = dlctrans(t1, t2, zfill_width, feature_dict)
        dist = (dist + 1) / 2

        return -dist

    base_weight_func = weight_func
    for video in vids:
        print("Processing... ", video)
        nframe = len(VideoWriter(video))
        videofolder = str(Path(video).parents[0])
        dest = destfolder or videofolder
        deeplabcut.utils.auxiliaryfunctions.attempt_to_make_folder(dest)
        vname = Path(video).stem

        feature_dict_path = str(Path(dest) / (vname + DLCscorer + "_bpt_features.pickle"))
        # should only exist one
        if transformer_checkpoint:
            import dbm

            try:
                feature_dict = shelve.open(feature_dict_path, flag="r")
            except dbm.error as err:
                raise FileNotFoundError(f"{feature_dict_path} does not exist. Did you run transformer_reID()?") from err

        dataname = str(Path(dest) / (vname + DLCscorer + ".h5"))

        method = TRACK_METHODS[track_method]
        pickle_file = dataname.split(".h5")[0] + f"{method}.pickle"
        try:
            stitcher = TrackletStitcher.from_pickle(
                pickle_file, n_tracks, min_length, split_tracklets, prestitch_residuals
            )
            current_weight_func = base_weight_func
            with_id = any(tracklet.identity != -1 for tracklet in stitcher)
            if with_id and weight_func is None:
                # Add in identity weighing before building the graph
                def current_weight_func(t1, t2, stitcher=stitcher):
                    w = 0.01 if t1.identity == t2.identity else 1
                    return w * stitcher.calculate_edge_weight(t1, t2)

            if transformer_checkpoint:
                stitcher.build_graph(
                    max_gap=max_gap,
                    weight_func=partial(trans_weight_func, nframe=nframe, feature_dict=feature_dict),
                )
            else:
                stitcher.build_graph(max_gap=max_gap, weight_func=current_weight_func)

            stitcher.stitch()
            if transformer_checkpoint:
                stitcher.write_tracks(
                    output_name=output_name,
                    animal_names=animal_names,
                    suffix="tr",
                    save_as_csv=save_as_csv,
                )
            else:
                stitcher.write_tracks(
                    output_name=output_name,
                    animal_names=animal_names,
                    suffix="",
                    save_as_csv=save_as_csv,
                )
        except FileNotFoundError as e:
            print(e, "\nSkipping...")

train_network

train_network(
    config,
    shuffle=1,
    trainingsetindex=0,
    max_snapshots_to_keep=5,
    displayiters=None,
    saveiters=None,
    maxiters=None,
    allow_growth=True,
    gputouse=None,
    autotune=False,
    keepdeconvweights=True,
    modelprefix="",
    superanimal_name="",
    superanimal_transfer_learning=False,
)

Trains the network with the labels in the training dataset.

Parameters:

Name	Type	Description	Default
config	string	Full path of the config.yaml file as a string.	required
shuffle	int	Integer value specifying the shuffle index to select for training. Defaults to 1.	1
trainingsetindex	int	Integer specifying which TrainingsetFraction to use. Note that TrainingFraction is a list in config.yaml. Defaults to 0.	0
max_snapshots_to_keep	int or None	Sets how many snapshots are kept, i.e. states of the trained network. Every saving iteration many times a snapshot is stored, however only the last max_snapshots_to_keep many are kept! If you change this to None, then all are kept. See: https://github.com/DeepLabCut/DeepLabCut/issues/8#issuecomment-387404835	5
display_iters	int	This variable is actually set in pose_config.yaml. However, you can overwrite it with this hack. Don't use this regularly, just if you are too lazy to dig out the pose_config.yaml file for the corresponding project. If None, the value from there is used, otherwise it is overwritten! Defaults to None.	required
saveiters	int	This variable is actually set in pose_config.yaml. However, you can overwrite it with this hack. Don't use this regularly, just if you are too lazy to dig out the pose_config.yaml file for the corresponding project. If None, the value from there is used, otherwise it is overwritten! Defaults to None.	None
maxiters	int	This variable is actually set in pose_config.yaml. However, you can overwrite it with this hack. Don't use this regularly, just if you are too lazy to dig out the pose_config.yaml file for the corresponding project. If None, the value from there is used, otherwise it is overwritten! Defaults to None.	None
allow_growth	bool	For some smaller GPUs the memory issues happen. If True, the memory allocator does not pre-allocate the entire specified GPU memory region, instead starting small and growing as needed. See issue: https://forum.image.sc/t/how-to-stop-running-out-of-vram/30551/2. Defaults to True.	True
gputouse	int	Natural number indicating the number of your GPU (see number in nvidia-smi). If you do not have a GPU put None. See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries. Defaults to None.	None
autotune	bool	Property of TensorFlow, somehow faster if False (as Eldar found out, see https://github.com/tensorflow/tensorflow/issues/13317). Defaults to False.	False
keepdeconvweights	bool	Restores deconvolution layer (and backbone) weights when training from a snapshot. Set to false if bodypart count changes. Defaults to True.	True
modelprefix	str	Directory containing the deeplabcut models to use when evaluating the network. By default, the models are assumed to exist in the project folder. Defaults to "".	''
superanimal_name	str	Specified if transfer learning with superanimal is desired. Defaults to "".	''
superanimal_transfer_learning	bool	If true, transfer learning (new decoding layer). If false and superanimal_name is set, fine-tuning (reuse decoding layer). Defaults to False.	False

Returns:

Type	Description
	None

Examples:

To train the network for first shuffle of the training dataset

deeplabcut.train_network("/analysis/project/reaching-task/config.yaml")

To train the network for second shuffle of the training dataset

deeplabcut.train_network(
    '/analysis/project/reaching-task/config.yaml',
    shuffle=2,
    keepdeconvweights=True,
)

Source code in deeplabcut/pose_estimation_tensorflow/training.py

def train_network(
    config,
    shuffle=1,
    trainingsetindex=0,
    max_snapshots_to_keep=5,
    displayiters=None,
    saveiters=None,
    maxiters=None,
    allow_growth=True,
    gputouse=None,
    autotune=False,
    keepdeconvweights=True,
    modelprefix="",
    superanimal_name="",
    superanimal_transfer_learning=False,
):
    """Trains the network with the labels in the training dataset.

    Args:
        config (string): Full path of the config.yaml file as a string.
        shuffle (int, optional): Integer value specifying the shuffle index to select for training. Defaults to 1.
        trainingsetindex (int, optional): Integer specifying which TrainingsetFraction to use.
            Note that TrainingFraction is a list in config.yaml. Defaults to 0.
        max_snapshots_to_keep (int or None): Sets how many snapshots are kept, i.e. states of the trained network. Every
            saving iteration many times a snapshot is stored, however only the last
            ``max_snapshots_to_keep`` many are kept! If you change this to None, then all
            are kept.
            See: https://github.com/DeepLabCut/DeepLabCut/issues/8#issuecomment-387404835
        display_iters (int, optional): This variable is actually set in ``pose_config.yaml``. However, you can
            overwrite it with this hack. Don't use this regularly, just if you are too lazy
            to dig out the ``pose_config.yaml`` file for the corresponding project. If
            ``None``, the value from there is used, otherwise it is overwritten! Defaults to None.
        saveiters (int, optional): This variable is actually set in ``pose_config.yaml``. However, you can
            overwrite it with this hack. Don't use this regularly, just if you are too lazy
            to dig out the ``pose_config.yaml`` file for the corresponding project.
            If ``None``, the value from there is used, otherwise it is overwritten! Defaults to None.
        maxiters (int, optional): This variable is actually set in ``pose_config.yaml``. However, you can
            overwrite it with this hack. Don't use this regularly, just if you are too lazy
            to dig out the ``pose_config.yaml`` file for the corresponding project.
            If ``None``, the value from there is used, otherwise it is overwritten! Defaults to None.
        allow_growth (bool, optional): For some smaller GPUs the memory issues happen. If ``True``, the memory
            allocator does not pre-allocate the entire specified GPU memory region, instead
            starting small and growing as needed.
            See issue: https://forum.image.sc/t/how-to-stop-running-out-of-vram/30551/2. Defaults to True.
        gputouse (int, optional): Natural number indicating the number of your GPU (see number in nvidia-smi).
            If you do not have a GPU put None.
            See: https://nvidia.custhelp.com/app/answers/detail/a_id/3751/~/useful-nvidia-smi-queries. Defaults to None.
        autotune (bool, optional): Property of TensorFlow, somehow faster if ``False``
            (as Eldar found out, see https://github.com/tensorflow/tensorflow/issues/13317). Defaults to False.
        keepdeconvweights (bool, optional): Restores deconvolution layer (and backbone) weights when
            training from a snapshot. Set to false if bodypart count changes. Defaults to True.
        modelprefix (str, optional): Directory containing the deeplabcut models to use when evaluating the network.
            By default, the models are assumed to exist in the project folder. Defaults to "".
        superanimal_name (str, optional): Specified if transfer learning with superanimal is desired. Defaults to "".
        superanimal_transfer_learning (bool, optional): If true, transfer learning (new decoding layer).
            If false and superanimal_name is set, fine-tuning (reuse decoding layer). Defaults to False.

    Returns:
        None

    Examples:
        To train the network for first shuffle of the training dataset

            deeplabcut.train_network("/analysis/project/reaching-task/config.yaml")

        To train the network for second shuffle of the training dataset

            deeplabcut.train_network(
                '/analysis/project/reaching-task/config.yaml',
                shuffle=2,
                keepdeconvweights=True,
            )
    """
    if allow_growth:
        os.environ["TF_FORCE_GPU_ALLOW_GROWTH"] = "true"

    # reload logger.
    import importlib
    import logging

    import tensorflow as tf

    importlib.reload(logging)
    logging.shutdown()

    from deeplabcut.utils import auxiliaryfunctions

    tf.compat.v1.reset_default_graph()
    start_path = Path.cwd()

    # Read file path for pose_config file. >> pass it on
    cfg = auxiliaryfunctions.read_config(config)
    modelfoldername = auxiliaryfunctions.get_model_folder(
        cfg["TrainingFraction"][trainingsetindex], shuffle, cfg, modelprefix=modelprefix
    )
    poseconfigfile = Path(cfg["project_path"]) / str(modelfoldername) / "train" / "pose_cfg.yaml"
    if not poseconfigfile.is_file():
        print("The training datafile ", poseconfigfile, " is not present.")
        print("Probably, the training dataset for this specific shuffle index was not created.")
        print(
            "Try with a different shuffle/trainingsetfraction or use function 'create_training_dataset' to create a new"
            "trainingdataset with this shuffle index."
        )
    else:
        # Set environment variables
        if autotune is not False:  # see: https://github.com/tensorflow/tensorflow/issues/13317
            os.environ["TF_CUDNN_USE_AUTOTUNE"] = "0"
        if gputouse is not None:
            os.environ["CUDA_VISIBLE_DEVICES"] = str(gputouse)
    try:
        cfg_dlc = auxiliaryfunctions.read_plainconfig(poseconfigfile)

        if superanimal_name != "":
            from dlclibrary.dlcmodelzoo.modelzoo_download import (
                MODELOPTIONS,
                download_huggingface_model,
            )

            from deeplabcut.modelzoo.utils import parse_available_supermodels

            dlc_root_path = auxiliaryfunctions.get_deeplabcut_path()
            parse_available_supermodels()
            weight_folder = str(
                Path(dlc_root_path)
                / "pose_estimation_tensorflow"
                / "models"
                / "pretrained"
                / (superanimal_name + "_weights")
            )

            if superanimal_name in MODELOPTIONS:
                if not Path(weight_folder).exists():
                    download_huggingface_model(superanimal_name, weight_folder)
                else:
                    print(f"{weight_folder} exists, using the downloaded weights")
            else:
                print(
                    f"{superanimal_name} not available. Available ones are: ",
                    MODELOPTIONS,
                )

            snapshots = list(Path(weight_folder).glob("snapshot-*.index"))
            init_weights = auxiliaryfunctions.safe_resolve(snapshots[0].with_suffix(""))

            from deeplabcut.pose_estimation_tensorflow.core.train_multianimal import (
                train,
            )

            print("Selecting multi-animal trainer")
            train(
                str(poseconfigfile),
                displayiters,
                saveiters,
                maxiters,
                max_to_keep=max_snapshots_to_keep,
                keepdeconvweights=keepdeconvweights,
                allow_growth=allow_growth,
                init_weights=str(init_weights),
                remove_head=(True if superanimal_name != "" and superanimal_transfer_learning else False),
            )  # pass on path and file name for pose_cfg.yaml!

        elif "multi-animal" in cfg_dlc["dataset_type"]:
            from deeplabcut.pose_estimation_tensorflow.core.train_multianimal import (
                train,
            )

            print("Selecting multi-animal trainer")
            train(
                str(poseconfigfile),
                displayiters,
                saveiters,
                maxiters,
                max_to_keep=max_snapshots_to_keep,
                keepdeconvweights=keepdeconvweights,
                allow_growth=allow_growth,
            )  # pass on path and file name for pose_cfg.yaml!
        else:
            from deeplabcut.pose_estimation_tensorflow.core.train import train

            print("Selecting single-animal trainer")
            train(
                str(poseconfigfile),
                displayiters,
                saveiters,
                maxiters,
                max_to_keep=max_snapshots_to_keep,
                keepdeconvweights=keepdeconvweights,
                allow_growth=allow_growth,
            )  # pass on path and file name for pose_cfg.yaml!

    except BaseException as e:
        raise e
    finally:
        os.chdir(str(start_path))
    print(
        "The network is now trained and ready to evaluate. Use the function 'evaluate_network' to evaluate the network."
    )

visualize_locrefs

visualize_locrefs(
    image: ndarray, scmap: ndarray, locref_x: ndarray, locref_y: ndarray, step: int = 5, zoom_width: int = 0
) -> tuple[plt.Figure, plt.Axes]

Plots a scoremap and the corresponding location refinement field on an image.

Parameters:

Name	Type	Description	Default
image	ndarray	An image as a numpy array of shape (h, w, channels)	required
scmap	ndarray	A scoremap of shape (h, w)	required
locref_x	ndarray	The x-coordinate of the location refinement field, of shape (h, w)	required
locref_y	ndarray	The y-coordinate of the location refinement field, of shape (h, w)	required
step	int	The step with which to plot the location refinement field.	5
zoom_width	int	The zoom width with which to plot the scoremaps.	0

Returns:

Type	Description
tuple[Figure, Axes]	The figure and axis on which the image scoremap and locref field were plot.

Source code in deeplabcut/core/visualization.py

def visualize_locrefs(
    image: np.ndarray,
    scmap: np.ndarray,
    locref_x: np.ndarray,
    locref_y: np.ndarray,
    step: int = 5,
    zoom_width: int = 0,
) -> tuple[plt.Figure, plt.Axes]:
    """Plots a scoremap and the corresponding location refinement field on an image.

    Args:
        image: An image as a numpy array of shape (h, w, channels)
        scmap: A scoremap of shape (h, w)
        locref_x: The x-coordinate of the location refinement field, of shape (h, w)
        locref_y: The y-coordinate of the location refinement field, of shape (h, w)
        step: The step with which to plot the location refinement field.
        zoom_width: The zoom width with which to plot the scoremaps.

    Returns:
        The figure and axis on which the image scoremap and locref field were plot.
    """
    fig, ax = visualize_scoremaps(image, scmap)
    X, Y = np.meshgrid(np.arange(locref_x.shape[1]), np.arange(locref_x.shape[0]))
    M = np.zeros(locref_x.shape, dtype=bool)
    M[scmap < 0.5] = True
    U = np.ma.masked_array(locref_x, mask=M)
    V = np.ma.masked_array(locref_y, mask=M)
    ax.quiver(
        X[::step, ::step],
        Y[::step, ::step],
        U[::step, ::step],
        V[::step, ::step],
        color="r",
        units="x",
        scale_units="xy",
        scale=1,
        angles="xy",
    )
    if zoom_width > 0:
        maxloc = np.unravel_index(np.argmax(scmap), scmap.shape)
        ax.set_xlim(maxloc[1] - zoom_width, maxloc[1] + zoom_width)
        ax.set_ylim(maxloc[0] + zoom_width, maxloc[0] - zoom_width)
    return fig, ax

visualize_paf

visualize_paf(image: ndarray, paf: ndarray, step: int = 5, colors: list | None = None) -> tuple[plt.Figure, plt.Axes]

Plots the PAF on top of the image.

Parameters:

Name	Type	Description	Default
image	ndarray	Shape (height, width, channels). The image on which the model was run.	required
paf	ndarray	Shape (height, width, 2 * len(paf_graph)). The PAF output by the model.	required
step	int	The step with which to plot the scoremaps.	5
colors	list | None	The colormap to use.	None

Returns:

Type	Description
tuple[Figure, Axes]	The figure and axis on which the image PAF was plot.

Source code in deeplabcut/core/visualization.py

def visualize_paf(
    image: np.ndarray,
    paf: np.ndarray,
    step: int = 5,
    colors: list | None = None,
) -> tuple[plt.Figure, plt.Axes]:
    """Plots the PAF on top of the image.

    Args:
        image: Shape (height, width, channels). The image on which the model was run.
        paf: Shape (height, width, 2 * len(paf_graph)). The PAF output by the model.
        step: The step with which to plot the scoremaps.
        colors: The colormap to use.

    Returns:
        The figure and axis on which the image PAF was plot.
    """
    ny, nx = np.shape(image)[:2]
    fig, ax = form_figure(nx, ny)
    ax.imshow(image)
    n_fields = paf.shape[2]
    if colors is None:
        colors = ["r"] * n_fields
    for n in range(n_fields):
        U = paf[:, :, n, 0]
        V = paf[:, :, n, 1]
        X, Y = np.meshgrid(np.arange(U.shape[1]), np.arange(U.shape[0]))
        M = np.zeros(U.shape, dtype=bool)
        M[U**2 + V**2 < 0.5 * 0.5**2] = True
        U = np.ma.masked_array(U, mask=M)
        V = np.ma.masked_array(V, mask=M)
        ax.quiver(
            X[::step, ::step],
            Y[::step, ::step],
            U[::step, ::step],
            V[::step, ::step],
            scale=50,
            headaxislength=4,
            alpha=1,
            width=0.002,
            color=colors[n],
            angles="xy",
        )
    return fig, ax

visualize_scoremaps

visualize_scoremaps(image: ndarray, scmap: ndarray) -> tuple[plt.Figure, plt.Axes]

Plots scoremaps as an image overlay.

Parameters:

Name	Type	Description	Default
image	ndarray	An image as a numpy array of shape (h, w, channels)	required
scmap	ndarray	A scoremap of shape (h, w)	required

Returns:

Type	Description
tuple[Figure, Axes]	The figure and axis on which the image scoremap was plot.

Source code in deeplabcut/core/visualization.py

def visualize_scoremaps(
    image: np.ndarray,
    scmap: np.ndarray,
) -> tuple[plt.Figure, plt.Axes]:
    """Plots scoremaps as an image overlay.

    Args:
        image: An image as a numpy array of shape (h, w, channels)
        scmap: A scoremap of shape (h, w)

    Returns:
        The figure and axis on which the image scoremap was plot.
    """
    ny, nx = np.shape(image)[:2]
    fig, ax = form_figure(nx, ny)
    ax.imshow(image)
    ax.imshow(scmap, alpha=0.5)
    return fig, ax