Time-walk Correction: Matrix

The timewalk matrix gives important insight into your data. More specifically it shows you where to put the cToT cut parameter. The timewalk lookup table is an average of this matrix, “cut” at a particular cToT point. That is, values lower than this are discarded and values higher are averaged and used to construct the lookup table.

Luna Command

./tpx3dump process -i /Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/example_data.tpx3 -o /Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/example_data.h5 --eps-t 160ns --eps-s 1 --calculate-timewalk-statistics --layout single

Python Script

Plot the timewalk matrix

import os, sys
from typing import *
import lmfit
import pandas as pd  # ensure you have `pip install pandas`
import warnings
import seaborn as sns
import matplotlib.pyplot as plt
from matplotlib import cm
from matplotlib.colors import Normalize
import numpy as np


warnings.filterwarnings("ignore")  # suppress warnings from plotting libraries.

sns.set_context(context="talk")

# add some paths to PYTHONPATH
for directory in ["..", "."]:
    sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), directory)))

# on our system "EXAMPLE_DATA_HDF5" refers to the absolute path
# to a hdf5 file generated by luna. Replace with your own!
from env_vars_for_docs_examples import EXAMPLE_DATA_HDF5, PLOTS_DIRECTORY

# re-use functions from previous example
from ex2_read_data_time_units import load_timewalk_matrix, TimeUnit


def plot_timewalk_matrix_heatmap(timewalk_matrix: pd.DataFrame, max_ctot_and_tot_to_show: Optional[List[int]] = None):
    """
    Plots a heatmap of the timewalk matrix.

    This function generates and saves heatmaps of the timewalk matrix, showing the relationship between
    CToT (Corrected Time over Threshold) and ToT (Time over Threshold) bins, and their corresponding
    AverageDToA (Average Delta Time of Arrival) values. The heatmaps are saved as PNG files.

    Parameters:
    -----------
    timewalk_matrix : pd.DataFrame
        A DataFrame containing the timewalk matrix data. The DataFrame should have the following columns:
        - 'CToT': Corrected Time over Threshold
        - 'ToT': Time over Threshold
        - 'AverageDToA': Average Delta Time of Arrival
        - Other columns may include 'SumSquareDiff', 'Count', 'Std', 'Sem', etc.

    max_ctot_and_tot_to_show : Optional[List[int]], default=None
        A list of integers specifying the maximum CToT and ToT values to show in the heatmap. If None,
        the default value [1024 * 25] is used. Each value in the list will generate a separate heatmap.

    Returns:
    --------
    None
        The function does not return any value. It saves the generated heatmaps as PNG files in the
        specified directory.

    Raises:
    -------
    AssertionError
        If `max_ctot_and_tot_to_show` is not a list of integers.

    """
    if max_ctot_and_tot_to_show is None:
        max_ctot_and_tot_to_show = [1024 * 25]

    for max_tot in max_ctot_and_tot_to_show:
        assert isinstance(max_ctot_and_tot_to_show, list), "max_ctot_and_tot_to_show should be a list of integers"

        timewalk_matrix_avg = timewalk_matrix.pivot(index="ToT", columns="CToT", values="AverageDToA")
        print(timewalk_matrix_avg.head())
        data = timewalk_matrix_avg.loc[timewalk_matrix_avg.index <= max_tot, timewalk_matrix_avg.columns <= max_tot]

        fig, ax = plt.subplots()
        sns.despine(fig=fig)
        img = ax.imshow(data.values,
                        origin="lower",
                        cmap="BuPu",
                        aspect="auto",
                        )
        ax.set_ylabel("ToT bin (ns)")
        ax.set_xlabel("cToT bin (ns)")

        xticks = np.linspace(0, data.shape[1] - 1, num=min(10, data.shape[1]), dtype=int)
        yticks = np.linspace(0, data.shape[0] - 1, num=min(10, data.shape[0]), dtype=int)

        ax.set_xticks(xticks)
        ax.set_yticks(yticks)
        ax.set_xticklabels(data.columns[xticks], rotation=45)
        ax.set_yticklabels(data.index[yticks])

        c = plt.colorbar(img)
        c.set_label('average(dToA) ns')
        plt.title(f"cToT vs ToT vs dToA \n(Showing max cToT = max ToT = {max_tot})")
        fname = os.path.join(PLOTS_DIRECTORY, f"ex8_timewalk_matrix_heatmap_{max_tot}.png")
        plt.savefig(fname, bbox_inches='tight', dpi=300)



if __name__ == "__main__":
    toa_unit = TimeUnit.Nanoseconds

    timewalk_matrix = load_timewalk_matrix(EXAMPLE_DATA_HDF5, toa_unit=toa_unit)
    print("Timewalk Matrix:")
    print(timewalk_matrix.head().to_string())
    if timewalk_matrix is not None:
        tot_thresholds = [ 500, 1000, 1500, 2000]
        plot_timewalk_matrix_heatmap(timewalk_matrix, tot_thresholds)

Script Output

Example Output

<KeysViewHDF5 ['Clusters', 'ExposureTimeBoundaries', 'PixelHits', 'TimewalkLookupTable', 'TimewalkMatrix']>
Timewalk Matrix:
   CToT  ToT  AverageDToA  SumSquareDiff  Count        Std       Sem
0   450   25    96.223969     53196212.0     12  21.054733  6.077978
1   450   50    83.750008    119628920.0     25  21.875000  4.375000
2   450   75    76.627617     66283176.0     24  16.618660  3.392270
3   450  100    62.467461     91039528.0     48  13.771915  1.987805
4   450  125    51.258690    118634712.0     72  12.836293  1.512772
CToT       450        475        500   ...       6675       6825       6850
ToT                                    ...                                 
25    96.223969  94.140625  99.356606  ...        NaN  23.437500  21.875000
50    83.750008  76.041664  84.742653  ...  15.625000  17.187500  23.437500
75    76.627617  72.968750  72.727280  ...  14.062500  16.406252  15.104167
100   62.467461  60.044651  63.733555  ...  11.979167   9.375000   8.593750
125   51.258690  52.926594  54.313362  ...  10.937500        NaN  10.937500

[5 rows x 191 columns]
CToT       450        475        500   ...       6675       6825       6850
ToT                                    ...                                 
25    96.223969  94.140625  99.356606  ...        NaN  23.437500  21.875000
50    83.750008  76.041664  84.742653  ...  15.625000  17.187500  23.437500
75    76.627617  72.968750  72.727280  ...  14.062500  16.406252  15.104167
100   62.467461  60.044651  63.733555  ...  11.979167   9.375000   8.593750
125   51.258690  52.926594  54.313362  ...  10.937500        NaN  10.937500

[5 rows x 191 columns]
CToT       450        475        500   ...       6675       6825       6850
ToT                                    ...                                 
25    96.223969  94.140625  99.356606  ...        NaN  23.437500  21.875000
50    83.750008  76.041664  84.742653  ...  15.625000  17.187500  23.437500
75    76.627617  72.968750  72.727280  ...  14.062500  16.406252  15.104167
100   62.467461  60.044651  63.733555  ...  11.979167   9.375000   8.593750
125   51.258690  52.926594  54.313362  ...  10.937500        NaN  10.937500

[5 rows x 191 columns]
CToT       450        475        500   ...       6675       6825       6850
ToT                                    ...                                 
25    96.223969  94.140625  99.356606  ...        NaN  23.437500  21.875000
50    83.750008  76.041664  84.742653  ...  15.625000  17.187500  23.437500
75    76.627617  72.968750  72.727280  ...  14.062500  16.406252  15.104167
100   62.467461  60.044651  63.733555  ...  11.979167   9.375000   8.593750
125   51.258690  52.926594  54.313362  ...  10.937500        NaN  10.937500

[5 rows x 191 columns]