Accumulate timewalk statistics

Timewalk correction is in Luna is statistical and as such relies on large sample sizes for accuracy. You can accumulate a timewalk matrix across similar experiments to increase the N.

Luna Command

./tpx3dump process --input-files /Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/example_data.tpx3 --layout single --clustering-algorithm DFSCluster --eps-t 100ns --eps-s 1 --calculate-timewalk-statistics --output-file /Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/example_data.h5
./tpx3dump process --input-files /Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/example_data.tpx3 --output-file /Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/example_data2.h5 --layout single --clustering-algorithm DFSCluster --eps-t 100ns --eps-s 1 --timewalk-file /Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/example_data.h5 --ctot-cut 450

Python Script

Accumulation of timewalk statistics across experiments
 1import os, sys
 2import h5py  # ensure you have `pip install h5py`
 3import pandas as pd  # ensure you have `pip install pandas`
 4from typing import *
 5import matplotlib.pyplot as plt
 6import seaborn as sns
 7import warnings
 8
 9
10
11warnings.filterwarnings("ignore")  # suppress warnings from plotting libraries.
12
13sns.set_context(context="talk")
14
15# add some paths to PYTHONPATH
16for directory in ["..", "."]:
17    sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), directory)))
18
19# on our system "EXAMPLE_DATA_HDF5" refers to the absolute path
20# to a hdf5 file generated by luna. Replace with your own!
21from env_vars_for_docs_examples import EXAMPLE_DATA_HDF5, EXAMPLE2_DATA_HDF5, PLOTS_DIRECTORY
22
23# re-use functions from previous example
24from ex2_read_data_time_units import TimeUnit, load_pixel_hits
25from ex11_raw_only_analysis import plot_toa
26
27TOA_UNIT = TimeUnit.Nanoseconds
28
29
30if __name__ == "__main__":
31
32    toa_before_correction_fname = os.path.join(PLOTS_DIRECTORY, f"ex14_apply_timewalk_correction_toa_before_correction.png")
33    toa_after_correction_fname = os.path.join(PLOTS_DIRECTORY, f"ex14_apply_timewalk_correction_toa_after_correction.png")
34
35    pixel_hits_before_correction: pd.DataFrame = load_pixel_hits(EXAMPLE_DATA_HDF5, toa_unit=TimeUnit.Microseconds)
36    pixel_hits_after_correction: pd.DataFrame = load_pixel_hits(EXAMPLE2_DATA_HDF5, toa_unit=TimeUnit.Microseconds)
37
38    start = 0
39    stop = 50
40    plot_toa(pixel_hits_before_correction, start=start, stop=stop, fname=toa_before_correction_fname,time_units=TOA_UNIT)
41    plot_toa(pixel_hits_after_correction, start=start, stop=stop, fname=toa_after_correction_fname,time_units=TOA_UNIT)
42

Script Output

Example Output
hdf5 datasets: ['Clusters', 'ExposureTimeBoundaries', 'PixelHits', 'TimewalkLookupTable', 'TimewalkMatrix']
hdf5 datasets: ['Clusters', 'ExposureTimeBoundaries', 'PixelHits']
Plot saved to "/Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/examples_output/plots/ex14_apply_timewalk_correction_toa_before_correction.png"
Plot saved to "/Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/examples_output/plots/ex14_apply_timewalk_correction_toa_after_correction.png"