Repository navigation
A few issues with pintk Chi2 calculation and fitting #1963
Description
I found a few issues with pintk. They are demonstrated with the following files (remove .txt from the filenames after downloading).
J0437phoff.par.txt
test.tim.txt
J0437test.par.txt
- Post-Fit Chi2 is bonkers. To see this, run
pintk J0437test.par test.tim, then click "Fit". Here is what is printed:
------------------------------------
Pre-Fit Chi2: 127.87657
Pre-Fit DOF: 97
Pre-Fit reduced-Chi2: 1.3183152
Pre-Fit Weighted RMS: 5.6247845 us
------------------------------------
Post-Fit Chi2: 1465222
Has Correlated Errors False
Post-Fit DOF: 97
Post-Fit Reduced-Chi2: 15105.382
Post-Fit Weighted RMS: 5.5043546 us
------------------------------------
Parameter Pre-Fit Post-Fit Uncertainty Difference Diff/Unc
------------------------------------------------------------------------------------------------------------------------------------
F0 Hz 173.6879456649309113 173.6879456650772781 1.1830554e-10 1.4636679e-10 134.85374
F1 Hz / s -1.7283484555202134978e-15 -1.7309811914243010753e-15 1.9126623e-18 -2.6327359e-18 -821.45156
Akaike information criterion = -2314.022850977091
Notice that the weighted rms drops from 5.62 to 5.50 µs, but Chi2 is crazy.
I found that this is reversed if there is a PHOFF in the initial .par file, pintk J0437phoff.par test.tim. Here the pre-fit is huge, but the post-fit is fine.
------------------------------------
Pre-Fit Chi2: 1328707.7
Pre-Fit DOF: 96
Pre-Fit reduced-Chi2: 13840.705
Pre-Fit Weighted RMS: 5.6247845 us
------------------------------------
Post-Fit Chi2: 121.19168
Has Correlated Errors False
Post-Fit DOF: 96
Post-Fit Reduced-Chi2: 1.2624134
Post-Fit Weighted RMS: 5.5043547 us
------------------------------------
Parameter Pre-Fit Post-Fit Uncertainty Difference Diff/Unc
------------------------------------------------------------------------------------------------------------------------------------
F0 Hz 173.6879456649309113 173.68794566507727815 1.1830554e-10 1.4636685e-10 134.85379
F1 Hz / s -1.7283484555202134978e-15 -1.7309811924439748781e-15 1.9126623e-18 -2.6327369e-18 -821.45188
PHOFF 0.1 0.10511710065360719 0.0035830273 0.0051171007
Akaike information criterion = -2314.02284612503
-
In the previous output, notice that Diff/Unc is computed using the uncertainty that was in the initial par file, rather than the uncertainty on that parameter resulting from the current fit. This is confusing and wrong.
-
If you edit J0437phoff.par to have
PHOFF 0.0then the DownhillWLSFitter does not work, with a StepProblem. Changing to the WLSFitter does work, however. Here is what happens:
------------------------------------
Pre-Fit Chi2: 127.87657
Pre-Fit DOF: 96
Pre-Fit reduced-Chi2: 1.3320476
Pre-Fit Weighted RMS: 5.6247845 us
------------------------------------
Post-Fit Chi2: 127.87657
WARNING (pint.logging ): /Users/paulr/src/PINT/src/pint/fitter.py:1031 StepProblem: Unable to improve chi2 even with very small steps
Has Correlated Errors False
Post-Fit DOF: 96
Post-Fit Reduced-Chi2: 1.3320476
Post-Fit Weighted RMS: 5.6247845 us
------------------------------------
Parameter Pre-Fit Post-Fit Uncertainty Difference Diff/Unc
------------------------------------------------------------------------------------------------------------------------------------
F0 Hz 173.6879456649309113 173.6879456649309113 1.1830554e-10 0 0
F1 Hz / s -1.7283484555202134978e-15 -1.7283484555202134978e-15 1.9126623e-18 0 0
PHOFF 0.0 0.0 0.0035830273 0
Akaike information criterion = -2307.3379556298382
So, something is wrong in how Chi2 is computed and maybe it got introduced when PHOFF was added?
Activity
dlakaplan commentedon May 25, 2026
So there is now a "subtract mean" box to control some of this behavior. i.e., if I do the first test you suggested but I click "subtract mean" first, I get:
which makes sense. So I'm not sure if your issue #1 is really a problem? When you have a
PHOFFin the file it fits for it and then the fits looks good once it has a real value.dlakaplan commentedon May 25, 2026
For the "Diff/Unc" (#2 above): see #1999