'LIKELIHOOD AND MINIMIZATION' RooFit tutorial macro #606
Understanding and customizing error handling in likelihood evaluations
Author: Clemens Lange, Wouter Verkerke (C version)
This notebook tutorial was automatically generated with ROOTBOOK-izer from the macro found in the ROOT repository on Monday, May 13, 2024 at 11:36 AM.
import ROOT
Observable
m = ROOT.RooRealVar("m", "m", 5.20, 5.30)
Parameters
m0 = ROOT.RooRealVar("m0", "m0", 5.291, 5.20, 5.30)
k = ROOT.RooRealVar("k", "k", -30, -50, -10)
argus = ROOT.RooArgusBG("argus", "argus", m, m0, k)
Sample 1000 events in m from argus
data = argus.generate({m}, 1000)
frame1 = m.frame(Bins=40, Title="Argus model and data")
data.plotOn(frame1)
argus.plotOn(frame1)
<cppyy.gbl.RooPlot object at 0xaf1f700>
The ARGUS background shape has a sharp kinematic cutoff at m=m0 and is prone to evaluation errors if the cutoff parameter m0 is floated: when the pdf cutoff value is lower than that in data events with m>m0 will have zero probability
Perform unbinned ML fit. Print detailed error messages for up to 10 events per likelihood evaluation. The default error handling strategy is to return a very high value of the likelihood to MINUIT if errors occur, which will force MINUIT to retreat from the problematic area
argus.fitTo(data, PrintEvalErrors=10)
<cppyy.gbl.RooFitResult object at 0x(nil)>
[#1] INFO:Fitting -- RooAbsPdf::fitTo(argus_over_argus_Int[m]) fixing normalization set for coefficient determination to observables in data [#1] INFO:Fitting -- using CPU computation library compiled with -mavx2 [#1] INFO:Fitting -- RooAddition::defaultErrorLevel(nll_argus_over_argus_Int[m]_argusData) Summation contains a RooNLLVar, using its error level [#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: activating const optimization Minuit2Minimizer: Minimize with max-calls 1000 convergence for edm < 1 strategy 1 RooAbsMinimizerFcn: Minimized function has error status. Returning maximum FCN so far (-2417.08) to force MIGRAD to back out of this region. Error log follows. Parameter values: k=-33.8075 m0=5.29014 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] getLogVal() top-level p.d.f not greater than zero @ numerator=argus=0.387202, denominator=argus_Int[m]=0.0305662 RooAbsMinimizerFcn: Minimized function has error status. Returning maximum FCN so far (-2417.08) to force MIGRAD to back out of this region. Error log follows. Parameter values: k=-36.7074 m0=5.2901 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] getLogVal() top-level p.d.f not greater than zero @ numerator=argus=0.370612, denominator=argus_Int[m]=0.0290505 Minuit2Minimizer : Valid minimum - status = 0 FVAL = -2419.30692128725559 Edm = 2.90497786168460102e-06 Nfcn = 47 k = -35.3713 +/- 3.51942 (limited) m0 = 5.2904 +/- 0.000261877 (limited) [#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: deactivating const optimization
Info in <Minuit2>: MnSeedGenerator Computing seed using NumericalGradient calculator Info in <Minuit2>: MnSeedGenerator Initial state: FCN = -2417.48203 Edm = 1.59436124 NCalls = 7 Info in <Minuit2>: MnSeedGenerator Initial state Minimum value : -2417.48203 Edm : 1.59436124 Internal parameters: [ 0 0.9614110188] Internal gradient : [ 6.430040774 66.49119995] Internal covariance matrix: [[ 0.059577243 0] [ 0 0.00088534902]]] Info in <Minuit2>: VariableMetricBuilder Start iterating until Edm is < 0.001 with call limit = 1000 Info in <Minuit2>: VariableMetricBuilder 0 - FCN = -2417.48203 Edm = 1.59436124 NCalls = 7 Info in <Minuit2>: VariableMetricBuilder 1 - FCN = -2418.669621 Edm = 0.6597753344 NCalls = 15 Info in <Minuit2>: VariableMetricBuilder 2 - FCN = -2419.092375 Edm = 0.3262165039 NCalls = 23 Info in <Minuit2>: VariableMetricBuilder 3 - FCN = -2419.306527 Edm = 0.001182979929 NCalls = 29 Info in <Minuit2>: VariableMetricBuilder 4 - FCN = -2419.306921 Edm = 3.048074013e-06 NCalls = 35 Info in <Minuit2>: VariableMetricBuilder After Hessian Info in <Minuit2>: VariableMetricBuilder 5 - FCN = -2419.306921 Edm = 2.904977862e-06 NCalls = 47 Info in <Minuit2>: Minuit2Minimizer::Hesse Using max-calls 1000 Info in <Minuit2>: Minuit2Minimizer::Hesse Hesse is valid - matrix is accurate
Perform another fit. In self configuration only the number of errors per likelihood evaluation is shown, it is greater than zero. The EvalErrorWall(kFALSE) arguments disables the default error handling strategy and will cause the actual (problematic) value of the likelihood to be passed to MINUIT.
NB: Use of self option is NOT recommended as default strategt as broken -log(L) values can often be lower than 'good' ones because offending events are removed. ROOT.This may effectively create a False minimum in problem areas. ROOT.This is clearly illustrated in the second plot
m0.setError(0.1)
argus.fitTo(data, PrintEvalErrors=0, EvalErrorWall=False)
<cppyy.gbl.RooFitResult object at 0x(nil)>
[#1] INFO:Fitting -- RooAbsPdf::fitTo(argus_over_argus_Int[m]) fixing normalization set for coefficient determination to observables in data [#1] INFO:Fitting -- RooAddition::defaultErrorLevel(nll_argus_over_argus_Int[m]_argusData) Summation contains a RooNLLVar, using its error level [#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: activating const optimization Minuit2Minimizer: Minimize with max-calls 1000 convergence for edm < 1 strategy 1 RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=5.28877 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 4 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=nan m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=nan RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 2000 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=5.27133 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 166 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=5.28726 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 9 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=5.2898 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 1 errors Minuit2Minimizer : Invalid minimum - status = 3 FVAL = -2419.31 Edm = -nan Nfcn = 34 RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=5.2898 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 1 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=5.28374 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 44 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=5.27133 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 166 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=5.28726 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 9 errors RooAbsMinimizerFcn: Minimized function has error status but is ignored. Parameter values: k=-35.3713 m0=5.2898 RooAbsPdf::argus_over_argus_Int[m][ numerator=argus denominator=argus_Int[m] ] has 1 errors [#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: deactivating const optimization
Info in <Minuit2>: MnSeedGenerator Computing seed using NumericalGradient calculator Info in <Minuit2>: MnSeedGenerator Initial state: FCN = -2419.306921 Edm = nan NCalls = 9 Info in <Minuit2>: MnSeedGenerator Initial state Minimum value : -2419.306921 Edm : nan Internal parameters: [ -0.2719048895 0.940796957] Internal gradient : [ -0.0131804372 nan] Internal covariance matrix: [[ 0.065014457 0] [ 0 2]]] Info in <Minuit2>: VariableMetricBuilder Start iterating until Edm is < 0.001 with call limit = 1000 Info in <Minuit2>: VariableMetricBuilder 0 - FCN = -2419.306921 Edm = nan NCalls = 9 Warning in <Minuit2>: VariableMetricBuilder No improvement in line search Info in <Minuit2>: VariableMetricBuilder 1 - FCN = -2419.306921 Edm = nan NCalls = 20 Warning in <Minuit2>: MnPosDef Matrix forced pos-def by adding to diagonal nan Info in <Minuit2>: VariableMetricBuilder After Hessian Info in <Minuit2>: VariableMetricBuilder 2 - FCN = -2419.306921 Edm = -nan NCalls = 34 Info in <Minuit2>: VariableMetricBuilder Edm has been re-computed after Hesse; Edm -nan is now within tolerance Warning in <Minuit2>: Minuit2Minimizer::Minimize Minimization did NOT converge, Edm is above max Info in <Minuit2>: Minuit2Minimizer::Hesse Using max-calls 1000 Warning in <Minuit2>: MnPosDef Matrix forced pos-def by adding to diagonal nan Info in <Minuit2>: Minuit2Minimizer::Hesse Hesse is valid - matrix is full but made positive defined
Construct likelihood function of model and data
nll = argus.createNLL(data)
[#1] INFO:Fitting -- RooAbsPdf::fitTo(argus_over_argus_Int[m]) fixing normalization set for coefficient determination to observables in data
Plot likelihood in m0 in range that includes problematic values In self configuration no messages are printed for likelihood evaluation errors, but if an likelihood value evaluates with error, corresponding value on the curve will be set to the value given in EvalErrorValue().
frame2 = m0.frame(Range=(5.288, 5.293), Title="-log(L) scan vs m0, regions masked")
nll.plotOn(frame2, ShiftToZero=True, PrintEvalErrors=-1, EvalErrorValue=(nll.getVal() + 10), LineColor="r")
frame2.SetMaximum(15)
frame2.SetMinimum(0)
c = ROOT.TCanvas("rf606_nllerrorhandling", "rf606_nllerrorhandling", 1200, 400)
c.Divide(2)
c.cd(1)
ROOT.gPad.SetLeftMargin(0.15)
frame1.GetYaxis().SetTitleOffset(1.4)
frame1.Draw()
c.cd(2)
ROOT.gPad.SetLeftMargin(0.15)
frame2.GetYaxis().SetTitleOffset(1.4)
frame2.Draw()
c.SaveAs("rf606_nllerrorhandling.png")
Info in <TCanvas::Print>: png file rf606_nllerrorhandling.png has been created
Draw all canvases
from ROOT import gROOT
gROOT.GetListOfCanvases().Draw()