This tutorial explains the structure and usage of the pygsti.report.Results
objects which are returned from the high-level algorithm functions (e.g. do_long_sequence_gst
, do_stdpractice_gst
, and do_model_test
).
pygsti.report.Results
¶A Results
object is used to store estimated Model
s for a single DataSet
, along with the associated input parameters which led to each estimate. As a concrete example, we'll explore one of the Results
objects generated by the GST methods tutorial (so if you haven't run this tutorial, go do it now).
from __future__ import print_function
import pygsti
import pickle
results = pickle.load(open('../../tutorial_files/exampleResults.pkl','rb'))
print(results)
---------------------------------------------------------- ---------------- pyGSTi Results Object ------------------- ---------------------------------------------------------- How to access my contents: .dataset -- the DataSet used to generate these results .circuit_lists -- a dict of Circuit lists w/keys: --------------------------------------------------------- iteration final all iteration delta prep fiducials effect fiducials germs .circuit_structs -- a dict of CircuitStructures w/keys: --------------------------------------------------------- iteration final .estimates -- a dictionary of Estimate objects: --------------------------------------------------------- default
As you can see, printing a Results
object gives you a summary of its structure and what you can do with it. The single DataSet
can be accessed via the .dataset
member, and the estimated Model
objects can be found within the pygsti.report.Estimate
objects contained within the .estimates
member. As the summary states, .estimates
is a dictionary of Estimate
objects, and can contain as multiple estimates of the data with the caveat that all of these estimates must use the same circuits, that is, the same Circuit
lists and/or CircuitStructure
s for each algorithm iteration, and the same number of iterations. The reasoning behind this limitation is that when you alter the operation sequences which are used, this alters which data is used, which is similar to altering the data set itself - and we've already drawn the line that each Results
object holds information for just a single data set.
pygsti.report.Estimate
¶The Estimate
objects represent different gauge-unfixed or "up-to-gauge" estimates, and each holds one or more Model
and associated ConfidenceRegion
objects, and dictionaries containing the parameters used to generate the estimate. They also may contain multiple gauge-optimized "versions" of their single gauge-unfixed estimate. They can be printed to display a summary of their contents:
print(results.estimates['default'].goparameters['go0'].keys())
odict_keys(['_gaugeGroupEl', 'comm', 'itemWeights', 'model', 'returnAll', 'targetModel'])
Estimate
objects do not store the operation sequences - rather, since these must be the same for all the estimates of a Results
object, the Results
object holds them separately in its .circuit_lists
and .circuit_structs
members (both of which are dictionaries like .estimates
). Furthermore, since varying the gauge optimization parameters is such a common variation, a single Estimate
may hold multiple dictionaries of gauge-optimization parameters as the elements of its .goparameters
dictionary. The keys of goparameters
will and must correspond to keys within the .models
member (the Model
estimate of that gauge optimization). The .confidence_regions
dictionary (empty in the above example and so not included in the summary) holds ConfidenceRegion
objects, each associated with 1) one the Model
s in .models
, 2) one of the Circuit
lists in the parent Results
object's .circuit_lists
, and 3) a confidence level. The keys of .confidence_regions
are complicated because they must include all three of these associations, and so the .get_confidence_region(...)
method is preferred to directly accessing .confidence_regions
. Different Estimate
objects typically hold estimates for different model parameterizations and/or algorithm parameters.
In our example, results
contains only a single Estimate
called default
(the default estimate label created by do_long_sequence_gst
). This Estimate
contains the raw un-gauge-optimized Model
labeled "final iteration estimate"
, as well as a single gauge-optimized Model
labeled go0
(again, the default created by do_long_sequence_gst
).
Creating your own Results
object is as simple as
pygsti.report.Results()
to create an empty objectinit_dataset
and init_circuits
Estimate
objects by calling add_estimate
with the essential components of a new gauge-unfixed estimate (the target model, the starting model, the estimated models by iteration, and the parameter dictionary).This is demonstrated with dummy parameters below.
my_results = pygsti.objects.Results()
my_results.init_dataset( results.dataset ) #use the same data and strings as our results
my_results.init_circuits( results.circuit_structs['iteration'] )
target_model = results.estimates['default'].models['target']
mdl_initial = target_model.copy()
my_estimate = target_model.depolarize(op_noise=0.01, spam_noise=0.1)
my_estimate_by_iter = [my_estimate]*len(my_results.circuit_structs['iteration'])
my_parameters = {'someParam': 1.0 }
my_results.add_estimate(target_model, mdl_initial, my_estimate_by_iter, my_parameters, estimate_key="myTestEstimate")
print(my_results)
---------------------------------------------------------- ---------------- pyGSTi Results Object ------------------- ---------------------------------------------------------- How to access my contents: .dataset -- the DataSet used to generate these results .circuit_lists -- a dict of Circuit lists w/keys: --------------------------------------------------------- iteration final all iteration delta prep fiducials effect fiducials germs .circuit_structs -- a dict of CircuitStructures w/keys: --------------------------------------------------------- iteration final .estimates -- a dictionary of Estimate objects: --------------------------------------------------------- myTestEstimate
In many circumstances, one may want to perform a new gauge optimization on an existing gauge-unfixed Estimate
, est
, creating a new gauge-optimized Model
to be stored in est
. This is accomplished using the est.add_gaugeoptimized
which lightly wraps a call to pygsti.gaugeopt_to_target
. You specify the arguments to gaugeopt_to_target
as a dictionary to add_gaugeoptimized
, but you're allowed to leave out the first two: the Model
to be optimized, (model
, taken to be the est.models['final iteration estimate']
) and the model to optimize toward (targetModel
, taken to be the est.models['target']
). Note that these arguments can still be specified to override their defaults. In particular, setting targetModel
in the dictionary of parameters allows one to independently specify the model to optimize toward (and this need not be a perfect, ideal model!).
The optional label
argument of add_gaugeoptimized
specifies the key within est.goparameters
and est.models
where the gauge optimization argument dictionary and resulting gauge-optimized Model
will be stored. If the label given already exists, that gauge-optimized estimate is replaced with the new one. If label
is left as None
, then "goX" is used as the label, where X is the next available integer.
If the model
argument of add_gaugeoptimized
is supplied, then this is taken to be the result of the described gauge optimization and no call to gaugeopt_to_target
is made. (In this case, one could simply pass an empty dictionary of as goparams
.)
Below we demonstrate how to add gauge-optimized models to an Estimate
in several ways. Please refer to the previous tutorial on low-level algorithms for an explanation of the various arguments to gaugeopt_to_target
.
est = results.estimates['default']
est.add_gaugeoptimized({'itemWeights': {'gates': 1.0, 'spam': 1.0}}, label="equal_footing")
est.add_gaugeoptimized({'itemWeights': {'gates': 1.0, 'spam': 1.0, 'Gx': 10.0}}, label="Gx_heavy")
mdl_guess = est.models['target'].depolarize(op_noise=0.05, spam_noise=0.02) # a guess at what gates should be...
est.add_gaugeoptimized({'targetModel': mdl_guess, 'itemWeights': {'spam': 0.01}}, label="imperfect gopt target")
print(est)
---------------------------------------------------------- ---------------- pyGSTi Estimate Object ------------------ ---------------------------------------------------------- How to access my contents: .models -- a dictionary of Model objects w/keys: --------------------------------------------------------- target seed iteration estimates final iteration estimate go0 equal_footing Gx_heavy imperfect gopt target .parameters -- a dictionary of simulation parameters: --------------------------------------------------------- objective memLimit starting point profiler minProbClip minProbClipForWeighting probClipInterval radius weights cptpPenaltyFactor spamPenaltyFactor distributeMethod depolarizeStart randomizeStart contractStartToCPTP tolerance maxIterations useFreqWeightedChiSq nestedCircuitLists profile check truncScheme opLabelAliases includeLGST max length list .goparameters -- a dictionary of gauge-optimization parameter dictionaries: --------------------------------------------------------- go0 equal_footing Gx_heavy imperfect gopt target
In order to compute confidence regions and intervals within reports (see later tutorials), an Estimate
object must be equipped with one or more "confidence region factory" objects. These factories are instances of pygsti.objects.ConfidenceRegionFactory
(suprise, suprise). Their purpose is to generate confidence regions and intervals (for any confidence level) for quantities computed from a particular Model
that in turn resulted from optimizaing the likelihood function corresponding to a particular set of circuits. Thus, a confidence region factory has associated with it three things: 1) a Model
, 2) a list of Circuit
s, and 3) a DataSet
. A dictionary of factories is held as the .confidence_region_factories
member of an Estimate
object. Each factory within this dictionary is associated with the one-and-only DataSet
of the Estimate
's parent Results
object, and the associated Model
and Circuit
list are given by the keys of .confidence_region_factories
(model-key, circuit-list-key tuples). Here model-key is the key of a Model
within the Estimate
's .models
member and circuit-list key is the key of a list within the parent Results
object's .circuit_lists
member.
Thankfully, you won't usually need to deal with the .confidence_region_factories
member directly. To create a new factory for a given Model
, Circuit
-list pair you can simply call the add_confidence_region_factory
with the appropriate key labels. Once a factory is created, it must be initialized for computing confidence regions. The only non-experimental way to do this currently is to compute the Hessian of the log-likelihood (often computationally intensive) and then projecting the inverse of this Hessian onto the non-gauge space of the model. These two steps are performed via the compute_hessian
and project_hessian
member functions of a ConfidenceRegionFactory
object.
model_label = "go0"
clist_label = "final"
crfactory = results.estimates['default'].add_confidence_region_factory(model_label, clist_label)
Note that there are different ways of projecting the Hessian which have different strengths and weakenesses. The "optimal gate CIs" method is the most robust method for giving the smallest error bars possible, but it takes significant computation time. The "intrinsic error" method is fast and usually reliable, but may not always give the smallest possible error bars.
crfactory.compute_hessian(comm=None) #could use lots of processor here...
inv_proj_H = crfactory.project_hessian('intrinsic error')
--- Hessian Projector Optimization from separate SPAM and Gate weighting --- Resulting intrinsic errors: 0.00385301 (gates), 0.00222961 (spam) Resulting sqrt(mean(operationCIs**2)): 0.00432743 Resulting sqrt(mean(spamCIs**2)): 0.00415411
Alternate way: In the special case of constructing factories for Model
s which are gauge-equivalent to one another, one can skip the compute_hessian
step for all but the first Model
, so long as the gauge optimization parameters and the final gauge-tranformation element are stored in the Estimate
s .goparameters
dictionary (automatically populated when adding a gauge optimization via add_gaugeoptimized
). Instead, one must gauge-propagate the Hessian from the first Model
to the others using the gauge_propagate_confidence_region_factory
method of the Estimate
object.
Below, we show how this might usually be done: first a confidence region factory for the "final iteration estimate" Model
and 'final' operation sequence list (the defaults) is created and a Hessian is computed. Then, when a factory is needed for the gauge-equivalent Model
"go0", the Hessian is propagated from the "final iteration estimate" Model
. Note that the propagated Hessian must still be projected for the "go0" model.
crfact_final = results.estimates['default'].add_confidence_region_factory() #default 'final iteration estimate'
crfact_final.compute_hessian(comm=None)
results.estimates['default'].gauge_propagate_confidence_region_factory('go0', verbosity=1) #instead of computing one
crfact_go0 = results.estimates['default'].get_confidence_region_factory('go0')
inv_proj_H = crfact_go0.project_hessian('intrinsic error')
*** Propagating Hessian from 'final iteration estimate' to 'go0' *** Column: [##################################################] 100.0% Successfully transported Hessian and ConfidenceRegionFactory. --- Hessian Projector Optimization from separate SPAM and Gate weighting --- Resulting intrinsic errors: 0.00385301 (gates), 0.00222961 (spam) Resulting sqrt(mean(operationCIs**2)): 0.00432743 Resulting sqrt(mean(spamCIs**2)): 0.00415411
In summary, when thinking about Results
and Estimate
objects, remember:
Results
object represents the results for a single set of data (or "effective" set of data as defined by the circuits used).Estimate
object represents a single gauge-unfixed estimate based on the data. An Estimate
may also contain one or more gauge-optimized versions of the gauge-invariant estimate.Estimate
can construct confidence intervals only after a ConfidenceRegionFactory
object is created and initialized using a multi-step process. Because it may be computationally expensive, these steps are not performed automatically when reports are generated.