Results Object Tutorial

This tutorial explains the structure and usage of the pygsti.report.Results objects which are returned from the high-level driver functions (see prior tutorial).

pygsti.report.Results

A Results object is used to store estimated GateSets for a single DataSet, along with the associated input parameters which led to each estimate. As a concrete example, we'll explore one of the Results objects generated by the high-level algorithms tutorial.

In [1]:
from __future__ import print_function
import pygsti
import pickle
/usr/local/lib/python3.6/site-packages/pyGSTi-0.9.4.4-py3.6.egg/pygsti/tools/matrixtools.py:23: UserWarning: Could not import Cython extension - falling back to slower pure-python routines
  _warnings.warn("Could not import Cython extension - falling back to slower pure-python routines")
In [2]:
results = pickle.load(open('tutorial_files/exampleResults.pkl','rb'))
print(results)
---------------------------------------
FileNotFoundErrorTraceback (most recent call last)
<ipython-input-2-afe31d41f232> in <module>()
----> 1 results = pickle.load(open('tutorial_files/exampleResults.pkl','rb'))
      2 print(results)

FileNotFoundError: [Errno 2] No such file or directory: 'tutorial_files/exampleResults.pkl'

As you can see, printing a Results object gives you a summary of its structure and what you can do with it. The single DataSet can be accessed via the .dataset member, and the estimated GateSet objects can be found within the pygsti.report.Estimate objects contained within the .estimates member. As the summary states, .estimates is a dictionary of Estimate objects, and can contain as multiple estimates of the data with the caveat that all of these estimates must use the same gate sequences, that is, the same GateString lists and/or GateStringStructures for each algorithm iteration, and the same number of iterations. The reasoning behind this limitation is that when you alter the gate sequences which are used, this alters which data is used, which is similar to altering the data set itself - and we've already drawn the line that each Results object holds information for just a single data set.

pygsti.report.Estimate

The Estimate objects represent different gauge-unfixed or "up-to-gauge" estimates, and each holds one or more GateSet and associated ConfidenceRegion objects, and dictionaries containing the parameters used to generate the estimate. They also may contain multiple gauge-optimized "versions" of their single gauge-unfixed estimate. They can be printed to display a summary of their contents:

In [3]:
print(results.estimates['default'].goparameters['go0'].keys())
odict_keys(['_gaugeGroupEl', 'gateset', 'itemWeights', 'returnAll', 'targetGateset'])

Estimate objects do not store the gate sequences - rather, since these must be the same for all the estimates of a Results object, the Results object holds them separately in its .gatestring_lists and .gatestring_structs members (both of which are dictionaries like .estimates). Furthermore, since varying the gauge optimization parameters is such a common variation, a single Estimate may hold multiple dictionaries of gauge-optimization parameters as the elements of its .goparameters dictionary. The keys of goparameters will and must correspond to keys within the .gatesets member (the GateSet estimate of that gauge optimization). The .confidence_regions dictionary (empty in the above example and so not included in the summary) holds ConfidenceRegion objects, each associated with 1) one the GateSets in .gatesets, 2) one of the GateString lists in the parent Results object's .gatestring_lists, and 3) a confidence level. The keys of .confidence_regions are complicated because they must include all three of these associations, and so the .get_confidence_region(...) method is preferred to directly accessing .confidence_regions. Different Estimate objects typically hold estimates for different gate set parameterizations and/or algorithm parameters.

In our example, results contains only a single Estimate called default (the default estimate label created by do_long_sequence_gst). This Estimate contains the raw un-gauge-optimized GateSet labeled "final iteration estimate", as well as a single gauge-optimized GateSet labeled go0 (again, the default created by do_long_sequence_gst).

Rolling your own...

Creating your own Results object is as simple as

  1. calling pygsti.report.Results() to create an empty object
  2. initializing the data set and gate sequences via calls to init_dataset and init_gatestrings
  3. creating contained Estimate objects by calling add_estimate with the essential components of a new gauge-unfixed estimate (the target gate set, the starting gate set, the estimated gate sets by iteration, and the parameter dictionary).

This is demonstrated with dummy parameters below.

In [4]:
my_results = pygsti.objects.Results()
my_results.init_dataset( results.dataset ) #use the same data and strings as our results
my_results.init_gatestrings( results.gatestring_structs['iteration'] )

gs_target = results.estimates['default'].gatesets['target']
gs_initial = gs_target.copy()
my_estimate = gs_target.depolarize(gate_noise=0.01, spam_noise=0.1)
my_estimate_by_iter = [my_estimate]*len(my_results.gatestring_structs['iteration'])
my_parameters = {'someParam': 1.0 }
my_results.add_estimate(gs_target, gs_initial, my_estimate_by_iter, my_parameters, estimate_key="myTestEstimate")

print(my_results)
----------------------------------------------------------
---------------- pyGSTi Results Object -------------------
----------------------------------------------------------

How to access my contents:

 .dataset    -- the DataSet used to generate these results

 .gatestring_lists   -- a dict of GateString lists w/keys:
 ---------------------------------------------------------
  iteration
  final
  all
  iteration delta
  prep fiducials
  effect fiducials
  germs

 .gatestring_structs   -- a dict of GatestringStructures w/keys:
 ---------------------------------------------------------
  iteration
  final

 .estimates   -- a dictionary of Estimate objects:
 ---------------------------------------------------------
  myTestEstimate


Creating a new gauge-optimized estimate

In many circumstances, one may want to perform a new gauge optimization on an existing gauge-unfixed Estimate, est, creating a new gauge-optimized GateSet to be stored in est. This is accomplished using the est.add_gaugeoptimized which lightly wraps a call to pygsti.gaugeopt_to_target. You specify the arguments to gaugeopt_to_target as a dictionary to add_gaugeoptimized, but you're allowed to leave out the first two: the GateSet to be optimized, (gateset, taken to be the est.gatesets['final iteration estimate']) and the gate set to optimize toward (targetGateset, taken to be the est.gatesets['target']). Note that these arguments can still be specified to override their defaults. In particular, setting targetGateset in the dictionary of parameters allows one to independently specify the gate set to optimize toward (and this need not be a perfect, ideal gate set!).

The optional label argument of add_gaugeoptimized specifies the key within est.goparameters and est.gatesets where the gauge optimization argument dictionary and resulting gauge-optimized GateSet will be stored. If the label given already exists, that gauge-optimized estimate is replaced with the new one. If label is left as None, then "goX" is used as the label, where X is the next available integer.

If the gateset argument of add_gaugeoptimized is supplied, then this is taken to be the result of the described gauge optimization and no call to gaugeopt_to_target is made. (In this case, one could simply pass an empty dictionary of as goparams.)

Below we demonstrate how to add gauge-optimized gate sets to an Estimate in several ways. Please refer to the previous tutorial on low-level algorithms for an explanation of the various arguments to gaugeopt_to_target.

In [5]:
est = results.estimates['default']
est.add_gaugeoptimized({'itemWeights': {'gates': 1.0, 'spam': 1.0}}, label="equal_footing")
est.add_gaugeoptimized({'itemWeights': {'gates': 1.0, 'spam': 1.0, 'Gx': 10.0}}, label="Gx_heavy")

gs_guess = est.gatesets['target'].depolarize(gate_noise=0.05, spam_noise=0.02) # a guess at what gates should be...
est.add_gaugeoptimized({'targetGateset': gs_guess, 'itemWeights': {'spam': 0.01}}, label="imperfect gopt target")

print(est)
----------------------------------------------------------
---------------- pyGSTi Estimate Object ------------------
----------------------------------------------------------

How to access my contents:

 .gatesets   -- a dictionary of GateSet objects w/keys:
 ---------------------------------------------------------
  target
  seed
  iteration estimates
  final iteration estimate
  go0
  equal_footing
  Gx_heavy
  imperfect gopt target

 .parameters   -- a dictionary of simulation parameters:
 ---------------------------------------------------------
  objective
  memLimit
  starting point
  profiler
  minProbClip
  minProbClipForWeighting
  probClipInterval
  radius
  weights
  cptpPenaltyFactor
  spamPenaltyFactor
  distributeMethod
  depolarizeStart
  contractStartToCPTP
  tolerance
  maxIterations
  useFreqWeightedChiSq
  nestedGateStringLists
  profile
  check
  truncScheme
  gateLabelAliases
  includeLGST
  max length list

 .goparameters   -- a dictionary of gauge-optimization parameter dictionaries:
 ---------------------------------------------------------
  go0
  equal_footing
  Gx_heavy
  imperfect gopt target


Confidence Region Factories

In order to compute confidence regions and intervals within reports (see later tutorials), an Estimate object must be equipped with one or more "confidence region factory" objects. These factories are instances of pygsti.objects.ConfidenceRegionFactory (suprise, suprise). Their purpose is to generate confidence regions and intervals (for any confidence level) for quantities computed from a particular GateSet that in turn resulted from optimizaing the likelihood function corresponding to a particular set of gate sequences. Thus, a confidence region factory has associated with it three things: 1) a GateSet, 2) a list of GateStrings, and 3) a DataSet. A dictionary of factories is held as the .confidence_region_factories member of an Estimate object. Each factory within this dictionary is associated with the one-and-only DataSet of the Estimate's parent Results object, and the associated GateSet and GateString list are given by the keys of .confidence_region_factories (gateset-key, gatestring-list-key tuples). Here gateset-key is the key of a GateSet within the Estimate's .gatesets member and gatestring-list key is the key of a list within the parent Results object's .gatestring_lists member.

Thankfully, you won't usually need to deal with the .confidence_region_factories member directly. To create a new factory for a given GateSet, GateString-list pair you can simply call the add_confidence_region_factory with the appropriate key labels. Once a factory is created, it must be initialized for computing confidence regions. The only non-experimental way to do this currently is to compute the Hessian of the log-likelihood (often computationally intensive) and then projecting the inverse of this Hessian onto the non-gauge space of the gate set. These two steps are performed via the compute_hessian and project_hessian member functions of a ConfidenceRegionFactory object.

In [6]:
gateset_label = "go0"
gslist_label = "final"
crfactory = results.estimates['default'].add_confidence_region_factory(gateset_label, gslist_label)

Note that there are different ways of projecting the Hessian which have different strengths and weakenesses. The "optimal gate CIs" method is the most robust method for giving the smallest error bars possible, but it takes significant computation time. The "intrinsic error" method is fast and usually reliable, but may not always give the smallest possible error bars.

In [7]:
crfactory.compute_hessian(comm=None) #could use lots of processor here...
inv_proj_H = crfactory.project_hessian('intrinsic error')
    
--- Hessian Projector Optimization from separate SPAM and Gate weighting ---
  Resulting intrinsic errors: 0.00381267 (gates), 0.00134802 (spam)
  Resulting sqrt(mean(gateCIs**2)): 0.00454592
  Resulting sqrt(mean(spamCIs**2)): 0.00303855

Alternate way: In the special case of constructing factories for GateSets which are gauge-equivalent to one another, one can skip the compute_hessian step for all but the first GateSet, so long as the gauge optimization parameters and the final gauge-tranformation element are stored in the Estimates .goparameters dictionary (automatically populated when adding a gauge optimization via add_gaugeoptimized). Instead, one must gauge-propagate the Hessian from the first GateSet to the others using the gauge_propagate_confidence_region_factory method of the Estimate object.

Below, we show how this might usually be done: first a confidence region factory for the "final iteration estimate" GateSet and 'final' gate string list (the defaults) is created and a Hessian is computed. Then, when a factory is needed for the gauge-equivalent GateSet "go0", the Hessian is propagated from the "final iteration estimate" GateSet. Note that the propagated Hessian must still be projected for the "go0" gate set.

In [8]:
crfact_final = results.estimates['default'].add_confidence_region_factory() #default 'final iteration estimate'
crfact_final.compute_hessian(comm=None)

results.estimates['default'].gauge_propagate_confidence_region_factory('go0', verbosity=1) #instead of computing one
crfact_go0 = results.estimates['default'].get_confidence_region_factory('go0')
inv_proj_H = crfact_go0.project_hessian('intrinsic error')
 *** Propagating Hessian from 'final iteration estimate' to 'go0' ***
Column:  [##################################################] 100.0% 
   Successfully transported Hessian and ConfidenceRegionFactory.
    
--- Hessian Projector Optimization from separate SPAM and Gate weighting ---
  Resulting intrinsic errors: 0.00381267 (gates), 0.00134802 (spam)
  Resulting sqrt(mean(gateCIs**2)): 0.00454592
  Resulting sqrt(mean(spamCIs**2)): 0.00303855

Summary

In summary, when thinking about Results and Estimate objects, remember:

  • each Results object represents the results for a single set of data (or "effective" set of data as defined by the sequences used).
  • each contained Estimate object represents a single gauge-unfixed estimate based on the data. An Estimate may also contain one or more gauge-optimized versions of the gauge-invariant estimate.
  • an Estimate can construct confidence intervals only after a ConfidenceRegionFactory object is created and initialized using a multi-step process. Because it may be computationally expensive, these steps are not performed automatically when reports are generated.
In [ ]: