PyGSTi is able to construct polished report documents, which provide high-level summaries as well as detailed analyses of GST results. Reports are intended to be quick and easy way of analyzing a GST estimate, and pyGSTi's report generation functions are specifically designed to interact with its high-level driver functions (see the high-level algorithms tutorial). Currently there is only a single report generation function, pygsti.report.create_general_report
, which takes one or more Results
objects as input and produces an HTML file as output. The HTML format allows the reports to include interactive plots and switches, making it easy to compare different types of analysis or data sets.
PyGSTi's "general" report creates a stand-alone HTML document which cannot run Python. Thus, all the results displayed in the report must be pre-computed (in Python). If you find yourself wanting to fiddle with things and feel that the general report is too static, please consider using a Workspace
object (see following tutorials) within a Jupyter notebook, where you can intermix report tables/plots and Python. Internally, create_general_report
is just a canned routine which uses a WorkSpace
object to generate various tables and plots and then inserts them into a HTML template.
Note to veteran users: PyGSTi has recently transitioned to producing HTML (rather than LaTeX/PDF) reports. The way to generate such report is largely unchanged, with one important exception. Previously, the Results
object had various report-generation methods included within it. We've found this is too restrictive, as we'd sometimes like to generate a report which utilizes the results from multiple runs of GST (to compare them, for instance). Thus, the Results
class is now just a container for a DataSet
and its related GateSet
s, GatestringStructure
s, etc. All of the report-generation capability is now housed in within separate report functions, which we now demonstrate.
Results
¶We start by performing GST using do_long_sequence_gst
, as usual, to create a Results
object (we could also have just loaded one from file).
import pygsti
from pygsti.construction import std1Q_XYI
gs_target = std1Q_XYI.gs_target
fiducials = std1Q_XYI.fiducials
germs = std1Q_XYI.germs
maxLengths = [1,2,4,8,16]
ds = pygsti.io.load_dataset("tutorial_files/Example_Dataset.txt", cache=True)
#Run GST
gs_target.set_all_parameterizations("TP") #TP-constrained
results = pygsti.do_long_sequence_gst(ds, gs_target, fiducials, fiducials, germs,
maxLengths, verbosity=3)
Loading from cache file: tutorial_files/Example_Dataset.txt.cache --- Gate Sequence Creation --- 1282 sequences created Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing --- LGST --- Singular values of I_tilde (truncating to first 4 of 6) = 4.245030583357432 1.1797105733752995 0.9564978918311123 0.9423535266759971 0.04708902142849772 0.015314932955168468 Singular values of target I_tilde (truncating to first 4 of 6) = 4.242640687119284 1.4142135623730958 1.4142135623730947 1.4142135623730945 2.31214371751647e-16 4.4245234371241965e-17 Resulting gate set: rho0 = 0.7071 -0.0302 0.0396 0.7480 Mdefault = TP-POVM with effect vectors: 0: 0.73 0 0 0.65 1: 0.69 0 0 -0.65 Gi = 1.0000 0 0 0 0.0094 0.9238 0.0542 -0.0155 0.0285 -0.0149 0.9021 0.0200 -0.0142 0.0280 0.0009 0.9057 Gx = 1.0000 0 0 0 0.0064 0.9053 0.0281 -0.0044 -0.0006 0.0215 -0.0471 -0.9983 -0.0692 -0.0056 0.8095 0.0090 Gy = 1.0000 0 0 0 -0.0152 -0.0245 0.0379 0.9906 0.0076 -0.0126 0.8876 -0.0257 -0.0771 -0.8084 -0.0476 0.0210 --- Iterative MLGST: Iter 1 of 5 92 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 62.7837, mu=0, |J|=1362.27 --- Outer Iter 1: norm_f = 42.2737, mu=172.193, |J|=4003.22 --- Outer Iter 2: norm_f = 41.0782, mu=57.3976, |J|=4002.28 --- Outer Iter 3: norm_f = 41.0771, mu=19.1325, |J|=4002.28 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Sum of Chi^2 = 41.0771 (92 data params - 31 model params = expected mean of 61; p-value = 0.976519) Completed in 0.2s 2*Delta(log(L)) = 41.2329 Iteration 1 took 0.3s --- Iterative MLGST: Iter 2 of 5 168 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 175.573, mu=0, |J|=4118.16 --- Outer Iter 1: norm_f = 124.232, mu=1805.46, |J|=4115.29 --- Outer Iter 2: norm_f = 119.708, mu=601.819, |J|=4114.91 --- Outer Iter 3: norm_f = 119.299, mu=200.606, |J|=4114.85 --- Outer Iter 4: norm_f = 119.288, mu=66.8688, |J|=4114.84 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Sum of Chi^2 = 119.288 (168 data params - 31 model params = expected mean of 137; p-value = 0.859758) Completed in 0.3s 2*Delta(log(L)) = 119.601 Iteration 2 took 0.5s --- Iterative MLGST: Iter 3 of 5 450 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 498.161, mu=0, |J|=4505.2 --- Outer Iter 1: norm_f = 416.334, mu=2013.07, |J|=4507.21 --- Outer Iter 2: norm_f = 415.465, mu=671.024, |J|=4506.99 --- Outer Iter 3: norm_f = 415.46, mu=223.675, |J|=4506.99 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Sum of Chi^2 = 415.46 (450 data params - 31 model params = expected mean of 419; p-value = 0.539658) Completed in 0.5s 2*Delta(log(L)) = 415.96 Iteration 3 took 0.8s --- Iterative MLGST: Iter 4 of 5 862 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 860.476, mu=0, |J|=5095.16 --- Outer Iter 1: norm_f = 814.571, mu=2303.12, |J|=5078.86 --- Outer Iter 2: norm_f = 814.34, mu=767.708, |J|=5078.57 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Sum of Chi^2 = 814.34 (862 data params - 31 model params = expected mean of 831; p-value = 0.65359) Completed in 0.6s 2*Delta(log(L)) = 815.742 Iteration 4 took 1.1s --- Iterative MLGST: Iter 5 of 5 1282 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 1265.58, mu=0, |J|=5735.19 --- Outer Iter 1: norm_f = 1252.76, mu=2582.97, |J|=5736.35 --- Outer Iter 2: norm_f = 1252.69, mu=860.99, |J|=5736.93 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Sum of Chi^2 = 1252.69 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.481202) Completed in 0.9s 2*Delta(log(L)) = 1254.49 Iteration 5 took 1.7s Switching to ML objective (last iteration) --- MLGST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 627.245, mu=0, |J|=3702.41 --- Outer Iter 1: norm_f = 627.238, mu=3782.07, |J|=3378.33 --- Outer Iter 2: norm_f = 627.234, mu=4.13103e+07, |J|=3299.12 --- Outer Iter 3: norm_f = 627.225, mu=2.26458e+07, |J|=3086.64 --- Outer Iter 4: norm_f = 627.218, mu=2.09848e+07, |J|=3021.71 --- Outer Iter 5: norm_f = 627.217, mu=2.05917e+07, |J|=3128.49 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Maximum log(L) = 627.217 below upper bound of -2.13594e+06 2*Delta(log(L)) = 1254.43 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.467364) Completed in 1.8s 2*Delta(log(L)) = 1254.43 Final MLGST took 1.8s Iterative MLGST Total Time: 6.1s -- Adding Gauge Optimized (go0) --
Now that we have results
, we use the create_standard_report
method within pygsti.report.factory
to generate a report. If the given filename ends in ".pdf
" then a PDF-format report is generated; otherwise the file name specifies a folder that will be filled with HTML pages. To open a HTML-format report, you open the main.html
file directly inside the report's folder. Setting auto_open=True
makes the finished report open in your web browser automatically.
#HTML
pygsti.report.create_standard_report(results, "tutorial_files/exampleReport",
title="GST Example Report", verbosity=1, auto_open=True)
print("\n")
#PDF
pygsti.report.create_standard_report(results, "tutorial_files/exampleReport.pdf",
title="GST Example Report", verbosity=1, auto_open=True)
*** Creating workspace *** *** Generating switchboard *** Found standard clifford compilation from std1Q_XYI *** Generating tables *** *** Generating plots *** *** Merging into template file *** Output written to tutorial_files/exampleReport directory Opening tutorial_files/exampleReport/main.html... *** Report Generation Complete! Total time 37.8488s *** *** Creating workspace *** *** Generating switchboard *** Found standard clifford compilation from std1Q_XYI *** Generating tables *** *** Generating plots *** *** Merging into template file ***
/usr/local/lib/python3.6/site-packages/matplotlib/cbook/deprecation.py:107: MatplotlibDeprecationWarning: Passing one of 'on', 'true', 'off', 'false' as a boolean is deprecated; use an actual boolean (True/False) instead.
Latex file(s) successfully generated. Attempting to compile with pdflatex...
------------ FileNotFoundErrorTraceback (most recent call last) <ipython-input-2-3cc7d2a54f4e> in <module>() 7 #PDF 8 pygsti.report.create_standard_report(results, "tutorial_files/exampleReport.pdf", ----> 9 title="GST Example Report", verbosity=1, auto_open=True) ~/Projects/pyGSTi/packages/pygsti/report/factory.py in create_standard_report(results, filename, title, confidenceLevel, comm, ws, auto_open, link_to, brevity, advancedOptions, verbosity) 1042 assert(cmd), "Cannot render PDF documents: no `latex_cmd` render option." 1043 printer.log("Latex file(s) successfully generated. Attempting to compile with %s..." % cmd) -> 1044 _merge.compile_latex_report(base, [cmd] + flags, printer, auto_open) 1045 else: 1046 raise ValueError("Unrecognized format: %s" % fmt) ~/Projects/pyGSTi/packages/pygsti/report/merge_helpers.py in compile_latex_report(report_filename, latex_call, printer, auto_open) 804 try: 805 #Run latex --> 806 stdout, stderr, returncode = process_call(call) 807 evaluate_call(call, stdout, stderr, returncode, printer) 808 printer.log("Initial output PDF %s successfully generated." % ~/Projects/pyGSTi/packages/pygsti/report/merge_helpers.py in process_call(call) 675 """ 676 process = _subprocess.Popen(call, stdout=_subprocess.PIPE, --> 677 stderr=_subprocess.PIPE) 678 stdout, stderr = process.communicate() 679 return stdout, stderr, process.returncode /usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors) 707 c2pread, c2pwrite, 708 errread, errwrite, --> 709 restore_signals, start_new_session) 710 except: 711 # Cleanup if the child failed starting. /usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session) 1342 if errno_num == errno.ENOENT: 1343 err_msg += ': ' + repr(err_filename) -> 1344 raise child_exception_type(errno_num, err_msg, err_filename) 1345 raise child_exception_type(err_msg) 1346 FileNotFoundError: [Errno 2] No such file or directory: 'pdflatex': 'pdflatex'
There are several remarks about these reports worth noting:
Results
objects that have multiple estimates and/or gauge optimizations, consider using the Results
object's view
method to single out the estimate and gauge optimization you're after.pdflatex
on your system to compile PDF reports.Next, let's analyze the same data two different ways: with and without the TP-constraint (i.e. whether the gates must be trace-preserving) and furthermore gauge optmimize each case using several different SPAM-weights. In each case we'll call do_long_sequence_gst
with gaugeOptParams=False
, so that no gauge optimization is done, and then perform several gauge optimizations separately and add these to the Results
object via its add_gaugeoptimized
function.
#Case1: TP-constrained GST
tpTarget = gs_target.copy()
tpTarget.set_all_parameterizations("TP")
results_tp = pygsti.do_long_sequence_gst(ds, tpTarget, fiducials, fiducials, germs,
maxLengths, gaugeOptParams=False, verbosity=1)
#Gauge optimize
est = results_tp.estimates['default']
gsFinal = est.gatesets['final iteration estimate']
gsTarget = est.gatesets['target']
for spamWt in [1e-4,1e-2,1.0]:
gs = pygsti.gaugeopt_to_target(gsFinal,gsTarget,{'gates':1, 'spam':spamWt})
est.add_gaugeoptimized({'itemWeights': {'gates':1, 'spam':spamWt}}, gs, "Spam %g" % spamWt)
#Case2: "Full" GST
fullTarget = gs_target.copy()
fullTarget.set_all_parameterizations("full")
results_full = pygsti.do_long_sequence_gst(ds, fullTarget, fiducials, fiducials, germs,
maxLengths, gaugeOptParams=False, verbosity=1)
#Gauge optimize
est = results_full.estimates['default']
gsFinal = est.gatesets['final iteration estimate']
gsTarget = est.gatesets['target']
for spamWt in [1e-4,1e-2,1.0]:
gs = pygsti.gaugeopt_to_target(gsFinal,gsTarget,{'gates':1, 'spam':spamWt})
est.add_gaugeoptimized({'itemWeights': {'gates':1, 'spam':spamWt}}, gs, "Spam %g" % spamWt)
We'll now call the same create_standard_report
function but this time instead of passing a single Results
object as the first argument we'll pass a dictionary of them. This will result in a HTML report that includes switches to select which case ("TP" or "Full") as well as which gauge optimization to display output quantities for. PDF reports cannot support this interactivity, and so if you try to generate a PDF report you'll get an error.
ws = pygsti.report.create_standard_report({'TP': results_tp, "Full": results_full},
"tutorial_files/exampleMultiEstimateReport",
title="Example Multi-Estimate Report",
verbosity=2, auto_open=True)
In the above call we capture the return value in the variable ws
- a Workspace
object. PyGSTi's Workspace
objects function as both a factory for figures and tables as well as a smart cache for computed values. Within create_standard_report
a Workspace
object is created and used to create all the figures in the report. As an intended side effect, each of these figures is cached, along with some of the intermediate results used to create it. As we'll see below, a Workspace
can also be specified as input to create_standard_report
, allowing it to utilize previously cached quantities.
Note to veteran users: Other report formats such as beamer
-class PDF presentation and Powerpoint presentation have been dropped from pyGSTi. These presentation formats were rarely used and moreover we feel that the HTML format is able to provide all of the functionality that was present in these discontinued formats.
Another way: Because both results_tp
and results_full
above used the same dataset and gate sequences, we could have combined them as two estimates in a single Results
object (see the previous tutorial on pyGSTi's Results
object). This can be done by renaming at least one of the "default"
-named estimates in results_tp
or results_full
(below we rename both) and then adding the estimate within results_full
to the estimates already contained in results_tp
:
results_tp.rename_estimate('default','TP')
results_full.rename_estimate('default','Full')
results_both = results_tp.copy() #copy just for neatness
results_both.add_estimates(results_full, estimatesToAdd=['Full'])
Creating a report using results_both
will result in the same report we just generated. We'll demonstrate this anyway, but in addition we'll supply create_standard_report
a ws
argument, which tells it to use any cached values contained in a given input Workspace
to expedite report generation. Since our workspace object has the exact quantities we need cached in it, you'll notice a significant speedup. Finally, note that even though there's just a single Results
object, you still can't generate a PDF report from it because it contains multiple estimates.
pygsti.report.create_standard_report(results_both,
"tutorial_files/exampleMultiEstimateReport2",
title="Example Multi-Estimate Report (v2)",
verbosity=2, auto_open=True, ws=ws)
do_stdpractice_gst
¶It's no coincidence that a Results
object containing multiple estimates using the same data is precisely what's returned from do_stdpractice_gst
(see docstring for information on its arguments). This allows one to run GST multiple times, creating several different "standard" estimates and gauge optimizations, and plot them all in a single (HTML) report.
results_std = pygsti.do_stdpractice_gst(ds, gs_target, fiducials, fiducials, germs,
maxLengths, verbosity=4, modes="TP,CPTP,Target",
gaugeOptSuite=('single','toggleValidSpam'))
# Generate a report with "TP", "CPTP", and "Target" estimates
pygsti.report.create_standard_report(results_std, "tutorial_files/exampleStdReport",
title="Post StdPractice Report", auto_open=True,
verbosity=1)
-- Std Practice: Iter 1 of 3 (TP) --: --- Gate Sequence Creation --- 1282 sequences created Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing --- LGST --- Singular values of I_tilde (truncating to first 4 of 6) = 4.245030583357432 1.1797105733752995 0.9564978918311123 0.9423535266759971 0.04708902142849772 0.015314932955168468 Singular values of target I_tilde (truncating to first 4 of 6) = 4.242640687119284 1.4142135623730958 1.4142135623730947 1.4142135623730945 2.31214371751647e-16 4.4245234371241965e-17 Resulting gate set: rho0 = 0.7071 -0.0302 0.0396 0.7480 Mdefault = TP-POVM with effect vectors: 0: 0.73 0 0 0.65 1: 0.69 0 0 -0.65 Gi = 1.0000 0 0 0 0.0094 0.9238 0.0542 -0.0155 0.0285 -0.0149 0.9021 0.0200 -0.0142 0.0280 0.0009 0.9057 Gx = 1.0000 0 0 0 0.0064 0.9053 0.0281 -0.0044 -0.0006 0.0215 -0.0471 -0.9983 -0.0692 -0.0056 0.8095 0.0090 Gy = 1.0000 0 0 0 -0.0152 -0.0245 0.0379 0.9906 0.0076 -0.0126 0.8876 -0.0257 -0.0771 -0.8084 -0.0476 0.0210 --- Iterative MLGST: Iter 1 of 5 92 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 62.7837, mu=0, |J|=1362.27 --- Outer Iter 1: norm_f = 42.2737, mu=172.193, |J|=4003.22 --- Outer Iter 2: norm_f = 41.0782, mu=57.3976, |J|=4002.28 --- Outer Iter 3: norm_f = 41.0771, mu=19.1325, |J|=4002.28 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Sum of Chi^2 = 41.0771 (92 data params - 31 model params = expected mean of 61; p-value = 0.976519) Completed in 0.2s 2*Delta(log(L)) = 41.2329 Iteration 1 took 0.3s --- Iterative MLGST: Iter 2 of 5 168 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 175.573, mu=0, |J|=4118.16 --- Outer Iter 1: norm_f = 124.232, mu=1805.46, |J|=4115.29 --- Outer Iter 2: norm_f = 119.708, mu=601.819, |J|=4114.91 --- Outer Iter 3: norm_f = 119.299, mu=200.606, |J|=4114.85 --- Outer Iter 4: norm_f = 119.288, mu=66.8688, |J|=4114.84 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Sum of Chi^2 = 119.288 (168 data params - 31 model params = expected mean of 137; p-value = 0.859758) Completed in 0.3s 2*Delta(log(L)) = 119.601 Iteration 2 took 0.4s --- Iterative MLGST: Iter 3 of 5 450 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 498.161, mu=0, |J|=4505.2 --- Outer Iter 1: norm_f = 416.334, mu=2013.07, |J|=4507.21 --- Outer Iter 2: norm_f = 415.465, mu=671.024, |J|=4506.99 --- Outer Iter 3: norm_f = 415.46, mu=223.675, |J|=4506.99 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Sum of Chi^2 = 415.46 (450 data params - 31 model params = expected mean of 419; p-value = 0.539658) Completed in 0.5s 2*Delta(log(L)) = 415.96 Iteration 3 took 0.8s --- Iterative MLGST: Iter 4 of 5 862 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 860.476, mu=0, |J|=5095.16 --- Outer Iter 1: norm_f = 814.571, mu=2303.12, |J|=5078.86 --- Outer Iter 2: norm_f = 814.34, mu=767.708, |J|=5078.57 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Sum of Chi^2 = 814.34 (862 data params - 31 model params = expected mean of 831; p-value = 0.65359) Completed in 0.7s 2*Delta(log(L)) = 815.742 Iteration 4 took 1.2s --- Iterative MLGST: Iter 5 of 5 1282 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 1265.58, mu=0, |J|=5735.19 --- Outer Iter 1: norm_f = 1252.76, mu=2582.97, |J|=5736.35 --- Outer Iter 2: norm_f = 1252.69, mu=860.99, |J|=5736.93 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Sum of Chi^2 = 1252.69 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.481202) Completed in 1.0s 2*Delta(log(L)) = 1254.49 Iteration 5 took 1.8s Switching to ML objective (last iteration) --- MLGST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 627.245, mu=0, |J|=3702.41 --- Outer Iter 1: norm_f = 627.238, mu=3782.07, |J|=3378.33 --- Outer Iter 2: norm_f = 627.234, mu=4.13103e+07, |J|=3299.12 --- Outer Iter 3: norm_f = 627.225, mu=2.26458e+07, |J|=3086.64 --- Outer Iter 4: norm_f = 627.218, mu=2.09848e+07, |J|=3021.71 --- Outer Iter 5: norm_f = 627.217, mu=2.05917e+07, |J|=3128.49 Least squares message = Both actual and predicted relative reductions in the sum of squares are at most 1e-06 Maximum log(L) = 627.217 below upper bound of -2.13594e+06 2*Delta(log(L)) = 1254.43 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.467364) Completed in 1.9s 2*Delta(log(L)) = 1254.43 Final MLGST took 1.9s Iterative MLGST Total Time: 6.3s -- Performing 'single' gauge optimization on TP estimate -- -- Adding Gauge Optimized (single) -- -- Performing 'Spam 0.001' gauge optimization on TP estimate -- -- Adding Gauge Optimized (Spam 0.001) -- -- Performing 'Spam 0.001+v' gauge optimization on TP estimate -- -- Adding Gauge Optimized (Spam 0.001+v) -- -- Std Practice: Iter 2 of 3 (CPTP) --: --- Gate Sequence Creation --- 1282 sequences created Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing --- Iterative MLGST: Iter 1 of 5 92 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 1.10824e+07, mu=0, |J|=21595.7 --- Outer Iter 1: norm_f = 1055.69, mu=51712.5, |J|=1014.07 --- Outer Iter 2: norm_f = 906.343, mu=17237.5, |J|=958.022 --- Outer Iter 3: norm_f = 890.808, mu=5745.83, |J|=941.716 --- Outer Iter 4: norm_f = 887.155, mu=1915.28, |J|=940.07 --- Outer Iter 5: norm_f = 886.12, mu=638.426, |J|=940.129 --- Outer Iter 6: norm_f = 886.031, mu=212.809, |J|=940.16 --- Outer Iter 7: norm_f = 885.532, mu=70.9362, |J|=940.053 --- Outer Iter 8: norm_f = 829.351, mu=189.163, |J|=926.197 --- Outer Iter 9: norm_f = 639.664, mu=504.435, |J|=905.287 --- Outer Iter 10: norm_f = 468.911, mu=1020.24, |J|=863.757 --- Outer Iter 11: norm_f = 144.284, mu=340.081, |J|=1014.76 --- Outer Iter 12: norm_f = 134.688, mu=113.36, |J|=982.031 --- Outer Iter 13: norm_f = 133.055, mu=113.81, |J|=968.439 --- Outer Iter 14: norm_f = 127.127, mu=95.4437, |J|=966.489 --- Outer Iter 15: norm_f = 101.018, mu=63.6291, |J|=951.557 --- Outer Iter 16: norm_f = 79.1423, mu=57.8456, |J|=949.601 --- Outer Iter 17: norm_f = 55.2669, mu=19744.6, |J|=3971.7 --- Outer Iter 18: norm_f = 53.4873, mu=8123.49, |J|=3974.72 --- Outer Iter 19: norm_f = 52.6336, mu=5158.47, |J|=3975.96 --- Outer Iter 20: norm_f = 51.6616, mu=2895.56, |J|=3977.28 --- Outer Iter 21: norm_f = 50.1915, mu=965.188, |J|=3979 --- Outer Iter 22: norm_f = 47.4829, mu=321.729, |J|=3981.8 --- Outer Iter 23: norm_f = 44.9263, mu=107.243, |J|=3985.99 --- Outer Iter 24: norm_f = 43.5176, mu=35.7477, |J|=3992.24 --- Outer Iter 25: norm_f = 43.1156, mu=31.473, |J|=1802.81 --- Outer Iter 26: norm_f = 41.9408, mu=10.491, |J|=4017.9 --- Outer Iter 27: norm_f = 41.2585, mu=8.25944, |J|=4022.01 --- Outer Iter 28: norm_f = 41.0862, mu=2.75315, |J|=4023.48 --- Outer Iter 29: norm_f = 41.0859, mu=4.98069, |J|=4025.25 --- Outer Iter 30: norm_f = 41.0858, mu=9.8871, |J|=4027.21 Least squares message = Relative change in |x| is at most 1e-08 Sum of Chi^2 = 41.1114 (92 data params - 31 model params = expected mean of 61; p-value = 0.976293) Completed in 0.9s 2*Delta(log(L)) = 44.6096 Iteration 1 took 0.9s --- Iterative MLGST: Iter 2 of 5 168 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 179.784, mu=0, |J|=4102.96 --- Outer Iter 1: norm_f = 138.536, mu=2285.72, |J|=4101.71 --- Outer Iter 2: norm_f = 125.246, mu=761.907, |J|=4104.28 --- Outer Iter 3: norm_f = 120.441, mu=253.969, |J|=4105.63 --- Outer Iter 4: norm_f = 119.465, mu=84.6563, |J|=4106.02 --- Outer Iter 5: norm_f = 119.322, mu=62.5071, |J|=4106.74 --- Outer Iter 6: norm_f = 119.317, mu=81.3216, |J|=4107.73 --- Outer Iter 7: norm_f = 119.315, mu=139.502, |J|=4108.97 --- Outer Iter 8: norm_f = 119.314, mu=242.128, |J|=4110.48 --- Outer Iter 9: norm_f = 119.313, mu=426.252, |J|=4112.26 --- Outer Iter 10: norm_f = 119.312, mu=771.941, |J|=4114.3 --- Outer Iter 11: norm_f = 119.312, mu=1478.24, |J|=4116.6 Least squares message = Relative change in |x| is at most 1e-08 Sum of Chi^2 = 119.375 (168 data params - 31 model params = expected mean of 137; p-value = 0.858461) Completed in 0.6s 2*Delta(log(L)) = 122.407 Iteration 2 took 0.7s --- Iterative MLGST: Iter 3 of 5 450 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 520.731, mu=0, |J|=4388.01 --- Outer Iter 1: norm_f = 425.747, mu=2348.99, |J|=4387.99 --- Outer Iter 2: norm_f = 418.289, mu=782.998, |J|=4387.4 --- Outer Iter 3: norm_f = 416.05, mu=365.617, |J|=4387.75 --- Outer Iter 4: norm_f = 415.687, mu=355.573, |J|=4389.64 --- Outer Iter 5: norm_f = 415.658, mu=503.681, |J|=4392.13 --- Outer Iter 6: norm_f = 415.643, mu=802.345, |J|=4395.23 --- Outer Iter 7: norm_f = 415.629, mu=1291.1, |J|=4399.08 --- Outer Iter 8: norm_f = 415.617, mu=2101.57, |J|=4403.66 --- Outer Iter 9: norm_f = 415.607, mu=3490.92, |J|=4408.96 --- Outer Iter 10: norm_f = 415.599, mu=6019.34, |J|=4414.94 --- Outer Iter 11: norm_f = 415.596, mu=11138.8, |J|=4421.59 Least squares message = Relative change in |x| is at most 1e-08 Sum of Chi^2 = 415.944 (450 data params - 31 model params = expected mean of 419; p-value = 0.532983) Completed in 1.0s 2*Delta(log(L)) = 431.785 Iteration 3 took 1.3s --- Iterative MLGST: Iter 4 of 5 862 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 893.164, mu=0, |J|=4750.25 --- Outer Iter 1: norm_f = 815.33, mu=2372.71, |J|=4738.51 --- Outer Iter 2: norm_f = 814.675, mu=2184.19, |J|=4739.75 --- Outer Iter 3: norm_f = 814.612, mu=2654.57, |J|=4742.24 --- Outer Iter 4: norm_f = 814.583, mu=3911.8, |J|=4746.52 --- Outer Iter 5: norm_f = 814.565, mu=6296.89, |J|=4752.54 --- Outer Iter 6: norm_f = 814.552, mu=10677.9, |J|=4760.2 --- Outer Iter 7: norm_f = 814.545, mu=19253, |J|=4769.42 Least squares message = Relative change in |x| is at most 1e-08 Sum of Chi^2 = 815.074 (862 data params - 31 model params = expected mean of 831; p-value = 0.646841) Completed in 1.3s 2*Delta(log(L)) = 823.722 Iteration 4 took 1.8s --- Iterative MLGST: Iter 5 of 5 1282 gate strings ---: --- Minimum Chi^2 GST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 1289.5, mu=0, |J|=5010.07 --- Outer Iter 1: norm_f = 1254.01, mu=2582.86, |J|=5006.93 --- Outer Iter 2: norm_f = 1253.18, mu=2427.07, |J|=5010.14 --- Outer Iter 3: norm_f = 1253.05, mu=2664.24, |J|=5016.29 --- Outer Iter 4: norm_f = 1253.01, mu=3781.49, |J|=5025.33 --- Outer Iter 5: norm_f = 1252.98, mu=5981.45, |J|=5037.22 --- Outer Iter 6: norm_f = 1252.96, mu=9875.24, |J|=5051.82 --- Outer Iter 7: norm_f = 1252.95, mu=16863.4, |J|=5068.99 --- Outer Iter 8: norm_f = 1252.94, mu=30266.9, |J|=5088.58 --- Outer Iter 9: norm_f = 1252.94, mu=59685.2, |J|=5110.48 Least squares message = Relative change in |x| is at most 1e-08 Sum of Chi^2 = 1253.54 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.47442) Completed in 2.2s 2*Delta(log(L)) = 1270.99 Iteration 5 took 3.0s Switching to ML objective (last iteration) --- MLGST --- Created evaluation tree with 1 subtrees. Will divide 1 procs into 1 (subtree-processing) groups of ~1 procs each, to distribute over 43 params (taken as 1 param groups of ~43 params). --- Outer Iter 0: norm_f = 635.496, mu=0, |J|=3112.36 --- Outer Iter 1: norm_f = 627.361, mu=979.324, |J|=2703.11 Least squares message = Relative change in |x| is at most 1e-08 Maximum log(L) = 627.761 below upper bound of -2.13594e+06 2*Delta(log(L)) = 1255.52 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.458742) Completed in 1.3s 2*Delta(log(L)) = 1255.52 Final MLGST took 1.3s Iterative MLGST Total Time: 8.9s -- Performing 'single' gauge optimization on CPTP estimate -- -- Adding Gauge Optimized (single) -- -- Performing 'Spam 0.001' gauge optimization on CPTP estimate -- -- Adding Gauge Optimized (Spam 0.001) -- -- Performing 'Spam 0.001+v' gauge optimization on CPTP estimate -- -- Adding Gauge Optimized (Spam 0.001+v) -- -- Std Practice: Iter 3 of 3 (Target) --: --- Gate Sequence Creation --- 1282 sequences created Dataset has 3382 entries: 1282 utilized, 0 requested sequences were missing -- Performing 'single' gauge optimization on Target estimate -- -- Adding Gauge Optimized (single) -- -- Performing 'Spam 0.001' gauge optimization on Target estimate -- -- Adding Gauge Optimized (Spam 0.001) -- -- Performing 'Spam 0.001+v' gauge optimization on Target estimate -- -- Adding Gauge Optimized (Spam 0.001+v) -- *** Creating workspace *** *** Generating switchboard *** Found standard clifford compilation from std1Q_XYI Found standard clifford compilation from std1Q_XYI Found standard clifford compilation from std1Q_XYI *** Generating tables *** *** Generating plots *** *** Merging into template file *** Output written to tutorial_files/exampleStdReport directory Opening tutorial_files/exampleStdReport/main.html... *** Report Generation Complete! Total time 122.583s ***
<pygsti.report.workspace.Workspace at 0x11ce6a0b8>
To display confidence intervals for reported quantities, you must do two things:
confidenceLevel
argument to create_standard_report
.Constructing a factory often means computing a Hessian, which can be time consuming, and so this is not done automatically. Here we demonstrate how to construct a valid factory for the "Spam 0.001" gauge-optimization of the "CPTP" estimate by computing and then projecting the Hessian of the likelihood function.
#Construct and initialize a "confidence region factory" for the CPTP estimate
crfact = results_std.estimates["CPTP"].add_confidence_region_factory('Spam 0.001', 'final')
crfact.compute_hessian(comm=None) #we could use more processors
crfact.project_hessian('intrinsic error')
pygsti.report.create_standard_report(results_std, "tutorial_files/exampleStdReport2",
title="Post StdPractice Report (w/CIs on CPTP)",
confidenceLevel=95, auto_open=True, verbosity=1)
--- Hessian Projector Optimization from separate SPAM and Gate weighting --- Resulting intrinsic errors: 0.0119628 (gates), 0.000159745 (spam) Resulting sqrt(mean(gateCIs**2)): 0.0144373 Resulting sqrt(mean(spamCIs**2)): 0.00181812 *** Creating workspace *** *** Generating switchboard *** Found standard clifford compilation from std1Q_XYI Found standard clifford compilation from std1Q_XYI Found standard clifford compilation from std1Q_XYI *** Generating tables *** *** Generating plots *** *** Merging into template file *** Output written to tutorial_files/exampleStdReport2 directory Opening tutorial_files/exampleStdReport2/main.html... *** Report Generation Complete! Total time 131.073s ***
<pygsti.report.workspace.Workspace at 0x1198c3cc0>
We've already seen above that create_standard_report
can be given a dictionary of Results
objects instead of a single one. This allows the creation of reports containing estimates for different DataSet
s (each Results
object only holds estimates for a single DataSet
). Furthermore, when the data sets have the same gate sequences, they will be compared within a tab of the HTML report.
Below, we generate a new data set with the same sequences as the one loaded at the beginning of this tutorial, proceed to run standard-practice GST on that dataset, and create a report of the results along with those of the original dataset. Look at the "Data Comparison" tab within the gauge-invariant error metrics category.
#Make another dataset & estimates
depol_gateset = gs_target.depolarize(gate_noise=0.1)
datagen_gateset = depol_gateset.rotate((0.05,0,0.03))
#Compute the sequences needed to perform Long Sequence GST on
# this GateSet with sequences up to lenth 512
gatestring_list = pygsti.construction.make_lsgst_experiment_list(
std1Q_XYI.gs_target, std1Q_XYI.prepStrs, std1Q_XYI.effectStrs,
std1Q_XYI.germs, [1,2,4,8,16,32,64,128,256,512])
ds2 = pygsti.construction.generate_fake_data(datagen_gateset, gatestring_list, nSamples=1000,
sampleError='binomial', seed=2018)
results_std2 = pygsti.do_stdpractice_gst(ds2, gs_target, fiducials, fiducials, germs,
maxLengths, verbosity=3, modes="TP,CPTP,Target",
gaugeOptSuite=('single','toggleValidSpam'))
pygsti.report.create_standard_report({'DS1': results_std, 'DS2': results_std2},
"tutorial_files/exampleMultiDataSetReport",
title="Example Multi-Dataset Report",
auto_open=True, verbosity=1)
create_standard_report
options¶Finally, let us highlight a few of the additional arguments one can supply to create_standard_report
that allows further control over what gets reported.
Setting the link_to
argument to a tuple of 'pkl'
, 'tex'
, and/or 'pdf'
will create hyperlinks within the plots or below the tables of the HTML linking to Python pickle, LaTeX source, and PDF versions of the content, respectively. The Python pickle files for tables contain pickled pandas DataFrame
objects, wheras those of plots contain ordinary Python dictionaries of the data that is plotted. Applies to HTML reports only.
Setting the brevity
argument to an integer higher than $0$ (the default) will reduce the amount of information included in the report (for details on what is included for each value, see the doc string). Using brevity > 0
will reduce the time required to create, and later load, the report, as well as the output file/folder size. This applies to both HTML and PDF reports.
Below, we demonstrate both of these options in very brief (brevity=4
) report with links to pickle and PDF files. Note that to generate the PDF files you must have pdflatex
installed.
pygsti.report.create_standard_report(results_std,
"tutorial_files/exampleBriefReport",
title="Example Brief Report",
auto_open=True, verbosity=1,
brevity=4, link_to=('pkl','pdf'))
create_report_notebook
¶In addition to the standard HTML-page reports demonstrated above, pyGSTi is able to generate a Jupyter notebook containing the Python commands to create the figures and tables within a general report. This is facilitated
by Workspace
objects, which are factories for figures and tables (see previous tutorials). By calling create_report_notebook
, all of the relevant Workspace
initialization and calls are dumped to a new notebook file, which can be run (either fully or partially) by the user at their convenience. Creating such "report notebooks" has the advantage that the user may insert Python code amidst the figure and table generation calls to inspect or modify what is display in a highly customizable fashion. The chief disadvantages of report notebooks is that they require the user to 1) have a Jupyter server up and running and 2) to run the notebook before any figures are displayed.
The line below demonstrates how to create a report notebook using create_report_notebook
. Note that the argument list is very similar to create_general_report
.
pygsti.report.create_report_notebook(results, "tutorial_files/exampleReport.ipynb",
title="GST Example Report Notebook", confidenceLevel=None,
auto_open=True, connected=False, verbosity=3)