An example showing how to generate bootstrapped error bars.

In [1]:
from __future__ import print_function
In [2]:
import os
import sys
import time
import json

import pygsti
from pygsti.construction import std1Q_XYI

%pylab inline
Populating the interactive namespace from numpy and matplotlib
In [3]:
#Get a GST estimate (similar to Tutorial 0)

# 1) get the target GateSet
gs_target = std1Q_XYI.gs_target

# 2) get the building blocks needed to specify which gate sequences are needed
prep_fiducials, meas_fiducials = std1Q_XYI.prepStrs, std1Q_XYI.effectStrs
germs = std1Q_XYI.germs
maxLengths = [1,2,4,8,16]

# 3) generate "fake" data from a depolarized version of gs_target
gs_datagen = gs_target.depolarize(gate_noise=0.1, spam_noise=0.001)
listOfExperiments = pygsti.construction.make_lsgst_experiment_list(
    gs_target, prep_fiducials, meas_fiducials, germs, maxLengths)
ds = pygsti.construction.generate_fake_data(gs_datagen, listOfExperiments, nSamples=1000,
                                            sampleError="binomial", seed=1234)


results = pygsti.do_stdpractice_gst(ds, gs_target, prep_fiducials, meas_fiducials,
                                    germs, maxLengths, modes="TP")
estimated_gateset = results.estimates['TP'].gatesets['single']
-- Std Practice:  Iter 1 of 1  (TP) --: 
  --- Gate Sequence Creation ---
  --- LGST ---
  --- Iterative MLGST: [##################################################] 100.0%  1282 gate strings ---
  Iterative MLGST Total Time: 3.8s
  --- Re-optimizing logl after robust data scaling ---
  -- Performing 'single' gauge optimization on TP estimate --
  -- Performing 'single' gauge optimization on TP.Robust+ estimate --

Parametric Bootstrapping

Here we do parametric bootstrapping, as indicated by the 'parametric' argument below. The output is eventually stored in the "mean" and "std" GateSets, which hold the mean and standard deviation values of the set of bootstrapped gatesets (after gauge optimization). It is this latter "standard deviation Gateset" which holds the collection of error bars. Note: due to print setting issues, the outputs that are printed here will not necessarily reflect the true accuracy of the estimates made.

In [4]:
#The number of simulated datasets & gatesets made for bootstrapping purposes.  
# For good statistics, should probably be greater than 10.
numGatesets=10

param_boot_gatesets = pygsti.drivers.make_bootstrap_gatesets(
                        numGatesets, ds, 'parametric', prep_fiducials, meas_fiducials, germs, maxLengths,
                        inputGateSet=estimated_gateset, startSeed=0, returnData=False,
                        verbosity=2)
Creating DataSets: 
0 Generating parametric dataset.
1 Generating parametric dataset.
2 Generating parametric dataset.
3 Generating parametric dataset.
4 Generating parametric dataset.
5 Generating parametric dataset.
6 Generating parametric dataset.
7 Generating parametric dataset.
8 Generating parametric dataset.
9 Generating parametric dataset.
Creating GateSets: 
Running MLGST Iteration 0 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244087505140386
  1.1933106324899718
  0.9684847697487777
  0.9222856401382363
  0.049523451272192875
  0.007406728777226977
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 60.5326 (92 data params - 31 model params = expected mean of 61; p-value = 0.492813)
  Completed in 0.2s
  2*Delta(log(L)) = 60.6361
  Iteration 1 took 0.3s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 135.857 (168 data params - 31 model params = expected mean of 137; p-value = 0.511528)
  Completed in 0.3s
  2*Delta(log(L)) = 136.111
  Iteration 2 took 0.5s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 409.969 (450 data params - 31 model params = expected mean of 419; p-value = 0.614543)
  Completed in 0.6s
  2*Delta(log(L)) = 410.797
  Iteration 3 took 1.0s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 859.977 (862 data params - 31 model params = expected mean of 831; p-value = 0.236076)
  Completed in 1.4s
  2*Delta(log(L)) = 861.781
  Iteration 4 took 2.3s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1263.53 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.396258)
  Completed in 1.2s
  2*Delta(log(L)) = 1265.25
  Iteration 5 took 2.7s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 632.576 below upper bound of -2.13596e+06
      2*Delta(log(L)) = 1265.15 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.383928)
    Completed in 3.2s
  2*Delta(log(L)) = 1265.15
  Final MLGST took 3.2s
  
Iterative MLGST Total Time: 10.0s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 1 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244300079957604
  1.1595780865916032
  0.9791702801863404
  0.8913830952512019
  0.02415791540153395
  0.011118785548090178
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 52.0333 (92 data params - 31 model params = expected mean of 61; p-value = 0.786463)
  Completed in 0.2s
  2*Delta(log(L)) = 52.0829
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 130 (168 data params - 31 model params = expected mean of 137; p-value = 0.651672)
  Completed in 0.3s
  2*Delta(log(L)) = 130.033
  Iteration 2 took 0.6s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 416.305 (450 data params - 31 model params = expected mean of 419; p-value = 0.527998)
  Completed in 0.7s
  2*Delta(log(L)) = 415.948
  Iteration 3 took 1.2s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 843.134 (862 data params - 31 model params = expected mean of 831; p-value = 0.37732)
  Completed in 1.1s
  2*Delta(log(L)) = 843.282
  Iteration 4 took 2.0s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1233.87 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.629566)
  Completed in 1.5s
  2*Delta(log(L)) = 1234.11
  Iteration 5 took 3.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 617.018 below upper bound of -2.13642e+06
      2*Delta(log(L)) = 1234.04 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.628303)
    Completed in 3.3s
  2*Delta(log(L)) = 1234.04
  Final MLGST took 3.3s
  
Iterative MLGST Total Time: 10.5s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 2 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244703087785422
  1.1510593527507826
  0.957713209547531
  0.9051849882976878
  0.03290750589393019
  0.018566212978513904
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 49.8959 (92 data params - 31 model params = expected mean of 61; p-value = 0.84432)
  Completed in 0.2s
  2*Delta(log(L)) = 49.9217
  Iteration 1 took 0.3s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 123.304 (168 data params - 31 model params = expected mean of 137; p-value = 0.792772)
  Completed in 0.3s
  2*Delta(log(L)) = 124.044
  Iteration 2 took 0.5s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 443.365 (450 data params - 31 model params = expected mean of 419; p-value = 0.197996)
  Completed in 0.7s
  2*Delta(log(L)) = 444.826
  Iteration 3 took 1.2s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 869.774 (862 data params - 31 model params = expected mean of 831; p-value = 0.170297)
  Completed in 1.0s
  2*Delta(log(L)) = 871.308
  Iteration 4 took 1.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1311.76 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.113354)
  Completed in 1.6s
  2*Delta(log(L)) = 1313.49
  Iteration 5 took 2.9s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 656.705 below upper bound of -2.13652e+06
      2*Delta(log(L)) = 1313.41 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.107338)
    Completed in 2.8s
  2*Delta(log(L)) = 1313.41
  Final MLGST took 2.8s
  
Iterative MLGST Total Time: 9.5s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 3 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244287281337175
  1.2109117947458583
  0.9770264234304016
  0.9265254882893396
  0.05343908226796799
  0.022276686921616513
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 64.4923 (92 data params - 31 model params = expected mean of 61; p-value = 0.355521)
  Completed in 0.2s
  2*Delta(log(L)) = 65.4324
  Iteration 1 took 0.3s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 127.333 (168 data params - 31 model params = expected mean of 137; p-value = 0.711504)
  Completed in 0.3s
  2*Delta(log(L)) = 128.596
  Iteration 2 took 0.5s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 393.266 (450 data params - 31 model params = expected mean of 419; p-value = 0.811844)
  Completed in 0.6s
  2*Delta(log(L)) = 394.093
  Iteration 3 took 1.0s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 792.284 (862 data params - 31 model params = expected mean of 831; p-value = 0.828539)
  Completed in 1.1s
  2*Delta(log(L)) = 793.237
  Iteration 4 took 2.0s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1182.22 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.917392)
  Completed in 1.5s
  2*Delta(log(L)) = 1183.13
  Iteration 5 took 2.9s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 591.534 below upper bound of -2.13647e+06
      2*Delta(log(L)) = 1183.07 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.914669)
    Completed in 3.1s
  2*Delta(log(L)) = 1183.07
  Final MLGST took 3.1s
  
Iterative MLGST Total Time: 9.8s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 4 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244795252519304
  1.1646337119621837
  0.9481210587529844
  0.9117106295074608
  0.04492344439713607
  0.016662871495333537
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 69.3051 (92 data params - 31 model params = expected mean of 61; p-value = 0.217692)
  Completed in 0.3s
  2*Delta(log(L)) = 69.3131
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 128.579 (168 data params - 31 model params = expected mean of 137; p-value = 0.68405)
  Completed in 0.2s
  2*Delta(log(L)) = 128.776
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 452.142 (450 data params - 31 model params = expected mean of 419; p-value = 0.127373)
  Completed in 0.5s
  2*Delta(log(L)) = 452.807
  Iteration 3 took 0.9s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 871.949 (862 data params - 31 model params = expected mean of 831; p-value = 0.15752)
  Completed in 0.8s
  2*Delta(log(L)) = 872.888
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1313.7 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.106321)
  Completed in 1.2s
  2*Delta(log(L)) = 1314.79
  Iteration 5 took 2.3s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 657.355 below upper bound of -2.13569e+06
      2*Delta(log(L)) = 1314.71 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.102764)
    Completed in 4.6s
  2*Delta(log(L)) = 1314.71
  Final MLGST took 4.6s
  
Iterative MLGST Total Time: 10.0s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 5 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.2444581573163145
  1.1581941205791029
  0.9540176133865252
  0.9266371861891638
  0.04681379479826308
  0.017254477195332543
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 49.018 (92 data params - 31 model params = expected mean of 61; p-value = 0.865164)
  Completed in 0.2s
  2*Delta(log(L)) = 49.1744
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 105.521 (168 data params - 31 model params = expected mean of 137; p-value = 0.978799)
  Completed in 0.2s
  2*Delta(log(L)) = 105.764
  Iteration 2 took 0.5s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 434.844 (450 data params - 31 model params = expected mean of 419; p-value = 0.286532)
  Completed in 0.5s
  2*Delta(log(L)) = 435.729
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 799.01 (862 data params - 31 model params = expected mean of 831; p-value = 0.781887)
  Completed in 0.8s
  2*Delta(log(L)) = 800.172
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1203.93 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.82634)
  Completed in 1.3s
  2*Delta(log(L)) = 1205.23
  Iteration 5 took 2.4s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 602.589 below upper bound of -2.13631e+06
      2*Delta(log(L)) = 1205.18 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.819679)
    Completed in 3.1s
  2*Delta(log(L)) = 1205.18
  Final MLGST took 3.1s
  
Iterative MLGST Total Time: 8.5s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 6 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.243757410532886
  1.1899026885039474
  0.9654612474337722
  0.9140723325132796
  0.03929480284131846
  0.029135915041683117
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 85.8212 (92 data params - 31 model params = expected mean of 61; p-value = 0.0198118)
  Completed in 0.2s
  2*Delta(log(L)) = 85.7269
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 150.406 (168 data params - 31 model params = expected mean of 137; p-value = 0.20472)
  Completed in 0.2s
  2*Delta(log(L)) = 150.698
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 425.417 (450 data params - 31 model params = expected mean of 419; p-value = 0.40379)
  Completed in 0.4s
  2*Delta(log(L)) = 425.598
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 855.63 (862 data params - 31 model params = expected mean of 831; p-value = 0.26941)
  Completed in 0.8s
  2*Delta(log(L)) = 856.189
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1268.35 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.359944)
  Completed in 1.1s
  2*Delta(log(L)) = 1269.05
  Iteration 5 took 2.0s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 634.487 below upper bound of -2.13586e+06
      2*Delta(log(L)) = 1268.97 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.355339)
    Completed in 2.1s
  2*Delta(log(L)) = 1268.97
  Final MLGST took 2.1s
  
Iterative MLGST Total Time: 6.8s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 7 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244690901529703
  1.1599087281947678
  0.9357112178080124
  0.8833533639509482
  0.04886079924026652
  0.011952668779571548
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 55.8109 (92 data params - 31 model params = expected mean of 61; p-value = 0.663847)
  Completed in 0.1s
  2*Delta(log(L)) = 55.9415
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 112.694 (168 data params - 31 model params = expected mean of 137; p-value = 0.936314)
  Completed in 0.2s
  2*Delta(log(L)) = 112.498
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 367.871 (450 data params - 31 model params = expected mean of 419; p-value = 0.965649)
  Completed in 0.4s
  2*Delta(log(L)) = 368.374
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 816.801 (862 data params - 31 model params = expected mean of 831; p-value = 0.630784)
  Completed in 0.7s
  2*Delta(log(L)) = 817.562
  Iteration 4 took 1.3s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1179.44 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.925846)
  Completed in 1.1s
  2*Delta(log(L)) = 1179.96
  Iteration 5 took 2.0s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 589.951 below upper bound of -2.13627e+06
      2*Delta(log(L)) = 1179.9 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.92448)
    Completed in 2.7s
  2*Delta(log(L)) = 1179.9
  Final MLGST took 2.7s
  
Iterative MLGST Total Time: 7.3s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 8 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.243614095083225
  1.149707353458093
  0.9518126910613969
  0.8989235827830663
  0.06159972556576604
  0.016275155191118898
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 50.0227 (92 data params - 31 model params = expected mean of 61; p-value = 0.841165)
  Completed in 0.1s
  2*Delta(log(L)) = 50.0376
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 132.473 (168 data params - 31 model params = expected mean of 137; p-value = 0.593344)
  Completed in 0.3s
  2*Delta(log(L)) = 132.446
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 382.12 (450 data params - 31 model params = expected mean of 419; p-value = 0.90151)
  Completed in 0.4s
  2*Delta(log(L)) = 382.402
  Iteration 3 took 0.7s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 748.451 (862 data params - 31 model params = expected mean of 831; p-value = 0.981219)
  Completed in 0.8s
  2*Delta(log(L)) = 748.733
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1154.38 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.975604)
  Completed in 1.0s
  2*Delta(log(L)) = 1154.97
  Iteration 5 took 1.9s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 577.448 below upper bound of -2.13616e+06
      2*Delta(log(L)) = 1154.9 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.974969)
    Completed in 2.3s
  2*Delta(log(L)) = 1154.9
  Final MLGST took 2.3s
  
Iterative MLGST Total Time: 6.9s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 9 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244983881707133
  1.1711044612769528
  0.9662160886864828
  0.9246310051231014
  0.03192329072831494
  0.016942923637104233
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 46.7488 (92 data params - 31 model params = expected mean of 61; p-value = 0.910761)
  Completed in 0.3s
  2*Delta(log(L)) = 46.6947
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 117.689 (168 data params - 31 model params = expected mean of 137; p-value = 0.882119)
  Completed in 0.2s
  2*Delta(log(L)) = 117.676
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 399.817 (450 data params - 31 model params = expected mean of 419; p-value = 0.742133)
  Completed in 0.4s
  2*Delta(log(L)) = 400.028
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 816.018 (862 data params - 31 model params = expected mean of 831; p-value = 0.638084)
  Completed in 0.6s
  2*Delta(log(L)) = 816.523
  Iteration 4 took 1.3s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1184.49 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.909959)
  Completed in 1.0s
  2*Delta(log(L)) = 1185.1
  Iteration 5 took 2.0s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 592.528 below upper bound of -2.13651e+06
      2*Delta(log(L)) = 1185.06 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.908056)
    Completed in 2.3s
  2*Delta(log(L)) = 1185.06
  Final MLGST took 2.4s
  
Iterative MLGST Total Time: 7.1s
  -- Adding Gauge Optimized (go0) --
In [5]:
gauge_opt_pboot_gatesets = pygsti.drivers.gauge_optimize_gs_list(param_boot_gatesets, estimated_gateset,
                                                                 plot=False) #plotting support removed w/matplotlib
Spam weight 0
Spam weight 1
Spam weight 2
Spam weight 3
Spam weight 4
Spam weight 5
Spam weight 6
Spam weight 7
Spam weight 8
Spam weight 9
Spam weight 10
Spam weight 11
Spam weight 12
Best SPAM weight is 1.0
In [6]:
pboot_mean = pygsti.drivers.to_mean_gateset(gauge_opt_pboot_gatesets, estimated_gateset)
pboot_std  = pygsti.drivers.to_std_gateset(gauge_opt_pboot_gatesets, estimated_gateset)

#Summary of the error bars
print("Parametric bootstrapped error bars, with", numGatesets, "resamples\n")
print("Error in rho vec:") 
print(pboot_std['rho0'], end='\n\n')
print("Error in effect vecs:")
print(pboot_std['Mdefault'], end='\n\n')
print("Error in Gi:")
print(pboot_std['Gi'], end='\n\n')
print("Error in Gx:")
print(pboot_std['Gx'], end='\n\n')
print("Error in Gy:")
print(pboot_std['Gy'])
Parametric bootstrapped error bars, with 10 resamples

Error in rho vec:
TP-Parameterized spam vector with length 4
 0.71
   0
   0
   0


Error in effect vecs:
TP-POVM with effect vectors:
0:
   0
   0
   0
   0

1:
 1.41
   0
   0
   0



Error in Gi:
TP Parameterized gate with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gx:
TP Parameterized gate with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gy:
TP Parameterized gate with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0

Non-parametric Bootstrapping

Here we do non-parametric bootstrapping, as indicated by the 'nonparametric' argument below. The output is again eventually stored in the "mean" and "std" GateSets, which hold the mean and standard deviation values of the set of bootstrapped gatesets (after gauge optimization). It is this latter "standard deviation Gateset" which holds the collection of error bars. Note: due to print setting issues, the outputs that are printed here will not necessarily reflect the true accuracy of the estimates made.

(Technical note: ddof = 1 is by default used when computing the standard deviation -- see numpy.std -- meaning that we are computing a standard deviation of the sample, not of the population.)

In [7]:
#The number of simulated datasets & gatesets made for bootstrapping purposes.  
# For good statistics, should probably be greater than 10.
numGatesets=10

nonparam_boot_gatesets = pygsti.drivers.make_bootstrap_gatesets(
                          numGatesets, ds, 'nonparametric', prep_fiducials, meas_fiducials, germs, maxLengths,
                          targetGateSet=estimated_gateset, startSeed=0, returnData=False, verbosity=2)
Creating DataSets: 
0 Generating non-parametric dataset.
1 Generating non-parametric dataset.
2 Generating non-parametric dataset.
3 Generating non-parametric dataset.
4 Generating non-parametric dataset.
5 Generating non-parametric dataset.
6 Generating non-parametric dataset.
7 Generating non-parametric dataset.
8 Generating non-parametric dataset.
9 Generating non-parametric dataset.
Creating GateSets: 
Running MLGST Iteration 0 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.245022884279376
  1.1802897474075076
  0.9890829898766544
  0.8982444923224625
  0.11322258617756151
  0.021520716470784768
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 173.063 (92 data params - 31 model params = expected mean of 61; p-value = 1.14941e-12)
  Completed in 0.2s
  2*Delta(log(L)) = 173.696
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 351.92 (168 data params - 31 model params = expected mean of 137; p-value = 0)
  Completed in 0.2s
  2*Delta(log(L)) = 352.866
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 974.476 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.4s
  2*Delta(log(L)) = 976.819
  Iteration 3 took 0.7s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1816.6 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 1819.62
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2711.68 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.1s
  2*Delta(log(L)) = 2715
  Iteration 5 took 2.0s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1357.31 below upper bound of -2.13485e+06
      2*Delta(log(L)) = 2714.63 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 2.0s
  2*Delta(log(L)) = 2714.63
  Final MLGST took 2.0s
  
Iterative MLGST Total Time: 6.6s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 636.32 below upper bound of -2.13485e+06
      2*Delta(log(L)) = 1272.64 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.328717)
    Completed in 2.1s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 1 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.245169041294076
  1.1518727885462865
  0.9672776687757828
  0.901071397315401
  0.06224476106466454
  0.010036333629296372
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 158.235 (92 data params - 31 model params = expected mean of 61; p-value = 1.42309e-10)
  Completed in 0.1s
  2*Delta(log(L)) = 159.411
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 288.424 (168 data params - 31 model params = expected mean of 137; p-value = 7.83706e-13)
  Completed in 0.2s
  2*Delta(log(L)) = 289.857
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 854.852 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.4s
  2*Delta(log(L)) = 857.541
  Iteration 3 took 0.7s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1652.99 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.9s
  2*Delta(log(L)) = 1655.06
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2448.38 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.0s
  2*Delta(log(L)) = 2451.02
  Iteration 5 took 1.8s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1225.36 below upper bound of -2.13563e+06
      2*Delta(log(L)) = 2450.72 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 2.0s
  2*Delta(log(L)) = 2450.72
  Final MLGST took 2.0s
  
Iterative MLGST Total Time: 6.5s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 636.487 below upper bound of -2.13563e+06
      2*Delta(log(L)) = 1272.97 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.326335)
    Completed in 2.4s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 2 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.2449518950819165
  1.1416876758215317
  0.9849600533667989
  0.8768896933983193
  0.06572648432632268
  0.028050801401548413
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 158.939 (92 data params - 31 model params = expected mean of 61; p-value = 1.13813e-10)
  Completed in 0.2s
  2*Delta(log(L)) = 159.568
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 287.603 (168 data params - 31 model params = expected mean of 137; p-value = 9.76885e-13)
  Completed in 0.2s
  2*Delta(log(L)) = 288.708
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 933.123 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.4s
  2*Delta(log(L)) = 937.164
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1730.51 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 1735.87
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2649.6 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.0s
  2*Delta(log(L)) = 2655.74
  Iteration 5 took 1.9s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1327.67 below upper bound of -2.13523e+06
      2*Delta(log(L)) = 2655.35 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.8s
  2*Delta(log(L)) = 2655.35
  Final MLGST took 1.8s
  
Iterative MLGST Total Time: 6.5s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 634.38 below upper bound of -2.13523e+06
      2*Delta(log(L)) = 1268.76 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.356924)
    Completed in 1.9s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 3 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.245665131823529
  1.2021122366330264
  0.9739552285286952
  0.9264658537547215
  0.047574335152868644
  0.018129430161165774
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 148.816 (92 data params - 31 model params = expected mean of 61; p-value = 2.6785e-09)
  Completed in 0.2s
  2*Delta(log(L)) = 149.169
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 319.244 (168 data params - 31 model params = expected mean of 137; p-value = 1.11022e-16)
  Completed in 0.2s
  2*Delta(log(L)) = 319.783
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 831.176 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 832.446
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1696.18 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 1698.62
  Iteration 4 took 1.5s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2505.89 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.0s
  2*Delta(log(L)) = 2508.85
  Iteration 5 took 2.5s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1254.26 below upper bound of -2.13547e+06
      2*Delta(log(L)) = 2508.52 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 2.4s
  2*Delta(log(L)) = 2508.52
  Final MLGST took 2.4s
  
Iterative MLGST Total Time: 7.7s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 635.801 below upper bound of -2.13547e+06
      2*Delta(log(L)) = 1271.6 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.336169)
    Completed in 1.8s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 4 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244967407534078
  1.1519164538560829
  0.9795568132377331
  0.9038710521155845
  0.11836490138898303
  0.04009380519753089
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 152.215 (92 data params - 31 model params = expected mean of 61; p-value = 9.40295e-10)
  Completed in 0.2s
  2*Delta(log(L)) = 153.774
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 294.682 (168 data params - 31 model params = expected mean of 137; p-value = 1.43441e-13)
  Completed in 0.2s
  2*Delta(log(L)) = 296.817
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 890.84 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.4s
  2*Delta(log(L)) = 893.259
  Iteration 3 took 0.7s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1729.8 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 1733.12
  Iteration 4 took 1.3s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2611.36 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.0s
  2*Delta(log(L)) = 2615.21
  Iteration 5 took 1.9s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1307.46 below upper bound of -2.13509e+06
      2*Delta(log(L)) = 2614.92 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.7s
  2*Delta(log(L)) = 2614.92
  Final MLGST took 1.7s
  
Iterative MLGST Total Time: 6.1s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 635.626 below upper bound of -2.13509e+06
      2*Delta(log(L)) = 1271.25 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.338692)
    Completed in 1.8s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 5 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244966614375648
  1.1437961373874967
  0.962527682687429
  0.947364235918003
  0.05166521027472325
  0.016401510440440772
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 121.921 (92 data params - 31 model params = expected mean of 61; p-value = 5.97884e-06)
  Completed in 0.1s
  2*Delta(log(L)) = 122.227
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 249.751 (168 data params - 31 model params = expected mean of 137; p-value = 1.35374e-08)
  Completed in 0.3s
  2*Delta(log(L)) = 250.509
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 839.724 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.4s
  2*Delta(log(L)) = 843.673
  Iteration 3 took 0.7s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1658.72 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 1662.54
  Iteration 4 took 1.2s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2447.74 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 0.9s
  2*Delta(log(L)) = 2452.24
  Iteration 5 took 1.7s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1225.97 below upper bound of -2.13534e+06
      2*Delta(log(L)) = 2451.94 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.5s
  2*Delta(log(L)) = 2451.94
  Final MLGST took 1.5s
  
Iterative MLGST Total Time: 5.7s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 636.209 below upper bound of -2.13534e+06
      2*Delta(log(L)) = 1272.42 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.330301)
    Completed in 2.0s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 6 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244722834250942
  1.1795372972478275
  0.9651028595686848
  0.9179307909363903
  0.036402259680446275
  0.018172335659510378
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 159.074 (92 data params - 31 model params = expected mean of 61; p-value = 1.09027e-10)
  Completed in 0.1s
  2*Delta(log(L)) = 159.125
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 313.928 (168 data params - 31 model params = expected mean of 137; p-value = 6.66134e-16)
  Completed in 0.2s
  2*Delta(log(L)) = 314.378
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 902.566 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.3s
  2*Delta(log(L)) = 903.827
  Iteration 3 took 0.6s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1818.02 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.6s
  2*Delta(log(L)) = 1821.3
  Iteration 4 took 1.0s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2777.26 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 2782.01
  Iteration 5 took 1.5s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1390.82 below upper bound of -2.13458e+06
      2*Delta(log(L)) = 2781.65 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.7s
  2*Delta(log(L)) = 2781.65
  Final MLGST took 1.7s
  
Iterative MLGST Total Time: 5.3s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 635.76 below upper bound of -2.13458e+06
      2*Delta(log(L)) = 1271.52 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.336759)
    Completed in 1.5s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 7 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.2451643045836125
  1.149112945116303
  0.9469147673011137
  0.882676441039575
  0.09178053685609519
  0.03356372164197511
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 180.044 (92 data params - 31 model params = expected mean of 61; p-value = 1.10467e-13)
  Completed in 0.1s
  2*Delta(log(L)) = 180.682
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 340.192 (168 data params - 31 model params = expected mean of 137; p-value = 0)
  Completed in 0.2s
  2*Delta(log(L)) = 340.825
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 902.141 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.4s
  2*Delta(log(L)) = 904.644
  Iteration 3 took 0.6s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1711.24 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 1714.98
  Iteration 4 took 1.3s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2587.53 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.0s
  2*Delta(log(L)) = 2591.98
  Iteration 5 took 1.6s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1295.85 below upper bound of -2.13547e+06
      2*Delta(log(L)) = 2591.71 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 4.7s
  2*Delta(log(L)) = 2591.71
  Final MLGST took 4.7s
  
Iterative MLGST Total Time: 8.7s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 636.16 below upper bound of -2.13547e+06
      2*Delta(log(L)) = 1272.32 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.331005)
    Completed in 2.2s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 8 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.245096780919539
  1.1359650265619394
  0.9425792600583035
  0.9065276561807704
  0.09735417945308723
  0.013888480299966716
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 119.831 (92 data params - 31 model params = expected mean of 61; p-value = 1.03602e-05)
  Completed in 0.1s
  2*Delta(log(L)) = 120.106
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 268.089 (168 data params - 31 model params = expected mean of 137; p-value = 1.56678e-10)
  Completed in 0.2s
  2*Delta(log(L)) = 267.809
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 827.631 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.3s
  2*Delta(log(L)) = 828.114
  Iteration 3 took 0.6s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1595.31 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 1596.33
  Iteration 4 took 1.1s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2468.71 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 0.9s
  2*Delta(log(L)) = 2470.38
  Iteration 5 took 1.6s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1235.05 below upper bound of -2.13539e+06
      2*Delta(log(L)) = 2470.09 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 3.2s
  2*Delta(log(L)) = 2470.09
  Final MLGST took 3.2s
  
Iterative MLGST Total Time: 7.0s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 635.842 below upper bound of -2.13539e+06
      2*Delta(log(L)) = 1271.68 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.335571)
    Completed in 2.1s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 9 
--- Gate Sequence Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.246104964081442
  1.157968846812025
  0.976656376489257
  0.9170906635373808
  0.10583791291942916
  0.005114403198416756
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244260334128999
  1.1703566281126876
  0.9480786823090673
  0.9404187467171867
  2.485157467463535e-16
  1.397518013374202e-16
  
--- Iterative MLGST: Iter 1 of 5  92 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 156.732 (92 data params - 31 model params = expected mean of 61; p-value = 2.28922e-10)
  Completed in 0.1s
  2*Delta(log(L)) = 157.548
  Iteration 1 took 0.1s
  
--- Iterative MLGST: Iter 2 of 5  168 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 288.639 (168 data params - 31 model params = expected mean of 137; p-value = 7.39631e-13)
  Completed in 0.2s
  2*Delta(log(L)) = 290.224
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 843.872 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.4s
  2*Delta(log(L)) = 847.728
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1676.99 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.6s
  2*Delta(log(L)) = 1681.64
  Iteration 4 took 1.2s
  
--- Iterative MLGST: Iter 5 of 5  1282 gate strings ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2484.05 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 0.9s
  2*Delta(log(L)) = 2489.36
  Iteration 5 took 1.6s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1244.55 below upper bound of -2.13552e+06
      2*Delta(log(L)) = 2489.09 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.6s
  2*Delta(log(L)) = 2489.09
  Final MLGST took 1.6s
  
Iterative MLGST Total Time: 5.6s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 636.501 below upper bound of -2.13552e+06
      2*Delta(log(L)) = 1273 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.326137)
    Completed in 1.6s
  -- Adding Gauge Optimized (go0) --
In [8]:
gauge_opt_npboot_gatesets = pygsti.drivers.gauge_optimize_gs_list(nonparam_boot_gatesets, estimated_gateset,
                                                                 plot=False) #plotting removed w/matplotlib
Spam weight 0
Spam weight 1
Spam weight 2
Spam weight 3
Spam weight 4
Spam weight 5
Spam weight 6
Spam weight 7
Spam weight 8
Spam weight 9
Spam weight 10
Spam weight 11
Spam weight 12
Best SPAM weight is 1.0
In [9]:
npboot_mean = pygsti.drivers.to_mean_gateset(gauge_opt_npboot_gatesets, estimated_gateset)
npboot_std  = pygsti.drivers.to_std_gateset(gauge_opt_npboot_gatesets, estimated_gateset)

#Summary of the error bars
print("Non-parametric bootstrapped error bars, with", numGatesets, "resamples\n")
print("Error in rho vec:")
print(npboot_std['rho0'], end='\n\n')
print("Error in effect vecs:")
print(npboot_std['Mdefault'], end='\n\n')
print("Error in Gi:")
print(npboot_std['Gi'], end='\n\n')
print("Error in Gx:")
print(npboot_std['Gx'], end='\n\n')
print("Error in Gy:")
print(npboot_std['Gy'])
Non-parametric bootstrapped error bars, with 10 resamples

Error in rho vec:
TP-Parameterized spam vector with length 4
 0.71
   0
   0
   0


Error in effect vecs:
TP-POVM with effect vectors:
0:
   0
   0
   0
   0

1:
 1.41
   0
   0
   0



Error in Gi:
TP Parameterized gate with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gx:
TP Parameterized gate with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gy:
TP Parameterized gate with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0

In [10]:
loglog(npboot_std.to_vector(),pboot_std.to_vector(),'.')
loglog(np.logspace(-4,-2,10),np.logspace(-4,-2,10),'--')
xlabel('Non-parametric')
ylabel('Parametric')
xlim((1e-4,1e-2)); ylim((1e-4,1e-2))
title('Scatter plot comparing param vs. non-param bootstrapping error bars.')
Out[10]:
Text(0.5,1,'Scatter plot comparing param vs. non-param bootstrapping error bars.')
In [ ]: