An example showing how to generate bootstrapped error bars.

In [1]:
import os
import sys
import time
import json

import pygsti
from pygsti.modelpacks.legacy import std1Q_XYI

%pylab inline
Populating the interactive namespace from numpy and matplotlib
In [2]:
#Get a GST estimate (similar to Tutorial 0)

# 1) get the target Model
target_model = std1Q_XYI.target_model()

# 2) get the building blocks needed to specify which operation sequences are needed
prep_fiducials, meas_fiducials = std1Q_XYI.prepStrs, std1Q_XYI.effectStrs
germs = std1Q_XYI.germs
maxLengths = [1,2,4,8,16]

# 3) generate "fake" data from a depolarized version of target_model
mdl_datagen = target_model.depolarize(op_noise=0.1, spam_noise=0.001)
listOfExperiments = pygsti.construction.create_lsgst_circuits(
    target_model, prep_fiducials, meas_fiducials, germs, maxLengths)
ds = pygsti.construction.simulate_data(mdl_datagen, listOfExperiments, nSamples=1000,
                                            sampleError="binomial", seed=1234)


results = pygsti.run_stdpractice_gst(ds, target_model, prep_fiducials, meas_fiducials,
                                    germs, maxLengths, modes="TP")
estimated_model = results.estimates['TP'].models['stdgaugeopt']
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
-- Std Practice:  Iter 1 of 1  (TP) --: 
  --- Iterative MLGST: [##################################################] 100.0%  1282 operation sequences ---
  Iterative MLGST Total Time: 7.2s

Parametric Bootstrapping

Here we do parametric bootstrapping, as indicated by the 'parametric' argument below. The output is eventually stored in the "mean" and "std" Models, which hold the mean and standard deviation values of the set of bootstrapped models (after gauge optimization). It is this latter "standard deviation Model" which holds the collection of error bars. Note: due to print setting issues, the outputs that are printed here will not necessarily reflect the true accuracy of the estimates made.

In [3]:
#The number of simulated datasets & models made for bootstrapping purposes.  
# For good statistics, should probably be greater than 10.
numGatesets=10

param_boot_models = pygsti.drivers.make_bootstrap_models(
                        numGatesets, ds, 'parametric', prep_fiducials, meas_fiducials, germs, maxLengths,
                        inputModel=estimated_model, startSeed=0, returnData=False,
                        verbosity=2)
Creating DataSets: 
0 Generating parametric dataset.
1 Generating parametric dataset.
2 Generating parametric dataset.
3 Generating parametric dataset.
4 Generating parametric dataset.
5 Generating parametric dataset.
6 Generating parametric dataset.
7 Generating parametric dataset.
8 Generating parametric dataset.
9 Generating parametric dataset.
Creating Models: 
Running MLGST Iteration 0 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244653976670047
  1.192830953128653
  0.9689326363609111
  0.9189157715091224
  0.07422658287182586
  0.011606078537692615
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 54.3315 (92 data params - 31 model params = expected mean of 61; p-value = 0.7144)
  Completed in 0.3s
  2*Delta(log(L)) = 54.4952
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 139.054 (168 data params - 31 model params = expected mean of 137; p-value = 0.434979)
  Completed in 0.3s
  2*Delta(log(L)) = 139.281
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 433.775 (450 data params - 31 model params = expected mean of 419; p-value = 0.298928)
  Completed in 0.6s
  2*Delta(log(L)) = 434.566
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 871.047 (862 data params - 31 model params = expected mean of 831; p-value = 0.16274)
  Completed in 1.2s
  2*Delta(log(L)) = 872.842
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1273.04 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.325873)
  Completed in 1.8s
  2*Delta(log(L)) = 1274.82
  Iteration 5 took 2.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 637.362 below upper bound of -2.136e+06
      2*Delta(log(L)) = 1274.72 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.313964)
    Completed in 2.1s
  2*Delta(log(L)) = 1274.72
  Final MLGST took 2.1s
  
Iterative MLGST Total Time: 7.2s
Running MLGST Iteration 1 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.243869714012627
  1.16340135495376
  0.9505771532444359
  0.9192169139457478
  0.03669226197655027
  0.007493766406182375
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 62.8464 (92 data params - 31 model params = expected mean of 61; p-value = 0.410693)
  Completed in 0.4s
  2*Delta(log(L)) = 62.9639
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 141.38 (168 data params - 31 model params = expected mean of 137; p-value = 0.38131)
  Completed in 0.3s
  2*Delta(log(L)) = 141.59
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 417.084 (450 data params - 31 model params = expected mean of 419; p-value = 0.517254)
  Completed in 0.9s
  2*Delta(log(L)) = 417.095
  Iteration 3 took 1.0s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 870.4 (862 data params - 31 model params = expected mean of 831; p-value = 0.166552)
  Completed in 1.2s
  2*Delta(log(L)) = 870.82
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1245.19 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.540991)
  Completed in 2.1s
  2*Delta(log(L)) = 1245.74
  Iteration 5 took 2.5s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 622.828 below upper bound of -2.13648e+06
      2*Delta(log(L)) = 1245.66 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.537301)
    Completed in 6.6s
  2*Delta(log(L)) = 1245.66
  Final MLGST took 6.6s
  
Iterative MLGST Total Time: 12.3s
Running MLGST Iteration 2 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244707863399396
  1.1537403437788096
  0.9521888844551547
  0.9084465475528515
  0.03371022411959681
  0.02563577347463864
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 48.434 (92 data params - 31 model params = expected mean of 61; p-value = 0.878044)
  Completed in 0.4s
  2*Delta(log(L)) = 48.3704
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 122.032 (168 data params - 31 model params = expected mean of 137; p-value = 0.815624)
  Completed in 0.4s
  2*Delta(log(L)) = 122.378
  Iteration 2 took 0.5s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 419.72 (450 data params - 31 model params = expected mean of 419; p-value = 0.480909)
  Completed in 0.8s
  2*Delta(log(L)) = 420.806
  Iteration 3 took 1.0s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 845.378 (862 data params - 31 model params = expected mean of 831; p-value = 0.356808)
  Completed in 1.3s
  2*Delta(log(L)) = 847.099
  Iteration 4 took 1.5s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1301.32 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.157176)
  Completed in 2.0s
  2*Delta(log(L)) = 1303.25
  Iteration 5 took 2.4s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 651.59 below upper bound of -2.13642e+06
      2*Delta(log(L)) = 1303.18 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.148633)
    Completed in 5.5s
  2*Delta(log(L)) = 1303.18
  Final MLGST took 5.5s
  
Iterative MLGST Total Time: 11.3s
Running MLGST Iteration 3 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244050277155162
  1.2124619829658454
  0.9762089809043478
  0.9203172425509635
  0.03313244495774025
  0.02235228395337996
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 63.0797 (92 data params - 31 model params = expected mean of 61; p-value = 0.402678)
  Completed in 0.3s
  2*Delta(log(L)) = 63.7862
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 135.696 (168 data params - 31 model params = expected mean of 137; p-value = 0.515429)
  Completed in 0.3s
  2*Delta(log(L)) = 136.6
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 417.539 (450 data params - 31 model params = expected mean of 419; p-value = 0.510963)
  Completed in 0.7s
  2*Delta(log(L)) = 418.44
  Iteration 3 took 0.9s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 805.317 (862 data params - 31 model params = expected mean of 831; p-value = 0.732436)
  Completed in 1.0s
  2*Delta(log(L)) = 806.567
  Iteration 4 took 1.3s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1185.16 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.907703)
  Completed in 1.7s
  2*Delta(log(L)) = 1186.39
  Iteration 5 took 2.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 593.158 below upper bound of -2.1362e+06
      2*Delta(log(L)) = 1186.32 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.90367)
    Completed in 1.4s
  2*Delta(log(L)) = 1186.32
  Final MLGST took 1.4s
  
Iterative MLGST Total Time: 6.4s
Running MLGST Iteration 4 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.24484375709879
  1.1649382938018655
  0.9568638742519098
  0.9031869835559561
  0.033200958666622024
  0.017284232230193083
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 56.7701 (92 data params - 31 model params = expected mean of 61; p-value = 0.629839)
  Completed in 0.3s
  2*Delta(log(L)) = 56.8667
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 130.755 (168 data params - 31 model params = expected mean of 137; p-value = 0.634092)
  Completed in 0.3s
  2*Delta(log(L)) = 131.182
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 442.102 (450 data params - 31 model params = expected mean of 419; p-value = 0.209906)
  Completed in 0.7s
  2*Delta(log(L)) = 442.695
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 888.55 (862 data params - 31 model params = expected mean of 831; p-value = 0.0812685)
  Completed in 1.3s
  2*Delta(log(L)) = 889.66
  Iteration 4 took 1.6s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1313.66 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.106446)
  Completed in 1.7s
  2*Delta(log(L)) = 1314.88
  Iteration 5 took 2.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 657.405 below upper bound of -2.1359e+06
      2*Delta(log(L)) = 1314.81 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.102417)
    Completed in 4.8s
  2*Delta(log(L)) = 1314.81
  Final MLGST took 4.9s
  
Iterative MLGST Total Time: 10.1s
Running MLGST Iteration 5 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244643403566763
  1.1562039604688137
  0.9496844028666662
  0.9318264748631984
  0.07270157916508631
  0.008013391048464916
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 46.2692 (92 data params - 31 model params = expected mean of 61; p-value = 0.918875)
  Completed in 0.4s
  2*Delta(log(L)) = 46.4106
  Iteration 1 took 0.5s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 98.0874 (168 data params - 31 model params = expected mean of 137; p-value = 0.995035)
  Completed in 0.3s
  2*Delta(log(L)) = 98.0483
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 414.136 (450 data params - 31 model params = expected mean of 419; p-value = 0.557892)
  Completed in 0.8s
  2*Delta(log(L)) = 414.601
  Iteration 3 took 1.0s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 807.782 (862 data params - 31 model params = expected mean of 831; p-value = 0.711756)
  Completed in 1.3s
  2*Delta(log(L)) = 808.505
  Iteration 4 took 1.6s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1206.65 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.811649)
  Completed in 1.9s
  2*Delta(log(L)) = 1207.53
  Iteration 5 took 2.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 603.738 below upper bound of -2.13676e+06
      2*Delta(log(L)) = 1207.48 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.80705)
    Completed in 4.7s
  2*Delta(log(L)) = 1207.48
  Final MLGST took 4.8s
  
Iterative MLGST Total Time: 10.4s
Running MLGST Iteration 6 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.243488108495374
  1.165012987691403
  0.9474366699518069
  0.9203189657954223
  0.04090095083314043
  0.0070200724732477505
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 72.5965 (92 data params - 31 model params = expected mean of 61; p-value = 0.146998)
  Completed in 0.4s
  2*Delta(log(L)) = 72.4835
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 148.693 (168 data params - 31 model params = expected mean of 137; p-value = 0.23353)
  Completed in 0.5s
  2*Delta(log(L)) = 148.773
  Iteration 2 took 0.6s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 435.476 (450 data params - 31 model params = expected mean of 419; p-value = 0.279334)
  Completed in 0.7s
  2*Delta(log(L)) = 435.899
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 867.184 (862 data params - 31 model params = expected mean of 831; p-value = 0.186384)
  Completed in 1.1s
  2*Delta(log(L)) = 868.062
  Iteration 4 took 1.3s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1278.84 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.285778)
  Completed in 1.8s
  2*Delta(log(L)) = 1279.9
  Iteration 5 took 2.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 639.908 below upper bound of -2.13566e+06
      2*Delta(log(L)) = 1279.82 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.27927)
    Completed in 5.8s
  2*Delta(log(L)) = 1279.82
  Final MLGST took 5.8s
  
Iterative MLGST Total Time: 11.0s
Running MLGST Iteration 7 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244825840755222
  1.1607854217596698
  0.9292803921438999
  0.8958478552912302
  0.03648710217525403
  0.005481843557321346
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 61.4652 (92 data params - 31 model params = expected mean of 61; p-value = 0.45923)
  Completed in 0.4s
  2*Delta(log(L)) = 61.3186
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 126.314 (168 data params - 31 model params = expected mean of 137; p-value = 0.733237)
  Completed in 0.3s
  2*Delta(log(L)) = 126.31
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 408.23 (450 data params - 31 model params = expected mean of 419; p-value = 0.637659)
  Completed in 0.7s
  2*Delta(log(L)) = 408.489
  Iteration 3 took 0.9s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 846.207 (862 data params - 31 model params = expected mean of 831; p-value = 0.349347)
  Completed in 1.2s
  2*Delta(log(L)) = 846.906
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1204.55 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.823037)
  Completed in 1.8s
  2*Delta(log(L)) = 1205.22
  Iteration 5 took 2.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 602.585 below upper bound of -2.13604e+06
      2*Delta(log(L)) = 1205.17 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.819723)
    Completed in 4.0s
  2*Delta(log(L)) = 1205.17
  Final MLGST took 4.0s
  
Iterative MLGST Total Time: 9.3s
Running MLGST Iteration 8 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244918327291839
  1.1497175414386247
  0.9499958939687557
  0.8863416645506449
  0.03990669419647161
  0.028278621122238244
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 38.1221 (92 data params - 31 model params = expected mean of 61; p-value = 0.99049)
  Completed in 0.5s
  2*Delta(log(L)) = 38.2328
  Iteration 1 took 0.5s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 117.187 (168 data params - 31 model params = expected mean of 137; p-value = 0.888629)
  Completed in 0.3s
  2*Delta(log(L)) = 117.364
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 397.889 (450 data params - 31 model params = expected mean of 419; p-value = 0.763857)
  Completed in 0.7s
  2*Delta(log(L)) = 398.011
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 771.555 (862 data params - 31 model params = expected mean of 831; p-value = 0.930295)
  Completed in 1.1s
  2*Delta(log(L)) = 772.308
  Iteration 4 took 1.3s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1147.5 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.982818)
  Completed in 1.8s
  2*Delta(log(L)) = 1148.41
  Iteration 5 took 2.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 574.173 below upper bound of -2.13653e+06
      2*Delta(log(L)) = 1148.35 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.982039)
    Completed in 1.4s
  2*Delta(log(L)) = 1148.35
  Final MLGST took 1.4s
  
Iterative MLGST Total Time: 6.6s
Running MLGST Iteration 9 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244329086925476
  1.170786859388005
  0.9552770215877492
  0.9397649029334779
  0.030389543292175643
  0.017057158675289986
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 45.9565 (92 data params - 31 model params = expected mean of 61; p-value = 0.923886)
  Completed in 0.4s
  2*Delta(log(L)) = 46.0754
  Iteration 1 took 0.5s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 112.293 (168 data params - 31 model params = expected mean of 137; p-value = 0.939692)
  Completed in 0.3s
  2*Delta(log(L)) = 112.386
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 400.962 (450 data params - 31 model params = expected mean of 419; p-value = 0.728788)
  Completed in 0.6s
  2*Delta(log(L)) = 400.815
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 835.78 (862 data params - 31 model params = expected mean of 831; p-value = 0.446954)
  Completed in 1.3s
  2*Delta(log(L)) = 836.156
  Iteration 4 took 1.5s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1205.73 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.816695)
  Completed in 1.7s
  2*Delta(log(L)) = 1205.95
  Iteration 5 took 2.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 602.944 below upper bound of -2.13618e+06
      2*Delta(log(L)) = 1205.89 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.815828)
    Completed in 4.8s
  2*Delta(log(L)) = 1205.89
  Final MLGST took 4.8s
  
Iterative MLGST Total Time: 10.0s
In [4]:
gauge_opt_pboot_models = pygsti.drivers.gauge_optimize_model_list(param_boot_models, estimated_model,
                                                                 plot=False) #plotting support removed w/matplotlib
Spam weight 0
Spam weight 1
Spam weight 2
Spam weight 3
Spam weight 4
Spam weight 5
Spam weight 6
Spam weight 7
Spam weight 8
Spam weight 9
Spam weight 10
Spam weight 11
Spam weight 12
Best SPAM weight is 1.0
In [5]:
pboot_mean = pygsti.drivers.to_mean_model(gauge_opt_pboot_models, estimated_model)
pboot_std  = pygsti.drivers.to_std_model(gauge_opt_pboot_models, estimated_model)

#Summary of the error bars
print("Parametric bootstrapped error bars, with", numGatesets, "resamples\n")
print("Error in rho vec:") 
print(pboot_std['rho0'], end='\n\n')
print("Error in effect vecs:")
print(pboot_std['Mdefault'], end='\n\n')
print("Error in Gi:")
print(pboot_std['Gi'], end='\n\n')
print("Error in Gx:")
print(pboot_std['Gx'], end='\n\n')
print("Error in Gy:")
print(pboot_std['Gy'])
Parametric bootstrapped error bars, with 10 resamples

Error in rho vec:
TPSPAMVec with dimension 4
 0.71   0   0   0


Error in effect vecs:
TPPOVM with effect vectors:
0: FullSPAMVec with dimension 4
   0   0   0   0

1: ComplementSPAMVec with dimension 4
 1.41   0   0   0



Error in Gi:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gx:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gy:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0

Non-parametric Bootstrapping

Here we do non-parametric bootstrapping, as indicated by the 'nonparametric' argument below. The output is again eventually stored in the "mean" and "std" Models, which hold the mean and standard deviation values of the set of bootstrapped models (after gauge optimization). It is this latter "standard deviation Model" which holds the collection of error bars. Note: due to print setting issues, the outputs that are printed here will not necessarily reflect the true accuracy of the estimates made.

(Technical note: ddof = 1 is by default used when computing the standard deviation -- see numpy.std -- meaning that we are computing a standard deviation of the sample, not of the population.)

In [6]:
#The number of simulated datasets & models made for bootstrapping purposes.  
# For good statistics, should probably be greater than 10.
numModels=10

nonparam_boot_models = pygsti.drivers.make_bootstrap_models(
                          numModels, ds, 'nonparametric', prep_fiducials, meas_fiducials, germs, maxLengths,
                          targetModel=estimated_model, startSeed=0, returnData=False, verbosity=2)
Creating DataSets: 
0 Generating non-parametric dataset.
1 Generating non-parametric dataset.
2 Generating non-parametric dataset.
3 Generating non-parametric dataset.
4 Generating non-parametric dataset.
5 Generating non-parametric dataset.
6 Generating non-parametric dataset.
7 Generating non-parametric dataset.
8 Generating non-parametric dataset.
9 Generating non-parametric dataset.
Creating Models: 
Running MLGST Iteration 0 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244845695931912
  1.180285364189147
  0.9862995636431281
  0.902101822920102
  0.09233562824970798
  0.06946279872379223
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 163.213 (92 data params - 31 model params = expected mean of 61; p-value = 2.89595e-11)
  Completed in 0.4s
  2*Delta(log(L)) = 163.979
  Iteration 1 took 0.5s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 366.335 (168 data params - 31 model params = expected mean of 137; p-value = 0)
  Completed in 0.6s
  2*Delta(log(L)) = 367.229
  Iteration 2 took 0.7s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1002.45 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 1004.83
  Iteration 3 took 0.9s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1811.85 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 1.3s
  2*Delta(log(L)) = 1815.26
  Iteration 4 took 1.5s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2698.48 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.8s
  2*Delta(log(L)) = 2701.78
  Iteration 5 took 2.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1350.7 below upper bound of -2.13485e+06
      2*Delta(log(L)) = 2701.4 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.4s
  2*Delta(log(L)) = 2701.4
  Final MLGST took 1.4s
  
Iterative MLGST Total Time: 7.0s
Running MLGST Iteration 1 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244188636580856
  1.1517827075400349
  0.9655135770562866
  0.9082976321749557
  0.04860263960205439
  0.02186639107735652
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 146.601 (92 data params - 31 model params = expected mean of 61; p-value = 5.25726e-09)
  Completed in 0.3s
  2*Delta(log(L)) = 147.869
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 274.874 (168 data params - 31 model params = expected mean of 137; p-value = 2.78093e-11)
  Completed in 0.3s
  2*Delta(log(L)) = 276.479
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 861.635 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 864.309
  Iteration 3 took 0.9s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1662.57 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 1.1s
  2*Delta(log(L)) = 1664.8
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2456.71 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 2.8s
  2*Delta(log(L)) = 2459.77
  Iteration 5 took 3.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1229.73 below upper bound of -2.13563e+06
      2*Delta(log(L)) = 2459.47 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.7s
  2*Delta(log(L)) = 2459.47
  Final MLGST took 1.7s
  
Iterative MLGST Total Time: 7.8s
Running MLGST Iteration 2 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.2448073300699045
  1.141572198045367
  0.9828516150265091
  0.8821924040403455
  0.03264139759550893
  0.018800305608607182
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 152.32 (92 data params - 31 model params = expected mean of 61; p-value = 9.10147e-10)
  Completed in 0.4s
  2*Delta(log(L)) = 152.971
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 298.087 (168 data params - 31 model params = expected mean of 137; p-value = 5.62883e-14)
  Completed in 0.3s
  2*Delta(log(L)) = 299.453
  Iteration 2 took 0.5s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 937.718 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 941.677
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1719.38 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 1.3s
  2*Delta(log(L)) = 1724.64
  Iteration 4 took 1.6s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2633.13 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.7s
  2*Delta(log(L)) = 2639.19
  Iteration 5 took 2.0s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1319.4 below upper bound of -2.13523e+06
      2*Delta(log(L)) = 2638.8 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.4s
  2*Delta(log(L)) = 2638.8
  Final MLGST took 1.4s
  
Iterative MLGST Total Time: 6.7s
Running MLGST Iteration 3 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.243897328409793
  1.2017515643930934
  0.9736351960551161
  0.9362239850082709
  0.029479066656238114
  0.006692004333867255
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 174.837 (92 data params - 31 model params = expected mean of 61; p-value = 6.36491e-13)
  Completed in 0.3s
  2*Delta(log(L)) = 175.493
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 325.315 (168 data params - 31 model params = expected mean of 137; p-value = 0)
  Completed in 0.4s
  2*Delta(log(L)) = 326.067
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 858.545 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 859.868
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1692.02 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 1.3s
  2*Delta(log(L)) = 1694.63
  Iteration 4 took 1.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2494.42 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 2.2s
  2*Delta(log(L)) = 2497.55
  Iteration 5 took 2.6s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1248.61 below upper bound of -2.13547e+06
      2*Delta(log(L)) = 2497.23 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.5s
  2*Delta(log(L)) = 2497.23
  Final MLGST took 1.5s
  
Iterative MLGST Total Time: 7.5s
Running MLGST Iteration 4 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.246202328766042
  1.1520853452470667
  0.9764545650798362
  0.9066489771225429
  0.07354357830082349
  0.019302775885633176
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 176.392 (92 data params - 31 model params = expected mean of 61; p-value = 3.78142e-13)
  Completed in 0.3s
  2*Delta(log(L)) = 177.609
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 316.436 (168 data params - 31 model params = expected mean of 137; p-value = 3.33067e-16)
  Completed in 0.3s
  2*Delta(log(L)) = 318.349
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 913.377 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 915.855
  Iteration 3 took 0.8s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1743.39 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 1.3s
  2*Delta(log(L)) = 1746.68
  Iteration 4 took 1.5s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2606.38 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.7s
  2*Delta(log(L)) = 2610.2
  Iteration 5 took 2.0s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1304.96 below upper bound of -2.13509e+06
      2*Delta(log(L)) = 2609.92 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 3.5s
  2*Delta(log(L)) = 2609.92
  Final MLGST took 3.5s
  
Iterative MLGST Total Time: 8.6s
Running MLGST Iteration 5 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244598260247877
  1.1437330330943303
  0.9616678165967568
  0.9507864730743012
  0.032108008334019424
  0.01837190929970882
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 116.694 (92 data params - 31 model params = expected mean of 61; p-value = 2.32753e-05)
  Completed in 0.3s
  2*Delta(log(L)) = 117.694
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 249.287 (168 data params - 31 model params = expected mean of 137; p-value = 1.50925e-08)
  Completed in 0.4s
  2*Delta(log(L)) = 250.142
  Iteration 2 took 0.5s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 884.985 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.9s
  2*Delta(log(L)) = 888.74
  Iteration 3 took 1.0s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1681.69 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 1.3s
  2*Delta(log(L)) = 1685.49
  Iteration 4 took 1.5s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2479.15 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 2.1s
  2*Delta(log(L)) = 2483.66
  Iteration 5 took 2.5s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1241.68 below upper bound of -2.13534e+06
      2*Delta(log(L)) = 2483.36 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 2.1s
  2*Delta(log(L)) = 2483.36
  Final MLGST took 2.1s
  
Iterative MLGST Total Time: 8.0s
Running MLGST Iteration 6 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244215220611
  1.1794071130858117
  0.9621288682814436
  0.9237222775504998
  0.03227630273126832
  0.017188885795577733
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 160.089 (92 data params - 31 model params = expected mean of 61; p-value = 7.8896e-11)
  Completed in 0.3s
  2*Delta(log(L)) = 160.35
  Iteration 1 took 0.3s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 321.056 (168 data params - 31 model params = expected mean of 137; p-value = 1.11022e-16)
  Completed in 0.4s
  2*Delta(log(L)) = 321.985
  Iteration 2 took 0.5s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 907.37 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 908.307
  Iteration 3 took 0.9s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1832.64 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 1.4s
  2*Delta(log(L)) = 1836.11
  Iteration 4 took 1.7s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2766.72 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.9s
  2*Delta(log(L)) = 2771.52
  Iteration 5 took 2.3s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1385.58 below upper bound of -2.13458e+06
      2*Delta(log(L)) = 2771.16 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.3s
  2*Delta(log(L)) = 2771.16
  Final MLGST took 1.3s
  
Iterative MLGST Total Time: 7.0s
Running MLGST Iteration 7 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244815425123308
  1.149149730793523
  0.9449066135785853
  0.8893549404955351
  0.05783633057230394
  0.03235959131326644
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 186.944 (92 data params - 31 model params = expected mean of 61; p-value = 1.04361e-14)
  Completed in 0.3s
  2*Delta(log(L)) = 187.248
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 361.082 (168 data params - 31 model params = expected mean of 137; p-value = 0)
  Completed in 0.4s
  2*Delta(log(L)) = 361.613
  Iteration 2 took 0.5s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 916.064 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 918.454
  Iteration 3 took 0.9s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1693.04 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 1.4s
  2*Delta(log(L)) = 1696.61
  Iteration 4 took 1.6s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2570.75 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.8s
  2*Delta(log(L)) = 2575.11
  Iteration 5 took 2.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1287.42 below upper bound of -2.13547e+06
      2*Delta(log(L)) = 2574.84 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 5.7s
  2*Delta(log(L)) = 2574.84
  Final MLGST took 5.7s
  
Iterative MLGST Total Time: 11.4s
Running MLGST Iteration 8 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244035041550426
  1.1358903361549433
  0.9427163127298118
  0.9110426801402571
  0.09879829799276398
  0.02505584386042377
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 112.981 (92 data params - 31 model params = expected mean of 61; p-value = 5.91678e-05)
  Completed in 0.3s
  2*Delta(log(L)) = 113.15
  Iteration 1 took 0.4s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 264.276 (168 data params - 31 model params = expected mean of 137; p-value = 4.06604e-10)
  Completed in 0.4s
  2*Delta(log(L)) = 264.144
  Iteration 2 took 0.5s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 837.798 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 838.614
  Iteration 3 took 1.0s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1605.14 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 1.6s
  2*Delta(log(L)) = 1606.16
  Iteration 4 took 1.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2494.35 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.8s
  2*Delta(log(L)) = 2496.13
  Iteration 5 took 2.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1247.91 below upper bound of -2.13539e+06
      2*Delta(log(L)) = 2495.82 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 3.3s
  2*Delta(log(L)) = 2495.82
  Final MLGST took 3.3s
  
Iterative MLGST Total Time: 9.1s
Running MLGST Iteration 9 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244395806039566
  1.1575590718440645
  0.9761127108175292
  0.9288056665120381
  0.07242344826070647
  0.02966632169440047
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.244164062089174
  1.169453419312999
  0.9475116980891475
  0.9403516426340927
  2.709096598579531e-16
  1.7684881472925075e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 144.031 (92 data params - 31 model params = expected mean of 61; p-value = 1.14063e-08)
  Completed in 0.3s
  2*Delta(log(L)) = 144.887
  Iteration 1 took 0.3s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 311.047 (168 data params - 31 model params = expected mean of 137; p-value = 1.44329e-15)
  Completed in 0.4s
  2*Delta(log(L)) = 313.029
  Iteration 2 took 0.4s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 885.685 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 889.834
  Iteration 3 took 0.9s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1697.91 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 1.2s
  2*Delta(log(L)) = 1702.77
  Iteration 4 took 1.4s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2511.4 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.9s
  2*Delta(log(L)) = 2516.97
  Iteration 5 took 2.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1258.34 below upper bound of -2.13552e+06
      2*Delta(log(L)) = 2516.69 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 1.6s
  2*Delta(log(L)) = 2516.69
  Final MLGST took 1.6s
  
Iterative MLGST Total Time: 6.8s
In [7]:
gauge_opt_npboot_models = pygsti.drivers.gauge_optimize_model_list(nonparam_boot_models, estimated_model,
                                                                 plot=False) #plotting removed w/matplotlib
Spam weight 0
Spam weight 1
Spam weight 2
Spam weight 3
Spam weight 4
Spam weight 5
Spam weight 6
Spam weight 7
Spam weight 8
Spam weight 9
Spam weight 10
Spam weight 11
Spam weight 12
Best SPAM weight is 1.0
In [8]:
npboot_mean = pygsti.drivers.to_mean_model(gauge_opt_npboot_models, estimated_model)
npboot_std  = pygsti.drivers.to_std_model(gauge_opt_npboot_models, estimated_model)

#Summary of the error bars
print("Non-parametric bootstrapped error bars, with", numGatesets, "resamples\n")
print("Error in rho vec:")
print(npboot_std['rho0'], end='\n\n')
print("Error in effect vecs:")
print(npboot_std['Mdefault'], end='\n\n')
print("Error in Gi:")
print(npboot_std['Gi'], end='\n\n')
print("Error in Gx:")
print(npboot_std['Gx'], end='\n\n')
print("Error in Gy:")
print(npboot_std['Gy'])
Non-parametric bootstrapped error bars, with 10 resamples

Error in rho vec:
TPSPAMVec with dimension 4
 0.71   0   0   0


Error in effect vecs:
TPPOVM with effect vectors:
0: FullSPAMVec with dimension 4
   0   0   0   0

1: ComplementSPAMVec with dimension 4
 1.41   0   0   0



Error in Gi:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gx:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gy:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0

In [9]:
loglog(npboot_std.to_vector(),pboot_std.to_vector(),'.')
loglog(np.logspace(-4,-2,10),np.logspace(-4,-2,10),'--')
xlabel('Non-parametric')
ylabel('Parametric')
xlim((1e-4,1e-2)); ylim((1e-4,1e-2))
title('Scatter plot comparing param vs. non-param bootstrapping error bars.')
Out[9]:
Text(0.5,1,'Scatter plot comparing param vs. non-param bootstrapping error bars.')
In [ ]: