An example showing how to generate bootstrapped error bars.

In [1]:
from __future__ import print_function
In [2]:
import os
import sys
import time
import json

import pygsti
from pygsti.construction import std1Q_XYI

%pylab inline
Populating the interactive namespace from numpy and matplotlib
In [3]:
#Get a GST estimate (similar to Tutorial 0)

# 1) get the target Model
target_model = std1Q_XYI.target_model()

# 2) get the building blocks needed to specify which operation sequences are needed
prep_fiducials, meas_fiducials = std1Q_XYI.prepStrs, std1Q_XYI.effectStrs
germs = std1Q_XYI.germs
maxLengths = [1,2,4,8,16]

# 3) generate "fake" data from a depolarized version of target_model
mdl_datagen = target_model.depolarize(op_noise=0.1, spam_noise=0.001)
listOfExperiments = pygsti.construction.make_lsgst_experiment_list(
    target_model, prep_fiducials, meas_fiducials, germs, maxLengths)
ds = pygsti.construction.generate_fake_data(mdl_datagen, listOfExperiments, nSamples=1000,
                                            sampleError="binomial", seed=1234)


results = pygsti.do_stdpractice_gst(ds, target_model, prep_fiducials, meas_fiducials,
                                    germs, maxLengths, modes="TP")
estimated_model = results.estimates['TP'].models['single']
-- Std Practice:  Iter 1 of 1  (TP) --: 
  --- Circuit Creation ---
  --- LGST ---
  --- Iterative MLGST: [##################################################] 100.0%  1282 operation sequences ---
  Iterative MLGST Total Time: 2.3s
  --- Re-optimizing logl after robust data scaling ---
  -- Performing 'single' gauge optimization on TP estimate --
  -- Conveying 'single' gauge optimization to TP.Robust+ estimate --

Parametric Bootstrapping

Here we do parametric bootstrapping, as indicated by the 'parametric' argument below. The output is eventually stored in the "mean" and "std" Models, which hold the mean and standard deviation values of the set of bootstrapped models (after gauge optimization). It is this latter "standard deviation Model" which holds the collection of error bars. Note: due to print setting issues, the outputs that are printed here will not necessarily reflect the true accuracy of the estimates made.

In [4]:
#The number of simulated datasets & models made for bootstrapping purposes.  
# For good statistics, should probably be greater than 10.
numGatesets=10

param_boot_models = pygsti.drivers.make_bootstrap_models(
                        numGatesets, ds, 'parametric', prep_fiducials, meas_fiducials, germs, maxLengths,
                        inputModel=estimated_model, startSeed=0, returnData=False,
                        verbosity=2)
Creating DataSets: 
0 Generating parametric dataset.
1 Generating parametric dataset.
2 Generating parametric dataset.
3 Generating parametric dataset.
4 Generating parametric dataset.
5 Generating parametric dataset.
6 Generating parametric dataset.
7 Generating parametric dataset.
8 Generating parametric dataset.
9 Generating parametric dataset.
Creating Models: 
Running MLGST Iteration 0 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.243740981718673
  1.196825270321894
  0.9803977850297132
  0.9189829897008691
  0.04573617221601241
  0.02684031856476036
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 54.6619 (92 data params - 31 model params = expected mean of 61; p-value = 0.703349)
  Completed in 0.2s
  2*Delta(log(L)) = 54.714
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 129.189 (168 data params - 31 model params = expected mean of 137; p-value = 0.670269)
  Completed in 0.2s
  2*Delta(log(L)) = 129.277
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 390.627 (450 data params - 31 model params = expected mean of 419; p-value = 0.83646)
  Completed in 0.4s
  2*Delta(log(L)) = 390.768
  Iteration 3 took 0.4s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 834.331 (862 data params - 31 model params = expected mean of 831; p-value = 0.46099)
  Completed in 0.7s
  2*Delta(log(L)) = 834.617
  Iteration 4 took 0.7s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1249.69 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.505154)
  Completed in 1.0s
  2*Delta(log(L)) = 1250.2
  Iteration 5 took 1.0s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 625.055 below upper bound of -2.13561e+06
      2*Delta(log(L)) = 1250.11 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.501789)
    Completed in 0.4s
  2*Delta(log(L)) = 1250.11
  Final MLGST took 0.4s
  
Iterative MLGST Total Time: 3.0s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 1 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244955776409719
  1.1654832900216188
  0.956272084807351
  0.9195971979216845
  0.04819500202308558
  0.019674658291966154
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 69.4303 (92 data params - 31 model params = expected mean of 61; p-value = 0.214644)
  Completed in 0.2s
  2*Delta(log(L)) = 69.5064
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 139.522 (168 data params - 31 model params = expected mean of 137; p-value = 0.424013)
  Completed in 0.2s
  2*Delta(log(L)) = 139.487
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 425.208 (450 data params - 31 model params = expected mean of 419; p-value = 0.406563)
  Completed in 0.4s
  2*Delta(log(L)) = 424.889
  Iteration 3 took 0.4s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 860.208 (862 data params - 31 model params = expected mean of 831; p-value = 0.234373)
  Completed in 0.6s
  2*Delta(log(L)) = 860.294
  Iteration 4 took 0.7s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1236.14 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.612136)
  Completed in 1.0s
  2*Delta(log(L)) = 1236.41
  Iteration 5 took 1.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 618.162 below upper bound of -2.136e+06
      2*Delta(log(L)) = 1236.32 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.610735)
    Completed in 0.4s
  2*Delta(log(L)) = 1236.32
  Final MLGST took 0.4s
  
Iterative MLGST Total Time: 2.9s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 2 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244105950849598
  1.1579831871684374
  0.9724007421046054
  0.9008232174607133
  0.04747934275633835
  0.02335927755344803
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 53.7231 (92 data params - 31 model params = expected mean of 61; p-value = 0.734336)
  Completed in 0.2s
  2*Delta(log(L)) = 53.767
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 118.871 (168 data params - 31 model params = expected mean of 137; p-value = 0.865831)
  Completed in 0.2s
  2*Delta(log(L)) = 118.848
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 410.566 (450 data params - 31 model params = expected mean of 419; p-value = 0.606523)
  Completed in 0.4s
  2*Delta(log(L)) = 411.16
  Iteration 3 took 0.4s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 846.655 (862 data params - 31 model params = expected mean of 831; p-value = 0.345336)
  Completed in 0.6s
  2*Delta(log(L)) = 847.325
  Iteration 4 took 0.7s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1285.78 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.241266)
  Completed in 1.2s
  2*Delta(log(L)) = 1286.8
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 643.359 below upper bound of -2.13608e+06
      2*Delta(log(L)) = 1286.72 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.235548)
    Completed in 0.5s
  2*Delta(log(L)) = 1286.72
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.2s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 3 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.245026706871426
  1.1581551644431194
  0.9751123814360809
  0.917116445045877
  0.04948557817173936
  0.017700857933000646
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 54.7477 (92 data params - 31 model params = expected mean of 61; p-value = 0.700454)
  Completed in 0.2s
  2*Delta(log(L)) = 54.7464
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 123.896 (168 data params - 31 model params = expected mean of 137; p-value = 0.781645)
  Completed in 0.2s
  2*Delta(log(L)) = 123.873
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 375.888 (450 data params - 31 model params = expected mean of 419; p-value = 0.9358)
  Completed in 0.4s
  2*Delta(log(L)) = 376.466
  Iteration 3 took 0.4s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 787.336 (862 data params - 31 model params = expected mean of 831; p-value = 0.858584)
  Completed in 0.7s
  2*Delta(log(L)) = 787.672
  Iteration 4 took 0.7s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1173.37 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.942023)
  Completed in 1.1s
  2*Delta(log(L)) = 1173.91
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 586.913 below upper bound of -2.1359e+06
      2*Delta(log(L)) = 1173.83 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.940902)
    Completed in 0.5s
  2*Delta(log(L)) = 1173.83
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.2s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 4 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.243957692250352
  1.1690547567407383
  0.965190202696205
  0.9062510088586136
  0.058565422676061636
  0.020128461433915477
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 73.1954 (92 data params - 31 model params = expected mean of 61; p-value = 0.136202)
  Completed in 0.2s
  2*Delta(log(L)) = 73.3169
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 145.055 (168 data params - 31 model params = expected mean of 137; p-value = 0.302405)
  Completed in 0.2s
  2*Delta(log(L)) = 145.31
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 450.681 (450 data params - 31 model params = expected mean of 419; p-value = 0.137667)
  Completed in 0.4s
  2*Delta(log(L)) = 451.235
  Iteration 3 took 0.5s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 863.504 (862 data params - 31 model params = expected mean of 831; p-value = 0.21086)
  Completed in 0.8s
  2*Delta(log(L)) = 864.535
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1293.67 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.195791)
  Completed in 1.1s
  2*Delta(log(L)) = 1294.9
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 647.394 below upper bound of -2.13607e+06
      2*Delta(log(L)) = 1294.79 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.189784)
    Completed in 0.5s
  2*Delta(log(L)) = 1294.79
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.5s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 5 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244076886752041
  1.1641097446903883
  0.9656589990825546
  0.9256958692572328
  0.05988709636348685
  0.0018089980940076643
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 46.7157 (92 data params - 31 model params = expected mean of 61; p-value = 0.911338)
  Completed in 0.3s
  2*Delta(log(L)) = 46.8907
  Iteration 1 took 0.3s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 110.427 (168 data params - 31 model params = expected mean of 137; p-value = 0.953691)
  Completed in 0.2s
  2*Delta(log(L)) = 110.699
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 421.079 (450 data params - 31 model params = expected mean of 419; p-value = 0.462264)
  Completed in 0.5s
  2*Delta(log(L)) = 421.678
  Iteration 3 took 0.5s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 836.041 (862 data params - 31 model params = expected mean of 831; p-value = 0.444428)
  Completed in 0.8s
  2*Delta(log(L)) = 837.086
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1227.62 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.676188)
  Completed in 1.2s
  2*Delta(log(L)) = 1228.84
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 614.366 below upper bound of -2.13643e+06
      2*Delta(log(L)) = 1228.73 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.668034)
    Completed in 0.5s
  2*Delta(log(L)) = 1228.73
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.6s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 6 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244075532131559
  1.167915782338968
  0.9557843550642068
  0.9176164826710592
  0.07089776735122219
  0.012139475083936863
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 63.587 (92 data params - 31 model params = expected mean of 61; p-value = 0.385456)
  Completed in 0.2s
  2*Delta(log(L)) = 63.6718
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 134.64 (168 data params - 31 model params = expected mean of 137; p-value = 0.541046)
  Completed in 0.2s
  2*Delta(log(L)) = 134.834
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 422.608 (450 data params - 31 model params = expected mean of 419; p-value = 0.441453)
  Completed in 0.5s
  2*Delta(log(L)) = 422.757
  Iteration 3 took 0.5s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 844.495 (862 data params - 31 model params = expected mean of 831; p-value = 0.364826)
  Completed in 0.8s
  2*Delta(log(L)) = 844.732
  Iteration 4 took 0.9s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1269.48 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.351608)
  Completed in 1.1s
  2*Delta(log(L)) = 1269.93
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 634.909 below upper bound of -2.13574e+06
      2*Delta(log(L)) = 1269.82 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.349133)
    Completed in 0.5s
  2*Delta(log(L)) = 1269.82
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.6s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 7 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244480338214524
  1.165763527245094
  0.9338843507514768
  0.894817674590485
  0.03560561553467106
  0.011817184934139907
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 50.8873 (92 data params - 31 model params = expected mean of 61; p-value = 0.818698)
  Completed in 0.2s
  2*Delta(log(L)) = 51.09
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 118.749 (168 data params - 31 model params = expected mean of 137; p-value = 0.867575)
  Completed in 0.2s
  2*Delta(log(L)) = 118.844
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 404.585 (450 data params - 31 model params = expected mean of 419; p-value = 0.684631)
  Completed in 0.5s
  2*Delta(log(L)) = 404.878
  Iteration 3 took 0.5s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 825.414 (862 data params - 31 model params = expected mean of 831; p-value = 0.548141)
  Completed in 0.9s
  2*Delta(log(L)) = 826.085
  Iteration 4 took 0.9s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1179.53 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.925572)
  Completed in 1.2s
  2*Delta(log(L)) = 1180.31
  Iteration 5 took 1.3s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 590.119 below upper bound of -2.13605e+06
      2*Delta(log(L)) = 1180.24 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.923479)
    Completed in 0.6s
  2*Delta(log(L)) = 1180.24
  Final MLGST took 0.6s
  
Iterative MLGST Total Time: 3.8s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 8 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244787150940367
  1.1501560865775227
  0.9566739054733346
  0.9046650260494242
  0.030204804282964466
  0.005737486195909679
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 60.9529 (92 data params - 31 model params = expected mean of 61; p-value = 0.477613)
  Completed in 0.2s
  2*Delta(log(L)) = 60.9843
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 131.827 (168 data params - 31 model params = expected mean of 137; p-value = 0.608762)
  Completed in 0.2s
  2*Delta(log(L)) = 131.864
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 400.15 (450 data params - 31 model params = expected mean of 419; p-value = 0.738292)
  Completed in 0.4s
  2*Delta(log(L)) = 400.548
  Iteration 3 took 0.5s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 785.227 (862 data params - 31 model params = expected mean of 831; p-value = 0.870266)
  Completed in 0.7s
  2*Delta(log(L)) = 786.042
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1167.95 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.954012)
  Completed in 1.2s
  2*Delta(log(L)) = 1168.9
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 584.416 below upper bound of -2.13586e+06
      2*Delta(log(L)) = 1168.83 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.952212)
    Completed in 0.5s
  2*Delta(log(L)) = 1168.83
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.4s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 9 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244086871227867
  1.1742535252889312
  0.9742543570194043
  0.9330064213489485
  0.048119822313218055
  0.014464994594877137
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 47.5036 (92 data params - 31 model params = expected mean of 61; p-value = 0.896923)
  Completed in 0.2s
  2*Delta(log(L)) = 47.5543
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 115.65 (168 data params - 31 model params = expected mean of 137; p-value = 0.90705)
  Completed in 0.2s
  2*Delta(log(L)) = 115.791
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 398.455 (450 data params - 31 model params = expected mean of 419; p-value = 0.757579)
  Completed in 0.4s
  2*Delta(log(L)) = 398.689
  Iteration 3 took 0.5s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 806.048 (862 data params - 31 model params = expected mean of 831; p-value = 0.726382)
  Completed in 0.7s
  2*Delta(log(L)) = 806.647
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1182.69 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.915882)
  Completed in 1.2s
  2*Delta(log(L)) = 1183.65
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 591.796 below upper bound of -2.13645e+06
      2*Delta(log(L)) = 1183.59 (1282 data params - 31 model params = expected mean of 1251; p-value = 0.912964)
    Completed in 0.5s
  2*Delta(log(L)) = 1183.59
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.4s
  -- Adding Gauge Optimized (go0) --
In [5]:
gauge_opt_pboot_models = pygsti.drivers.gauge_optimize_model_list(param_boot_models, estimated_model,
                                                                 plot=False) #plotting support removed w/matplotlib
Spam weight 0
Spam weight 1
Spam weight 2
Spam weight 3
Spam weight 4
Spam weight 5
Spam weight 6
Spam weight 7
Spam weight 8
Spam weight 9
Spam weight 10
Spam weight 11
Spam weight 12
Best SPAM weight is 1.0
In [6]:
pboot_mean = pygsti.drivers.to_mean_model(gauge_opt_pboot_models, estimated_model)
pboot_std  = pygsti.drivers.to_std_model(gauge_opt_pboot_models, estimated_model)

#Summary of the error bars
print("Parametric bootstrapped error bars, with", numGatesets, "resamples\n")
print("Error in rho vec:") 
print(pboot_std['rho0'], end='\n\n')
print("Error in effect vecs:")
print(pboot_std['Mdefault'], end='\n\n')
print("Error in Gi:")
print(pboot_std['Gi'], end='\n\n')
print("Error in Gx:")
print(pboot_std['Gx'], end='\n\n')
print("Error in Gy:")
print(pboot_std['Gy'])
Parametric bootstrapped error bars, with 10 resamples

Error in rho vec:
TPSPAMVec with dimension 4
 0.71   0   0   0


Error in effect vecs:
TPPOVM with effect vectors:
0: FullSPAMVec with dimension 4
   0   0   0   0

1: ComplementSPAMVec with dimension 4
 1.41   0   0   0



Error in Gi:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gx:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gy:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0

Non-parametric Bootstrapping

Here we do non-parametric bootstrapping, as indicated by the 'nonparametric' argument below. The output is again eventually stored in the "mean" and "std" Models, which hold the mean and standard deviation values of the set of bootstrapped models (after gauge optimization). It is this latter "standard deviation Model" which holds the collection of error bars. Note: due to print setting issues, the outputs that are printed here will not necessarily reflect the true accuracy of the estimates made.

(Technical note: ddof = 1 is by default used when computing the standard deviation -- see numpy.std -- meaning that we are computing a standard deviation of the sample, not of the population.)

In [7]:
#The number of simulated datasets & models made for bootstrapping purposes.  
# For good statistics, should probably be greater than 10.
numModels=10

nonparam_boot_models = pygsti.drivers.make_bootstrap_models(
                          numModels, ds, 'nonparametric', prep_fiducials, meas_fiducials, germs, maxLengths,
                          targetModel=estimated_model, startSeed=0, returnData=False, verbosity=2)
Creating DataSets: 
0 Generating non-parametric dataset.
1 Generating non-parametric dataset.
2 Generating non-parametric dataset.
3 Generating non-parametric dataset.
4 Generating non-parametric dataset.
5 Generating non-parametric dataset.
6 Generating non-parametric dataset.
7 Generating non-parametric dataset.
8 Generating non-parametric dataset.
9 Generating non-parametric dataset.
Creating Models: 
Running MLGST Iteration 0 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244212085138186
  1.1843719301091324
  0.9857947532159174
  0.9013030041875327
  0.09547108379285918
  0.04886490428154151
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 170.59 (92 data params - 31 model params = expected mean of 61; p-value = 2.60802e-12)
  Completed in 0.2s
  2*Delta(log(L)) = 171.166
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 344.819 (168 data params - 31 model params = expected mean of 137; p-value = 0)
  Completed in 0.2s
  2*Delta(log(L)) = 345.262
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 991.904 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 993.352
  Iteration 3 took 0.5s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1815.39 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.9s
  2*Delta(log(L)) = 1817.16
  Iteration 4 took 0.9s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2701.38 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.1s
  2*Delta(log(L)) = 2703.31
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1351.47 below upper bound of -2.13485e+06
      2*Delta(log(L)) = 2702.94 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.5s
  2*Delta(log(L)) = 2702.94
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.6s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 1351.47 below upper bound of -2.13485e+06
      2*Delta(log(L)) = 2702.94 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.3s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 1 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244167171173532
  1.153514315008981
  0.9636315553213499
  0.908474545441188
  0.046056950903486917
  0.014789568961852528
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 147.684 (92 data params - 31 model params = expected mean of 61; p-value = 3.78334e-09)
  Completed in 0.2s
  2*Delta(log(L)) = 148.749
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 274.913 (168 data params - 31 model params = expected mean of 137; p-value = 2.75325e-11)
  Completed in 0.2s
  2*Delta(log(L)) = 276.308
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 866.238 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 868.748
  Iteration 3 took 0.5s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1658.63 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 1661.19
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2467.46 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.2s
  2*Delta(log(L)) = 2470.98
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1235.33 below upper bound of -2.13563e+06
      2*Delta(log(L)) = 2470.66 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.5s
  2*Delta(log(L)) = 2470.66
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.5s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 1235.33 below upper bound of -2.13563e+06
      2*Delta(log(L)) = 2470.66 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.3s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 2 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244617505701195
  1.1425181984723756
  0.9821383251044142
  0.8825511904531211
  0.03538596563546793
  0.019645266130539504
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 152.339 (92 data params - 31 model params = expected mean of 61; p-value = 9.04838e-10)
  Completed in 0.2s
  2*Delta(log(L)) = 152.658
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 287.202 (168 data params - 31 model params = expected mean of 137; p-value = 1.08791e-12)
  Completed in 0.2s
  2*Delta(log(L)) = 287.882
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 932.984 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 934.913
  Iteration 3 took 0.6s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1720.38 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 1723.17
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2634.67 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.3s
  2*Delta(log(L)) = 2638.42
  Iteration 5 took 1.3s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1318.98 below upper bound of -2.13523e+06
      2*Delta(log(L)) = 2637.96 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.5s
  2*Delta(log(L)) = 2637.96
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.7s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 1318.98 below upper bound of -2.13523e+06
      2*Delta(log(L)) = 2637.96 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.4s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 3 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.243988002748227
  1.2013993194238075
  0.9736701526820217
  0.9363076178129426
  0.027172808326342654
  0.005255940106641545
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 159.939 (92 data params - 31 model params = expected mean of 61; p-value = 8.27658e-11)
  Completed in 0.2s
  2*Delta(log(L)) = 160.22
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 289.265 (168 data params - 31 model params = expected mean of 137; p-value = 6.24945e-13)
  Completed in 0.2s
  2*Delta(log(L)) = 289.212
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 849.321 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 849.302
  Iteration 3 took 0.6s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1686.29 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 1687.79
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2496.17 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.2s
  2*Delta(log(L)) = 2498.46
  Iteration 5 took 1.3s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1249.03 below upper bound of -2.13547e+06
      2*Delta(log(L)) = 2498.05 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.5s
  2*Delta(log(L)) = 2498.05
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.6s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 1249.03 below upper bound of -2.13547e+06
      2*Delta(log(L)) = 2498.05 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.3s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 4 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.246180502104294
  1.1521822289971684
  0.9764404175045424
  0.906610245735585
  0.07369893163659891
  0.020240938685679086
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 160.269 (92 data params - 31 model params = expected mean of 61; p-value = 7.44789e-11)
  Completed in 0.2s
  2*Delta(log(L)) = 161.452
  Iteration 1 took 0.3s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 268.207 (168 data params - 31 model params = expected mean of 137; p-value = 1.52095e-10)
  Completed in 0.2s
  2*Delta(log(L)) = 269.635
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 906.473 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 908.592
  Iteration 3 took 0.6s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1752.98 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 1756.15
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2618.09 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.1s
  2*Delta(log(L)) = 2621.8
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1310.73 below upper bound of -2.13509e+06
      2*Delta(log(L)) = 2621.45 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.5s
  2*Delta(log(L)) = 2621.45
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.6s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 1310.73 below upper bound of -2.13509e+06
      2*Delta(log(L)) = 2621.45 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.4s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 5 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244326073733323
  1.1441327581502785
  0.9622582316708493
  0.9502766969909989
  0.04152115972837583
  0.029558409756115962
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 123.979 (92 data params - 31 model params = expected mean of 61; p-value = 3.45234e-06)
  Completed in 0.3s
  2*Delta(log(L)) = 125.106
  Iteration 1 took 0.3s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 247.859 (168 data params - 31 model params = expected mean of 137; p-value = 2.10431e-08)
  Completed in 0.2s
  2*Delta(log(L)) = 248.561
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 872.89 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.6s
  2*Delta(log(L)) = 876.046
  Iteration 3 took 0.6s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1682.54 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 1686.15
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2484.67 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.2s
  2*Delta(log(L)) = 2489.05
  Iteration 5 took 1.3s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1244.35 below upper bound of -2.13534e+06
      2*Delta(log(L)) = 2488.7 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.5s
  2*Delta(log(L)) = 2488.7
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.7s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 1244.35 below upper bound of -2.13534e+06
      2*Delta(log(L)) = 2488.7 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.4s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 6 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.243826627679486
  1.181000471428712
  0.9620835176798799
  0.9235835321348868
  0.029932975327267443
  0.01795053898160025
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 167.863 (92 data params - 31 model params = expected mean of 61; p-value = 6.39e-12)
  Completed in 0.2s
  2*Delta(log(L)) = 168.443
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 305.858 (168 data params - 31 model params = expected mean of 137; p-value = 6.43929e-15)
  Completed in 0.2s
  2*Delta(log(L)) = 306.043
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 897.008 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 897.009
  Iteration 3 took 0.5s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1832.6 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.7s
  2*Delta(log(L)) = 1834.65
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2774.87 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.1s
  2*Delta(log(L)) = 2778.26
  Iteration 5 took 1.1s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1388.92 below upper bound of -2.13458e+06
      2*Delta(log(L)) = 2777.85 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.5s
  2*Delta(log(L)) = 2777.85
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.4s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 1388.92 below upper bound of -2.13458e+06
      2*Delta(log(L)) = 2777.85 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.3s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 7 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244526056080279
  1.151251520412904
  0.9439701878450454
  0.8888666392207678
  0.06087224198184396
  0.030748143014395605
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 181.945 (92 data params - 31 model params = expected mean of 61; p-value = 5.79536e-14)
  Completed in 0.2s
  2*Delta(log(L)) = 182.096
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 329.174 (168 data params - 31 model params = expected mean of 137; p-value = 0)
  Completed in 0.2s
  2*Delta(log(L)) = 329.668
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 926.384 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 928.499
  Iteration 3 took 0.5s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1717.46 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.8s
  2*Delta(log(L)) = 1720.98
  Iteration 4 took 0.8s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2590.11 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.2s
  2*Delta(log(L)) = 2594.67
  Iteration 5 took 1.3s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1297.17 below upper bound of -2.13547e+06
      2*Delta(log(L)) = 2594.34 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.5s
  2*Delta(log(L)) = 2594.34
  Final MLGST took 0.5s
  
Iterative MLGST Total Time: 3.6s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 1297.17 below upper bound of -2.13547e+06
      2*Delta(log(L)) = 2594.34 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.3s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 8 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.243854590804392
  1.1359961161784795
  0.9430730695955586
  0.9111704896479723
  0.09720933554376518
  0.03639925023022892
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 115.208 (92 data params - 31 model params = expected mean of 61; p-value = 3.39228e-05)
  Completed in 0.3s
  2*Delta(log(L)) = 115.295
  Iteration 1 took 0.3s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 256.019 (168 data params - 31 model params = expected mean of 137; p-value = 3.0592e-09)
  Completed in 0.2s
  2*Delta(log(L)) = 255.846
  Iteration 2 took 0.3s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 837.178 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 837.916
  Iteration 3 took 0.6s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1599.73 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.9s
  2*Delta(log(L)) = 1600.86
  Iteration 4 took 0.9s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2488.63 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.2s
  2*Delta(log(L)) = 2490.51
  Iteration 5 took 1.2s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1245.07 below upper bound of -2.13539e+06
      2*Delta(log(L)) = 2490.15 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.6s
  2*Delta(log(L)) = 2490.15
  Final MLGST took 0.6s
  
Iterative MLGST Total Time: 3.8s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 1245.07 below upper bound of -2.13539e+06
      2*Delta(log(L)) = 2490.15 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.3s
  -- Adding Gauge Optimized (go0) --
Running MLGST Iteration 9 
--- Circuit Creation ---
   1282 sequences created
   Dataset has 1282 entries: 1282 utilized, 0 requested sequences were missing
--- LGST ---
  Singular values of I_tilde (truncating to first 4 of 6) = 
  4.244088493382835
  1.158491440342675
  0.9765988971692087
  0.9291379020211377
  0.06900998568202234
  0.015665820756577954
  
  Singular values of target I_tilde (truncating to first 4 of 6) = 
  4.2440765871547805
  1.1775300369228388
  0.9531813852473716
  0.9453339713427231
  3.092369762442044e-16
  2.0221726661332703e-16
  
--- Iterative MLGST: Iter 1 of 5  92 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 164.287 (92 data params - 31 model params = expected mean of 61; p-value = 2.04635e-11)
  Completed in 0.2s
  2*Delta(log(L)) = 165.06
  Iteration 1 took 0.2s
  
--- Iterative MLGST: Iter 2 of 5  168 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 296.859 (168 data params - 31 model params = expected mean of 137; p-value = 7.89369e-14)
  Completed in 0.2s
  2*Delta(log(L)) = 298.794
  Iteration 2 took 0.2s
  
--- Iterative MLGST: Iter 3 of 5  450 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 869.949 (450 data params - 31 model params = expected mean of 419; p-value = 0)
  Completed in 0.5s
  2*Delta(log(L)) = 873.417
  Iteration 3 took 0.6s
  
--- Iterative MLGST: Iter 4 of 5  862 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 1691.71 (862 data params - 31 model params = expected mean of 831; p-value = 0)
  Completed in 0.9s
  2*Delta(log(L)) = 1696.13
  Iteration 4 took 0.9s
  
--- Iterative MLGST: Iter 5 of 5  1282 operation sequences ---: 
  --- Minimum Chi^2 GST ---
  Sum of Chi^2 = 2508.01 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
  Completed in 1.2s
  2*Delta(log(L)) = 2513.11
  Iteration 5 took 1.3s
  
  Switching to ML objective (last iteration)
  --- MLGST ---
    Maximum log(L) = 1256.4 below upper bound of -2.13552e+06
      2*Delta(log(L)) = 2512.81 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.6s
  2*Delta(log(L)) = 2512.81
  Final MLGST took 0.6s
  
Iterative MLGST Total Time: 3.8s
  -- Adding Gauge Optimized (go0) --
--- Re-optimizing logl after robust data scaling ---
  --- MLGST ---
    Maximum log(L) = 1256.4 below upper bound of -2.13552e+06
      2*Delta(log(L)) = 2512.81 (1282 data params - 31 model params = expected mean of 1251; p-value = 0)
    Completed in 0.3s
  -- Adding Gauge Optimized (go0) --
In [8]:
gauge_opt_npboot_models = pygsti.drivers.gauge_optimize_model_list(nonparam_boot_models, estimated_model,
                                                                 plot=False) #plotting removed w/matplotlib
Spam weight 0
Spam weight 1
Spam weight 2
Spam weight 3
Spam weight 4
Spam weight 5
Spam weight 6
Spam weight 7
Spam weight 8
Spam weight 9
Spam weight 10
Spam weight 11
Spam weight 12
Best SPAM weight is 1.0
In [9]:
npboot_mean = pygsti.drivers.to_mean_model(gauge_opt_npboot_models, estimated_model)
npboot_std  = pygsti.drivers.to_std_model(gauge_opt_npboot_models, estimated_model)

#Summary of the error bars
print("Non-parametric bootstrapped error bars, with", numGatesets, "resamples\n")
print("Error in rho vec:")
print(npboot_std['rho0'], end='\n\n')
print("Error in effect vecs:")
print(npboot_std['Mdefault'], end='\n\n')
print("Error in Gi:")
print(npboot_std['Gi'], end='\n\n')
print("Error in Gx:")
print(npboot_std['Gx'], end='\n\n')
print("Error in Gy:")
print(npboot_std['Gy'])
Non-parametric bootstrapped error bars, with 10 resamples

Error in rho vec:
TPSPAMVec with dimension 4
 0.71   0   0   0


Error in effect vecs:
TPPOVM with effect vectors:
0: FullSPAMVec with dimension 4
   0   0   0   0

1: ComplementSPAMVec with dimension 4
 1.41   0   0   0



Error in Gi:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gx:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0


Error in Gy:
TPDenseOp with shape (4, 4)
 1.00   0   0   0
   0   0   0   0
   0   0   0   0
   0   0   0   0

In [10]:
loglog(npboot_std.to_vector(),pboot_std.to_vector(),'.')
loglog(np.logspace(-4,-2,10),np.logspace(-4,-2,10),'--')
xlabel('Non-parametric')
ylabel('Parametric')
xlim((1e-4,1e-2)); ylim((1e-4,1e-2))
title('Scatter plot comparing param vs. non-param bootstrapping error bars.')
Out[10]:
Text(0.5, 1.0, 'Scatter plot comparing param vs. non-param bootstrapping error bars.')
In [ ]: