GoNB is a Go notebook kernel. It allows one to easily run Go code in a Jupyter Notebook and variations.
In this tutorial we will walk through most of its features, and explain how it works.
See the README.md's Installation section. It also includes a Docker with Jupyter+GoNB pre-installed, that makes it trivial.
Whenever a cell is executed, gonb saves the cell content of the cell to a Go file, auto-imports missing dependencies (when it can guess), compiles and runs it. It may seem a lot, but its pretty fast (except maybe the first cell run that is usually a little slower) and feels interactive.
func main() {
fmt.Printf("Hello World!")
}
Hello World!
Easy, right ? Now when trying different things, to avoid having to write func main()
at every cell, gonb provides a short cut: "%%". Anything after a "%%" will be wrapped inside a func main() { ... }
.
So, let's redo our example above:
%%
fmt.Printf("Hello World!")
Hello World!
Imports, functions, constants, types and variables global declarations are memorized once executed, and carry over from one cell to another.
So one can work on different functions let's say on different cells.
Let's test it out:
func incr[T interface{constraints.Float|constraints.Integer}](x T) T {
return x+T(1)
}
Ok, now we have incr
defined to any numeric type, we can use it in all our future cells.
Some quick tests:
%%
x := incr(1)
y := incr(math.Pi)
fmt.Printf("incr: x=%d, y=%f\n", x, y)
incr: x=2, y=4.141593
Note: Only the various declarations are carried over from one cell to another, not the results of the execution, including updates to variables.
So for instance, if we initialize a variable
startValue
with 1, then increment it in one cell. Next time we execute a new cell, it will be again initialized to 1.
var startValue = float32(1)
%%
startValue = incr(startValue)
fmt.Printf("current startValue=%f\n", startValue)
current startValue=2.000000
Now if we execute again, startValue
is again initialized to 1:
%%
fmt.Printf("current startValue=%f\n", startValue)
current startValue=1.000000
If one wants to save results calculated from one cell to another, GoNB includes the github.com/janpfeifer/gonb/cache
package that makes it trivial to save and load previously generated results.
Example: Below VeryExpensive
is only called once for CachedValue
, so you will notice that if you run the cell multiple times, it will display always the same number, while NonCachedValue
will always call VeryExpensive
again, and display another number. So the string "...calculating..." is printed twice only the first time.
// Temporary fix until new release v0.6.0 propagates.
import (
"math/rand"
"github.com/janpfeifer/gonb/cache"
)
func VeryExpensive() int {
fmt.Println("\t...VeryExpensive() call...")
return rand.Intn(1000)
}
var (
CachedValue = cache.Cache("expensive", VeryExpensive)
NonCachedValue = VeryExpensive()
)
%%
fmt.Printf("NonCachedValue=%d\n", NonCachedValue)
fmt.Printf(" CachedValue=%d\n", CachedValue)
...VeryExpensive() call... ...VeryExpensive() call... NonCachedValue=976 CachedValue=16
The cache
package has many more features, check out its documentation.
Now, we don't want to have NonCachedValue
execute VeryExpensive
at every cell, so let's remove its definition.
See how to manage memorized definitions using %help
, in the "Managing Memorized Definitions" section, it's displayed at the end of the tutorial.
%rm NonCachedValue CachedValue
. removed var NonCachedValue . removed var CachedValue
A few things to remember from imports in gonb:
import "fmt"
, and it just worked).go get
before compiling the code. This automatically fetches an external import dependency. That is convenient in most cases, but in case you want to get an external Go module at an specific version, you can do it manually with something like !*go get <github.com/user/my_go_module>@<my_version>
. See below on running shell commands.Let's create a simple example that imports a delighful progress-bar library. Notice it automatically fetches the lastest version of the library github.com/schollz/progressbar/v3
-- and the execution of the cell the first time may take a few seconds because of that.
import progressbar "github.com/schollz/progressbar/v3"
%%
bar := progressbar.NewOptions(100,
progressbar.OptionUseANSICodes(true),
progressbar.OptionShowIts(),
progressbar.OptionSetItsString("steps"))
for i := 0; i < 100; i++ {
bar.Add(1)
time.Sleep(40 * time.Millisecond)
}
fmt.Printf("\nDone\n")
100% |████████████████████████████████████████| (25 steps/s) [3s:0s]:0s] Done
One of the things that makes working in Notebooks better than using a terminal is that one can display rich content, like dynamically generated images, plots, HTML, even videos and sound.
We'll follow with a few examples of what is already supported.
gonb includes the accompanying library gonbUI that handles the interfacing to the Notebook through a very simple API:
import "github.com/janpfeifer/gonb/gonbui"
%%
gonbui.DisplayHtml(`<span style="background:pink; color:#111; border-radius: 3px; border: 3px solid orange; font-size: 18px;">I 🧡 GoNB!</span>`)
Markdown output is also supported -- and used for the %help
command, see the bottom of the tutorial.
This also renders math formulas using latex, use $x^2$
for formulas inlined in text, or
$$x^2$$
for formulas in their own line.
Example:
%%
gonbui.DisplayMarkdown("#### Objective\n\n1. Have fun coding **Go**;\n1. Profit...\n"+
`$$f(x) = \int_{-\infty}^{\infty} e^{-x^2} dx$$`)
Let's draw a fractal, using another fun package: github.com/benc-uk/gofract
import "github.com/benc-uk/gofract/pkg/fractals"
import "github.com/benc-uk/gofract/pkg/colors"
%%
imgWidth := 320
// Default fractal
f := fractals.Fractal{
FractType: "mandelbrot",
Center: fractals.ComplexPair{-0.6, 0.0},
MagFactor: 1.0,
MaxIter: 90,
W: 3.0,
H: 2.0,
ImgWidth: imgWidth,
JuliaSeed: fractals.ComplexPair{0.355, 0.355},
InnerColor: "#000000",
FullScreen: false,
ColorRepeats: 2,
}
gradient := colors.GradientTable{}
gradient.AddToTable("#000762", 0.0)
gradient.AddToTable("#0B48C3", 0.2)
gradient.AddToTable("#ffffff", 0.4)
gradient.AddToTable("#E3A000", 0.5)
gradient.AddToTable("#000762", 0.9)
imgHeight := int(float64(imgWidth) * float64(f.H/f.W))
img := image.NewRGBA(image.Rect(0, 0, f.ImgWidth, imgHeight))
lastRenderTime := f.Render(img, gradient)
fmt.Printf("lastRenderTime=%v\n", lastRenderTime)
gonbui.DisplayImage(img)
lastRenderTime=1.684685
From the amazing SVGo library, I really wish I was that creative. Below is Antony Stark's Shining example, demoed here
import "bytes"
import svgo "github.com/ajstarks/svgo"
import "github.com/janpfeifer/gonb/gonbui"
func Shining(width, height int) string {
buf := bytes.NewBuffer(nil)
canvas := svgo.New(buf)
xp := []int{50, 70, 70, 50, 30, 30}
yp := []int{40, 50, 75, 85, 75, 50}
xl := []int{0, 0, 50, 100, 100}
yl := []int{100, 40, 10, 40, 100}
bgcolor := "rgb(227,78,25)"
bkcolor := "rgb(153,29,40)"
stcolor := "rgb(65,52,44)"
stwidth := 12
stylefmt := "stroke:%s;stroke-width:%d;fill:%s"
canvas.Start(width, height)
canvas.Def()
canvas.Gid("unit")
canvas.Polyline(xl, yl, "fill:none")
canvas.Polygon(xp, yp)
canvas.Gend()
canvas.Gid("runit")
canvas.TranslateRotate(150, 180, 180)
canvas.Use(0, 0, "#unit")
canvas.Gend()
canvas.Gend()
canvas.DefEnd()
canvas.Rect(0, 0, width, height, "fill:"+bgcolor)
canvas.Gstyle(fmt.Sprintf(stylefmt, stcolor, stwidth, bkcolor))
for y := 0; y < height; y += 130 {
for x := -50; x < width; x += 100 {
canvas.Use(x, y, "#unit")
canvas.Use(x, y, "#runit")
}
}
canvas.Gend()
canvas.End()
return buf.String()
}
%%
gonbui.DisplaySvg(Shining(500, 500))
A real pearl!
Since its latest update is not yet "released" (tagged in Git), we needed to get the version on the specific commit. See "Executing Shell Commands" below.
!*go get -u github.com/erkkah/margaid@d60b2efd2f5acc5d8fbbe13eaf85f1532e11a2fb
go: added github.com/erkkah/margaid v0.1.1-0.20230128143048-d60b2efd2f5a
import "bytes"
import "github.com/janpfeifer/gonb/gonbui"
import mg "github.com/erkkah/margaid"
func mgPlot(width, height int) string {
randomSeries := mg.NewSeries()
rand.Seed(time.Now().Unix())
for i := float64(0); i < 10; i++ {
randomSeries.Add(mg.MakeValue(i+1, 200*rand.Float64()))
}
testSeries := mg.NewSeries()
multiplier := 2.1
v := 0.33
for i := float64(0); i < 10; i++ {
v *= multiplier
testSeries.Add(mg.MakeValue(i+1, v))
}
diagram := mg.New(width, height,
mg.WithAutorange(mg.XAxis, testSeries),
mg.WithAutorange(mg.YAxis, testSeries),
mg.WithAutorange(mg.Y2Axis, testSeries),
mg.WithProjection(mg.YAxis, mg.Log),
mg.WithInset(70),
mg.WithPadding(2),
mg.WithColorScheme(90),
mg.WithBackgroundColor("#f8f8f8"),
)
diagram.Line(testSeries, mg.UsingAxes(mg.XAxis, mg.YAxis), mg.UsingMarker("square"), mg.UsingStrokeWidth(1))
diagram.Smooth(testSeries, mg.UsingAxes(mg.XAxis, mg.Y2Axis), mg.UsingStrokeWidth(3.14))
diagram.Smooth(randomSeries, mg.UsingAxes(mg.XAxis, mg.YAxis), mg.UsingMarker("filled-circle"))
diagram.Axis(testSeries, mg.XAxis, diagram.ValueTicker('f', 0, 10), false, "X")
diagram.Axis(testSeries, mg.YAxis, diagram.ValueTicker('f', 1, 2), true, "Y")
diagram.Frame()
diagram.Title("A diagram of sorts 📊 📈")
buf := bytes.NewBuffer(nil)
diagram.Render(buf)
return buf.String()
}
%%
gonbui.DisplaySvg(mgPlot(640, 480))
UpdateHtml
¶Still using Margaid but now we animate a Sin(x)
plot varying the frequency from 0.0 to 10.0, every 10 milliseconds. This demonstrates gonbui.UpdateHtml(id, html)
: it allows a transient HTML cell to be updated in the middle of the execution of a cell.
Note: This also works with markdown, use
gonbui.UpdateMarkdown(id, markdown)
instead.
import (
"bytes"
"math"
"time"
"github.com/janpfeifer/gonb/gonbui"
mg "github.com/erkkah/margaid"
)
func mgSinPlot(width, height int, freq float64) string {
series := mg.NewSeries()
const numPoints = 100
for i := 0; i < numPoints; i++ {
x := float64(i) / float64(numPoints) * 2.0 * math.Pi * freq
series.Add(mg.MakeValue(x, math.Sin(x)))
}
diagram := mg.New(width, height,
mg.WithAutorange(mg.XAxis, series),
mg.WithAutorange(mg.YAxis, series),
mg.WithBackgroundColor("#f8f8f8"),
)
diagram.Smooth(series, mg.UsingAxes(mg.XAxis, mg.YAxis), mg.UsingStrokeWidth(3.14))
diagram.Frame()
diagram.Title("Animated Sine")
buf := bytes.NewBuffer(nil)
diagram.Render(buf)
return buf.String()
}
%%
htmlCellId := "sin_plot"+gonbui.UniqueId()
plotSvg := ""
ticker := time.NewTicker(10 * time.Millisecond)
for freq := 0.0; freq <= 10.0; freq += 0.05 {
plotSvg = mgSinPlot(1024, 400, freq)
gonbui.UpdateHtml(htmlCellId, plotSvg)
<-ticker.C
}
ticker.Stop()
// Erase transient image and re-display it so it can be persisted.
gonbui.UpdateHtml(htmlCellId, "")
gonbui.DisplayHtml(plotSvg)
Another great plotting library.
import (
"bytes"
"math/rand"
"github.com/janpfeifer/gonb/gonbui"
"gonum.org/v1/plot"
"gonum.org/v1/plot/plotter"
"gonum.org/v1/plot/plotutil"
"gonum.org/v1/plot/vg"
)
// randomPoints returns some random x, y points.
func randomPoints(n int) plotter.XYs {
pts := make(plotter.XYs, n)
for i := range pts {
if i == 0 {
pts[i].X = rand.Float64()
} else {
pts[i].X = pts[i-1].X + rand.Float64()
}
pts[i].Y = pts[i].X + 10*rand.Float64()
}
return pts
}
func GonumPlotExample(width, height int, format string) []byte {
rand.Seed(int64(0))
p := plot.New()
p.Title.Text = "Plotutil example"
p.X.Label.Text = "X"
p.Y.Label.Text = "Y"
err := plotutil.AddLinePoints(p,
"First", randomPoints(15),
"Second", randomPoints(15),
"Third", randomPoints(15))
if err != nil {
panic(err)
}
buf := bytes.NewBuffer(nil)
writerTo, err := p.WriterTo(vg.Points(float64(width)), vg.Points(float64(height)), format)
if err != nil {
panic(err)
}
writerTo.WriteTo(buf)
return buf.Bytes()
}
%%
gonbui.DisplayPng(GonumPlotExample(400, 200, "png"))
The version in SVG looks better though:
%%
gonbui.DisplaySvg(string(GonumPlotExample(400, 200, "svg")))
Plotly has really fancy plots, worth exploring for generating exquisite reports.
Currently supported using github.com/MetalBlueberry/go-plotly
library.
Experimental: but if you bump into any issues please report, and we'll try to fix it.
import (
"github.com/janpfeifer/gonb/gonbui/plotly"
grob "github.com/MetalBlueberry/go-plotly/graph_objects"
)
%%
fig := &grob.Fig{
Data: grob.Traces{
&grob.Bar{
Type: grob.TraceTypeBar,
X: []float64{1, 2, 3},
Y: []float64{1, 2, 3},
},
},
Layout: &grob.Layout{
Title: &grob.LayoutTitle{
Text: "A Figure Specified By Go Struct",
},
},
}
plotly.DisplayFig(fig)
If using a Jupyter-Lab ran with access to X11, one can also experiment with desktop UI programs. Let's try the Hello World from Fyne, a popular, high quality graphical application toolkit for Go.
Important Note: Disabled by default: since this have many requirements, and won't run in a test environment. But feel free to copy&paste the code to a new cell and run it!
Note 1: Fyne needs to link many C++ libraries, so the firt time this is used it takes a few minutes to compile everything. But afterwards it becomes immediate and interactive.
Note 2: To compile it it requires some libraries present in the system. See Fyne's Getting Started.
Note 3: Remember to close the newly created small "Hello World!" window: GoNB is blocking while running a cell, and the cell only finishes to execute when the window is closed. Here we add a timeout just in case.
import (
"log"
"time"
"fyne.io/fyne/v2/app"
"fyne.io/fyne/v2/widget"
)
%%
a := app.New()
w := a.NewWindow("Hello World")
go func() {
<- time.Tick(10 * time.Second)
log.Printf("Timed out, exiting...")
os.Exit(1)
}()
w.SetContent(widget.NewLabel("Hello World!"))
w.ShowAndRun()
fmt.Println("Goodbye!")
Since there is always only one definition per function name, it's not possible for
each cell to have it's own init() function.
Instead GoNB converts any function named func init_something()
to init()
before
compiling and executing the cells.
This way each cell can create its own func init_...()
and have it called at every
cell execution.
Example:
func init_a() {
fmt.Println("init_a")
}
%%
fmt.Println("main")
init_a main
func init_b() {
fmt.Println("init_b")
}
%%
fmt.Println("main")
init_a init_b main
Now, we don't want to have init_a
and init_b
on all of our cells below. So we remove those definitions -- see how to manage memorized definitions using %help
, in the "Managing Memorized Definitions" section, it's displayed at the end of the tutorial.
%rm init_a init_b
. removed func init_a . removed func init_b
The %%
command can also be used to set arguments to the execution of the cell. This makes it easy to configure
different runs of the same code using flags. This is something handy when testing or developing code that is shared
with a normal code that already used flags.
Also,%%
not only wraps the code following it in a func main() { ... }
but also automatically adds a call to flag.Parse()
.
Example:
import (
"flag"
"fmt"
)
var flagWho = flag.String("who", "", "Your name!")
%% --who=world
fmt.Printf("Hello %s!\n", *flagWho)
Hello world!
Alternatively one can also set the arguments for execution with %args
, if not using %%
, as in:
%args --who=Wally
func main() {
flag.Parse()
fmt.Printf("Where is %s?", *flagWho)
}
Where is Wally?
The package gonbui/dom
allows one to manipulate the DOM in the browser in arbitrary ways.
It is very useful if implementing new widgets. Things to check out for:
dom.TransientJavascript
: Executes Javascript in a transient cell output and then erases it (so it's not saved with the notebook).dom.Append
: Appends HTML to a any element identified by an Id.dom.SetInnerHtml
and dom.GetInnerHtml
: Manipulate the innerHTML
of any element.dom.Persist
: Takes the innerHTML
of a transient element, and then displays it normally with gonbui.DisplayHtml
, so it can be saved with the notebook. Useful if using Javascript libraries (plotting for instance) that dyncamically generate content.Check the library documentation for more!
GoNB allows you to use the browser not only to output rich content (HTML, Widgets), but also interact with HTML UI elements.
It provides the widgets
package with a few basic widgets for you to use, and a gonbui/comms
package that you can use to create new widgets -- see docs/FrontEndCommunication.md
file for more details on that.
Let's reuse our Sine curve plotting function below, and connect it to a slider. And then create a button to stop the interaction when we are done.
(To allow this notebook to run automatically, we add a timeout -- if your playing with the tutorial with a live Jupyter Notebook, feel free to change it or remove the timeout)
import (
"github.com/janpfeifer/gonb/gonbui"
"github.com/janpfeifer/gonb/gonbui/dom"
"github.com/janpfeifer/gonb/gonbui/widgets"
)
%%
freq := 5.5 // From 1.0 to 10.0, controlled by a slider
divId := dom.CreateTransientDiv()
slider := widgets.Slider(0, 100, 50).AppendTo(divId).Done()
freqHtml := fmt.Sprintf(` <span id="sin_freq" style="font-family: monospace; font-style: italic; font-size: small; border: 1px solid; border-style: inset; padding-right:5px;">%6.2f</span> `, freq)
dom.Append(divId, freqHtml)
button := widgets.Button("Ok").AppendTo(divId).Done()
// Initial plot.
htmlCellId := "sin_plot"+gonbui.UniqueId()
plotSvg := mgSinPlot(800, 240, freq) // Defined in the cell above.
gonbui.UpdateHtml(htmlCellId, plotSvg)
// Listen to slider, button and a timeout timer.
sliderChan := slider.Listen().LatestOnly()
buttonChan := button.Listen()
loop:
for {
select {
case <- buttonChan.C:
break loop
case value := <-sliderChan.C:
freq := 1.0 + 9.0 * float64(value) / 100.0
dom.SetInnerText("sin_freq", fmt.Sprintf("%6.2f", freq))
plotSvg = mgSinPlot(800, 240, freq) // Defined in the cell above.
gonbui.UpdateHtml(htmlCellId, plotSvg)
case <- time.After(1000 * time.Millisecond):
break loop
}
}
// Erase transient UI and image and re-display it so it can be persisted.
gonbui.UpdateHtml(htmlCellId, "")
dom.Persist(divId)
gonbui.DisplayHtml(plotSvg)
gonbui.DisplayHtml("<p>Done.</p>")
Done.
There are different ways to provide input to a program in GoNB. We list them below and introduce a new one:
%%
command. For instance %% --x=10
will run your cell with the flag x
set to 10. This is handy for instance to test a function with different values, each one in a different cell.os.Stdin
to the desired file.gonbui/widgets
package, described above).gonbui
package has a function to do that, the results of which can be read in from the stdin
afterwards. See the following example:import (
"fmt"
"github.com/janpfeifer/gonb/gonbui"
)
%%
gonbui.RequestInput("Tell me a number: ", false)
var x int
_, err := fmt.Scan(&x)
if err != nil { panic(err) }
fmt.Printf("The number you typed was %d\n", x)
gonbui.RequestInput("Tell me a secret: ", true)
var secret string
_, err = fmt.Scan(&secret)
if err != nil { panic(err) }
fmt.Printf("Shh! Your secret was %q\n", secret)
The output would be something like:
Tell me a number: 42
The number you typed was 42
Tell me a secret: ······
Shh! Your secret was "I🧡GoNB!"
Note: Not executed by default because it breaks the automatic tests, but try it out on a new cell!
There are two variations to execute shell commands. They differ only on the directory from where they are executed.
!
prefix executes what comes next should be executed as a shell command, on the same directory
where the kernel is executed -- typically the same directory where the notebook files is saved.!go version
!pwd ; ls -l
go version go1.22.1 linux/amd64 /home/janpf/Projects/gonb/examples total 320 -rwxr-xr-x 1 janpf janpf 87160 Dec 15 08:39 google_colab_demo.ipynb drwxr-xr-x 3 janpf janpf 4096 Apr 7 10:34 tests -rw-r--r-- 1 janpf janpf 218236 Mar 25 15:26 tutorial.ipynb -rw-r--r-- 1 janpf janpf 10090 Dec 15 08:39 wasm_demo.ipynb
!*
prefix executes what comes next as a shell command, on the temporary directory used
to compile the Go program when executing the cells. This includes the go.mod
file, that
can be manipulated for special use cases, like importing a specific version of a module,
or to redirect
a module to a local directory for development (see Replace
section below)Example:
!*pwd ; ls -l
/tmp/gonb_b75e8f56 total 13720 -rw-r--r-- 1 janpf janpf 1379 Apr 7 10:40 go.mod -rwxr-xr-x 1 janpf janpf 14017798 Apr 7 10:40 gonb_b75e8f56 srwxr-xr-x 1 janpf janpf 0 Apr 7 10:39 gopls_socket -rw-r--r-- 1 janpf janpf 15907 Apr 7 10:40 go.sum -rw-r--r-- 1 janpf janpf 6128 Apr 7 10:40 main.go
You can also use a \
at the end of the line to extend the shell command to multiple lines.
Example:
!((ii=0)) ;\
while ((ii < 5)) ; do \
printf "\rCounting: ${ii} ..." ;\
sleep 1;\
((ii+=1));\
done;\
echo
Counting: 4 ...
If some shell program requires some input from the user, you can precede it with a %with_inputs
or %with_password
(for hidden input) and it will open a text field for typing some arbitrary text input. Example:
%with_password
!sudo -S apt update
For convenience, GoNB defines the following environment variables -- available for the shell scripts (!
and !*
) and for the Go cells:
GONB_DIR
: the directory where commands are executed from. This can be changed with %cd
.GONB_TMP_DIR
: the directory where the temporary Go code, with the cell code, is stored and compiled. This is the directory where !*
scripts are executed. It only changes when a kernel is restarted, and a new temporary directory is created.GONB_PIPE
: is the named pipe directory used to communicate rich content (HTML, images) to the kernel. Only available for Go cells, and a new one is created at every execution. This is used by the gonbui
functions described above, and doesn't need to be accessed directly.GoNB allows also executing the cell with go test
, to demo or debug tests and benchmarks. For that just mark the cell with %test
and if no other parameters are given, it will run the tests and benchmarks defined in the current cell.
If it is the only command in the cell, it will run all tests defined so far.
import "github.com/stretchr/testify/require"
func TestIncr(t *testing.T) {
require.Equal(t, 2, incr(1))
}
func BenchmarkIncr(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = incr(i)
}
}
%test
=== RUN TestIncr --- PASS: TestIncr (0.00s) goos: linux goarch: amd64 pkg: gonb_b75e8f56 cpu: 12th Gen Intel(R) Core(TM) i9-12900K BenchmarkIncr BenchmarkIncr-24 1000000000 0.1362 ns/op PASS
Alternatively, you can provide extra flags that are passed to the test binary. For instance, to execute all benchmarks defined so far, use:
%test -test.bench=. -test.run=Bechmark
go.mod
and go.work
¶GoNB uses go.mod
and understands go.work
-- but won't create it by default.
go.mod
¶Using the !*
command above we can easily "replace" a module to a local directory. This can be very handy
for developing a library in a powerful IDE on the side, and using the GoNB notebook to execute tests
and experiments. Changes in the library (in the IDE) when saved immediate take effect on the next cell execution.
The accompaining library gonbui was implemented mostly in this fashion using a notebook, that started with:
!*go mod edit -replace github.com/janpfeifer/gonb=/home/janpf/Projects/gonb
Check out the results with:
!*cat go.mod
!*go mod edit -replace "github.com/janpfeifer/gonb=${HOME}/Projects/gonb"
!*cat go.mod
module gonb_b75e8f56 go 1.22.1 require ( github.com/MetalBlueberry/go-plotly v0.4.0 github.com/ajstarks/svgo v0.0.0-20211024235047-1546f124cd8b github.com/benc-uk/gofract v0.0.0-20230120162050-a6f644f92fd6 github.com/erkkah/margaid v0.1.1-0.20230128143048-d60b2efd2f5a github.com/janpfeifer/gonb v0.10.0 github.com/schollz/progressbar/v3 v3.14.2 github.com/stretchr/testify v1.8.1 golang.org/x/exp v0.0.0-20240404231335-c0f41cb1a7a0 gonum.org/v1/plot v0.14.0 ) require ( git.sr.ht/~sbinet/gg v0.5.0 // indirect github.com/campoy/embedmd v1.0.0 // indirect github.com/davecgh/go-spew v1.1.1 // indirect github.com/go-fonts/liberation v0.3.1 // indirect github.com/go-latex/latex v0.0.0-20230307184459-12ec69307ad9 // indirect github.com/go-logr/logr v1.4.1 // indirect github.com/go-pdf/fpdf v0.8.0 // indirect github.com/gofrs/uuid v4.4.0+incompatible // indirect github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect github.com/lucasb-eyer/go-colorful v1.0.3 // indirect github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect github.com/pkg/errors v0.9.1 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect github.com/rivo/uniseg v0.4.7 // indirect golang.org/x/image v0.11.0 // indirect golang.org/x/sys v0.17.0 // indirect golang.org/x/term v0.17.0 // indirect golang.org/x/text v0.14.0 // indirect gopkg.in/yaml.v2 v2.4.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect k8s.io/klog/v2 v2.120.1 // indirect ) replace github.com/janpfeifer/gonb => /home/janpf/Projects/gonb
go.work
¶Another way to refer to modules that are being co-developed in local file is using Go workspaces.
In GoNB, a convenient way to achive this is by first creating a go.work
and then selecting the modules to be use from the local disk. Example:
!*rm -f go.work && go work init && go work use . "${HOME}/Projects/gonb"
%goworkfix
- Replace rule for module "github.com/janpfeifer/gonb" to local directory "/home/janpf/Projects/gonb" already exists.
Note:
- GoNB uses
go get
to automatically fetch missing imports. Unfortunatelygo get
doesn't handlego.work
use
definitions. The special command%goworkfix
handles that by adding areplace
entry ingo.mod
for all modules pointed to bygo.work
.- One can use the env variable
GONB_DIR
to refer to the current kernel directory (changed with%cd
).
GoNB tracks for changes in local files in target directories of replace
rules in go.mod
or use
rules in go.work
. This way auto-complete and contextual help stay up-to-date with changes on local files you may be editing on a separate editor.
See %track
and %untrack
to list and control tracking. For instance, in our tutorial, this is what GoNB is tracking:
%track
go build
flags¶Sometimes it can be desirable to change the Go flags used to build the cells being executed. For instance to check for coverage, or enable race condition detection, etc.
GoNB supports this in two ways:
go build
flags with %goflags <values ...>
. %goflags
without any values displays current settings, and %goflags ""
will reset to the default (no extra flags).go build ...
to see the results. This is supported by simply running !*go build ...
, which is conveniently executed (the !*
prefix) in the temporary directory where the cells code maintained by GoNB is located (in a main.go
). E.g.: To look at optimization decisions, use !*go build -gcflags=-m
.See an example in the examples/tests/gcflags.ipynb
file, which is used as an integration test.
Some other features:
The library is new, and there is still development going on, with still some features in the pipeline (more on the displaying side, auto-complete). For those who enjoy coding, help in improving GoNB is very welcome!
Finally, there is also the %help
command, which lists all the currently supported features:
%help
GoNB is a Go kernel that compiles and executes on-the-fly Go code.
When executing a cell, GoNB will save the cell contents (except non-Go commands see
below) into a main.go
file, compile and execute it.
It also saves any global declarations (imports, functions, types, variables, constants)
and reuse them at the next cell execution -- so you can define a function in one
cell, and reuse in the next one. Just the func main()
is not reused.
A hello world
example would look like:
func main() {
fmt.Printf(`Hello world!\n`);
}
But to avoid having to type func main()
all the time, you can use %%
and everything
after is wrapped inside a func main() { ... }
.
So our revised hello world
looks like:
%%
fmt.Printf(`Hello world!\n`)
func init()
¶Since there is always only one definition per function name, it's not possible for
each cell to have its own init() function.
Instead, GoNB converts any function named init_something()
to init()
before
compiling and executing.
This way each cell can create its own init_...()
and have it called at every cell execution.
%%
or %main
: Marks the lines as follows to be wrapped in a func main() {...}
during
execution. A shortcut to quickly execute code. It also automatically includes flag.Parse()
as the very first statement. Anything %%
or %main
are taken as arguments
to be passed to the program -- it resets previous values given by %args
.%args
: Sets arguments to be passed when executing the Go code. This allows one to
use flags as a normal program. Notice that if a value after %%
or %main
is given, it will
overwrite the values here.%autoget
and %noautoget
: Default is %autoget
, which automatically does go get
for
packages not yet available.%cd [<directory>]
: Change current directory of the Go kernel, and the directory from where
the cells are executed. If no directory is given it reports the current directory.%env VAR value
: Sets the environment variable VAR to the given value. These variables
will be available both for Go code and for shell scripts.%goflags <values...>
: Configures list of extra arguments to pass to go build
when compiling the
code for execution of a cell.
If no values are given, it simply shows the current setting.
To reset its value, use %goflags """
.
See example on how to use this in the tutorial.%with_inputs
: will prompt for inputs for the next shell command. Use this if
the next shell command (!
) you execute reads the stdin. Jupyter will require
you to enter one last value after the shell script executes.%with_password
: will prompt for a password passed to the next shell command.
Do this is if your next shell command requires a password.Notice all these commands are executed before any Go code in the same cell.
%list
(or %ls
): Lists all memorized definitions (imports, constants, types, variables and
functions) that are carried from one cell to another.%remove <definitions>
(or %rm <definitions>
): Removes (forgets) given definition(s). Use as key the
value(s) listed with %ls
.%reset [go.mod]
clears all memorized definitions (imports, constants, types, functions, etc.)
as well as re-initializes the go.mod
file.
If the optional go.mod
parameter is given, it will re-initialize only the go.mod
file --
useful when testing different set up of versions of libraries.!<shell_cmd>
: executes the given command on a new shell. It makes it easy to run
commands on the kernels box, for instance to install requirements, or quickly
check contents of directories or files. Lines ending in \
are continued on
the next line -- so multi-line commands can be entered. But each command is
executed in its own shell, that is, variables and state is not carried over.!*<shell_cmd>
: same as !<shell_cmd>
except it first changes directory to
the temporary directory used to compile the go code -- the latest execution
is always saved in the file main.go
. It's also where the go.mod
file for
the notebook is created and maintained. Useful for manipulating go.mod
,
for instance to get a package from some specific version, something
like !*go get github.com/my/package@v3
.Notice that when the cell is executed, first all shell commands are executed, and only after that, if there is any Go code in the cell, it is executed.
A convenient way to develop programs or libraries in GoNB is to use replace
rules in GoNB's go.mod
to your program or library being developed and test
your program from GoNB -- see the
Tutorial's
section "Developing Go libraries with a notebook" for different ways of achieving this.
To manipulate the list of files tracked for changes:
%track [file_or_directory]
: add file or directory to list of tracked files,
which are monitored by GoNB (and 'gopls') for auto-complete or contextual help.
If no file is given, it lists the currently tracked files.%untrack [file_or_directory][...]
: remove file or directory from list of tracked files.
If suffixed with ...
it will remove all files prefixed with the string given (without the
...
). If no file is given, it lists the currently tracked files.For convenience, GoNB defines the following environment variables -- available for the shell
scripts (!
and !*
) and for the Go cells:
GONB_DIR
: the directory where commands are executed from. This can be changed with %cd
.GONB_TMP_DIR
: the directory where the temporary Go code, with the cell code, is stored
and compiled. This is the directory where !*
scripts are executed. It only changes when a kernel
is restarted, and a new temporary directory is created.GONB_PIPE
: is the named pipe directory used to communicate rich content (HTML, images)
to the kernel. Only available for Go cells, and a new one is created at every execution.
This is used by the `GoNBui`` functions described above, and doesn't need to be accessed directly.The package gonbui/widgets
offers widgets that can be used to interact in a more
dynamic way, using the HTML element in the browser. E.g.: buttons, sliders.
It's not necessary to do anything, but, to help debug the communication system with the front-end, GoNB offers a couple of special commands:
%widgets
- install the javascript needed to communicate with the frontend.
This is usually not needed, since it happens automatically when using Widgets.%widgets_hb
- send a heartbeat signal to the front-end and wait for the
reply.
Used for debugging only.GoNB can also compile to WASM and run in the notebook. This is experimental, and likely to change (feedback is very welcome), and can be used to write interactive widgets in Go, in the notebook.
When a cell with %wasm
is executed, a temporary directory is created under the Jupyter root directory
called jupyter_files/<kernel unique id>/
and the cell is compiled to a wasm file and put in that
directory.
Then GONB outputs the javascript needed to run the compiled wam.
In the Go code, the following extra constants/variables are created in the global namespace, and can be used in your Go code:
GonbWasmDir
, GonbWasmUrl
: the directory and url (served by Jupyter) where the generated .wasm
files are read.
Potentially, the user can use it to serve other files.
These are unique for the kernel, but shared among cells.GonbWasmDivId
: When a %wasm
cell is executed, an empty <div id="<unique_id>"></div>
is created with a unique id -- every cell will have a different one.
This is where the Wasm code can dynamically create content.The following environment variables are set when %wasm
is created:
GONB_WASM_SUBDIR
, GONB_WASM_URL
: the directory and url (served by Jupyter) where the generated .wasm
files are read.
Potentially, the user can use it to serve other files.
These environment variables are available for shell scripts (!...
and !*...
special commands) and non-wasm
programs if they want to serve different files from there.If a cell includes the %test
command (anywhere in cell), it is compiled with go test
(as opposed to go build
).
This can be very useful both to demonstrate tests, or simply help develop/debug them in a notebook.
If %test
is given without any flags, it uses by default the flags -test.v
(verbose) and -test.run
defined
with the list of the tests defined in the current cell.
That is, it will run only the tests in the current cell.
Also, if there are any benchmarks in the current cell, it appends the flag -test.bench=.
and runs the benchmarks
defined in the current cell.
Alternatively one can use %test <flags>
, and the flags
are passed to the binary compiled with go test
.
Remember that test flags require to be prefixed with test.
.
So for a verbose output, use %test -test.v
.
For benchmarks, run %test -test.bench=. -test.run=Benchmark
.
See examples in the gotest.ipynb
notebook here.
The following are special commands that change how the cell is interpreted, so they are prefixed with %%
(two '%'
symbols). They try to follow IPython's Cell Magic.
They must always appear as the first line of the cell.
The contents in the cells are not assumed to be Go, so auto-complete and contextual help are disabled in those cells.
%%writefile
¶%%writefile [-a] <filePath>
Write contents of the cell (except the first line with the '%%writefile') to the given <filePath>
. If -a
is given
it will append the cell contents to the file.
This can be handy if for instance the notebook needs to write a configuration file, or simply to dump the code inside the cell into some file.
File path passes through a tilde (~
) expansion to the user's home directory, as well as environment variable substitution (e.g.: ${HOME}
or $MY_DIR/a/b
).
%%script
, %%bash
and %%sh
¶%%script <command>
Execute <command>
and feed it (STDIN
) with the contents of the cell. The %%bash
and %%sh
magic is an alias to %%script bash
and %%script sh
respectively.
Generally, a convenient way to run larger scripts.
%goworkfix
: work around 'go get' inability to handle 'go.work' files. If you are
using 'go.work' file to point to locally modified modules, consider using this. It creates
'go mod edit --replace' rules to point to the modules pointed to the 'use' rules in 'go.work'
file.
It overwrites/updates 'replace' rules for those modules, if they already exist. See
tutorial for an example.