%%html
<script>
function code_toggle() {
if (code_shown){
$('div.input').hide('500');
$('#toggleButton').val('Show Code')
} else {
$('div.input').show('500');
$('#toggleButton').val('Hide Code')
}
code_shown = !code_shown
}
$( document ).ready(function(){
code_shown=false;
$('div.input').hide()
});
</script>
<form action="javascript:code_toggle()"><input type="submit" id="toggleButton" value="Show Code"></form>
import random
from IPython.display import Image
%%javascript
require(['base/js/utils'],
function(utils) {
utils.load_extensions('calico-spell-check', 'calico-document-tools', 'calico-cell-tools');
});
# Reveal.js
from notebook.services.config import ConfigManager
cm = ConfigManager()
cm.update('livereveal', {
'theme': 'white',
'transition': 'none',
'controls': 'true',
'progress': 'true',
})
{'theme': 'white', 'transition': 'none', 'controls': 'true', 'progress': 'true'}
%load_ext tikzmagic
%%html
<style>
.red { color: #E41A1C; }
.orange { color: #FF7F00 }
.yellow { color: #FFC020 }
.green { color: #4DAF4A }
.blue { color: #377EB8; }
.purple { color: #984EA3 }
h1 {
color: #377EB8;
}
ctb_global_show div.ctb_hideshow.ctb_show {
display: inline;
}
div.tabContent {
padding: 0px;
background: #ffffff;
border: 0px;
}
.left {
float: left;
width: 50%;
vertical-align: text-top;
}
.right {
margin-left: 50%;
vertical-align: text-top;
}
.small {
zoom: 0.9;
-ms-zoom: 0.9;
-webkit-zoom: 0.9;
-moz-transform: scale(0.9,0.9);
-moz-transform-origin: left center;
}
.verysmall {
zoom: 0.75;
-ms-zoom: 0.75;
-webkit-zoom: 0.75;
-moz-transform: scale(0.75,0.75);
-moz-transform-origin: left center;
}
.tiny {
zoom: 0.6;
-ms-zoom: 0.6;
-webkit-zoom: 0.6;
-moz-transform: scale(0.6,0.6);
-moz-transform-origin: left center;
}
.rendered_html blockquote {
border-left-width: 0px;
padding: 15px;
margin: 0px;
width: 100%;
}
.rendered_html th {
padding: 0.5em;
border: 0px;
}
.rendered_html td {
padding: 0.25em;
border: 0px;
}
.rise-enabled .reveal .progress span {
background: #377EB8;
}
.present .top {
position: fixed !important;
top: 0 !important;
}
.present .rendered_html * + p, .present .rendered_html p, .present .rendered_html * + br, .present .rendered_html br {
margin: 0.5em 0;
}
.present tr, .present td {
border: 0px;
padding: 0.35em;
}
.present th {
border: 1px;
}
present .prompt {
min-width: 0px !important;
transition-duration: 0s !important;
}
.prompt {
min-width: 0px !important;
transition-duration: 0s !important;
}
.rise-enabled .cell li {
line-height: 135%;
}
</style>
A LM computes a probability for a sequence of words
$$p(\langle w_{1}, \ldots, w_{d} \rangle)$$Useful in a myriad of NLP tasks involving text generation, e.g.
In $n$-gram language models, the probability $p(w_{1}, \ldots, w_{d}) = \prod_{i=1}^{d} p(w_{i} \mid w_{1}, \ldots, w_{i - 1})$ is approximated as:
$$ \begin{aligned} p(w_{1}, \ldots, w_{d}) & \approx \prod_{i=1}^{d} p(w_{i} \mid w_{i - (n - 1)}, \ldots, w_{i - 1}) \\ & \approx \prod_{i=1}^{d} \frac{\text{count}(w_{i - (n - 1)}, \ldots, w_{i})}{\text{count}(w_{i - (n - 1)}, \ldots, w_{i - 1})} \end{aligned} $$Example with a bigram ($n = 2$) language model:
$$ \begin{aligned} p(\langle \text{Natural}, & \text{Language}, \text{Processing} \rangle) \approx \\ & p(\text{Natural}){}\cdot{}p(\text{Language} \mid \text{Natural}) \cdot p(\text{Processing} \mid \text{Language}) \end{aligned} $$At each single time step $t$, the hidden state $\mathbf{h}_t$ is given by:
$$ \begin{aligned} \mathbf{h}_t & = \sigma(\mathbf{W}^h \mathbf{h}_{t-1}+ \mathbf{W}^x \mathbf{x}_t) \in \mathbb{R}^{d_{h}}\\ \end{aligned} $$Input matrix: $\mathbf{W}^x \in \mathbb{R}^{d_{h} \times d_{x}}$
Transition matrix: $\mathbf{W}^h \in \mathbb{R}^{d_{h} \times d_{h}}$
The output $\hat{\mathbf{y}}_t \in [0, 1]^{|V|}$, a probability distribution over words in $V$, is given by:
$$ \begin{aligned} \hat{\mathbf{y}}_{t} & = \text{softmax}(\mathbf{W}^o \mathbf{h}_{t}) \\ \end{aligned} $$Output matrix: $\mathbf{W}^o \in \mathbb{R}^{|V| \times d_{h}}$
and softmax is defined as: $$ \text{softmax}(z) = \frac{1}{\sum_{i=1}^{d_z} e^{z_i}} \left(e^{z_1}, e^{z_2},\ldots,e^{z_{d_z}}\right) $$
The probability that the $t$-th word in the sequence is $w_{j}$ is given by:
$$p(w_{j} \mid w_{t}, \ldots, w_{1}) = \hat{\mathbf{y}}_{t, j}$$Consider the word sequence $\text{encode}(\text{Natural}, \text{Language}, \text{Processing}) \rightarrow (\mathbf{x}_{1}, \mathbf{x}_{2}, \mathbf{x}_{3})$
Reminder: $\mathbf{h}_t = \sigma(\mathbf{W}^h \mathbf{h}_{t-1}+ \mathbf{W}^x \mathbf{x}_t + \mathbf{b})$
$$ \begin{aligned} \mathbf{h}_1 = \sigma(\mathbf{W}^h \mathbf{h}_{0} + \mathbf{W}^x \mathbf{x}_1) &\;& \hat{\mathbf{y}}_{1} = \text{softmax}(\mathbf{W}^o \mathbf{h}_{1}) \\ \mathbf{h}_2 = \sigma(\mathbf{W}^h \mathbf{h}_{1} + \mathbf{W}^x \mathbf{x}_2) &\;& \hat{\mathbf{y}}_{2} = \text{softmax}(\mathbf{W}^o \mathbf{h}_{2}) \\ \mathbf{h}_3 = \sigma(\mathbf{W}^h \mathbf{h}_{2} + \mathbf{W}^x \mathbf{x}_3) &\;& \hat{\mathbf{y}}_{3} = \text{softmax}(\mathbf{W}^o \mathbf{h}_{3}) \\ \end{aligned} $$$$p(\text{Natural}, \text{Language}, \text{Processing}) = \hat{\mathbf{y}}_{1, [\text{Natural}]} \; \hat{\mathbf{y}}_{2, [\text{Language}]} \; \hat{\mathbf{y}}_{3, [\text{Processing}]}$$Recall that $\hat{\mathbf{y}}_{t} \in \mathbb{R}^{|V|}$ is a probability distribution over the vocabulary $V$.
We can train a RNN by minimizing the cross-entropy loss, predicting words instead of classes:
$$ \begin{aligned} J_{t} = - \sum_{i = 1}^{|V|} \mathbf{y}_{t, i} \log \hat{\mathbf{y}}_{t, i}, \quad \text{where} \quad \mathbf{y}_{t, i} = \left\{\begin{array}{ll}1 \; \text{if the $t$-th word is $w_{i}$,}\\0 \, \text{otherwise.}\end{array} \right. \end{aligned} $$Why? Multiply the same matrix $\mathbf{W}^{h}$ at each time step during forward propagation. The norm of the gradient might either tend to 0 (vanish) or be too large (explode).
Words from time steps far away are hardly considered when training to predict the next word.
Example:
or
A RNN is very likely to e.g. put an uniform probability distributions over nouns in $V$, and a low probability everywhere else.
It's an issue with language modeling, question answering, and many other tasks.
Several solutions in the literature:
Bound the gradient to a threshold (Gradient Clipping)
[Pascanu et al. 2013]
Use $\text{ReLU}(x) = \max(0, x)$ (Rectified Linear Units) or similar non-linearities instead of $\text{sigmoid}(x)$ or $\text{tanh}(x)$
[Glorot et al. 2011].
Clever initialization of the transition matrix ($\mathbf{W}^h = \mathbf{I}$)
[Socher et al. 2013, Le et al. 2015].
Use different recurrent models that favour backpropagation
LSTM [Hochreiter et al. 1997], GRU [Chung et al. 2014].
RNN vs. LSTM gradients on the input matrix $\mathbf{W}^x$
%%html
<center>
<video controls autoplay loop>
<source src="rnn-figures/vanishing.mp4" type="video/mp4">
</video>
</center>
%%tikz -l arrows -s 1000,400 -sc 0.65
\newcommand{\lstm}{
\definecolor{nice-red}{HTML}{E41A1C}
\definecolor{nice-orange}{HTML}{FF7F00}
\definecolor{nice-yellow}{HTML}{FFC020}
\definecolor{nice-green}{HTML}{4DAF4A}
\definecolor{nice-blue}{HTML}{377EB8}
\definecolor{nice-purple}{HTML}{984EA3}
%lstm first step
%lstm module box
\draw[line width=3pt, color=black!50] (-6,-3) rectangle (1.5,5.25);
\draw[ultra thick] (0,0) rectangle (1,2);
%memory ct
\draw[ultra thick, color=nice-purple, fill=nice-purple!10] (0,0) rectangle (1,2);
%non-linearities
\foreach \w/\h/\color in {-2/4.25/nice-blue,-2/1/nice-red,-2/-1/nice-green,0.5/-2/nice-yellow,0.5/3/black} {
\begin{scope}[shift={(\w,\h)},scale=0.5]
\draw[ultra thick, yshift=-0.5cm, color=\color] plot [domain=-0.3:0.3](\x, {(0.8/(1+exp(-15*\x))+0.1)});
\draw[ultra thick, color=\color] (0,0) circle (0.5cm);
\end{scope}
}
%tanh
\draw[thick, color=black] (0.25,3) -- (0.75,3);
\draw[thick, color=nice-yellow] (0.25,-2) -- (0.75,-2);
%component-wise multiplications
\foreach \w/\h in {-1/1,0.5/-1,0.5/4.25} {
\begin{scope}[shift={(\w,\h)},scale=0.5]
\draw[ultra thick, color=black] (0,0) circle (0.05cm);
\draw[ultra thick, color=black] (0,0) circle (0.5cm);
\end{scope}
}
%vector concat
\begin{scope}[shift={(-4,1)},scale=0.5]
\draw[ultra thick,yshift=0.2cm] (0,0) circle (0.05cm);
\draw[ultra thick,yshift=-0.2cm] (0,0) circle (0.05cm);
\draw[ultra thick] (0,0) circle (0.5cm);
\end{scope}
\foreach \fx/\fy/\tx/\ty in {
-5/-3.5/-5/0.85, %xt
-5/0.85/-4.2/0.85,
-6.5/4.25/-5/4.25, %ht1
-5/4.25/-5/1.15,
-5/1.15/-4.2/1.15,
-3.75/1/-3/1, %H
-3/4.25/-3/-2,
-3/-2/0.25/-2, %i
0.5/-1.75/0.5/-1.25,
-3/-1/-2.25/-1, %it
-1.75/-1/0.25/-1,
-3/1/-2.25/1, %ft
-1.75/1/-1.25/1,
-0.75/1/0/1,
-3/4.25/-2.25/4.25, %ot
-1.75/4.25/0.25/4.25,
0.5/2/0.5/2.75, %ct
-5.5/2/-5.1/2, %ct1
-5.5/2/-5.5/1,
-6.5/1/-5.5/1,
-4.9/2/-3.1/2,
-2.9/2/-1/2,
-1/2/-1/1.25
} {
\draw[ultra thick] (\fx,\fy) -- (\tx,\ty);
}
\foreach \fx/\fy/\tx/\ty in {
0.5/-0.75/0.5/0, %it
-0.75/1/0/1, %ft
1/1/2.25/1,
0.5/3.25/0.5/4,
0.75/4.25/2.25/4.25, %ht
0.5/4.5/0.5/6
} {
\draw[->, >=stealth', ultra thick] (\fx,\fy) -- (\tx,\ty);
}
}
\begin{scope}[scale=0.8]
\foreach \d in {0} {
\foreach \t/\word in {0/A,1/wedding,2/party,3/taking,4/pictures} {
\node[font=\Huge, anchor=west] at (\t*8.5-5.75,-4.5) {$\mathbf{v}$\_\word};
\begin{scope}[shift={(\t*8.5+\d*5.5,\d*9.5)}]
\lstm
\end{scope}
}
}
\end{scope}
\node[font=\Huge, anchor=west] at (27,5.75) {$\mathbf{v}$\_Sentence};
%dummy node for left alignment
\node[] at (17,0) {};
This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017) (preloaded format=pdflatex) \write18 enabled. entering extended mode (./tikz.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/local/texlive/2017/texmf-dist/tex/latex/standalone/standalone.cls Document Class: standalone 2015/07/15 v1.2 Class to compile TeX sub-files stand alone (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/ifluatex.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/ifpdf.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/ifxetex/ifxetex.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/xkeyval/xkeyval.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/xkeyval.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/xkvutils.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/keyval.tex)))) (/usr/local/texlive/2017/texmf-dist/tex/latex/standalone/standalone.cfg) (/usr/local/texlive/2017/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/local/texlive/2017/texmf-dist/tex/latex/base/size10.clo))This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017) (preloaded format=pdflatex) \write18 enabled. entering extended mode LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (./tikz.tex (/usr/local/texlive/2017/texmf-dist/tex/latex/standalone/standalone.cls Document Class: standalone 2015/07/15 v1.2 Class to compile TeX sub-files stand alone (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/ifluatex.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/ifpdf.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/ifxetex/ifxetex.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/xkeyval/xkeyval.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/xkeyval.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/xkvutils.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/keyval.tex)))) (/usr/local/texlive/2017/texmf-dist/tex/latex/standalone/standalone.cfg) (/usr/local/texlive/2017/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/local/texlive/2017/texmf-dist/tex/latex/base/size10.clo))) (/usr/local/texlive/2017/texmf-dist/tex/latex/xcolor/xcolor.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics-cfg/color.cfg) (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics-def/pdftex.def (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/infwarerr.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/ltxcmds.sty))) (/usr/local/texlive/2017/texmf-dist/tex/latex/psnfss/helvet.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/frontendlayer/tikz.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/basiclayer/pgf.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/utilities/pgfrcs.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfutil-common.te x (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfutil-common-li sts.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfutil-latex.def (/usr/local/texlive/2017/texmf-dist/tex/latex/ms/everyshi.sty)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfrcs.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/basiclayer/pgfcore.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics/graphicx.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics/graphics.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics/trig.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics-cfg/graphics.cfg))) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/systemlayer/pgfsys.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgfsys.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfkeysfiltered.c ode.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgf.cfg) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-pdftex.d ef (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-common-p df.def))) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgfsyssoftpath. code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgfsysprotocol. code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcore.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathcalc.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathutil.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathparser.code.tex ) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.code. tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.basic .code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.trigo nometric.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.rando m.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.compa rison.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.base. code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.round .code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.misc. code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.integ erarithmetics.code.tex))) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfloat.code.tex) ) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepoints.co de.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathconst ruct.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathusage .code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorescopes.co de.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoregraphicst ate.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransform ations.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorequick.cod e.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreobjects.c ode.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathproce ssing.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorearrows.co de.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreshade.cod e.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreimage.cod e.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreexternal. code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorelayers.co de.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretranspare ncy.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepatterns. code.tex))) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/modules/pgfmoduleshapes.cod e.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/modules/pgfmoduleplot.code. tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version -0-65.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version -1-18.sty)) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/utilities/pgffor.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/utilities/pgfkeys.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex) ) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/math/pgfmath.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgffor.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex))) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/tikz.cod e.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/pgflibraryplothan dlers.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/modules/pgfmodulematrix.cod e.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarytopaths.code.tex))) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-euclide.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-base.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/etex-pkg/etex.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarycalc.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibraryarrows.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/pgflibraryarrows. code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibraryplotmarks.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/pgflibraryplotmar ks.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarypositioning.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibraryshapes.misc.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/shapes/pgflibrary shapes.misc.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarydecorations.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/modules/pgfmoduledecoration s.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarydecorations.markings.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/decorations/pgfli brarydecorations.markings.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarydecorations.pathreplacing.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/decorations/pgfli brarydecorations.pathreplacing.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarypatterns.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/pgflibrarypattern s.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/latex/numprint/numprint.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/tools/array.sty) No configuration file `numprint.cfg' found.) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp.sty `Fixed Point Package', Version 0.8, April 2, 1995 (C) Michael Mehlich (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/defpattern.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-basic.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-addons.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-snap.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-exp.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-trigo.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-pas.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-random.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-eqn.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-upn.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-eval.sty)) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-tools-utilities.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-tools-arith.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-tools-base.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-tools-misc.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-tools-math.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-obj-points.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-obj-segments.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-obj-marks.tex)) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-tools-intersectio ns.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-tools-transformat ions.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-lines.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-lib-symbols.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-circles.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-addpoints.tex )) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-polygons.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-vectors.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-arcs.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-sectors.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-angles.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-protractor.te x) No file tikz.aux. (/usr/local/texlive/2017/texmf-dist/tex/context/base/mkii/supp-pdf.mkii [Loading MPS to PDF converter (version 2006.09.02).] ) (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/pdftexcmds.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/oberdiek/epstopdf-base.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/oberdiek/grfext.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/kvdefinekeys.sty)) (/usr/local/texlive/2017/texmf-dist/tex/latex/oberdiek/kvoptions.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/kvsetkeys.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/etexcmds.sty))) (/usr/local/texlive/2017/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg)) ABD: EveryShipout initializing macros [1{/usr/local/texlive/2017/texmf-var/font s/map/pdftex/updmap/pdftex.map}] (./tikz.aux) )</usr/local/texlive/2017/texmf-d ist/fonts/type1/public/amsfonts/cm/cmbx12.pfb></usr/local/texlive/2017/texmf-di st/fonts/type1/public/amsfonts/cm/cmr17.pfb> Output written on tikz.pdf (1 page, 29752 bytes). Transcript written on tikz.log. (./tikz.aux) Class standalone: Output written on tikz.png. ) ) No pages of output. Transcript written on tikz.log.
%%tikz -l arrows -s 1000,400 -sc 0.65
\definecolor{nice-red}{HTML}{E41A1C}
\definecolor{nice-orange}{HTML}{FF7F00}
\definecolor{nice-yellow}{HTML}{FFC020}
\definecolor{nice-green}{HTML}{4DAF4A}
\definecolor{nice-blue}{HTML}{377EB8}
\definecolor{nice-purple}{HTML}{984EA3}
\newcommand{\lstm}{
%lstm first step
%lstm module box
\draw[line width=3pt, color=black!50] (-6,-3) rectangle (1.5,5.25);
\draw[ultra thick] (0,0) rectangle (1,2);
%memory ct
\draw[ultra thick, color=nice-purple, fill=nice-purple!10] (0,0) rectangle (1,2);
%non-linearities
\foreach \w/\h/\color in {-2/4.25/nice-blue,-2/1/nice-red,-2/-1/nice-green,0.5/-2/nice-yellow,0.5/3/black} {
\begin{scope}[shift={(\w,\h)},scale=0.5]
\draw[ultra thick, yshift=-0.5cm, color=\color] plot [domain=-0.3:0.3](\x, {(0.8/(1+exp(-15*\x))+0.1)});
\draw[ultra thick, color=\color] (0,0) circle (0.5cm);
\end{scope}
}
%tanh
\draw[thick, color=black] (0.25,3) -- (0.75,3);
\draw[thick, color=nice-yellow] (0.25,-2) -- (0.75,-2);
%component-wise multiplications
\foreach \w/\h in {-1/1,0.5/-1,0.5/4.25} {
\begin{scope}[shift={(\w,\h)},scale=0.5]
\draw[ultra thick, color=black] (0,0) circle (0.05cm);
\draw[ultra thick, color=black] (0,0) circle (0.5cm);
\end{scope}
}
%vector concat
\begin{scope}[shift={(-4,1)},scale=0.5]
\draw[ultra thick,yshift=0.2cm] (0,0) circle (0.05cm);
\draw[ultra thick,yshift=-0.2cm] (0,0) circle (0.05cm);
\draw[ultra thick] (0,0) circle (0.5cm);
\end{scope}
\foreach \fx/\fy/\tx/\ty in {
-5/-3.5/-5/0.85, %xt
-5/0.85/-4.2/0.85,
-6.5/4.25/-5/4.25, %ht1
-5/4.25/-5/1.15,
-5/1.15/-4.2/1.15,
-3.75/1/-3/1, %H
-3/4.25/-3/-2,
-3/-2/0.25/-2, %i
0.5/-1.75/0.5/-1.25,
-3/-1/-2.25/-1, %it
-1.75/-1/0.25/-1,
-3/1/-2.25/1, %ft
-1.75/1/-1.25/1,
-0.75/1/0/1,
-3/4.25/-2.25/4.25, %ot
-1.75/4.25/0.25/4.25,
0.5/2/0.5/2.75, %ct
-5.5/2/-5.1/2, %ct1
-5.5/2/-5.5/1,
-6.5/1/-5.5/1,
-4.9/2/-3.1/2,
-2.9/2/-1/2,
-1/2/-1/1.25
} {
\draw[ultra thick] (\fx,\fy) -- (\tx,\ty);
}
\foreach \fx/\fy/\tx/\ty in {
0.5/-0.75/0.5/0, %it
-0.75/1/0/1, %ft
1/1/2.25/1,
0.5/3.25/0.5/4,
0.75/4.25/2.25/4.25, %ht
0.5/4.5/0.5/6
} {
\draw[->, >=stealth', ultra thick] (\fx,\fy) -- (\tx,\ty);
}
}
\begin{scope}[scale=0.8]
\foreach \d in {0} {
\foreach \t/\word in {0/A,1/wedding,2/party,3/taking,4/pictures} {
\node[font=\Huge, anchor=west] at (\t*8.5-5.75,-4.5) {$\mathbf{v}$\_\word};
\begin{scope}[shift={(\t*8.5+\d*5.5,\d*9.5)}]
\lstm
\end{scope}
}
}
\end{scope}
\node[font=\Huge, anchor=west] at (27,5.75) {$\mathbf{v}$\_Sentence};
\draw[line width=10pt, color=nice-red, opacity=0.8] (27.6,5) -- (27.6,0.75);
\draw[line width=10pt, color=nice-red, opacity=0.8] (27.5,0.75) -- (3,0.75);
\draw[->, >=stealth', line width=10pt, color=nice-red, opacity=0.8] (2.75,0.75) -- (2.75,-3);
%dummy node for left alignment
\node[] at (17,0) {};
This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017) (preloaded format=pdflatex) \write18 enabled. entering extended mode (./tikz.tex LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (/usr/local/texlive/2017/texmf-dist/tex/latex/standalone/standalone.cls Document Class: standalone 2015/07/15 v1.2 Class to compile TeX sub-files stand alone (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/ifluatex.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/ifpdf.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/ifxetex/ifxetex.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/xkeyval/xkeyval.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/xkeyval.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/xkvutils.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/keyval.tex)))) (/usr/local/texlive/2017/texmf-dist/tex/latex/standalone/standalone.cfg) (/usr/local/texlive/2017/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/local/texlive/2017/texmf-dist/tex/latex/base/size10.clo))This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017) (preloaded format=pdflatex) \write18 enabled. entering extended mode LaTeX2e <2017-04-15> Babel <3.10> and hyphenation patterns for 84 language(s) loaded. (./tikz.tex (/usr/local/texlive/2017/texmf-dist/tex/latex/standalone/standalone.cls Document Class: standalone 2015/07/15 v1.2 Class to compile TeX sub-files stand alone (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/ifluatex.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/ifpdf.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/ifxetex/ifxetex.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/xkeyval/xkeyval.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/xkeyval.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/xkvutils.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/xkeyval/keyval.tex)))) (/usr/local/texlive/2017/texmf-dist/tex/latex/standalone/standalone.cfg) (/usr/local/texlive/2017/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/local/texlive/2017/texmf-dist/tex/latex/base/size10.clo))) (/usr/local/texlive/2017/texmf-dist/tex/latex/xcolor/xcolor.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics-cfg/color.cfg) (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics-def/pdftex.def (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/infwarerr.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/ltxcmds.sty))) (/usr/local/texlive/2017/texmf-dist/tex/latex/psnfss/helvet.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/frontendlayer/tikz.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/basiclayer/pgf.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/utilities/pgfrcs.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfutil-common.te x (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfutil-common-li sts.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfutil-latex.def (/usr/local/texlive/2017/texmf-dist/tex/latex/ms/everyshi.sty)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfrcs.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/basiclayer/pgfcore.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics/graphicx.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics/graphics.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics/trig.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/graphics-cfg/graphics.cfg))) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/systemlayer/pgfsys.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgfsys.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfkeysfiltered.c ode.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgf.cfg) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-pdftex.d ef (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-common-p df.def))) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgfsyssoftpath. code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/systemlayer/pgfsysprotocol. code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcore.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathcalc.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathutil.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathparser.code.tex ) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.code. tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.basic .code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.trigo nometric.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.rando m.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.compa rison.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.base. code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.round .code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.misc. code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.integ erarithmetics.code.tex))) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmathfloat.code.tex) ) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepoints.co de.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathconst ruct.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathusage .code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorescopes.co de.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoregraphicst ate.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransform ations.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorequick.cod e.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreobjects.c ode.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathproce ssing.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorearrows.co de.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreshade.cod e.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreimage.cod e.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreexternal. code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorelayers.co de.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretranspare ncy.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepatterns. code.tex))) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/modules/pgfmoduleshapes.cod e.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/modules/pgfmoduleplot.code. tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version -0-65.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version -1-18.sty)) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/utilities/pgffor.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/utilities/pgfkeys.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex) ) (/usr/local/texlive/2017/texmf-dist/tex/latex/pgf/math/pgfmath.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/utilities/pgffor.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex))) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/tikz.cod e.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/pgflibraryplothan dlers.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/modules/pgfmodulematrix.cod e.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarytopaths.code.tex))) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-euclide.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-base.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/etex-pkg/etex.sty) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarycalc.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibraryarrows.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/pgflibraryarrows. code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibraryplotmarks.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/pgflibraryplotmar ks.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarypositioning.code.tex) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibraryshapes.misc.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/shapes/pgflibrary shapes.misc.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarydecorations.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/modules/pgfmoduledecoration s.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarydecorations.markings.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/decorations/pgfli brarydecorations.markings.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarydecorations.pathreplacing.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/decorations/pgfli brarydecorations.pathreplacing.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/frontendlayer/tikz/librarie s/tikzlibrarypatterns.code.tex (/usr/local/texlive/2017/texmf-dist/tex/generic/pgf/libraries/pgflibrarypattern s.code.tex)) (/usr/local/texlive/2017/texmf-dist/tex/latex/numprint/numprint.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/tools/array.sty) No configuration file `numprint.cfg' found.) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp.sty `Fixed Point Package', Version 0.8, April 2, 1995 (C) Michael Mehlich (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/defpattern.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-basic.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-addons.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-snap.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-exp.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-trigo.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-pas.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-random.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-eqn.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-upn.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/fp/fp-eval.sty)) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-tools-utilities.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-tools-arith.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-tools-base.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-tools-misc.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-tools-math.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-obj-points.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-obj-segments.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-base/tkz-obj-marks.tex)) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-tools-intersectio ns.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-tools-transformat ions.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-lines.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-lib-symbols.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-circles.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-addpoints.tex )) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-polygons.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-vectors.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-arcs.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-sectors.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-angles.tex) (/usr/local/texlive/2017/texmf-dist/tex/latex/tkz-euclide/tkz-obj-protractor.te x) No file tikz.aux. (/usr/local/texlive/2017/texmf-dist/tex/context/base/mkii/supp-pdf.mkii [Loading MPS to PDF converter (version 2006.09.02).] ) (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/pdftexcmds.sty) (/usr/local/texlive/2017/texmf-dist/tex/latex/oberdiek/epstopdf-base.sty (/usr/local/texlive/2017/texmf-dist/tex/latex/oberdiek/grfext.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/kvdefinekeys.sty)) (/usr/local/texlive/2017/texmf-dist/tex/latex/oberdiek/kvoptions.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/kvsetkeys.sty (/usr/local/texlive/2017/texmf-dist/tex/generic/oberdiek/etexcmds.sty))) (/usr/local/texlive/2017/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg)) ABD: EveryShipout initializing macros [1{/usr/local/texlive/2017/texmf-var/font s/map/pdftex/updmap/pdftex.map}] (./tikz.aux) )</usr/local/texlive/2017/texmf-d ist/fonts/type1/public/amsfonts/cm/cmbx12.pfb></usr/local/texlive/2017/texmf-di st/fonts/type1/public/amsfonts/cm/cmr17.pfb> Output written on tikz.pdf (1 page, 29943 bytes). Transcript written on tikz.log. (./tikz.aux) Class standalone: Output written on tikz.png. ) ) No pages of output. Transcript written on tikz.log.
%%tikz -l arrows -s 1100,500 -sc 0.65
\definecolor{nice-red}{HTML}{E41A1C}
\definecolor{nice-orange}{HTML}{FF7F00}
\definecolor{nice-yellow}{HTML}{FFC020}
\definecolor{nice-green}{HTML}{4DAF4A}
\definecolor{nice-blue}{HTML}{377EB8}
\definecolor{nice-purple}{HTML}{984EA3}
\newcommand{\lstm}{
%lstm first step
%lstm module box
\draw[line width=3pt, color=black!50] (-6,-3) rectangle (1.5,5.25);
\draw[ultra thick] (0,0) rectangle (1,2);
%memory ct
\draw[ultra thick, color=nice-purple, fill=nice-purple!10] (0,0) rectangle (1,2);
%non-linearities
\foreach \w/\h/\color in {-2/4.25/nice-blue,-2/1/nice-red,-2/-1/nice-green,0.5/-2/nice-yellow,0.5/3/black} {
\begin{scope}[shift={(\w,\h)},scale=0.5]
\draw[ultra thick, yshift=-0.5cm, color=\color] plot [domain=-0.3:0.3](\x, {(0.8/(1+exp(-15*\x))+0.1)});
\draw[ultra thick, color=\color] (0,0) circle (0.5cm);
\end{scope}
}
%tanh
\draw[thick, color=black] (0.25,3) -- (0.75,3);
\draw[thick, color=nice-yellow] (0.25,-2) -- (0.75,-2);
%component-wise multiplications
\foreach \w/\h in {-1/1,0.5/-1,0.5/4.25} {
\begin{scope}[shift={(\w,\h)},scale=0.5]
\draw[ultra thick, color=black] (0,0) circle (0.05cm);
\draw[ultra thick, color=black] (0,0) circle (0.5cm);
\end{scope}
}
%vector concat
\begin{scope}[shift={(-4,1)},scale=0.5]
\draw[ultra thick,yshift=0.2cm] (0,0) circle (0.05cm);
\draw[ultra thick,yshift=-0.2cm] (0,0) circle (0.05cm);
\draw[ultra thick] (0,0) circle (0.5cm);
\end{scope}
\foreach \fx/\fy/\tx/\ty in {
-5/-3.5/-5/0.85, %xt
-5/0.85/-4.2/0.85,
-6.5/4.25/-5/4.25, %ht1
-5/4.25/-5/1.15,
-5/1.15/-4.2/1.15,
-3.75/1/-3/1, %H
-3/4.25/-3/-2,
-3/-2/0.25/-2, %i
0.5/-1.75/0.5/-1.25,
-3/-1/-2.25/-1, %it
-1.75/-1/0.25/-1,
-3/1/-2.25/1, %ft
-1.75/1/-1.25/1,
-0.75/1/0/1,
-3/4.25/-2.25/4.25, %ot
-1.75/4.25/0.25/4.25,
0.5/2/0.5/2.75, %ct
-5.5/2/-5.1/2, %ct1
-5.5/2/-5.5/1,
-6.5/1/-5.5/1,
-4.9/2/-3.1/2,
-2.9/2/-1/2,
-1/2/-1/1.25
} {
\draw[ultra thick] (\fx,\fy) -- (\tx,\ty);
}
\foreach \fx/\fy/\tx/\ty in {
0.5/-0.75/0.5/0, %it
-0.75/1/0/1, %ft
1/1/2.25/1,
0.5/3.25/0.5/4,
0.75/4.25/2.25/4.25, %ht
0.5/4.5/0.5/6
} {
\draw[->, >=stealth', ultra thick] (\fx,\fy) -- (\tx,\ty);
}
}
\begin{scope}[scale=0.8]
\foreach \d in {0,1,2} {
\foreach \t/\word in {0/A,1/wedding,2/party,3/taking,4/pictures} {
\node[font=\Huge, anchor=west] at (\t*8.5-5.75,-4.5) {$\mathbf{v}$\_\word};
\begin{scope}[shift={(\t*8.5+\d*5.5,\d*9.5)}]
\lstm
\end{scope}
}
}
\end{scope}
\node[font=\Huge, anchor=west] at (34,20.75) {$\mathbf{v}$\_Sentence};
\draw[line width=10pt, color=nice-red, opacity=0.8] (36.4,16) -- (36.4,20);
\draw[line width=10pt, color=nice-red, opacity=0.8] (25.25,16) -- (36.5,16);
\draw[line width=10pt, color=nice-red, opacity=0.8] (25.25,8.5) -- (25.25,16);
\draw[line width=10pt, color=nice-red, opacity=0.8] (14,8.5) -- (25.25,8.5);
\draw[line width=10pt, color=nice-red, opacity=0.8] (14,8.5) -- (14,0.75);
\draw[line width=10pt, color=nice-red, opacity=0.8] (14,0.75) -- (3,0.75);
\draw[->, >=stealth', line width=10pt, color=nice-red, opacity=0.8] (2.75,0.75) -- (2.75,-3);
%dummy node for left alignment
\node[] at (17,0) {};
LaTeX terminated with signal -1 No image generated.
This is pdfTeX, Version 3.14159265-2.6-1.40.20 (TeX Live 2019/Debian) (preloaded format=pdflatex 2020.5.27) 19 SEP 2022 09:15 entering extended mode \write18 enabled. %&-line parsing enabled. **tikz.tex (./tikz.tex LaTeX2e <2020-02-02> patch level 2 L3 programming layer <2020-02-14> (/usr/share/texlive/texmf-dist/tex/latex/standalone/standalone.cls Document Class: standalone 2018/03/26 v1.3a Class to compile TeX sub-files stan dalone (/usr/share/texlive/texmf-dist/tex/latex/tools/shellesc.sty Package: shellesc 2019/11/08 v1.0c unified shell escape interface for LaTeX Package shellesc Info: Unrestricted shell escape enabled on input line 75. ) (/usr/share/texlive/texmf-dist/tex/generic/iftex/ifluatex.sty Package: ifluatex 2019/10/25 v1.5 ifluatex legacy package. Use iftex instead. (/usr/share/texlive/texmf-dist/tex/generic/iftex/iftex.sty Package: iftex 2019/11/07 v1.0c TeX engine tests )) (/usr/share/texlive/texmf-dist/tex/latex/xkeyval/xkeyval.sty Package: xkeyval 2014/12/03 v2.7a package option processing (HA) (/usr/share/texlive/texmf-dist/tex/generic/xkeyval/xkeyval.tex (/usr/share/texlive/texmf-dist/tex/generic/xkeyval/xkvutils.tex \XKV@toks=\toks14 \XKV@tempa@toks=\toks15 (/usr/share/texlive/texmf-dist/tex/generic/xkeyval/keyval.tex)) \XKV@depth=\count167 File: xkeyval.tex 2014/12/03 v2.7a key=value parser (HA) )) \sa@internal=\count168 \c@sapage=\count169 (/usr/share/texlive/texmf-dist/tex/latex/standalone/standalone.cfg File: standalone.cfg 2018/03/26 v1.3a Default configuration file for 'standalon e' class ) (/usr/share/texlive/texmf-dist/tex/latex/base/article.cls Document Class: article 2019/12/20 v1.4l Standard LaTeX document class (/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo File: size10.clo 2019/12/20 v1.4l Standard LaTeX file (size option) ) \c@part=\count170 \c@section=\count171 \c@subsection=\count172 \c@subsubsection=\count173 \c@paragraph=\count174 \c@subparagraph=\count175 \c@figure=\count176 \c@table=\count177 \abovecaptionskip=\skip47 \belowcaptionskip=\skip48 \bibindent=\dimen134 ) \sa@box=\box45 runsystem(pdflatex -shell-escape -jobname 'tikz' '\expandafter\def\csname sa@ internal@run\endcsname{1}\input{tikz}')...executed. runsystem(convert -density 300 -units PixelsPerInch tikz.pdf -resize 1100x500 - quality 90 tikz.png)...executed. Class standalone Warning: Conversion unsuccessful! (standalone) There might be something wrong with your (standalone) conversation software or the file permissions! ) (/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty Package: xcolor 2016/05/11 v2.12 LaTeX color extensions (UK) (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg File: color.cfg 2016/01/02 v1.6 sample color configuration ) Package xcolor Info: Driver file: pdftex.def on input line 225. (/usr/share/texlive/texmf-dist/tex/latex/graphics-def/pdftex.def File: pdftex.def 2018/01/08 v1.0l Graphics/color driver for pdftex ) Package xcolor Info: Model `cmy' substituted by `cmy0' on input line 1348. Package xcolor Info: Model `hsb' substituted by `rgb' on input line 1352. Package xcolor Info: Model `RGB' extended on input line 1364. Package xcolor Info: Model `HTML' substituted by `rgb' on input line 1366. Package xcolor Info: Model `Hsb' substituted by `hsb' on input line 1367. Package xcolor Info: Model `tHsb' substituted by `hsb' on input line 1368. Package xcolor Info: Model `HSB' substituted by `hsb' on input line 1369. Package xcolor Info: Model `Gray' substituted by `gray' on input line 1370. Package xcolor Info: Model `wave' substituted by `hsb' on input line 1371. ) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/helvet.sty Package: helvet 2005/04/12 PSNFSS-v9.2a (WaS) ) (/usr/share/texlive/texmf-dist/tex/latex/pgf/frontendlayer/tikz.sty (/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgf.sty (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfrcs.sty (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-common.tex \pgfutil@everybye=\toks16 \pgfutil@tempdima=\dimen135 \pgfutil@tempdimb=\dimen136 (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-common-lists.t ex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-latex.def \pgfutil@abb=\box46 (/usr/share/texlive/texmf-dist/tex/latex/ms/everyshi.sty Package: everyshi 2001/05/15 v3.00 EveryShipout Package (MS) )) (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfrcs.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/pgf.revision.tex) Package: pgfrcs 2020/01/08 v3.1.5b (3.1.5b) )) Package: pgf 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgfcore.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty Package: graphicx 2019/11/30 v1.2a Enhanced LaTeX Graphics (DPC,SPQR) (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty Package: graphics 2019/11/30 v1.4a Standard LaTeX Graphics (DPC,SPQR) (/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty Package: trig 2016/01/03 v1.10 sin cos tan (DPC) ) (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg File: graphics.cfg 2016/06/04 v1.11 sample graphics configuration ) Package graphics Info: Driver file: pdftex.def on input line 105. ) \Gin@req@height=\dimen137 \Gin@req@width=\dimen138 ) (/usr/share/texlive/texmf-dist/tex/latex/pgf/systemlayer/pgfsys.sty (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys.code.tex Package: pgfsys 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex \pgfkeys@pathtoks=\toks17 \pgfkeys@temptoks=\toks18 (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeysfiltered.code.t ex \pgfkeys@tmptoks=\toks19 )) \pgf@x=\dimen139 \pgf@y=\dimen140 \pgf@xa=\dimen141 \pgf@ya=\dimen142 \pgf@xb=\dimen143 \pgf@yb=\dimen144 \pgf@xc=\dimen145 \pgf@yc=\dimen146 \pgf@xd=\dimen147 \pgf@yd=\dimen148 \w@pgf@writea=\write3 \r@pgf@reada=\read2 \c@pgf@counta=\count178 \c@pgf@countb=\count179 \c@pgf@countc=\count180 \c@pgf@countd=\count181 \t@pgf@toka=\toks20 \t@pgf@tokb=\toks21 \t@pgf@tokc=\toks22 \pgf@sys@id@count=\count182 (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgf.cfg File: pgf.cfg 2020/01/08 v3.1.5b (3.1.5b) ) Driver file for pgf: pgfsys-pdftex.def (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-pdftex.def File: pgfsys-pdftex.def 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-common-pdf.de f File: pgfsys-common-pdf.def 2020/01/08 v3.1.5b (3.1.5b) ))) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsyssoftpath.code. tex File: pgfsyssoftpath.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgfsyssoftpath@smallbuffer@items=\count183 \pgfsyssoftpath@bigbuffer@items=\count184 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsysprotocol.code. tex File: pgfsysprotocol.code.tex 2020/01/08 v3.1.5b (3.1.5b) )) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcore.code.tex Package: pgfcore 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathcalc.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathutil.code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathparser.code.tex \pgfmath@dimen=\dimen149 \pgfmath@count=\count185 \pgfmath@box=\box47 \pgfmath@toks=\toks23 \pgfmath@stack@operand=\toks24 \pgfmath@stack@operation=\toks25 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.basic.code .tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.trigonomet ric.code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.random.cod e.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.comparison .code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.base.code. tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.round.code .tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.misc.code. tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.integerari thmetics.code.tex))) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfloat.code.tex \c@pgfmathroundto@lastzeros=\count186 )) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfint.code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepoints.code.te x File: pgfcorepoints.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgf@picminx=\dimen150 \pgf@picmaxx=\dimen151 \pgf@picminy=\dimen152 \pgf@picmaxy=\dimen153 \pgf@pathminx=\dimen154 \pgf@pathmaxx=\dimen155 \pgf@pathminy=\dimen156 \pgf@pathmaxy=\dimen157 \pgf@xx=\dimen158 \pgf@xy=\dimen159 \pgf@yx=\dimen160 \pgf@yy=\dimen161 \pgf@zx=\dimen162 \pgf@zy=\dimen163 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathconstruct. code.tex File: pgfcorepathconstruct.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgf@path@lastx=\dimen164 \pgf@path@lasty=\dimen165 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathusage.code .tex File: pgfcorepathusage.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgf@shorten@end@additional=\dimen166 \pgf@shorten@start@additional=\dimen167 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorescopes.code.te x File: pgfcorescopes.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgfpic=\box48 \pgf@hbox=\box49 \pgf@layerbox@main=\box50 \pgf@picture@serial@count=\count187 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoregraphicstate.c ode.tex File: pgfcoregraphicstate.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgflinewidth=\dimen168 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransformation s.code.tex File: pgfcoretransformations.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgf@pt@x=\dimen169 \pgf@pt@y=\dimen170 \pgf@pt@temp=\dimen171 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorequick.code.tex File: pgfcorequick.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreobjects.code.t ex File: pgfcoreobjects.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathprocessing .code.tex File: pgfcorepathprocessing.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorearrows.code.te x File: pgfcorearrows.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgfarrowsep=\dimen172 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreshade.code.tex File: pgfcoreshade.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgf@max=\dimen173 \pgf@sys@shading@range@num=\count188 \pgf@shadingcount=\count189 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreimage.code.tex File: pgfcoreimage.code.tex 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreexternal.code. tex File: pgfcoreexternal.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgfexternal@startupbox=\box51 )) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorelayers.code.te x File: pgfcorelayers.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransparency.c ode.tex File: pgfcoretransparency.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepatterns.code. tex File: pgfcorepatterns.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorerdf.code.tex File: pgfcorerdf.code.tex 2020/01/08 v3.1.5b (3.1.5b) ))) (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleshapes.code.tex File: pgfmoduleshapes.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgfnodeparttextbox=\box52 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleplot.code.tex File: pgfmoduleplot.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-0-65 .sty Package: pgfcomp-version-0-65 2020/01/08 v3.1.5b (3.1.5b) \pgf@nodesepstart=\dimen174 \pgf@nodesepend=\dimen175 ) (/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-1-18 .sty Package: pgfcomp-version-1-18 2020/01/08 v3.1.5b (3.1.5b) )) (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgffor.sty (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfkeys.sty (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex)) (/usr/share/texlive/texmf-dist/tex/latex/pgf/math/pgfmath.sty (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgffor.code.tex Package: pgffor 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex) \pgffor@iter=\dimen176 \pgffor@skip=\dimen177 \pgffor@stack=\toks26 \pgffor@toks=\toks27 )) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/tikz.code.tex Package: tikz 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryplothandlers .code.tex File: pgflibraryplothandlers.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgf@plot@mark@count=\count190 \pgfplotmarksize=\dimen178 ) \tikz@lastx=\dimen179 \tikz@lasty=\dimen180 \tikz@lastxsaved=\dimen181 \tikz@lastysaved=\dimen182 \tikz@lastmovetox=\dimen183 \tikz@lastmovetoy=\dimen184 \tikzleveldistance=\dimen185 \tikzsiblingdistance=\dimen186 \tikz@figbox=\box53 \tikz@figbox@bg=\box54 \tikz@tempbox=\box55 \tikz@tempbox@bg=\box56 \tikztreelevel=\count191 \tikznumberofchildren=\count192 \tikznumberofcurrentchild=\count193 \tikz@fig@count=\count194 (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmodulematrix.code.tex File: pgfmodulematrix.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgfmatrixcurrentrow=\count195 \pgfmatrixcurrentcolumn=\count196 \pgf@matrix@numberofcolumns=\count197 ) \tikz@expandcount=\count198 (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarytopaths.code.tex File: tikzlibrarytopaths.code.tex 2020/01/08 v3.1.5b (3.1.5b) ))) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-euclide.sty 2020/01/24 3.02c tkz-euclide.sty Package: tkz-euclide 2020/01/24 3.02c for euclidan geometry (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-base.sty 2020/01/24 3.02c tkz-base.sty Package: tkz-base 2020/01/24 3.02c tkz-base (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryangles.code.tex File: tikzlibraryangles.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryarrows.code.tex File: tikzlibraryarrows.code.tex 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryarrows.code. tex File: pgflibraryarrows.code.tex 2020/01/08 v3.1.5b (3.1.5b) \arrowsize=\dimen187 )) (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryarrows.meta. code.tex File: pgflibraryarrows.meta.code.tex 2020/01/08 v3.1.5b (3.1.5b) \pgfarrowinset=\dimen188 \pgfarrowlength=\dimen189 \pgfarrowwidth=\dimen190 \pgfarrowlinewidth=\dimen191 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarybabel.code.tex File: tikzlibrarybabel.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarycalc.code.tex File: tikzlibrarycalc.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarydecorations.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduledecorations.cod e.tex \pgfdecoratedcompleteddistance=\dimen192 \pgfdecoratedremainingdistance=\dimen193 \pgfdecoratedinputsegmentcompleteddistance=\dimen194 \pgfdecoratedinputsegmentremainingdistance=\dimen195 \pgf@decorate@distancetomove=\dimen196 \pgf@decorate@repeatstate=\count199 \pgfdecorationsegmentamplitude=\dimen197 \pgfdecorationsegmentlength=\dimen198 ) \tikz@lib@dec@box=\box57 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarydecorations.markings.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/decorations/pgflibrary decorations.markings.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarydecorations.pathreplacing.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/decorations/pgflibrary decorations.pathreplacing.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarydecorations.shapes.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/decorations/pgflibrary decorations.shapes.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarydecorations.text.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/decorations/pgflibrary decorations.text.code.tex \pgf@lib@dec@text@box=\box58 ) \tikz@lib@dec@te@box=\box59 ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarydecorations.pathmorphing.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/decorations/pgflibrary decorations.pathmorphing.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryintersections.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryintersection s.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryfpu.code.tex ) \pgf@intersect@solutions=\count266 )) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarypatterns.code.tex File: tikzlibrarypatterns.code.tex 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibrarypatterns.cod e.tex File: pgflibrarypatterns.code.tex 2020/01/08 v3.1.5b (3.1.5b) )) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryplotmarks.code.tex File: tikzlibraryplotmarks.code.tex 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryplotmarks.co de.tex File: pgflibraryplotmarks.code.tex 2020/01/08 v3.1.5b (3.1.5b) )) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarypositioning.code.tex File: tikzlibrarypositioning.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryquotes.code.tex File: tikzlibraryquotes.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryshadows.code.tex File: tikzlibraryshadows.code.tex 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryfadings.code.tex File: tikzlibraryfadings.code.tex 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryfadings.code .tex File: pgflibraryfadings.code.tex 2020/01/08 v3.1.5b (3.1.5b) ))) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryshapes.misc.code.tex File: tikzlibraryshapes.misc.code.tex 2020/01/08 v3.1.5b (3.1.5b) (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/shapes/pgflibraryshape s.misc.code.tex File: pgflibraryshapes.misc.code.tex 2020/01/08 v3.1.5b (3.1.5b) )) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarysnakes.code.tex File: tikzlibrarysnakes.code.tex 2020/01/08 v3.1.5b (3.1.5b) Package pgf Warning: Snakes have been superseded by decorations. Please use the decoration libraries instead of the snakes library on input line 14. ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarythrough.code.tex File: tikzlibrarythrough.code.tex 2020/01/08 v3.1.5b (3.1.5b) ) (/usr/share/texlive/texmf-dist/tex/latex/numprint/numprint.sty Package: numprint 2012/08/20 v1.39 Print numbers (HH) (/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty Package: array 2019/08/31 v2.4l Tabular extension package (FMi) \col@sep=\dimen199 \ar@mcellbox=\box60 \extrarowheight=\dimen256 \NC@list=\toks28 \extratabsurround=\skip49 \backup@length=\skip50 \ar@cellbox=\box61 ) \c@nprt@mantissa@digitsbefore=\count267 \c@nprt@mantissa@digitsafter=\count268 \c@nprt@exponent@digitsbefore=\count269 \c@nprt@exponent@digitsafter=\count270 \nprt@digitwidth=\skip51 \nprt@sepwidth=\skip52 \nprt@decimalwidth=\skip53 \nprt@blockwidth=\skip54 \nprt@digittoks=\toks29 \nprt@pretoks=\toks30 \nprt@posttoks=\toks31 \nprt@thisdigit=\count271 \nprt@curpos=\count272 \nprt@rndpos=\count273 \c@nprt@digitsfirstblock=\count274 \c@nprt@blockcnt=\count275 \c@nprt@cntprint=\count276 No configuration file `numprint.cfg' found.) (/usr/share/texlive/texmf-dist/tex/latex/l3packages/xfp/xfp.sty (/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3.sty Package: expl3 2020-02-14 L3 programming layer (loader) (/usr/share/texlive/texmf-dist/tex/latex/l3backend/l3backend-pdfmode.def File: l3backend-pdfmode.def 2020-02-03 L3 backend support: PDF mode \l__kernel_color_stack_int=\count277 \l__pdf_internal_box=\box62 )) (/usr/share/texlive/texmf-dist/tex/latex/l3packages/xparse/xparse.sty Package: xparse 2020-02-14 L3 Experimental document command parser \l__xparse_current_arg_int=\count278 \g__xparse_grabber_int=\count279 \l__xparse_m_args_int=\count280 \l__xparse_v_nesting_int=\count281 ) Package: xfp 2020-02-14 L3 Floating point unit ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp.sty Package: fp 1995/04/02 `Fixed Point Package', Version 0.8, April 2, 1995 (C) Michael Mehlich (/usr/share/texlive/texmf-dist/tex/latex/fp/defpattern.sty Package: defpattern 1994/10/12 \actioncount=\count282 ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp-basic.sty Package: fp-basic 1996/05/13 \FP@xs=\count283 \FP@xia=\count284 \FP@xib=\count285 \FP@xfa=\count286 \FP@xfb=\count287 \FP@rega=\count288 \FP@regb=\count289 \FP@regs=\count290 \FP@times=\count291 ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp-addons.sty Package: fp-addons 1995/03/15 ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp-snap.sty Package: fp-snap 1995/04/05 ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp-exp.sty Package: fp-exp 1995/04/03 ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp-trigo.sty Package: fp-trigo 1995/04/14 ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp-pas.sty Package: fp-pas 1994/08/29 ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp-random.sty Package: fp-random 1995/02/23 \FPseed=\count292 ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp-eqn.sty Package: fp-eqn 1995/04/03 ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp-upn.sty Package: fp-upn 1996/10/21 ) (/usr/share/texlive/texmf-dist/tex/latex/fp/fp-eval.sty Package: fp-eval 1995/04/03 )) \tkzRadius=\dimen257 \tkzLength=\dimen258 \tkz@radi=\dimen259 \tkz@ax=\dimen260 \tkz@ay=\dimen261 \tkz@bx=\dimen262 \tkz@by=\dimen263 \tkz@cx=\dimen264 \tkz@cy=\dimen265 \tkz@dx=\dimen266 \tkz@dy=\dimen267 \tkz@tax=\dimen268 \tkz@tay=\dimen269 \tkz@tbx=\dimen270 \tkz@tby=\dimen271 \tkz@tcx=\dimen272 \tkz@tcy=\dimen273 \tkz@tdx=\dimen274 \tkz@tdy=\dimen275 \tkz@cntmk=\count293 Local configuration file tkz-base.cfg found and used (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-base.cfg 2020/02/04 3.02c tkz-base.cfg ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-tools-modules.tex 2020/01/24 3.02c tkz-tools-utilities.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-lib-marks.tex 2020/01/24 3.02c tkz-lib-symbols.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-tools-base.tex 2020/01/24 3.02c tkz-tools-base.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-tools-utilities.tex 2020/01/24 3.02c tkz-tools-utilities.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-tools-math.tex 2020/01/24 3.02c tkz-tools-math.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-tools-text.tex 2020/01/24 3.02c tkz-tools-text.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-tools-BB.tex 2020/01/24 3.02c tkz-obj-BB.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-tools-arith.tex 2020/01/24 3.02c tkz-tools-arith.tex File: tkz-tool-arith.tex tkz-tool-arith 3.02 c ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-tools-print.tex 2020/01/24 3.02c tkz-tools-print.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-tools-misc.tex 2020/01/24 3.02c tkz-tools-misc.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-obj-axes.tex 2020/01/24 3.02c tkz-obj-axes.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-obj-grids.tex 2020/01/24 3.02c tkz-obj-grids.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-obj-marks.tex 2020/01/24 3.02c tkz-obj-marks.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-obj-points.tex 2020/01/24 3.02c tkz-obj-points.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-base/tkz-obj-rep.tex 2020/01/24 3.02c tkz-obj-rep.tex )) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-tools-intersections.te x 2020/01/24 3.02c tkz-tools-intersections.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-tools-angles.tex 2020/01/24 3.02c tkz-tools-angles.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-angles.tex 2020/01/24 3.02c tkz-tool-eu-angles.tex \tkz@arcsize=\dimen276 \tkz@fillsize=\dimen277 ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-arcs.tex 2020/01/24 3.02c tkz-obj-eu-arcs.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-compass.tex 2020/01/24 3.02c tkz-obj-eu-compass.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-circles.tex 2020/01/24 3.02c tkz-obj-eu-circles.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-draw-circles.te x 2020/01/24 3.02c tkz-obj-eu-draw-circles.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-lines.tex 2020/01/24 3.02c tkz-obj-eu-lines.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-draw-lines.tex 2020/01/24 3.02c tkz-obj-eu-draw-lines.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-points.tex 2020/01/24 3.02c tkz-obj-eu-points.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-points-by.tex 2020/01/24 3.02c tkz-tools-eu-points-by.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-points-rnd.tex 2020/01/24 3.02c tkz-obj-eu-points-rnd.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-points-with.tex 2020/01/24 3.02c tkz-obj-eu-points-with.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-draw-polygons.t ex 2020/01/24 3.02c tkz-obj-eu-polygons.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-polygons.tex 2020/01/24 3.02c tkz-obj-eu-polygons.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-protractor.tex 2020/01/24 3.02c tkz-obj-eu-protractor.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-sectors.tex 2020/01/24 3.02c tkz-obj-eu-sectors.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-show.tex 2020/01/24 3.02c tkz-obj-eu-show.tex ) (/usr/share/texlive/texmf-dist/tex/latex/tkz-euclide/tkz-obj-eu-triangles.tex 2020/01/24 3.02c tkz-obj-eu-triangles.tex )) ! I can't find file `tkz-obj-angles.tex'. <argument> ...\|=12 \input tkz-obj-\tkz@temp .tex \catcode `\@ =\csname tkz@... l.6 \usetkzobj{all} (Press Enter to retry, or Control-D to exit) Please type another input file name: ! Emergency stop. <argument> ...\|=12 \input tkz-obj-\tkz@temp .tex \catcode `\@ =\csname tkz@... l.6 \usetkzobj{all} End of file on the terminal! Here is how much of TeX's memory you used: 18127 strings out of 481239 405187 string characters out of 5920377 637319 words of memory out of 5000000 32943 multiletter control sequences out of 15000+600000 532338 words of font info for 24 fonts, out of 8000000 for 9000 1141 hyphenation exceptions out of 8191 60i,1n,91p,466b,197s stack positions out of 5000i,500n,10000p,200000b,80000s ! ==> Fatal error occurred, no output PDF file produced!
Encoder-only:
Encoder-decoder:
Recurrent Neural Networks are extremely powerful and flexible
Seq2Seq models are composed by:
Seq2Seq models are widely popular in e.g.:
Problem - for sequence labeling, you may need to incorporate information from both the preceding and following words. Can RNNs do that?
Image(url='../img/deep_bilstm.png'+'?'+str(random.random()), width=800)
RNNs with tokenization at the character level for:
RNNs for sentence representation:
Image(url='../img/pinter_char_rnn.png'+'?'+str(random.random()), width=400)