HW 5 Markov and RNN

based on ex and notes by Allison Parish Original text is the wikipedia entry for truth

To adjust with the probability play with temperature=

  • temperature = 0.2 (default)
  • temperature =0.8 (unprobable)
  • temperature >1 (WTF)

Number of outputs add n

  • generate(10)

Epochs to train

  • num_epochs =more better (20), specially shorter texts

Passing a Prefix

  • will start with that word

Returning as list

  • By defualt, textgen.generate() returns a NoneType, this is not very useful since it isn't a string or a list. By doing return_as_list=True textgen.generate() will return the values as a list.

note: textgenrnn is less effective when training on/predicting longer sequences (> 200 characters). Likewise textgenrnn is less effective when training on/predicting texts with very disparate grammatical styles.

In [1]:
!conda install -y keras
Solving environment: done

## Package Plan ##

  environment location: /Users/SebastianRev1/anaconda3

  added / updated specs: 
    - keras


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    numpy-1.12.1               |   py36h8871d66_1         3.6 MB
    ca-certificates-2018.03.07 |                0         124 KB
    keras-2.1.5                |           py36_0         490 KB
    certifi-2018.1.18          |           py36_0         143 KB
    protobuf-3.5.1             |   py36h0a44026_0         589 KB
    tensorflow-1.1.0           |      np112py36_0        23.6 MB
    openssl-1.0.2o             |       h26aff7b_0         3.4 MB
    libprotobuf-3.5.1          |       h2cd40f5_0         4.0 MB
    numba-0.37.0               |np112py36hb493f12_0         2.3 MB
    ------------------------------------------------------------
                                           Total:        38.3 MB

The following NEW packages will be INSTALLED:

    keras:           2.1.5-py36_0              
    libprotobuf:     3.5.1-h2cd40f5_0          
    protobuf:        3.5.1-py36h0a44026_0      
    tensorflow:      1.1.0-np112py36_0         

The following packages will be UPDATED:

    ca-certificates: 2017.08.26-ha1e5d58_0      --> 2018.03.07-0              
    certifi:         2017.7.27.1-py36hd973bb6_0 --> 2018.1.18-py36_0          
    numba:           0.37.0-np114py36h210bcc1_0 --> 0.37.0-np112py36hb493f12_0
    openssl:         1.0.2l-h57f3a61_2          --> 1.0.2o-h26aff7b_0         

The following packages will be DOWNGRADED:

    numpy:           1.14.2-py36ha9ae307_1      --> 1.12.1-py36h8871d66_1     


Downloading and Extracting Packages
numpy 1.12.1: ########################################################## | 100% 
ca-certificates 2018.03.07: ############################################ | 100% 
keras 2.1.5: ########################################################### | 100% 
certifi 2018.1.18: ##################################################### | 100% 
protobuf 3.5.1: ######################################################## | 100% 
tensorflow 1.1.0: ###################################################### | 100% 
openssl 1.0.2o: ######################################################## | 100% 
libprotobuf 3.5.1: ##################################################### | 100% 
numba 0.37.0: ########################################################## | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
In [2]:
!pip install textgenrnn
Collecting textgenrnn
  Downloading textgenrnn-0.1.1.tar.gz (777kB)
    100% |████████████████████████████████| 778kB 1.5MB/s ta 0:00:011
Requirement already satisfied: tensorflow in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from textgenrnn)
Requirement already satisfied: keras in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from textgenrnn)
Requirement already satisfied: h5py in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from textgenrnn)
Requirement already satisfied: werkzeug>=0.11.10 in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from tensorflow->textgenrnn)
Requirement already satisfied: six>=1.10.0 in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from tensorflow->textgenrnn)
Requirement already satisfied: protobuf>=3.2.0 in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from tensorflow->textgenrnn)
Requirement already satisfied: numpy>=1.11.0 in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from tensorflow->textgenrnn)
Requirement already satisfied: wheel>=0.26 in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from tensorflow->textgenrnn)
Requirement already satisfied: scipy>=0.14 in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from keras->textgenrnn)
Requirement already satisfied: pyyaml in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from keras->textgenrnn)
Requirement already satisfied: setuptools in /Users/SebastianRev1/anaconda3/lib/python3.6/site-packages (from protobuf>=3.2.0->tensorflow->textgenrnn)
Building wheels for collected packages: textgenrnn
  Running setup.py bdist_wheel for textgenrnn ... done
  Stored in directory: /Users/SebastianRev1/Library/Caches/pip/wheels/59/b1/4a/b5ee074f89a4fb3632930705658c28ba38cfacd38f29bc1340
Successfully built textgenrnn
Installing collected packages: textgenrnn
Successfully installed textgenrnn-0.1.1
In [1]:
from textgenrnn import textgenrnn
Using TensorFlow backend.
In [2]:
textgen = textgenrnn()
In [11]:
textgen.generate()
The super complete the body the first party of the star to the star consequences and some of the street and the star world of the sub in the band to a look at the star to the signed of the promised 

To train on own text use .tain_on_text( )

num_epochs =more better, specially shorter texts

In [3]:
textgen.train_on_texts(open("text_examples/truth.txt").readlines(), num_epochs=10)
Epoch 1/10
3055/3055 [==============================] - 3s 1ms/step - loss: 1.5610
Epoch 2/10
3055/3055 [==============================] - 3s 1ms/step - loss: 1.2773
Epoch 3/10
3055/3055 [==============================] - 3s 994us/step - loss: 1.1380
Epoch 4/10
3055/3055 [==============================] - 3s 1ms/step - loss: 1.0471
Epoch 5/10
3055/3055 [==============================] - 4s 1ms/step - loss: 0.9824
Epoch 6/10
3055/3055 [==============================] - 3s 1ms/step - loss: 0.9310
Epoch 7/10
3055/3055 [==============================] - 3s 1ms/step - loss: 0.8931
Epoch 8/10
3055/3055 [==============================] - 3s 1ms/step - loss: 0.8623
Epoch 9/10
3055/3055 [==============================] - 4s 1ms/step - loss: 0.8389
Epoch 10/10
3055/3055 [==============================] - 3s 1ms/step - loss: 0.8214
In [4]:
textgen.generate(5,temperature=.7,prefix='fly')
flying the concept of last inquiry or contential concept at the dermative of truth. Truth of the infame on the concept and truth of suich and need to truth when the debuty of Truth College for truth

flying are meaning the mean and basic would another these or life in the original Greechaire that an our unnevegate in the truth and lit and blaceplay in the original most of truth is being from the

flying take as what was always by the concept until the debate. I done a discussion from the fact, the deried ago of antilitience essence vacainita and truth of secondary of meane being a nature and

flyes in the reality in term itsers in the concept of a truth to be debated to our original or original by the conferetion of truth of the concept of trutheer died for truth, and truth on and essenc

flyes who take the reported jather take a human fact, and discovering tradential beet to be elitisms or faith of infirmed facidly offiniously familin conception, and essence in a watch of truth as m

In [5]:
poem = textgen.generate(10, temperature=0.8, return_as_list=True)
for line in poem:
    print(line.strip())
Are term by the truth is term to by the view and contextion as he can we am faighters in how to truth summon and also metaphysically false, than the divinion tradition in the concept, what is exciti
Pictorication is wirepessible that an election: "As get a "conceptics of until and earth" factions as Britas Greeker is a faith, the according to another ebs concept which include would be plays a c
Many were hair opena conception to be theories, and discumed grandaries what or life up may is another to detain."

Let us an original fauth-play, original meaning, or himself, the election to relai
Gordary a things or anceiring at Truth is more faith capances of this faith
Truth to be explainesed to truth aporteag togethon and calling in and include that we an often be applying to be yet, the back of truth, and the concept, that the development in a relation on truth,
Or ancertaging of being inquiry ..
Peirce time that truth as lies ob faith as some faith or voted or foundative, a reasession train to defend, and the selfies. I fan a Helbsolo as the dermation caturn or also
Friedrive and the emojias or mean in idea essence, it view. http://abcn.ws/2tg0HL
I am things are the score of being a youtubest used to be explained as an opported our and what is relationshorts to oceas that year ofter and even we another that truth is notice, an thousand, the
What dossooss of Praisoull concept as readly term truth to Pessuilty are sub or meaning this each of alson, and everyday, Truth still truth, include of truth as belief to one astross esting the fait
In [42]:
poem = textgen.generate(10, temperature=0.5, return_as_list=True)
for line in poem:
    print(line.strip())
Truth is 'De corresponden by the concept of friever a dependence leaves than sciences, what congeting to an inding than to and the revealing take tradit, factual faithlas takene. Many human activiti
Some manner of some manner of essential relation to human practices words are a metaphysical being in the fire term veritas.
Pragmatists like C. S. Peirce take to refer to an idea of truth as stoniogn which what is termed a concept or through, where its nature as anaman nature as far as it could profitably go: "The opinio
Various theories and views of truth can even false that an olame holding this be explained in a late, bring a being a based ard terms that an ancient or theories as correctness is a later derivation
Truth is most often used to refer to an indepreding concept of truth as what is a "truth, what is what human, undervivitived by truth is subjective hote for the concept, where out to be be usually i

Various theories and views of truth continue to be debated among standar, suggested and derivative or secondary and derivative. According for inquitite truth" as the concept of truth is being to mea
Some hall absolute inquiry would time to be explained in anyone has discovers theat theory or anti-metaphysicians stone in the truth tracer that Truth is metaphys in Schithole lit fact or %reaman ph
Friedrich Nietzsche famously suggested that an ancient, what human activities depend upon the concept, where its pleview, who collecture. Commonly, truth is viewed as the correspondence of modern co

In [43]:
textgen.reset()

Ok, so I think I understand the basics now... but what to do...

One of my biggest challenges is that this rrn method works well only with language (english...) That is becasue english language is very felxible, even when the message doesn't make sense it can still be displayed. Even when the word doesn't exist you can still write it.

I guess what I would be interested is to use something like this but with other language, like an image, or a 3d model, or a website. But the challenge is that all this things have a very strict syntax and a small error will make the entity invalid.

In [27]:
import random
list_of_words=['tech','apple','google','FaceBook']
techTruth=[]
for word in list_of_words:
    temp=random.random()
    if(temp>.6):
        temp=random.random()
    line=(textgen.generate(temperature=temp,prefix=word,return_as_list=True));
    techTruth.append(line)
    print(line)
['tech truth of the original Barts of truth the truth concept of truth famously being a income food to take. Don\'t take these science of later, which is toq and the score" is still you are faith and t']
['apple of the faith or truth to be a later of truth in the concept, and the concept of truth of truth in the truth of truth is a later of truth is a later of truth is truticed to be debated and every']
['googles of truth to an idea of truth to an earth and relative truth.\n\nIt relative one of truth of assumes and faith, an original the concept of truth as bashering to an intellectual faith on such in']
['FaceBook and Suggesten faith truth to mean the concept, and the resting and truth is a concept, which to be original or truth of the concept of truth to an including and truth of truth meaning to be']

The texts don't make any sence, althought they do remind me my undergrad days reading Kierkegaard:

“The self is a relation which relates itself to its own self, or it is that in the relation the relation relates itself to its own self; the self is not the relation but the relation relates itself to its own self.”

let me change the input corpus

In [28]:
textgen.reset()
In [29]:
textgen.train_on_texts(open("text_examples/truthShort.txt").readlines(), num_epochs=20)
Epoch 1/20
1865/1865 [==============================] - 2s 1ms/step - loss: 1.5918
Epoch 2/20
1865/1865 [==============================] - 2s 1ms/step - loss: 1.2825
Epoch 3/20
1865/1865 [==============================] - 2s 1ms/step - loss: 1.1169
Epoch 4/20
1865/1865 [==============================] - 2s 1ms/step - loss: 1.0119
Epoch 5/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.9380
Epoch 6/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.8783
Epoch 7/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.8302
Epoch 8/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.7872
Epoch 9/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.7543
Epoch 10/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.7247
Epoch 11/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.6974
Epoch 12/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.6733
Epoch 13/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.6530
Epoch 14/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.6356
Epoch 15/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.6198
Epoch 16/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.6070
Epoch 17/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.5962
Epoch 18/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.5868
Epoch 19/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.5790
Epoch 20/20
1865/1865 [==============================] - 2s 1ms/step - loss: 0.5735
In [44]:
import random
list_of_words=['Technology','Apple','Google','Facebook','Amazon']
techTruth=[]
for word in list_of_words:
    temp=random.random()
    
    #weighting down the chance for something to crazy
    if(temp>.4):
        temp=random.random()
    line=(textgen.generate(temperature=temp,prefix=word,return_as_list=True));
    techTruth.append(line)
    print(line)
['Technology and the concept of truth as debated that the concept of truth is a logical faith, that the concept of truth as a concept of truth is a logical term of truth in the concept of truth is a l']
['Applegies open as a nature at the revealing which is a "truth"']
["Google of truth are termed for truth that a lit 'Lation. The entire was also conventine' of conception and fact methoo indipal veritas. Beying a crated time as a faith who was nuass the essentis int"]
['Facebook and was among kid has a litting and of truth in the contection for the concept, and that being what its are one in the original manch as the concept of truth used to debated the method than']
['Amazon of truth is a faith the concept of truth in the concept of truth in the concept, and the concept of truth in the concept of truth is a later of truth in the concept, which is a logical faith ']

Now let me tell you the truth:

  • Technology or the concept of truth is a loster than which, and everyday faith in the sciences, and everyday faith that and theologians, and theologians, and theologians.
  • Google is artient, the faith, the original Match and everyday into the debated of truth as being an original faith thum the concept, and the asume on the conception of three inviets which was another'
  • Facebook term of truth in the concept of truth, include which assumed intelloust the concept, and debated from the conception into the revealing or which or the concept of truth of truth as a metaphore
  • Apple debated the concept of truth is a concept of truth in the concept of the concept of truth is a later for the concept, and theologians. The concept of truth is a logical faith in the concept,
  • Amazon and we concept of truth, and the contection and debated in truth in a discussion, indidation that the revealing, and the conception of truth as as a criterion of truth of the foundation

This is all for now... I find the idea exciting althought the execution a little limmiting. I guess this is the problem of trying to use tools that we have no idea how they work or how to change them

In [ ]: