Terry Stewart
after(eight, nine)
chase(dogs, cats)
knows(Anne, thinks(Bill, likes(Charlie, Dave)))
[number:eight next:nine]
[subject:dogs action:chase object:cats]
[subject:Anne action:knows object:[subject:Bill action:thinks object:[subject:Charlie action:likes object:Dave]]]
Problems
Implementing Symbol Systems in Neurons
There is an alternate approach
Something that's similar to the symbolic approach, but much more tied to biology
Based on vectors and functions on those vectors
Example
BLUE
$\circledast$ SQUARE + RED
$\circledast$ CIRCLE
Lots of nice properties
[number:eight next:nine]
NUMBER
$\circledast$ EIGHT + NEXT
$\circledast$ NINE
[subject:Anne action:knows object:[subject:Bill action:thinks object:[subject:Charlie action:likes object:Dave]]]
SUBJ
$\circledast$ ANNE + ACT
$\circledast$ KNOWS + OBJ
$\circledast$ (SUBJ
$\circledast$ BILL + ACT
$\circledast$ THINKS + OBJ
$\circledast$ (SUBJ
$\circledast$ CHARLIE + ACT
$\circledast$ LIKES + OBJ
$\circledast$ DAVE))
RED
is similar to PINK
then RED
$\circledast$ CIRCLE
is similar to PINK
$\circledast$ CIRCLE
But rather complicated
RED
$\circledast$ CIRCLE + BLUE
$\circledast$ TRIANGLE
CIRCLE'
'
is "inverse"RED
$\circledast$ CIRCLE + BLUE
$\circledast$ TRIANGLE
) $\circledast$ CIRCLE'
RED
$\circledast$ CIRCLE
$\circledast$ CIRCLE' + BLUE
$\circledast$ TRIANGLE
$\circledast$ CIRCLE'
RED + BLUE
$\circledast$ TRIANGLE
$\circledast$ CIRCLE'
RED + noise
RED
OBJ1
$\circledast$ (TYPE
$\circledast$ STAR + SIZE
$\circledast$ LITTLE) + OBJ2
$\circledast$ (TYPE
$\circledast$ STAR + SIZE
$\circledast$ BIG) + BESIDE
$\circledast$ OBJ1
$\circledast$ OBJ2
BESIDE
$\circledast$ OBJ1
$\circledast$ OBJ2
= BESIDE
$\circledast$ OBJ2
$\circledast$ OBJ1
S
= RED
$\circledast$ NOUN
VAR
= BALL
$\circledast$ NOUN'
S
$\circledast$ VAR
= RED
$\circledast$ BALL
This is not an actual question on the test
How can we model people doing this task?
A fair number of different attempts
Does this vector approach offer an alternative?
First we need to represent the different patterns as a vector
How do we represent a picture?
SHAPE
$\circledast$ ARROW + NUMBER
$\circledast$ ONE + DIRECTION
$\circledast$ UP
We have shown that it's possible to build these sorts of representations up directly from visual stimuli
The memory of the list is built up by using a basal ganglia action selection system to control feeding values into an integrator
The same system can be used to do a version of the Raven's Matrices task
S1 = ONE
$\circledast$ P1
S2 = ONE
$\circledast$ P1 + ONE
$\circledast$ P2
S3 = ONE
$\circledast$ P1 + ONE
$\circledast$ P2 + ONE
$\circledast$ P3
S4 = FOUR
$\circledast$ P1
S5 = FOUR
$\circledast$ P1 + FOUR
$\circledast$ P2
S6 = FOUR
$\circledast$ P1 + FOUR
$\circledast$ P2 + FOUR
$\circledast$ P3
S7 = FIVE
$\circledast$ P1
S8 = FIVE
$\circledast$ P1 + FIVE
$\circledast$ P2
what is S9
?
Let's figure out what the transformation is
T1 = S2
$\circledast$ S1'
T2 = S3
$\circledast$ S2'
T3 = S5
$\circledast$ S4'
T4 = S6
$\circledast$ S5'
T5 = S8
$\circledast$ S7'
T = (T1 + T2 + T3 + T4 + T5)/5
S9 = S8
$\circledast$ T
S9 = FIVE
$\circledast$ P1 + FIVE
$\circledast$ P2 + FIVE
$\circledast$ P3
This becomes a novel way of manipulating structured information
D3 + A + D2 + B
D3 + B + D2 + A
D3
$\circledast$ A + D2
$\circledast$ B
D = 64
subdim = 8
N = 500
import nef
net=nef.Network('Symbols', fixed_seed=1, quick=True) #Create the network object
net.make('A',neurons=1,dimensions=D,mode='direct') # don't bother simulating these neurons
net.make('B',neurons=1,dimensions=D,mode='direct') # don't bother simulating these neurons
net.make_array('C',N,D/subdim,dimensions=subdim,radius=1.0/math.sqrt(D), seed=2)
conv = nef.convolution.make_convolution(net,'*','A','B','C',200, seed=3)
net.add_to_nengo()
D = 64
subdim = 8
N = 500
import nef
net=nef.Network('Symbols', fixed_seed=1, quick=True) #Create the network object
net.make('A',1,D,mode='direct')
net.make('B',1,D,mode='direct')
net.make_array('C',N,D/subdim,dimensions=subdim,radius=1.0/math.sqrt(D), seed=2)
conv = nef.convolution.make_convolution(net,'*','A','B','C',200, seed=3)
net.make('E',1,D,mode='direct')
net.make('F',1,D,mode='direct')
conv = nef.convolution.make_convolution(net,'/','C','E','F',200, invert_second=True, seed=3)
net.add_to_nengo()
D = 64
subdim = 8
N = 500
import nef
net=nef.Network('Symbols', fixed_seed=1, quick=True) #Create the network object
net.make('A',1,D,mode='direct')
net.make('B',1,D,mode='direct')
net.make_array('C',N,D/subdim,dimensions=subdim,radius=1.0/math.sqrt(D), seed=2)
net.connect('C', 'C', pstc=0.1)
conv = nef.convolution.make_convolution(net,'*','A','B','C',200, seed=3)
net.make('E',1,D,mode='direct')
net.make('F',1,D,mode='direct')
conv = nef.convolution.make_convolution(net,'/','C','E','F',200, invert_second=True, seed=3)
net.add_to_nengo()