5.2. IMPLEMENTING NEURAL NETWORKS 171
ìintersectionîB, and the set notation for the logical function OR isA∪B,also
calledAìunionîB.
Now consider another logical function. In the following example we consider
theexclusive OR, which, in set notation, is writtenAXORB=(A∩B^0 )∪
(B∩A^0 ). This set is also called thesymmetric differenceof the setsAand
B, often writtenA 4 B.
Example 5.2Theexclusive or(XOR) is the Boolean function with truth
table
g(x 1 ,x 2 )=
0 ifx 1 =x 2 =1
0 ifx 1 =x 2 =0
1 otherwiseWhen we display the input-output relationship expressed byg, we see that this
problem is not linearly separable.
That is, there is no line that can separate the two subsetsW={(0,0),(1,1)}
andB={(0,1),(1,0)}. Therefore, the functiongcannot be implemented by
using a perceptron, so we need to consider a hidden layer.
From Exercise 2 on page 193, you know how to implementS∩T^0 with weights
and bias(w 1 A,w 2 A,bA)and(w 1 B,w 2 B,bB), respectively. From Example 5.1, we
know how to implement OR with weights and bias(wAC,wBC,bC).Putting
these neurons together we get a solution forXOR, with the expense of having a
neural network architecture more complicated than that of perceptrons, namely,
using a two-layer neural network:
This is a multi-layer neural network in which the input layer (or layer 0 )has
two nodesx 1 ,x 2 (no computation), together withx 0 =1to implement the bias
for neuronsAandB; the hidden layer has two hidden neuronsAandB,with
x 0 =1to implement the bias for neuronC; and the output layer consists of