Forum | Blog | Wiki

Zoo Dataset Test

An example of a multivariate data type classification problem using Neuroph

by Alma Murankovic, Faculty of Organisation Sciences, University of Belgrade

an experiment for Intelligent Systems course

Introduction

A Neural network is an artificial system (made of artificial neuron cells). It is modeled after the way the human brain works.In this example we will be testing Neuroph with Zoo Dataset, which can be found here http://archive.ics.uci.edu/ml/. Several architectures will be tried out, and it will be determined which ones represent a good solution to the problem.

First here are some useful information about our Zoo Dataset:
Data Set Characteristics:Multivariate
Number of Instances: 101
Attribute Characteristics:Categorical,Integer
Number of Attributes: 17
Associated Tasks:Classification

Introducing the problem

The objective of this problem is to create and train neural network to study the feasibility of classification animal species.The name of data set is Zoo Data Set create by Richard Forsyth.The data set that we use in this experiment can be found at
This data set includes 101 instances.We have 17 input attributes and one output. Input attributes are:

animal name
hair
feathers
eggs
milk
airborne
aquatic
predator
toothed
backbone
bretahes
venomous
fins
legs
tail
domestic
catsize

Data Set Information:

A simple database containing 17 Boolean-valued attributes. The "type" attribute appears to be the class attribute. Here is a breakdown of which animals are in which type:

Class - Set of animals:

1 - aardvark, antelope, bear, boar, buffalo, calf, cavy, cheetah, deer, dolphin, elephant, fruitbat, giraffe, girl, goat, gorilla, hamster, hare, leopard, lion, lynx, mink, mole, mongoose, opossum, oryx, platypus, polecat, pony, porpoise, puma, pussycat, raccoon, reindeer, seal, sealion, squirrel, vampire, vole, wallaby,wolf
2 - chicken, crow, dove, duck, flamingo, gull, hawk, kiwi, lark, ostrich, parakeet, penguin, pheasant, rhea, skimmer, skua, sparrow, swan, vulture, wren
3 - pitviper, seasnake, slowworm, tortoise, tuatara
4 - bass, carp, catfish, chub, dogfish, haddock, herring, pike, piranha, seahorse, sole, stingray, tuna
5 - frog, frog, newt, toad
6 - flea, gnat, honeybee, housefly, ladybird, moth, termite, wasp
7 - clam, crab, crayfish, lobster, octopus, scorpion, seawasp, slug, starfish, worm

In order to train a neural network, there are six steps to be made:

1. Normalize the data

2. Create a Neuroph project

3. Create a training set

4. Create a neural network

5. Train the network

6. Test the network to make sure that it is trained properly

Step 1. Normalizing the data

First the dataset must be normalized. It means that value in this dataset must be a number beetween 0 and 1.

In this case, the thirteenth and seventeenth column must be normalized. After we finished that we got 28 columns,exactly we got 21 inputs and 7 outputs.

Step 2. Creating a new Neuroph project.

Now we need to create our project. First, you must start Neuroph Studio and create a new project select File --> New Project.

After that, we click button "Next" and enter Project name. The name of our project is "ZOOProjekt". Then press Finish button to create your project.

Step 3. Creating a Training set

To create training set, we choose Training --> New Training Set. We need training our dataset to teach our neural network to perform clasification of animals species.

Select training set file type and click "Next". After that, enter training set name. Select the type of supervised because we are using the dataset with both input and output attributes, enter number of inputs and number of outputs.

Number of inputs: 21

Number of outputs: 7

In our example the name of training set is"TrainingAllData".

After pressing "Next" we must insert some data into training set table. We need to click "Load From File" button.

Now we choose our dataset,select values separator(in our case select the ,) and press "Load" to load chosen dataset.

After we clicked to load we must go on button "Finish",and then we can start to train our dataset.

After this, everything is ready for the creation of neural networks. We will create several neural networks with different parameters, and determine which is the best solution for our problem by testing them.

Training attempt 1

Step 4.1 Creating a neural network

Now we are going to create our neural network.We must click on our project in the "Project" window, and then click "New", then "Neural Network". Now me must enter your neural network name and choose a Neural Network Type.

Network Type: Multi Layer Perceptron

Our network will call "zooMreza1" and choose "Multi Layer Perception" type. A multilayer perceptron (MLP) is a feedforward artificial neural network model that maps sets of input data onto a set of appropriate output.

By pressing the "Next" button, we will get next window.

Training Algorithm: Backpropagation with Momentum
Number of inputs: 21
Hidden neurons: 15
Number of outputs: 7

First, we enter number of input and output neurons.The number of input and output neurons are the same as in the training set. And now we have to choose number of hidden layers. Deciding the number of neurons in the hidden layers is a very important part of deciding your overall neural network architecture. Though these layers do not directly interact with the external environment, they have a tremendous influence on the final output. Both the number of hidden layers and the number of neurons in each of these hidden layers must be carefully considered.

There are many rule-of-thumb methods for determining the correct number of neurons to use in the hidden layers, such as the following:

The number of hidden neurons should be between the size of the input layer and the size of the output layer.

The number of hidden neurons should be 2/3 the size of the input layer, plus the size of the output layer.

The number of hidden neurons should be less than twice the size of the input layer.

Next, we will check "Use Bias Neurons" because Bias neurons are added to neural networks to help them learn patterns. Then, for Transfer function select "Sigmoid" because it is the best solution for our king of problem, and last select "Backpropogation with Momentum" for learning rule because Backpropagation With Momentum algorithm shows a much higher rate of convergence than the Backpropagation algorithm.

After pressing "Finish", you create new Neural network. In the picture below we can see the graph view of this neural network.

Figure shows the input, the output and hidden neurons and how they are connected with each other. Except for two neurons with activation level 1-bias activation, all other neurons have an activation level 0.

Step 5.1 Training the neural network

After we have created training set and our neural network we can train that network.In this case,we will train with 60% od datasets and test other 40. We select the training set and click "Train". Now we have to set learning parameters for training. The maximum error will be 0.01, learning rate will be 0.3 and momentum will be 0.4. Learning rate is a control parameter of some training algorithms, which controls the step size when weights are iteratively adjusted.

Momentum simply adds a fraction m of the previous weight update to the current one. The momentum parameter is used to prevent the system from converging to a local minimum or saddle point. A high momentum parameter can also help to increase the speed of convergence of the system. However, setting the momentum parameter too high can create a risk of overshooting the minimum, which can cause the system to become unstable. A momentum coefficient that is too low cannot reliably avoid local minima, and can also slow down the training of the system.

Minimum mean square error of the epoch. Square root of the sum of squared differences between the network targets and actual outputs divided by number of patterns (only for training by minimum error).

After 30 iterations train was reached the max error. Total Network Error is 0.008464153008441102.

Table 1. Training results using another learning parameters:

Training attempt	Hidden Neurons	Learning Rate	Momentum	Max Error	Number of iterations	Total Net Errors
1.	15	0.2	0.7	0.01	30	0.00846
2.	15	0.4	0.4	0.01	18	0.00838
3.	15	0.3	0.7	0.01	14	0.00902

Step 6.1 Testing the neural network

After the network is trained, we click "Test". It is recommended to select the part of dataset which haven't been trained for testing.

The Total Mean Square Error is 0.03243928212023193. This isn't the best choice and we should try some to reach the best solution for our problem.

Training attempt 2

Step 4.2 Creating a neural network

We will use the same network as we used in previous attempt,but now we wil train with 70% and test other 30.

Step 5.2 Training the neural network

We select the training set and click "Train". A new window will open. The maximum error will be 0.01, learning rate will be 0.2 and momentum will be 0.7.

A graph will appear where we can see how the error is changing in every iteration. After 29 iterations train was reached the max error. Total Network Error is 0.009610920907586279.

Table 2. Training results using another learning parameters:

Training attempt	Hidden Neurons	Learning Rate	Momentum	Max Error	Number of iterations	Total Net Errors
4.	15	0.2	0.7	0.01	29	0.00961
5.	15	0.4	0.7	0.01	15	0.00999
6.	15	0.3	0.4	0.01	38	0.00993

Step 6.2 Testing the neural network

After the network is trained, we will choose "Test30" for testing the neural network.

How we can press "Test" button.

The Total Mean Square Error is 0.035600107905252074, which indicates that the solution in the previous case is better.

Training attempt 3

Step 4.3 Creating a neural network

We will use the same network as we used in previous attempt,but now we wil train with 80% and test other 20.

Step 5.3 Training the neural network

We select the training set and click "Train". A new window will open. The maximum error will be 0.01, learning rate will be 0.2 and momentum will be 0.7.

A graph will appear where we can see how the error is changing in every iteration. After 27 iterations train was reached the max error. Total Network Error is 0.0098948868555994837.

Table 3. Training results using another learning parameters:

Training attempt	Hidden Neurons	Learning Rate	Momentum	Max Error	Number of iterations	Total Net Errors
7.	15	0.2	0.7	0.01	27	0.00989
8.	15	0.4	0.7	0.01	14	0.00758
9.	15	0.3	0.4	0.01	42	0.00952

Step 6.3 Testing the neural network

After the network is trained, we will choose "Test20" for testing the neural network.

How we can press "Test" button.

The Total Mean Square Error is 0.0028291107730972985, which indicates that is the better than previous cases.

Training attempt 4

Step 4.4 Creating a neural network

Now we are going to create new neural network, "zooMreza2". The only differences will be in the number of hidden neurons. In this case we will put 10 hidden neurons. This network will give worse results than the previous one.

Training Algorithm: Backpropagation with Momentum
Number of inputs: 21
Hidden neurons: 10
Number of outputs: 7

Step 5.4 Training the neural network

We select the training set and click "Train". A new window will open. The maximum error will be 0.01, learning rate will be 0.2 and momentum will be 0.7.

A graph will appear where we can see how the error is changing in every iteration. After 19 iterations train was reached the max error. Total Network Error is 0.008502727854145176.

Table 4. Training results using another learning parameters:

Training attempt	Hidden Neurons	Learning Rate	Momentum	Max Error	Number of iterations	Total Net Errors
10.	10	0.2	0.7	0.01	19	0.008502
11.	10	0.4	0.4	0.01	47	0.00846
12.	10	0.3	0.7	0.01	14	0.00955

Step 6.4 Testing the neural network

After the network is trained, we will choose "Test40" for testing the neural network.

How we can press "Test" button

The Total Mean Square Error is 0.034094787000198384.

Training attempt 5

Step 4.5 Creating a neural network

We are going the train the same network as in the preceding example.

Step 5.5 Training the neural network

We select the training set and click "Train". A new window will open. The maximum error will be 0.01, learning rate will be 0.2 and momentum will be 0.7.

A graph will appear where we can see how the error is changing in every iteration. After 32 iterations train was reached the max error. Total Network Error is 0.009906603376969936.

Table 5. Training results using another learning parameters:

Training attempt	Hidden Neurons	Learning Rate	Momentum	Max Error	Number of iterations	Total Net Errors
13.	10	0.2	0.7	0.01	32	0.00990
14.	10	0.4	0.4	0.01	36	0.00991
15.	10	0.3	0.7	0.01	22	0.00995

Step 6.5 Testing the neural network

After the network is trained, we will choose "Test30" for testing the neural network.

How we can press "Test" button.

The Total Mean Square Error is 0.048350400690786756.

Training attempt 6

Step 4.6 Creating a neural network

We are going the train the same network as in the preceding example.

Step 5.6 Training the neural network

We select the training set and click "Train". A new window will open. The maximum error will be 0.01, learning rate will be 0.2 and momentum will be 0.7.

A graph will appear where we can see how the error is changing in every iteration. After 49 iterations train was reached the max error. Total Network Error is 0.009876188297591385.

Table 6. Training results using another learning parameters:

Training attempt	Hidden Neurons	Learning Rate	Momentum	Max Error	Number of iterations	Total Net Errors
16.	10	0.2	0.7	0.01	49	0.00988
17.	10	0.4	0.4	0.01	10	0.00961
18.	10	0.3	0.7	0.01	11	0.00878

Step 6.6 Testing the neural network

After the network is trained, we will choose "Test20" for testing the neural network.

How we can press "Test" button.

The Total Mean Square Error is 0.04166680587996988.

Training attempt 7

Step 4.7 Creating a neural network

We are going to create a new neural network,"zooMreza3" which will have 40 hidden neurons.

Step 5.7 Training the neural network

We select the training set and click "Train". A new window will open. The maximum error will be 0.01, learning rate will be 0.2 and momentum will be 0.7.

A graph will appear where we can see how the error is changing in every iteration. After 20 iterations train was reached the max error. Total Network Error is 0.006822656757290463.

Table 7. Training results using another learning parameters:

Training attempt	Hidden Neurons	Learning Rate	Momentum	Max Error	Number of iterations	Total Net Errors
19.	40	0.2	0.7	0.01	20	0.00682
20.	40	0.4	0.4	0.01	24	0.00982
21.	40	0.3	0.7	0.01	17	0.00301

Step 6.7 Testing the neural network

After the network is trained, we will choose "Test40" for testing the neural network.

How we can press "Test" button.

The Total Mean Square Error is 0.0432385420946262.

Training attempt 8

Step 4.8 Creating a neural network

We are going the train the same network as in the preceding example.

Step 5.8 Training the neural network

We select the training set and click "Train". A new window will open. The maximum error will be 0.01, learning rate will be 0.2 and momentum will be 0.7.

A graph will appear where we can see how the error is changing in every iteration. After 29 iterations train was reached the max error. Total Network Error is 0.00877714677824164.

Table 8. Training results using another learning parameters:

Training attempt	Hidden Neurons	Learning Rate	Momentum	Max Error	Number of iterations	Total Net Errors
22.	40	0.2	0.7	0.01	29	0.00878
23.	40	0.4	0.4	0.01	27	0.00981
24.	40	0.3	0.7	0.01	17	0.00850

Step 6.8 Testing the neural network

After the network is trained, we will choose "Test30" for testing the neural network.

How we can press "Test" button.

The Total Mean Square Error is 0.04687419534448093.

Training attempt 9

Step 4.9 Creating a neural network

We are going the train the same network as in the preceding example.

Step 5.9 Training the neural network

We select the training set and click "Train". A new window will open. The maximum error will be 0.01, learning rate will be 0.2 and momentum will be 0.7.

A graph will appear where we can see how the error is changing in every iteration. After 33 iterations train was reached the max error. Total Network Error is 0.007149720580853874.

Table 9. Training results using another learning parameters:

Training attempt	Hidden Neurons	Learning Rate	Momentum	Max Error	Number of iterations	Total Net Errors
25.	40	0.2	0.7	0.01	33	0.00715
26.	40	0.4	0.4	0.01	23	0.00954
27.	40	0.3	0.7	0.01	16	0.00875

Step 6.9 Testing the neural network

After the network is trained, we will choose "Test20" for testing the neural network.

How we can press "Test" button.

The Total Mean Square Error is 0.0019519341890287078.

Conclusion

We explained in detail the creation, training and testing neural networks. We have created 3 different architectures of neural networks.
In our experiment we have tried by 10,15 and 40 hidden neurons. The results have shown that the best solution by using 40 hidden
neurons and so we trained 80% of dataset and test another 20%,The Total Mean Square Error was 0.0019519341890287078.

The best results of our experiment are given in the table below.

Training set	Testing set	Hidden neurons	Iterations	Total Net Error	Total Mean Square Error (during testing)
80%	20%	15	27	0.0098948868555994837	0.0028291107730972985
70%	30%	10	19	0.008502727854145176	0.034094787000198384
80%	20%	40	33	0.007149720580853874	0.0019519341890287078

DOWNLOAD