Author: Velibor Ilić
Abstract: In this text the appliance of neural networks at object-oriented programming is being discussed. NeuroVCL components for Delphi represent development kit for building programs with backpropagation neural networks. In this text the example of program developing using these components has been given.
Programming languages: Delphi 5
Operating systems: Windows 95, 98, NT
Date: Jun, 2000
NeuroVCL Components (Delphi 6)
Available for download since 25.11.2001.
Download!!! NeuroVCL V1.0 (NEUROVCLD6.ZIP 69Kb)
https://www.dropbox.com/s/d2becyna22c8il9/NEUROVCLD6.ZIP?dl=0
NeuroVCL Components (Delphi 5)
Available for download since 05.12.2000.
Download!!! NeuroVCL V1.0 (NEUROVCL.ZIP 68Kb)
https://www.dropbox.com/s/2l6a7pip3yc5ecc/NEUROVCL.ZIP?dl=0
NeuroVCL Examples
Available for download since 25.11.2001.
Download source code (23KB)
https://www.dropbox.com/s/t4vg5lyluoz4z2h/neuro_examples.zip?dl=0
This paper was presented on 5th seminar on neural networks (NEUREL)
| Ilić, V., (2000): “NeuroVCL components for Delphi”, Seminar on Neural Network Applications in Electrical Engineering “NEUREL 2000”, Belgrade, Sponsored by IEEE Signal Processing Society, pp 130-134, IEEE Catalog Number 00EX278, ISBN 0-7803-5512-1 (Softbounded Edition), ISBN 0-7803-5512-X (Microfishe Edition), Library of Congress: 99-60930, http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=902400
|
Introduction
Software requests
ANN V2.3 - neural net training program
Object VCL components for Delphi
Object TNNSignal
Object TNeuralNet
Example 1 - using NeuroVCL components
Example 2 - binary functions
Example 3 - pattern recognition
Conclusion
Referecnes
Theoretically a neural network can be trained to calculate every countable function. Practically, neural networks show best results in fields such as: classification, pattern recognition, voice recognition, processing information with noise, approximation functions, solving problems with a lot of available examples for training, and problems which need fast selection role depending on input.
This is exactly the class of problems, which is hard to handle with the usage of the common programming languages. Exposing a neural net as the object component is the best way to utilize advantages of this different way of solving problems.
1. Windows 95/98, NT/2000
2. Borland Delphi 5.0
3. NeuroVCL components
4. ANN V2.3 - neural net training program
The program ANN is based on the backpropagation neural networks. This program is used for training neural networks that are used in programs developed by NeuroVCL components.
Figure 1. - ANN V2.3 - Neural network training program
The program ANN is used to train neural networks with patterns from training set. The following files are necessary to assign neural network configurations and to determinate training set:
| Extension |
Description |
| *.INI |
This file describes neural network configuration: number of layers, number of neurons on layers, learning coefficient, coefficient
to survey training. |
| *.INP |
Inputs – Training data set which neural networks are trained to recognize |
| *.OUT |
Outputs - Training data set expected on output. |
Files created by ANN after training the neural network:
| Extension |
Description |
| *.NET |
This file is created with saving of all important coefficients of the neural network. It contains the states of all weights and thresholds of all the neurons in the network. |
| *.ERR |
This file contains recorded changes minimal, maximum and average error at the end of each iteration. |
| *.LOG |
In this file is given values of neural networks outputs and expected ones after XX iteration for all the patterns
from training set. |
| *.PAR |
In this file the weight and a threshold change of every neuron of the network has been saved. |
After the successful training weights are saved to file (with *.NET extension) and later are loaded in neuro components which are used in the final application.
ANN is developed with the Delphi 5 programming language. The program is developed with the usage of object-oriented methodology. During the development, the following objects have been created: TArray, TNeuron and TNet. These objects are used as a base for the development of the NeuroVCL components for Delphi.
More information...
Download!!! ANN V2.3 (ann.zip 463Kb)
https://www.dropbox.com/s/b3k6purhw2dl99y/ann.zip?dl=0
Classes' present a base for object oriented programming. Class is the set of the related fields and methods (functions and procedures), which are used to complete a certain program task. Classes represent a set of data, and the objects are particular specimens of the classes.
Programming in Delphi is based on VCL classes (VCL Visual Components Class).
With the help of TNNSignal type object, the values of the neural network inputs are changed and read on the network outputs after the calculation.
Look of components on form
Properties of TNNSignal:
| Properties name |
Type |
Description |
| Limited |
Boolean |
If this property is set True program does not allow overvaluing min/max in property Value |
| Max |
Integer |
Maximum value of the signal if value is limited |
| Min |
Integer |
Minimum value of the signal If value is limited |
| Name |
String |
Object name |
| Value |
Real |
Value of signal |
Events of TNNSignal
OnChange - Events are performing if value of property Value is changed.
Methods
| Method name |
Description |
| Constructor Create |
Create object TNNSignal |
| Destructor Destroy |
Free memory taken by object |
The object of the TNeuralNet type presents a neural network. This object does the processing of the inputs signals (TNNSignal) and calculates appropriate values, which are read by output signals (TNNSignal). Data from file (*.NET) with weights, generated by ANN determinates the function of this object (performance, “algorithm”), information on the way of outputs calculation, depending on inputs.
 VCL Net
Properties
| Properties name |
Type |
Description |
| Active |
Boolean |
Property determinates if network is active or not. |
| FileName |
TStrings |
Path to file (*.NET) that created after the neural network training with ANN program. This file contains the states of threshold and weights of the neurons of the network. |
| LayerHidden01 |
Integer |
Number of neurons on 1. hidden layer |
| LayerHidden02 |
Integer |
Number of neurons on 2. hidden layer |
| LayerHidden03 |
Integer |
Number of neurons on 3. hidden layer |
| LayerHidden04 |
Integer |
Number of neurons on 4. hidden layer |
| LayerHidden05 |
Integer |
Number of neurons on 5. hidden layer |
| LayerHidden06 |
Integer |
Number of neurons on 6. hidden layer |
| LayerHidden07 |
Integer |
Number of neurons on 7. hidden layer |
| LayerHidden08 |
Integer |
Number of neurons on 8. hidden layer |
| LayerHidden09 |
Integer |
Number of neurons on 9. hidden layer |
| LayerHidden10 |
Integer |
Number of neurons on 10. hidden layer |
| Name |
String |
Object name |
| Ni |
Real |
Learning coefficient |
| SignalInputs |
TStrings |
Names of input signals |
| SignalOutputs |
TStrings |
Names of output signals |
After placing the necessary objects in to the form it is necessary to connect the neural network with input and output signals. First we choose input signals by clicking on property SignalInputs. The given dialog box is used for an easy selection of object names on a neural network input. The same way we connect output signals with neural network.
By connecting signals with neural networks the value of property InputsLayer and OutuptsLayer is automatically changed.
Methods
| Method name |
Description |
| Constructor Create |
Creates object TNeuralNet |
| Destructor Destroy |
Free memory taken by object |
| Procedure LoadFromFile(FileName:String) |
Load weights and thresholds from file.
Note: Number of layers and neurons on every layer in the program must be the same as the number of layers and neurons on every layer in neural net what was trained by program ANN. |
| Procedure SaveToFile(FileName:String) |
Saves weights and thresholds to file. |
| Procedure Update |
Calculate values on networks outputs depending on input data. |
Using ANN on created appropriate training set is trained neural network for specific problem solving. Given weights and thresholds from neural networks are saved to file, which is later used in the program that is being created.
In this example neural network is trained to recognize 12 different patterns size 3×3 (figure 1.1) in array size 5×5. These patterns can be placed in 9 positions in array (figure 1.2.). The training set for this net is consisted of 108 elements (the number of positions × the number of shapes) and the exact examples (figure 1.1, and 1.2).
Neural network configuration:
Three layers neural net with backpropagation algorithm of training:
Number neurons at input layer (number of inputs): 5×5 = 25 (figure 1.1.)
Number neurons at hidden layer: 20
Number neurons at output layer (number of outputs): 12
Learning coefficient (alpha): 0.25
Number of patterns in training set: 12×9 = 108 (figure 1.1 and figure 1.2)
Training set:
a) Inputs:
figure 1.1. - positions of object in array
ANN has been trained by using 3 textual files (*.INI, *.INP, *.OUT). The details about file specification can be read in [5, 6].
After a successful training the neural network is necessary for saving weights and thresholds in the file (for example shapes.net).
figure 1.2. - different objects
This example shows the flow of the application building for the pattern recognition in array 5×5. It is necessary to set 37 objects type of TSignal (25 inputs + 12 outputs). Set 25 TImage that reacts when user clicks on it and object changes colors (black or white). By changing colors the value of the signals (TSignal) linked to image (Algorithm 1.1.) were changed.
TForm1.Image1Click(Sender: TObject);
begin
If Image1.Canvas.Pixels[12,12]=RGB(0,0,0) then
begin
Image1.Canvas.Brush.Color:=RGB(255,255,255);
In01.Value:=0;
end
else
begin
Image1.Canvas.brush.Color:=RGB(0,0,0);
In01.Value:=1;
end;
Image1.Canvas.Rectangle(5,5,46,53);
end; |
Algorithm 1.1. Procedure links the reaction of the user (clicking mouse button on image) with inputs of neural networks.
The TneuralNet type object must be set on the form. This object has to be linked with input and output signals (figure 4).
Two buttons (TButton) are placed so that the program could be controlled easier.
When designing of the user interface is finished, it is necessary to add programming code to link events.
In procedure that is performed during the start of the program, we should load network weights and thresholds and activate objects.
Procedure TForm1.FormCreate(Sender: TObject);
Begin
NeuralNet1.LoadFromFile(shapes.net');
NeuralNet1.Activate:=true;
End; |
Algorithm 1.2 - Loading network coefficients and activating object
Procedure that determinates what symbol is drawn on array (Algorithm 1.3.).
Procedure TForm1.Button1Click(Sender: TObject);
Begin
NeuralNet1.Update;//update outputs on neural net
Max:=Out01.Value; //determinate max output value
Pos:=1;
If max<Out02.Value then
Begin
Max:=Out02.Value;
Pos:=2;
End;
.....
Repeated for every 12 values
.....
If pos=1 then label1.caption:='1';
If pos=2 then label1.caption:='2';
...
End;
Str(max:4:2,SMax);
Label2.Caption:=SMax;// probability
End; |
Algorithm 1.3 – Determination of the symbol in array
Symbols that we want to determinate are brought on input of neural network using classical programming method. Comparisons of the values on neural network outputs are performed with IF command, by which the symbol in the array is determined. As we can see in this program neural networks can be viewed as complex function (black box) with 25 inputs and 12 outputs.
Description of problem
Program demonstrate calculation of 10 binary function with apliance neural networks.
In this program is used three layers neural net with backpropagation algorithm of training:
Number neurons at input layer (number of inputs): 2
Number neurons at hidden layer: 6
Number neurons at output layer (number of outputs): 10
Learning coefficient (alpha): 0.25
Number of patterns in training set: 10
Training set which is used for neural network training is shown in following table:
| input 1 |
input 2 |
output 1 |
output 2 |
output 3 |
output 4 |
output 5 |
output 6 |
output 7 |
output 8 |
output 9 |
output 10 |
| x |
y |
x and y |
x or y |
x xor y |
x => y |
x <=> y |
not(x) |
not(y) |
not
(x and y) |
not
(x or y) |
not
(x => y) |
| 0 |
0 |
0 |
0 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
0 |
| 0 |
1 |
0 |
1 |
1 |
1 |
0 |
1 |
0 |
1 |
0 |
0 |
| 1 |
0 |
0 |
1 |
1 |
0 |
0 |
0 |
1 |
1 |
0 |
1 |
| 1 |
1 |
1 |
1 |
0 |
1 |
1 |
0 |
0 |
0 |
0 |
0 |
Creating and training neural networks
First is created file BIN.INI that is used to define neural network structure, number of layer and number of neurons on every layer. Then is necessary to create two files to describe training set (BIN.INP, BIN.OUT).
ANN (neural network training program) is used to train created neural network and save states of connection weights in file (BIN.NET).
Figure 2.1. - ANN - neural network training program
Developing application in Delphi
On form was putted 12 object type TNNSignal (2 for inputs and 10 for outputs of network) and 1 object type TNeuralNet. In object inspector was adopted properties SignalInputs and SignalOutputs by dialog box to connect input and output signals with neural network. On hidden layer must be putted same number of neurons as in network during training with ANN.
Figure 2.2. - Form in Delphi during development application
When user click on button is executed following procedure:
procedure TForm1.Button1Click(Sender: TObject);
var s:string;
begin
Input1.Value:=StrToFloat(Edit1.text); //set new value on first network input
Input2.Value:=StrToFloat(Edit2.text); //set new value on second network input
NeuralNet1.Update; // count now values on network outputs
str(Output1.Value:2:2,s); // read value on first network output and convert to string
Label1.Caption:=s; // show value on first network output
str(Output2.Value:2:2,s);
Label2.Caption:=s;
str(Output3.Value:2:2,s);
Label3.Caption:=s;
str(Output4.Value:2:2,s);
Label4.Caption:=s;
str(Output5.Value:2:2,s);
Label5.Caption:=s;
str(Output6.Value:2:2,s);
Label6.Caption:=s;
str(Output7.Value:2:2,s);
Label7.Caption:=s;
str(Output8.Value:2:2,s);
Label8.Caption:=s;
str(Output9.Value:2:2,s);
Label9.Caption:=s;
str(Output10.Value:2:2,s);
Label10.Caption:=s;
end; |
Algorithm 2.1 - Loading network parameters and activating objects
Content of file with example N_SRC1.ZIP
/data // copy files from this directory in directory data of program ANN
bin.ini // information about network configuration
bin.inp // training set - inputs
bin.out // training set - outputs
/source
bin.net // connection weights count using ANN V2.3
Project1.cfg
Project1.dof
Project1.dpr
Project1.res
Unit1.dcu
Unit1.dfm
Unit1.pas
Description of problem
This program demonstrate using of NeuroVCL components for pattern recognition. Neural network is trained on examples to recognize 8 shapes in matrix 3×3.
In this program is used three layers neural net with backpropagation algorithm of training:
Number neurons at input layer (number of inputs): 9
Number neurons at hidden layer: 6
Number neurons at output layer (number of outputs): 8
Learning coefficient (alpha): 0.25
Number of patterns in training set: 8
Training set what is used for training neural network. Training set is consisting eight patterns (figure 3.1).
Figure 3.1 - pattrerns from training set
Neural network have 8 outputs (every output present one of patterns). Output of neural networks is one value 1 and rest of values is 0. Position of 1 determinate pattrern on input.
Creating and training neural networks
First is created file PATTERNS.INI that is used to define neural network structure, number of layer and number of neurons on every layer. Then is necessary to create two files to describe training set (PATERNS.INP, PATERNS.OUT).
ANN (neural network training program) is used to train created neural network and save states of connection weights in file (PATERNS.NET).
Figure 3.2 - ANN - neural network training program
Developing application in Delphi
On form was putted 17 object type TNNSignal (9 for inputs and 8 for outputs of network) and 1 object type TNeuralNet. In object inspector was adopted properties SignalInputs and SignalOutputs by dialog box to connect input and output signals with neural network. On hidden layer must be putted same number of neurons as in network during training with ANN.
Figure 3.2 - Form in Delphi during development application
procedure TForm1.Image1Click(Sender: TObject);
begin
if image1.Canvas.Pixels[12,12]=RGB(0,0,0) then
begin
image1.Canvas.brush.Color:=RGB(255,255,255);
In01.Value:=0;
// <- field color white
end
else
begin
image1.Canvas.brush.Color:=RGB(0,0,0);
In01.Value:=1;
// <- field color black
end;
image1.Canvas.Rectangle(5,5,46,53);
end; |
Algorithm 3.1 - connecting user reaction
(click on Image) with neural network
procedure TForm1.Button1Click(Sender: TObject);
var max:real;
pos:integer;
smax:string;
begin
NeuralNet1.Update; // <- update values on neural network output
max:=Out01.Value;
pos:=1;
if max<Out02.Value then
begin max:=Out02.Value; pos:=2;
end;
if max<Out03.Value then
begin max:=Out03.Value; pos:=3;
end;
if max<Out04.Value then
begin max:=Out04.Value; pos:=4;
end;
if max<Out05.Value then
begin max:=Out05.Value; pos:=5;
end;
if max<Out06.Value then
begin max:=Out06.Value; pos:=6;
end;
if max<Out07.Value then
begin max:=Out07.Value; pos:=7;
end;
if max<Out08.Value then
begin max:=Out08.Value; pos:=8;
end;
Case pos of
1:label9.caption:='1';
2:label9.caption:='2';
3:label9.caption:='3';
4:label9.caption:='4';
5:label9.caption:='5';
6:label9.caption:='6';
7:label9.caption:='7';
8:label9.caption:='8';
end;
str(max:4:2,smax);
label10.caption:=smax;
end; |
Algorithm 3.2 - updating network outputs and showing results
Content of file with example N_SRC2.ZIP
/data // copy files from this directory in directory data of program ANN
paterns.ini // information about network configuration
paterns.inp // training set - inputs
paterns.out // training set - outputs
/source
paterns.net // connection weights count using ANN V2.3
Project1.cfg
Project1.dof
Project1.dpr
Project1.res
Unit1.dcu
Unit1.dfm
Unit1.pas
NeuroVCL components in combination with ANN - neural network training program is not limited on patterns recognition. These components can be used for solving any problem, which can be presented in an appropriate way.
With the example given in this text we have seen that neural networks and classical methods for solving the problems (programming languages) are interlarded. The NeuroVCL components can be observed as an expansion of possibilities of Delphi programming languages.
The advantage of such combining is that the programmer does not have to know complex mathematics and logic functions that link inputs and outputs, as he must know that he solved the problem by classic method.
With the appliance of the object oriented programming, neural network is presented as an object, which is simply used for solving the exact problem. With such a method the programmer is free from necessity of knowing inputs and outputs of the inside projection principles of the neural network. It can be observed as a black box (unknown complex logic function).
The advantage of using the object components regarding to the other software that makes the appliance of the neural networks possible, is that the programmer may write an application of his own, not depending on other software (exp. MathLab).
Also, with the appliance of the object components, the programmer has freedom for creating the users interface, what differs from the specialized application.
Using ANN and objects, which are forgoing developing NeuroVCL components, is developed OCR
V1.1 program for recognizing Cyrillic letters [5, 6].
Please send me a comments, suggestion and critics about improvement NeuroVCL components in newer version. Also if you have interesting examples with NeuroVCL components please send me on mail ilicv@EUnet.rs.
[1] “Frequently asked questions about AI”, http://www.cs.cmu.edu/Web/Groups/AI/html/faqs/ai/ai_general/top.html
[2] “Neural Network Frequently asked questions”, ftp://ftp.sas.com/pub/neural/FAQ.html
[3] Hotomski, P., (1995): “Sistemi Veštačke inteligencije”, Tehnički fakultet “Mihajlo Pupin”, Zrenjanin
[4] Ilić, V., (1999) “ANN - Program za obučavanje neuronskih mreža sa backpropagation algoritmom učenja”, http://SOLAIR.EUnet.rs/~ilicv/ANN.html
[5] Ilić, V., (1999) “Obučavanje neuronskih mreža za prepoznavanje ćiriličnih slova”, magistarski rad, Tehnički Fakultet
“Mihajlo Pupin”, Zrenjanin
[6] Ilić, V., (1999) “OCR - Program za prepoznavanje ćiriličnih slova”, http://SOLAIR.EUnet.rs/~ilicv/ocr.html
[7] Jocković, M., Ognjanović Z., Stankovski S. (1997) “Veštačka inteligencija inteligentne mašine i sistemi”, Grafomed, Beograd
[8] Milenković, S., (1997): “Veštačke neuronske mreže”, Zadužbina Andrejević, Beograd
[9] Reisdorph, K., (1999): “Naučite Delphi 4 za 21dan”, Kompjuter biblioteka, Čačak
[10] Sajić, I., (1995): “Neuronske mreže” , časopis “Računari” br 108, BIGZ, Beograd
[11] Savić, D., (1992), “Uvod u objektno programiranje”, PC Program, Beograd
[12] Subašić, P., (1998): “Fazi logika i neuronske mreže”, Tehnička Knjiga, Beograd
|
