A NEW TECHNIQUE FOR SUPPORT VECTOR MACHINE PARAMETERS OPTIMIZATION BASED ON MODIEFIED PSO ALGORITHM
Sabah ………….., ………, ……..,
Support vector machine can determine the global finest solutions in many complicated problems and it is widely used for human face classification in the last years. Nevertheless, one of the main limitations of SVM is optimizing the training parameters especially when SVM used in face recognition domains. Various methodologies are used to deal with this issue, such as PSO, OPSO, AAPSO and AOPSO. Nevertheless, there is a room of advancements in this kind of optimization process. Lately, an improved version of PSO is developed which is called Modified PSO. In this paper, a new technique based on Modified PSO called (Modified PSO-SVM) is proposed to optimize the parameters of SVM. The proposed scheme utilizes Modified PSO to select the optimal parameters of SVM. Two human face datasets: SCface, CASIAV5 and CMU Multi-PIE face datasets are used in the experiments. Then, the proposed technique is compared with the PSO-SVM, OPSO-SVM and AOPSO-SVM and it showed promising results in terms of accuracy.
Keyword: biometric, optimization, face recognition, PSO
“Support vector machine (SVM)” is a machine learning approach and it is one of the famous classification methods. It deals with a high dimensional problems and works efficiently using minor number of training examples. It works on the mechanism of risk minimization, that leads to the “global optimum” solutions (Hasan et al 2013). The SVM approach have been adopted in many areas such as “text categorization” Joachim (2005), medical diagnostics Raikwal & Saxena (2012), “credit risk analysis” Yu et al (2008), “information extraction” Li et al (2005), face recognition Abdullah et al (2017), etc. Nonetheless, the training parameters selection in SVM is a big challenge due to its effect on the stability performance of SVM. Lately, the well-known artificial intelligence algorithm the “Particle swarm optimization (PSO)” that is introduced by Kennedy & Eberhart in 1995 has been applied to optimize the SVM parameters. The inspiration of PSO come from the social actions between folks same as the groups of fish or birds blocking. Recently, many studies have used PSO to optimize the training parameters of SVM, one of the earlier studies introduced by (Wei et al 2011) and used effectively in face recognition domain. However, this issue still an open issue and there is an area of advancements to optimize the parameters of SVM. Newly, a new approach called Modified PSO is proposed by Khadhraoui et al (2016) which is an enhancement of the standard PSO algorithm and it is applied powerfully with 2D face recognition. Therefore, this study proposes a new face recognition scheme based on the Modified PSO algorithm and SVM. The remainder parts of this paper planned as follows: section 2 focuses on literature review; the proposed method described in section 3. Section 4 clarifies the experimental analysis of the proposed technique while Section 5 presents the conclusions
For face recognition application, many attempts have been adopted various forms of PSO algorithm to select the optimal parameters of SVM. One of the recent studies that applied the standard PSO with SVM was introduced by (Wei et al 2011). They used PCA algorithm for feature extraction process and PSO utilized to find the optimal parameters of SVM. The FERET database is used in the experimental to evaluate their proposed PSO-SVM technique and the results shows that their PSO-SVM has higher recognition accuracy than the standard SVM and BPNN as well. Nevertheless, the standard PSO has a drawback which random population generation that might influence in the recognition result. Moreover, they adopted one database which is not enough to make sure that the PSO-SVM is reliable in the complex face recognition applications. Hasan et al (2013) have been proposed OPSO-SVM face recognition technique which used OPSO in order to solve the random generation of population problem.They utilized FERET and YALE face databases in order to examine the performance of OPSO-SVM and they compared it with conventional SVM and PSO-SVM methods. Their OPSO-SVM has achieved higher accuracy than the standard SVM and PSO-SVM. However, there is a limitation in OPSO which is the fixed value of velocity coefficients same as the standard PSO. In order to evade the velocity coefficients problem, Abdulameer et al (2014) were introduced AAPSO-SVM which is a combination of a new method (adaptive acceleration particle swarm optimization) and support vector machine. They introduced new mechanism to choose the velocity coefficients based on the fitness values of the particles. They used YALE and CASIA face databases to evaluate the effectiveness of the AAPSO-SVM and they achieved promising results better than OPSO-SVM and PSO-SVM. Recently, Abdulameer and his collages in 2018 have proposed AOPSO-SVM, which is a combination of OPSO-SVM and AAPSO-SVM in order to achieve high recognition results and they succeed to achieve higher recognition accuracy than the over mentioned methods. So that and in order to achieve higher recognition results, a new face recognition technique based on the recent developed method (Modified PSO) and (SVM) is introduced in this study .
MATERIALS AND METHODS
SUPPORT VECTOR MACHINE CLASSIFIER
A Support Vector Machine (SVM) is defined by a separating hyperplane and it is considered as discriminative classifier. It is a supervised learning which means if given labeled training data, the algorithm produce an optimum hyperplane which can classifies the new examples. The hyperplane is a line separating a plane into two parts in two dimensional space Abdulah et al (2017) ; Abdulameer et al (2018).
The classification procedure of SVM in brief is clarified in figure 1 below:
Fig. 1: the SVM classification procedure
The optimization problem for nonlinear decision surface is assumed as:
??([email protected],?) ???1/2? ?G?^2+P?_(i=1)^n??_i
y_i (G.x_i+bi)?1-?_i,?_i?0,i=1,2,…….,n (1)
Where P is represents the penalty parameter, which is a regularization constant that controls the “tradeoff” between the error minimization and margin maximization. So, the decision function of the classification becomes:
f(x)=sin??(?_(i=1)^n?h_i y_i ke(x_i,x_j )+bi )? (2)
In the previous formula, h_i is the “Lagrange multipliers” and the kernel function is represented by ke(x_i,x_j )=??x?_i,??x?_j. The goal of the kernel function is to map the data in to “higher dimensional space” through using the nonlinear mapping functions ?x. The RBF which is “Radial basis function ” is widely used in the literature and we used it in SVM creation. It is defined by exp?(-?x_i-x_j ?/2?^2), ? is means a positive real number.
Particle Swarm Optimization
The PSO algorithm that was presented by Eberhart and Kennedy in 1995 which is able to solve the problems that has continuous variables. In PSO algorithm, every particle is discover the “solution space” seeking for an optimal solution. Kennedy (2011); Hasanain (2018). The velocities and positions of the particles are updated according to the following formulas:
??Ve?_i?^(t+1)=U*?V_i?^t+q_1*?ran?_1*(?pe?_ibest-?A_i?^t )+q_2*?ran?_2*(?gl?_best-?A_i?^t) (3)
L is the size of the swarm and i= 1,2,…L . ?Ve?_i denotes to the velocity of the current particle and ??Ve?_i?^(t+1) is new velocity of the particle; U is represents the inertia weight. q1 and q2 are two positives constants named as cognitive components ; two independents random number in the range 0, 1 are represented by ran1 and ran 2; ?A_i?^(t+1) refers to the position of the particle in the swarm; p_(i_best) represents the finest gotten solution of the ith particle; ?gl?_best represents the finest solution of particle in the swarm.
The Modified PSO
In order to improve the global search quality of the standard PSO in the modified PSO algorithm, the cognitive components (q_1, q_2), and the inertia weight U have been formed Khadhraoui et al (2016). The updating on the velocity is represented as in the subsequent formula:
??Ve?_i?^(t+1)=U_new*?V_i?^t+q_1*?ran?_1*(?pe?_ibest-?A_i?^t )+q_2*?ran?_2*(?gl?_best-?A_i?^t) (5)
where the U is represented as in the equation below:
U=U_max-(U_max-U_min )*iterc/?iter?_m (7)
where U_min is the minimum inertia weight and U_max is the maximum inertia weight.
q_1=q_1max-(q_1max-q_1min )*(iterc/?iter?_m ) (8)
q_2=q_2max-(q_2max-q_2min )*(iterc/?iter?_m ) (9)
Where q_1min is the minimum, value cognitive components while q_1max is the maximum values of the cognitive component; q_2min is the minimum value of social component while q_2max is the maximum values of the social component; iterc represents the current of iterations while ?iter?_m represents the maximum values of iterations. For every particle in every generation process, and in order to evaluate the velocity and the position for selecting the best search value, the Fitness function is used. The Fitness function able to provide the finest location of swarm space and the optimum “threshold” value. Hence, for every particle, the fitness function is evaluated. The particles develop by refreshing their speeds and positions dependent on their capacity to exploit the class detachment term demonstrated by the “scatter index” among the diverse classes. The means Mi of the equivalent classes and the mean M0 in the “feature space” are computed the following equations (10) and (11) :
D_i=1/F_i ?_(j=1)^(F_i)??z_j?^((i)) (10)
D_o=1/F?_(i=1)^L?F_i D_i (11)
Where i = 1,2,…,L and j= 1, 2, …,F ; ?z_j?^((i))are the example images that belongs to class z_i ; F_i denotes to the number of images in every class; D_i are the mean of equivalent classes and D_0 indicate to the mean in the feature space. The fitness function FIT is calculated by very simple formula (12) :
FIT=?(?_(i=1)^L??(D_i-D_0 )^t (D_i-D_0)?) (12)
PARAMETER OPTIMIZATION OF SVM BY THE MODIFIED PSO
The SVM built by the “RBF kernel function” and it is based on P and ?, which are two user-determined parameters. At that time, the particle is involve P and ?. The process of the optimization process for SVM through the modified PSO is illustrated in figure 2 as follow:
Fig.2: The block diagram of the optimization using the Modified PSO
Experimental exploration for the proposed face recognition technique
The whole procedure of the proposed face recognition technique based on the Modified PSO-SVM is displayed in Figure 3. After reading the face image, the LDA is adopted to extracts face features from the face image Mosa (2018). The resulted vector is used for training and testing the Modified PSO-SVM model.
Figure 3: The block diagram of the proposed face recognition technique
We utilized three human face databases CMU Multi-PIE Gross et al (2008) , CASIAV5 from Chinese Academy of Sciences (2007) and SCface face Grgic et al (2011) dataset in order to test the performance of the proposed technique. The MATLAB environment is used for the experimentation. From every dataset, we chose 200 images for 50 subjects and the images were in different pose and illuminations. The images are divided in to 30% for training and 70% for testing. The figure 4 shows some of the samples images from each dataset.
samples from CMU Multi-PIE dataset
Figure 4: Samples from the three datasets : SCface, CASIAV5 and CMU Multi-PIE dataset
The accuracy measurement is utilized to examine the performance of the proposed technique. In the experiments, images from different pose and illumination are adopted and ten rounds are adopted in the implementation. A comparison process for the proposed Modified PSO-SVM, SVM, OPSO-SVM, and AAPSO-SVM is employed in order to illustrate the efficiency of the proposed method. The experimental results shows that modified PSO-SVM has attained the highest accuracy among the other methods as shown in Table 1 and Figure 5 respectively.
Table 1. The accuracy of the SVM, PSO-SVM, OPSO-SVM, AAPSO-SVM and the proposed Modified PSO-SVM
In addition, the optimal parameters of one round on the three datasets for the proposed modified PSO-SVM and the above-mentioned methods are demonstrated in Table 1.
Table 2. The sample of the optimal parameters for the SVM, PSO-SVM, OPSO-SVM, AAPSO-SVM and Modified PSO-SVM on SCface datasets (first round only)
A new technique to optimize the parameters of support vector machine has been presented in this paper. The proposed method used the Modified PSO method as a method to optimize the parameters of SVM through Modified PSO-SVM . Three human face datasets have been used to examine the overall performance of the proposed technique. The proposed Modified PSO-SVM was compared in terms of accuracy with SVM and the recent methods like PSO-SVM,OPSO-SVM and AAPSO-SVM. The proposed technique has gained 98% accuracy among the other methods such as SVM, PSO-SVM,OPSO-SVM and AAPSO-SVM. we will examine the proposed technique with the occlusion circumstance as a future work.