Genetic_Programming_Theory_and_Practice_XIII

Kaizen Programming for Feature Construction for Classification 45

Algorithm 1Pseudo-code of Kaizen Programming for feature construction
1.Readthe dataset and setnas the number of instances
2.SetCurrentStandardQuality 0 ,MaxStagnated,Stagnated 0 , Size of the Team (st),
number of New Ideas per Expert (NIE), Expansion Factor (EF),w stNIE
3.Definethe target and set it as not achieved
4.Generatestinitial random ideas asCurrentStandard
5.ApplytheCurrentStandard(calculate the results from the expressions) and create the feature
setSTDn;st
6.BestStandard CurrentStandard
7.BestStandardQuality CurrentStandardQualityonk-fold cross-validation
8.Do
a.Generate, via GP operators, theTrialIdeas, which areNIEvariations (ideas) of the
CurrentStandardthrough multiple crossover and mutation. Even the worst idea from
CurrentStandardmight have offspring
b.Applyeach new idea, resulting in theTRIALn;wfeature set
c.Createthe expanded feature setFn;stCwcontainingTRIALn;wandSTDn;st
d.Createnewkstratified folds fromFto reduce bias in the search
e.Foreach fold
i.Inducea decision tree via CART
ii.Calculatethe arrayFoldImportancesas the importance of each feature fromFusingGini
Importance
f.End For
g.SetTrialImportancesas the average of allFoldImportances
h.MostImportantTrialIdeasis the subset of thestmost importantTrialIdeas(considering
TrialImportances)
i.CreateMITIn;stas a subset ofF, and calculateMostImportantTrialIdeasQualityusing the
currentk-folds
j.IfMostImportantTrialIdeasQualityis better thanCurrentStandardQualitythen
i.CurrentStandard MostImportantTrialIdeas
ii.CurrentStandardQuality MostImportantTrialIdeasQuality
iii.STDn;st MITIn;st
iv.IfCurrentStandardQualityis better thanBestStandardQualitythen
A.BestStandard CurrentStandard
B.BestStandardQuality CurrentStandardQuality
v.End If
k.Else
i.Stagnated StagnatedC 1
l.End If
m.IfStagnated>MaxStagnationthen
i.Stagnated 0
ii.st stCdstEFeto increase the team of experts’ size
iii.Generatestinitial random ideas asCurrentStandard
iv.ApplytheCurrentStandard(calculate the results from the expressions) and createSTDn;st
v.CalculateCurrentStandardQualityonk-fold cross-validation
n.End If
9.Whiletarget is not achieved
10.ReturnBestStandard,BestStandardQuality