P1:OTA/XYZP2:ABC JWST102-fmJWSTI02-WebbSeptember8.2011 Printername:YettoCome StatisticalPatternRecognition P1:OTA/XYZP2:ABC JWST102-fmJWSTI02-WebbSeptember8.2011 Printername:YettoCome P1:OTA/XYZP2:ABC JWST102-fmJWSTI02-WebbSeptember8.2011 Printername:YettoCome StatisticalPatternRecognition Thirdedition AndrewR.Webb.KeithDCopsey MathematicsandDataAnalysisConsultancy,Malvern,UK 时WILEY AJohnwileySons,Ltd,Publication P1:OTA/XYZP2:ABC JWST102-fmJWST102-WebbSeptember8.20118:52PrinterName:YettoCome Thiseditionfirstpublished2011 o2011JohnWileysons,Ltd Registeredoffi JohnWileysonsltd,TheAtrium,SouthernGate,Chichester,WestSussex,Po198SQ,UnitedKingdom Fordetailsofourglobaleditorialoffices,forcustomerservicesandforinformationabouthowtoapplyforpermissionto reusethecopyrightmaterialinthisbookpleaseseeourwebsiteatwww.wiley.com TherightoftheauthortobeidentifiedastheauthorofthisworkhasbeenassertedinaccordancewiththeCopyright DesignsandPatentsAct1988 Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted,inanyform DesignsandPatentsAct1988,withoutthepriorpermissionofthepublished'se,exceptaspermittedbytheUKCopyright, orbyanymeans,electronic,mechanical,photocopying,recordingorotherw Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbeavailablein electronicbook Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.Allbrandnamesandproduct namesusedinthisbookaretradenames,servicemarks,trademarksorregisteredtrademarksoftheirrespectiveowners Thepublisherisnotassociatedwithanyproductorvendormentionedinthisbook.Thispublicationisdesignedtoprovide accurateandauthoritativeinformationinregardtothesuhjectmattercovered.Itissoldontheunderstandingthatthe publisherisnotengagedinrenderingprofessionalservices.Ifprofessionaladviceorotherexpertassistanceisrequired.the servicesofacompetentprofessionalshouldbesought LibraryofCongressCataloging-in-PublicationData Webb,A.R(AndrewR) Statisticalpatternrecognition/AndrewR.webb,KeithDCopsey.-3rded Includesbibliographicalreferencesandindex ISBN9780-470-68227-2(hardback)-ISBN9780-470-68228-9(paper) LPatternperception-Statisticalmethods.I.Copsey,KeithD.Il.Title Q327.W432011 006.4dc23 20110249 AcataloguerecordforthisbookisavailablefromtheBritishlibrary HBISBN:978-0-470-68227-2 PBISBN:978-0-470-68228-9 PDFISBN:978-1-119-95296-1 oBookISBN:978-1-11995295-4 ePubisbn:978-1-11996140-6 Mobiisbn:978-1-11996141-3 Typesetin10/12ptTimesbyAptaraInc.NewDelhi,India P1:OTA/XYZP2:ABC JWST102-fmJWSTI02-WebbSeptember8.2011 Printername:YettoCome ToRosemary Samuel.miriam,Jacobandethan P1:OTA/XYZP2:ABC JWST102-fmJWSTI02-WebbSeptember8.2011 Printername:YettoCome P1:OTA/XYZP2:ABC JWST102-fmJWSTI02-WebbSeptember8.2011 Printername:YettoCome Contents Preface XIX Notation IntroductiontoStatisticalPatternRecognition 1.1StatisticalPatternrecognition 1.1.1Introductio 1.1.2TheBasicmodel 1.2StagesinaPatternRecognitionProblem 4 1.3Issues 6 4ApproachestoStatisticalPatternRecognition 7 1.5ElementaryDecisionTheory 1.5.1Bayes'DecisionRuleforMinimumError 1.5.2Bayes'DecisionRuleforMinimumError-RejectOption 12 1.5.3Bayes'DecisionRuleforMinimumRisk 13 1.5.4Bayes'DecisionRuleforMinimumRisk-RejectOption 15 1.5.5Neyman-PearsonDecisionrule 15 1.5.6MinimaxCriterion 18 1.5.7Discussion 1.6Discriminantfunctions 20 1.6.1Introducti 20 1.6.2Lineardiscriminantfunctions 21 1.6.3PiecewiseLineardiscriminantFunctions 23 1.6.4GeneralisedLineardiscriminantFunction 24 1.6.5Summary 26 1.7Multipleregression 27 1.8Outlineofbook 29 1.9NotesandReferences 29 Exercises 2DensityEstimation-Parametric 2.1Introduction P1:OTA/XYZP2:ABC JWST102-fmJWST102-WebbSeptember8.20118:52PrinterName:YettoCome CONTENTS 2.2EstimatingtheParametersofthedistributions 2.2.1EstimativeApproach 34 2.2.2PredictiveApproach 2.3TheGaussianClassifier 2.3.1Specification 2.3.2DerivationoftheGaussianClassifierPlug-InEstimates 37 2.3.3ExampleApplicationStudy 39 2.4DealingwithSingularitiesinthegaussianClassifier 2.4.1Introduction 40 2.4.2Naiveb: 40 2.4.3ProjectionontoaSubspace 41 2.4.4Lineardiscriminantfunction 41 2.4.5RegularisedDiscriminantAnalysis 42 2.4.6ExampleApplicationStudy 2.4.7FurtherDevelopments 45 2.4.8Summary 46 2.5FinitemixtureModels 46 2.5.1Introduction 46 2.5.2MixtureModelsfordiscrimination 48 2.5.3ParameterEstimationforNormalmixturemodels 49 2.5.4NormalMixtureModelCovariancematrixConstraints 2.5.5HowManyComponents 52 2.5.6Maximumlikelihoodestimationviaem 2.5.7ExampleApplicationStudy 60 2.5.8FurtherDevelopments 62 2.5.9Summary 63 2.6ApplicationStudies 63 2.7SummaryandDiscussion 66 2.8Recommendations 66 2.9Notesandreferences 67 Exercises 67 3Densityestimation-bayesian 3.1Introduction 3.1.1Basics 3.1.2Recursivecalculation 3.1.3Proportionality 73 3.2AnalyticSolutions 3.2.1ConjugatePriors 3.2.2EstimatingtheMeanofaNormalDistributionwith Knownvariance 75 3.2.3EstimatingtheMeanandtheCovarianceMatrixofamultivariate Normaldistribution 3.2.4UnknownPriorClassprobabilities 3.2.5S ummary 87 3.3BayesianSamplingSchemes 87 3.3.1Introduction 87