;:THOMSON LEARNINGSponsoring EditqtCprolyn CrockettCover Design: Jennifer MackresMarketing Representytive: Tom Ziolkowski Interior Illustration: Lori HeckelmanEditorial Assistant: Jennifer JenkinsPrint Buyer: Vena DyerPreduction Editor: Tom NovackTypesetting: Integre Technical Publishing CoAssistant Editor: Ann DayCover Printing: R. R. Donnelley g Song CoManuscript Editor: Carol ReitzCrawfordsvillePermissions Editor: Sue EwingPrinting and Binding: R. R. Donnelley8 Sons Co., CrawfordsvilleAll products used herein are used for identification purposes only and may be trademarksor registered trademarks of their respective ownersCoPYRIGHT 2002 the Wadsworth Group. Duxbury is an imprint of the WadsworthGroup, a division of Thomson Learning IncThomson Learning is a trademark used herein under licenseFor more information about this or any other Duxbury products, contactDUXBURY511 Forest Lodge RoadPacific Grove, CA 93950 USAwww.duxbury.com800-423-0563(Thomson Learning Academic Resource Center)All rights reserved. No part of this work may be reproduced, transcribed or used in anyform or by any means-graphic, electronic, or mechanical, including photocopying,recording, taping, Web distribution, or information storage and or retrievalsystems--without the prior written permission of the pubisherFor permission to use material from tiontact . uswww.thomsonrights.comfax:1-8007302215phone1-800-7302214八知APrinted in United States of Amer了10987654321Library of Congress Cataloging-in-Publication DatsCasella, GeorgeStatistical inference/George Casella, Roger L. Berger.2nd edcliIncludes bibliographical references and indexesISBN0534243126I. Mathematical statistics. 2. Probabilities. I. Berger, Roger L.II. TitleQA276c3720015195dc212001025794To anne and vickiDuxbury titles of related interestaniel, applied Nonparametric Statistics 2ndDerr, Statistical Consulting: A Guide to Efective CommunicationDurrett, Probability: Theory and Examples 2ndGray bill, Theory and Application of the Linear modelJohnson, Applied Multivariate Methods for Data AnalystsKuehl, Design of Experiments: Statistical Principles of Research Design and Analysis 2ndLarsen, Marx, &e cooil, Statistics for Applied Problem Solving and Decision MakingLohr, Sampling: Design and analysisLunneborg, Data Analysis by Resampling: Concepts and ApplicationsMinh, Applied probability modeMinitab Inc, MINITAB Student version 12 for windowsMyers, Classical and Modern Regression with Applications 2Ramsey Schafer, The Statistical Sleuth 2 tr of Statistical IdeasNewton Harvill, StatConcepts: A Visual ToSAS Institute Inc, JMP-IN: Statistical Discovery SoftwareSavage, INSIGHT: Business Analysis Software for Microsoft@ excelScheafer, Mendenhall, Ott, Elementary Survey Sampling 5thShapiro, Modeling the Supply ChainWinston, Simulation Modeling Using, ORISKTo order copies contact your local bookstore or call 1-800-354-9706. For moreinformation contact Duxbury Press-at 511 Forest Lodger Road, Pacific Grove, CA 93950orgotowww.duxbury.comQPreface to the second EditionAlthough Sir Arthur Conan Doyle is responsible for most of the quotes in this book,perhaps the best description of the life of this book can be attributed to the gratefulDead sentiment, "What a long, strange trip it's beenPlans for the second edition started about six years ago, and for a long time westruggled with questions about what to add and what to delete Thankfully, as timepassed the answers became clearer as the fow of the discipline of statistics becameclearer. We see the trend moving away from elegant proofs of special cases to algo-rithmic solutions of more complex and practical cases. This does not undermine theimportance of mathematics and rigor; indeed, we have found that these have becomemore important. But the manner in which they are applied is changingFor those familiar with the first edition, we can summarize the changes succinctlyas follows. Discussion of asymptotic methods has been greatly expanded into its ownchapter. There is more emphasis on computing and simulation(see Section 5.5 andthe computer algebra Appendix ) coverage of the more applicable techniques hasbeen expanded or added (for example, bootstrapping the EM algorithm, p-valueslogistic and robust regression ); and there are many new Miscellanea and ExercisesWe have de-emphasized the more specialized theoretical topics, such as equivarianceand decision theory, and have restructured some material in Chapters 3-11 for clarityThere are two things that we want to note. First, with respect to computer algebraprograms, although we believe that they are becoming increasingly valuable toolswe did not want to force them on the instructor who does not share that beliefThus, the treatment is "unobtrusive"in that it appears only in an appendix, withsome hints throughout the book where it may be useful. Second, we have changedthe numbering system to one that facilitates finding things. Now theorems, lemmas,examples, and definitions are numbered together; for example, Definition 7. 2. 4 isfollowed by Example 7.2.5 and Theorem 10.1.3 precedes Example 10.1.4The first four chapters have received only minor changes. We reordered some material (in particular, the inequalities and identities have been split), added some newexamples and exercises, and did some general updating Chapter 5 has also been re-ordered, with the convergence section being moved further back, and a new section ongenerating random variables added TIprevious coverage of invariance, which wasin Chapters 7-9 of the first edition, has been greatly reduced and incorporated intoChapter 6, which otherwise has received only minor editing(mostly the addition ofnew exercises). Chapter 7 has been expanded and updated and includes a new sectionon the EM algorithm. Chapter 8 has also received minor editing and updating, andVi rPREFACE TO THE SECOND EDITIONhas a new seetion on p-values. In Chapter 9 we now put more emphasis on pivoting(havingreelized that "guaranteeing an interval " was merely"pivoting the cdf").Alsothe materiar that was in Chapter 10 of the first edition( decision theory) has been re-duced, and small sections on loss function optimality of point estimation, hypothesistesting, and interval estimation have been added to the appropriate chaptersChapter 10 is entirely new and attempts to lay out the fundamentals of large sampleinference, including the delta method, consistency and asymptotic normality, bootstrapping, robust estimators, score tests, etc. Chapter 11 is classic oneway ANOVAand linear regression (which was covered in two different chapters in the first edi-tion). Unfortunately, coverage of randomized block designs has been eliminated forspace reasons. Chapter 12 covers regression with errors-in-variables and contains newmaterial on robust and logistic regressionAfter teaching from the first edition for a number of years, we know(approximately)what can be covered in a one-year course. From the second edition, it should bepossible to cover the following in one yearChapter 1: Sections 1-7Chapter 6: Sections 1-3Chapter 2: Sections 1-3Chapter 7: Sections 1-3Chapter 3: Sections 1-6Chapter 8: Sections 1-3Chapter 4: Sections 1-7Chapter 9: Sections 1-3Chapter 5: Sections 1-6Chapter 10: Sections 1, 3, 4Classes that begin the course with some probability background can cover more ma-terial from the later chaptersFinally, it is almost impossible to thank all of the people who have contributed insome way to making the second edition a reality(and help us correct the mistakes inthe first edition). To all of our students, friends, and colleagues who took the time tosend us a note or an e-mail, we thank you. a number of people made key suggestionsthat led to substantial changes in presentation. Sometimes these suggestions were justshort notes or comments, and some were longer reviews. Some were so long ago thattheir authors may have forgotten, but we haven't So thanks to Arthur Cohen, SirDavid Cox, Steve Samuels, Rob Strawderman and Tom Wehrly. We also owe much toJay beder, who has sent us numerous comments and suggestions over the years andpossibly knows the first edition better than we do, and to Michael perlman and hisclass, who are sending comments and corrections even as we write thisThis book has seen a number of editors. We thank alex Kugashev, who in themid-1990s first suggested doing a second edition, and our editor, Carolyn Crockettwho constantly encouraged us. Perhaps the one person (other than us)who is mostresponsible for this book is our first editor, John Kimmel, who encouraged, publishedand marketed the first edition. Thanks.JohnGrge casellaRoger L. BergerPreface to the First EditionWhen someone discovers that you are writing a textbook, one(or both)of two questions will be asked The first is Why are you writing a book? "and the second is"How is your book different from what's out there? The first question is fairly easyto answer. You are writing a book because you are not entirely satisfied with theavailable texts. The second question is harder to answer. The answer cant be putin a few sentences so, in order not to bore your audience who may be asking thequestion only out of politeness), you try to say something quick and witty. It usuallydoesn’ t workThe purpose of this book is to build theoretical statistics(as different from mathematical statistics from the first principles of probability theory. Logical developmentproofs, ideas, themes, etc, evolve through statistical arguments. Thus, starting fromthe basics of probability, we develop the theory of statistical inference using tech-niques, definitions, and concepts that are statistical and are natural extensions andconsequences of previous concepts. When this endeavor was started, we were not surehow well it would work. The final judgment of our success is, of course, left to thereaderThe book is intended for first-year graduate students majoring in statistics or ina field where a statistics concentration is desirable. The prerequisite is one year ofcalculus.(Some familiarity with matrix manipulations would be useful, but is notessential. )The book can be used for a two-semester, or three-quarter, introductorycourse in statisticsThe first four chapters cover basics of probability theory and introduce many fundamentals that are later necessary. Chapters 5 and 6 are the first statistical chapters.Chapter 5 is transitional (between probability and statistics)and can be the startingpoint for a course in statistical theory for students with some probability backgroundChapter 6 is somewhat unique, detailing three statistical principles(sufficiency, like-lihood, and invariance)and showing how these principles are important in modelingdata. Not all instructors will cover this chapter in detail, although we strongly recom-mend spending some time here. In particular, the likelihood and invariance principlesare treated in detail. Along with the sufficiency principle, these principles, and thethinking behind them, are fundamental to total statistical understandingChapters 7-9 represent the central core of statistical inference, estimation (pointand interval) and hypothesis testing A major feature of these chapters is the divisionInto methods of finding appropriate statistical techniques and methods of evaluatingthese techniques. Finding and evaluating are of interest to both the theorist and thePREFACE TO THE FIRST EDITIONpractitioner but we feel that it is important to separate these endeavors. Differentconcerns are important and different rules are invoked of further interest may bethe sections of these chapters titled Other Considerations. Here, we indicate how therules of statistical inference may be relaxed (as is done every day) and still producemeaningful inferences. Many of the techniques covered in these sections are ones thatare used in consulting and are helpful in analyzing and inferring from actual problemsThe final three chapters can be thought of as special topics, although we feel thatsome familiarity with the material is important in anyone's statistical educationChapter 10 is a thorough introduction to decision theory and contains the most mod-ern material we could include. Chapter 11 deals with the analysis of variance(onewayand randomized block), building the theory of the complete analysis from the moresimple theory of treatment contrasts. Our experience has been that experimenters aremost interested in inferences from contrasts, and using principles developed earliermost tests and intervals can be derived from contrasts. Finally, Chapter 12 treatsthe theory of regression, dealing first with simple linear regression and then coveringegression with "errors in variables, This latter topic is quite important, not only toshow its own usefulness and inherent difficulties. but also to illustrate the limitationsof inferences from ordinary regressionAs more concrete guidelines for basing a one-year course on this book, we offer thefollowing suggestions. There can be two distinct types of courses taught from thisbook. One kind we might label "more mathematical, " being a course appropriate forstudents majoring in statistics and having a solid mathematics background (at least1 years of calculus, some matrix algebra, and perhaps a real analysis course. Forsuch students we recommend covering Chapters 1-9 in their entirety (which shouldtake approximately 22 weeks) and spend the remaining time customizing the coursewith selected topics from Chapters 10-12. Once the first nine chapters are coveredthe material in each of the last three chapters is self-contained, and can be coveredin any order.Another type of course is"more practical. Such a course may also be a first coursefor mathematically sophisticated students, but is aimed at students with one year ofcalculus who may not be majoring in statistics. It stresses the more practical uses ofstatistical theory, being more concerned with understanding basic statistical conceptsand deriving reasonable statistical procedures for a variety of situations, and lessconcerned with formal optimality investigations. Such a course will necessarily omita certain amount of material, but the following list of sections can be covered in aone-year course.Chapter SectionsAll22.1,2.2,233.1,3.24.1,4.2、4.3、4.551,52,5.31,5466.1.1,6.217,1,72.1,72.2,7.2.3,73.1,73.3,7481,8.2.1,8.2.3,8.2.4,8.3.1,8.3.2,8.4PREFACE TO THE FIRST EDITION91,9.2.1,922,9.24,9.3.1,911.1,11212.1,122If time permits, there can be some discussion(with little emphasis on details) of thematerial in Sections 4.4, 5.5, and 6.1.2, 6.1.3, 6.1.4. The material in Sections 11.3 and12. 3 may also be consideredThe exercises have been gathered from many sources and are quite plentiful. Wfeel that, perhaps the only way to master this material is through practice, and thuswe have included much opportunity to do so. The exercises are as varied as we couldmake them, and many of them illustrate points that are either new or complementaryto the material in the text. Some exercises are even taken from research papers. (Itmakes you feel old when you can include exercises based on papers that were newresearch during your own student days! Although the exercises are not subdividedlike the chapters, their ordering roughly follows that of the chapter.(Subdivisionsoften give too many hints. ) Furthermore, the exercises become(again, roughly) morechallenging as their numbers become higher.As this is an introductory book with a relatively broad scope, the topics are notcovered in great depth. However, we felt some obligation to guide the reader onestep further in the topics that may be of interest. Thus, we have included manyreferences, pointing to the path to deeper understanding of any particular topic.(TheEncyclopedia of Statistical Sciences, edited by Kotz, Johnson, and Read, provides afine introduction to many topics.To write this book, we have drawn on both our past teachings and current work. Wehave also drawn on many people, to whom we are extremely grateful. We thank ourcolleagues at Cornell, North Carolina State, and Purdue in particular, Jim BergerLarry Brown, Sir David Cox, Ziding Feng, Janet Johnson, Leon Gleser, Costas GoutisDave Lansky, George McCabe, Chuck McCulloch, Myra Samuels, Steve Schwagerand Shayle Searle, who have given their time and expertise in reading parts of thismanuscript, offered assistance, and taken part in many conversations leading to constructive suggestions. We also thank Shanti Gupta for his hospitality, and the library at Purdue, which was essential. We are grateful for the detailed reading andhelpful suggestions of Shayle Searle and of our reviewers, both anonymous and nonanonymous(Jim Albert, Dan Coster, and Tom Wehrly). We also thank David Mooreand George McCabe for allowing us to use their tables, and Steve hirdt for supplyingus with data. Since this book was written by two people who, for most of the time,were at least 600 miles apart, we lastly thank Bitnet for making this entire thingpossibleGeorge CasellaRoger L. Berge