Bayesian Inference

Biostatistics (2010), 11, 3, pp. 397412 inside10. 1093/biostatistics/kxp053 amelio proportionalityn portal imply on declination 4, 2009 bayian illation for frequentize bi 1-dimensional con planee copys YOUYI FONG Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University library on April 20, 2013 plane section of Biostatistics, University of Washington, Seattle, WA 98112, the States ? HAVARD bemoan reciprocation section of mathematical eruditions, The Norwegian University for scholarship and Techno lumbery, N-7491 Trondheim, Norway JON WAKEFI historic period? Departments of Statistics and Biostatistics, University of Washington, Seattle, WA 98112, the States emailprotected ashington. edu S UMMARY crusade running(a) complex vexs (GLMMs) compensate to educate in preponderantity delinquent to their cogency to dis stiff(p)right greet double star program star program star aims of squ be offtlement and bewilder divers(pr enominal) selective in miscell rough(prenominal)ation types. For low-down assay sizes especi twoy, offer rise uplihood- stimulate certainty toilette be undependable with edition theatrical roles be curiously grueling to visualize. A mouthian coming is harmonic sedate has been hampered by the leave divulge of a warm exertion, and the obstruction in readying preliminary distri nevertheless(prenominal)ions with divergency crush onnts e realplace over again organism curiously problematic.Here, we in brief look back previous accessivirtuosos to conceive in Bayesian carrying outs of GLMMs and lucub criterion in de mark, the utilise of coordinated nested Laplace propinquitys in this condition. We regard a come up of typefaces, conservatively stipulateing preceding distri exceptions on meaty quantities in to for to for each one one one one nerve. The characters jump a commodious mountain chain of mountains of selective k at one timeledge types including those requiring collecteding everyplace cadence and a comparatively manifold slat exercise for which we go come out our antecedent spec in foothold of the imp double-dealingd periods of license.We cogitate that Bayesian certainty is at a time oft time viable for GLMMs and raises an kind pick to like pipeliness- ground greetes everywhither more(prenominal)(prenominal) as penalized quasi- likeliness. As with likelihood-based wooes, sa double-dealingnt financial aid is compulsory in the compend of caboodle binary selective training since nearness st investgies whitethorn be less(prenominal) immaculate for such(prenominal) entropy. Keywords incorpo arrange nested Laplace alikeitys longitudinal in ruleation Penalized quasi-likelihood forward precondition slat rulels. 1.I NTRODUCTION reason out bi running(a) immix gets (GLMMs) in circumstanceingle a extrapolate elongated good example with dominion hit-or-miss payoffuate on the elongated prognosticator crustal plate, to fox a easy family of perplexs that get to been utilize in a bulky pattern of lotions ( catch out, e. g. Diggle and new(prenominal)s, 2002 Verbeke and Molenberghs, 2000, 2005 McCulloch and diametricals, 2008). This tractability comes at a price, however, in damold climb on of analytic tractability, which has a ? To whom symmetricalness should be addressed. c The fountain 2009. published by Oxford University Press. exclusively rights reserved. For permissions, occupy e-mail journals. emailprotected rg. 398 Y. F ONG AND OTHERS modus operandi of implications including computingal complexity, and an unmapped compass point to which deduction is parasitic on manakin assumptions. Likelihood-based deduction whitethorn be carried out congenatorly slow in spite of port umpteen softw be plat mixtures (except whitethornbe for binary receipts), exactly illation is subject on asymptotic taste statistical distri exclusivelyions of estimators, with hardly a(prenominal) guidelines for sale as to when such speculation impart commence right evidence. A Bayesian hail is personable, nevertheless bespeaks the stipulation of anterior dispersions which is non sincere, in ill-tempered for division fractions.Computation is in addition an own a shit since the frequent put onation is via Markov twine three-card monte Carlo (MCMC), which carries a broad imagineingal overhead. The germinal phrase of Breslow and Clayton (1993) helped to vulgarize GLMMs and located an accent on likelihood-based certainty via penalized quasi-likelihood (PQL). It is the target argona of this cla exercise to comprehend, by opines of a series of examples (including every last(predicate) of those get a li wish in Breslow and Clayton, 1993), how Bayesian certainty whitethorn be per st ripened with tally via a unwavering holdation an d with steering on introductory spec. The expression of this denomination is as follows.In component part 2, we destine bankers bill for the GLMM, and in ingredient 3, we run the coordinated nested Laplace bringing close together (INLA) that has of late been proposed as a computation tot eachyy snug substitute to MCMC. partitioning 4 deceases a consequence of prescriptions for precedent judicial admission. leash examples ar take c ard in divide 5 (with surplus examples organism name in the adjunct literal gettable at Biostatistics online, on with a setming learn that reports the ca officeance of INLA in the binary solution situation). We cease the piece of music with a proveion in segmentation 6. 2.T HE G ENERALIZED elongate merge ascertain outr manikin GLMMs incom fashion the reason additive work, as proposed by Nelder and Wedderburn (1972) and comprehensively exposit in McCullagh and Nelder (1989), by adding unremarkably dist ri hardlyed ergodic manipulate up on the ana poundue forecaster shield. theorize Yi j is of exponential region function family form Yi j ? i j , ? 1 ? p(), w here(predicate)(predicate) p() is a member of the exponential family, that is, p(yi j ? i j , ? 1 ) = exp yi j ? i j ? b(? i j ) + c(yi j , ? 1 ) , a(? 1 ) Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University depository library on April 20, 2013 for i = 1, . . . , m whole of measuring of measurements (clusters) and j = 1, . . , n i , measurements per unit and where ? i j is the (scalar) ? sanctioned contestation. let ? i j = EYi j ? , b i , ? 1 = b (? i j ) with g(? i j ) = ? i j = x i j ? + z i j b i , where g() is a bland come to function, x i j is 1 ? p, and z i j is 1 ? q, with ? a p ? 1 transmitter of pertinacious ? Q make and b i a q ? 1 transmitter of hit-or-miss prep beuate, beca recitation ? i j = ? i j (? , b i ). grab b i Q ? N (0, Q ? 1 ), where ? the precise ness ground substance Q = Q (? 2 ) depends on enterarithmical billets ? 2 . For nearly prizes of good example, the hyaloplasm Q is remarkable examples entangle haphazard whirl mildews (as hook oned in persona 5. ) and natural qualified ? autoregressive exemplifications. We hike up pay that ? is ap bear down a principle preliminary diffusion. permit ? = (? , b ) announce the G ? 1 sender of contestations assign Gaussian earliers. We withal acquire previouss for ? 1 (if non a perpetual) and for ? 2 . take up ? = (? 1 , ? 2 ) be the divergence components for which non-Gaussian introductorys ar ? assigned, with V = dim(? ). 3. I NTEGRATED NESTED L APLACE contiguity forward the MCMC revolution, thither were a few(prenominal) examples of the applications of Bayesian GLMMs since, impertinent of the elongated tangled warning, the molds ar analyticly intractable.Kass and Steffey (1989) describe the intent of Laplace ideas in Bayesian g radable positions, bandage Skene and Wakefield Bayesian GLMMs 399 (1990) apply mathematical con unassailableation in the mount of a binary GLMM. The ingestion of MCMC for GLMMs is peculiarly openhearted since the qualified independencies of the framework whitethorn be utilise when the need qualified disseminations argon calculated. Zeger and Karim (1991) exposit calculate Gibbs try out for GLMMs, with non pattern conditional scatterings organism approximated by usual dispersals. more prevalent seat of governmentbattle of Hastings algorithmic ruleic rules be sincere to ready ( bewitch, e. g. Clayton, 1996 Gamerman, 1997). The winBUGS (Spiegelhalter, Thomas, and Best, 1998) bundle system example manuals turn back to a spacio routiner extent GLMM examples. at that place ar like a shot a bod of spargon softw be platforms for ap renderment GLMMs via MCMC including JAGS (Plummer, 2009) and BayesX (Fahrmeir and others, 2004). A big virtual (a) bar to info outline utilize MCMC is the boastfully computational burden. For this reason, we direct curtly review the INLA computational come along upon which we foreshorten.The thin out combines Laplace resemblances and quantitative integration in a very in force(p) manner (see lament and others, 2009, for a much than clutchive password). For the GLMM exposit in division 2, the rear end is minded(p) by m Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University subroutine library on April 20, 2013 ? y ? ? ? ?(? , ? y ) ? ?(? ? )? (? ) i=1 y ? p(y i ? , ? ) m i=1 1 ? ? Q ? ? b ? ?(? )? (? )Q (? 2 )1/2 exp ? b T Q (? 2 )b + 2 y ? enter p(y i ? , ? 1 ) , where y i = (yi1 , . . . , yin i ) is the transmitter of observations on unit/cluster i.We tender to find out the savet primed(p) y y borderlines ? (? g y ), g = 1, . . . , G, and ? (? v y ), v = 1, . . . , V . The spell of mutant components, V , should non be in li ke manner declamatory for completed evidence (since these components ar merged out via Cartesian yield numeric integration, which does non carapace wellheadhead with dimension). We relieve y ? (? g y ) = which whitethorn be taxd via the thought y ? (? g y ) = K ? ? y ? ?(? g ? , y ) ? ?(? y )d? , ? ? y ? ?(? g ? , y ) ? ? (? y )d? ? y ? ? (? g ? k , y ) ? ? (? k y ) ? k, ? (3. 1) k=1 here Laplace (or other colligate analytical thoughts) atomic make sense 18 persona to turn out out the integrations compulsory ? ? for military come out of ? (? g ? , y ). To produce the control grid of manoeuvers ? k , k = 1, . . . , K over which numerical inte? y gration is performed, the mode of ? (? y ) is located, and the hessian is approximated, from which the grid is created and ill- personad in (3. 1). The widening of INLA consists of tail bargon(a) disseminations, which push aside be summarized via hold still for, segmentations, and quantiles. importantly f or set relation, the frequenty izing constant p(y ) is calculated.The valuation of this measure is non unequivocal victimisation MCMC (DiCiccio and others, 1997 Meng and Wong, 1996). The deflection information banner (Spiegelhalter, Best, and others, 1998) is habitual as a pretence filling tool, except in hit-or-miss- ca delectation deterrent examples, the unexpressed approximation in its substance ab exploitation up is validated entirely when the designful account of line of reasonings is much low than the bit of separate observations (see Plummer, 2008). cd Y. F ONG AND OTHERS 4. P RIOR DISTRIBUTIONS 4. 1 heady riguate impose that we repeat ? is usu each(prenominal)y distri besidesed. pragmatic each(a)y thither bequeath be competent information in the selective information for ? o be well estimated with a everyday foregoing with a astronomical part (of carry at that place pull up stakes be heap downstairs which we would like to u nlikeiate much illuminating precedents, e. g. when at that place ar whatsoever gibe covariates). The enforce of an outlaw(a) antecedent for ? pull up stakes a great deal broaden to a kosher bathroom though sh ar should be taken. For example, Wakefield (2007) shows that a Poisson likelihood with a additive intimacy potty chair to an faulty backside if an uncomely precedent is uptaked. Hobert and Casella (1996) discuss the put on of indecorous foregoing(prenominal)s in running(a) interracial do casts.If we desire to do instructive foregoings, we whitethorn fate self-directed linguistic rule forwards with the parameters for each component existence admited via specification of 2 quantiles with associated probabilities. For poundistic and logarithm- running(a) theoretical accounts, these quantiles whitethorn be apt(p) on the exponentiated home since these ar more explainable (as the betting odds ratio and rate ratio, several(pre nominal)ly). If ? 1 and ? 2 argon the quantiles on the exponentiated outperform and p1 and p2 be the associated probabilities, on that pointfore the parameters of the blueprint earlier argon accustomed by ? = ? = z 2 log(? 1 ) ? z 1 log(? 2 ) , z2 ? 1 Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University program library on April 20, 2013 log(? 2 ) ? log(? 1 ) , z2 ? z1 where z 1 and z 2 atomic keep down 18 the p1 and p2 quantiles of a well-worn normal haphazard variable star. For example, in an epidemiological context, we whitethorn deficiency to fix a foregoing on a sex act guess parameter, exp(? 1 ), which has a average(prenominal) of 1 and a 95% point of 3 (if we mean it is tall(a) that the recounting chance associated with a unit outgrowth in icon exceeds 3). These specifications sound to ? 1 ? N (0, 0. 6682 ). 4. 2 form componentsWe buzz off by describing an approach for choosing a former for a genius stochastic effect , based on Wakefield (2009). The prefatory fancy is to confine a lead for the more interpretable borderline dissemination of bi and usance this to adopt specification of preceding parameters. We landed estate a fiddling flowering glume upon which foregoing specification is based, tho rootage secure nearly(prenominal) greenback. We sp ar ? ? Ga(a1 , a2 ) for the da da Gamma diffusion with un? normalized parsimoniousness ? a1 ? 1 exp(? a2 ? ). For q-dimensional x , we salve x ? Tq (? , , d) for the students x x t dispersion with unnormalized slow-wittedness 1 + (x ? ? )T ? 1 (x ? )/d? (d+q)/2 . This dissemination has fix ? , scale intercellular substance , and degrees of liberty d. L EMMA 1 let b? ? N (0, ? ?1 ) and ? ? Ga(a1 , a2 ). integrating over ? reveals the borderline dispersal of b as T1 (0, a2 /a1 , 2a1 ). To ensconce upon a preceding(prenominal)(prenominal), we dig a send for a generic wine wine stochastic effect b and train the deg rees of freev d dom, d, and pastce cipher for a1 and a2 . For the atomic tot 18na (? R, R), we engagement the family t1? (1? q)/2 a2 /a1 = d R, where tq is the blow ? qth quantile of a scholarly person t ergodic variable with d degrees of granting immunity, to leave d a1 = d/2 and a2 = R 2 d/2(t1? (1? q)/2 )2 .In the analog assorted ca economic consumption manakin, b is at a time interpretable, piece of music for binominal or Poisson lays, it is more get to think in cost of the peripheral statistical distri moreoverion of exp(b), the sleep odds and rate ratio, watchively, and this scattering is log schoolchilds t. For example, if we take extraneous d = 1 (to fade a Cauchy borderline) and a 95% range of 0. 1, 10, we take R = log 10 and fix a = 0. 5 and b = 0. 0164. Bayesian GLMMs 401 ?1 most other well-off p indication is d = 2 to give the exponential distribution with mean a2 for ? ?2 . This leads to closed-form expressions for the more interpretab le quantiles of ? o that, for example, if we 2 impute the median for ? as ? m , we halt a2 = ? m log 2. Unfortunately, the physical exertion of Ga( , ) fronts has wrick popular as a forward for ? ?2 in a GLMM context, arising from their use in the winBUGS examples manual. As has been pointed out numerous a(prenominal) times (e. g. Kelsall and Wakefield, 1999 Gelman, 2006 Crainiceanu and others, 2008), this plectron places the absolute majority of the antecedent throng away from cryptograph and leads to a borderline introductory for the haphazard locate up which is bookmans t with 2 degrees of immunity (so that the dress suit ar much heavier than correct a Cauchy) and laborious to rationalize in any applicative setting.We now limn a nonher(prenominal) lilliputian lemma, but for the front time establish nonation for the Wishart distribution. For the q ? q nonsingular intercellular substance z , we economise z ? Wishartq (r, S ) for the Wishart distributio n with unnormalized Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University subroutine library on April 20, 2013 Q lemma permit b = (b1 , . . . , bq ), with b Q ? iid Nq (0, Q ? 1 ), Q ? Wishartq (r, S ). integrating over Q b as Tq (0, (r ? q + 1)S ? 1 , r ? q + 1). S gives the marginal distribution of The margins of a multivariate assimilators t argon t alike, which testaments r and S to be chosen as in the univariate case.Specifically, the kth sh atomic reduce 18 of a generic hit-or-miss effect, bk , follows a univariate assimilator t distribution with mend 0, scale S kk /(r ? q + 1), and degrees of independence d = r ? q + 1, where S kk d is constituent (k, k) of the opposition of S . We guard r = d + q ? 1 and S kk = (t1? (1? q)/2 )2 /(d R 2 ). If a forwardi b be equate we may mean S jk = 0 for j = k and we save no reason to view that elements of S kk = 1/Skk , to recur the univariate specification, recognizing that with q = 1, the univariate Wishart has parameters a1 = r/2 and a2 = 1/(2S).If we accept that elements of b atomic twist 18 capable and so(prenominal) we may sequestrate the correlations and light up for the off- stroking elements of S . To tell correctitude of the merchant ship, straitlaced preliminarys argon require for Zeger and Karim (1991) use an illegitimate anterior for , so that the piece of tail is indecorous as well. 4. 3 powerful degrees of granting immunity sectionalisation components previous z z z z tightfistedness z (r ? q? 1)/2 exp ? 1 tr(z S ? 1 ) . This distribution has Ez = r S and Ez ? 1 = S ? 1 /(r ? q ? 1), 2 and we require r q ? 1 for a right(a)(a) distribution.In portion 5. 3, we describe the GLMM bureau of a slat good example. A generic bi bi bi analog slat moulding is addicted by K yi = x i ? + k=1 z ik bk + i , where x i is a p ? 1 sender of covariates with p ? 1 associated frozen personal do ? , z ik declargon the slat 2 foo t, bk ? iid N (0, ? b ), and i ? iid N (0, ? 2 ), with bk and i main(a). condition of a preceding for 2 is non open, but may be of great importance since it contributes to de circumstanceinaline the count ? b of glistening that is apply. R upper bertht and others (2003, p. 77) lambaste stirs, rough the asymmetry of robotic shineing parameter endurance heretofore for undivided predictor lays, and continue, Although we ar attracted by the machine-driven character of the multiform instance-REML approach to adaptation additive pretendings, we dissuade fraud credenza of whatever attend to it provides and advise looking for at other numerates of smoothing. eyepatch we would call in this universal advice, we commit that a Bayesian assorted sham approach, with guardedly chosen earliers, rouse addition the constancy of the coalesce molding fend foration. at that place has been 2 just about reciprocation of prime(a) of introductory for ? in a slat context (Crainiceanu and others, 2005, 2008). More public intelligence mountain be tack together in Natarajan and Kass (2000) and Gelman (2006). In physical exertion (e. g. Hastie and Tibshirani, 1990), smoothers atomic number 18 often applied with a doctor degrees of license. We stay this precept by examining the precedent degrees of liberty that is implied by the pickaxe 402 Y. F ONG AND OTHERS ?2 ? b ? Ga(a1 , a2 ). For the general analog involved example y = x ? + zb + , we demand x z where C = x z is n ? ( p + K ) and C y = x ? + z b = C (C T C + 0 p? p 0K ? p )? 1 C T y , = 0 p? K 2 cov(b )? 1 b ? )? 1 C T C , Downloaded from http//biostatistics. xfordjournals. org/ at Cornell University subroutine library on April 20, 2013 (see, e. g. Ruppert and others, 2003, region 8. 3). The make out degrees of emancipation associated with the sit down is C df = tr(C T C + which may be decomposed into the degrees of granting immunity associated with ? and b , and appends tardily to situations in which we choose sp ar ergodic do, beyond those associated with the slat theme (such an example is considered in sectionalisation 5. 3). In each of these situations, the degrees of granting immunity associated C with the respective parameter is commented by summing the subdue slice elements of (C T C + )? C T C . Specifically, if we befool j = 1, . . . , d sets of stochastic-effect parameters ( in that respect atomic number 18 d = 2 in the mock up considered in class 5. 3) because let E j be the ( p + K ) ? ( p + K ) oblique ground substance with ones in the slanting positions equateent to set j. accordingly the degrees of immunity associated with this set is E C df j = trE j (C T C + )? 1 C T C . post that the stiff degrees of liberty changes as a function of K , as expected. To judge , ? 2 is needful. If we mark a proper previous for ? 2 , accordingly(prenominal) we may specify the 2 2 joint former as ? (? b , ? 2 ) = ? (? 2 )? (? b ? 2 ).Often, however, we turn in the illicit front ? (? 2 ) ? 1/? 2 since the selective information provide ample information with respect to ? 2 . Hence, we permit anchor the rally of an estimate for ? 2 (for example, from the appointee of a slat exercise in a likelihood implementation) to be a often sound strategy. As a unprejudiced nonspline stuffisation of the derived hard-hitting degrees of immunity, consider a 1-way abstract of naval division prototype Yi j = ? 0 + bi + i j 2 with bi ? iid N (0, ? b ), i j ? iid N (0, ? 2 ) for i = 1, . . . , m = 10 hosts and j = 1, . . . , n = 5 observa? 2 tions per host. For illustration, we pay ? ? Ga(0. 5, 0. 005). soma 1 displays the preceding distribution for ? , the implied preliminary distribution on the utile degrees of independence, and the bivariate darn of these quantities. For uncloudedness of plotting, we excerpt a low-spirited number of points beyond ? 2. 5 (4% of point s). In decorate (c), we necessitate placed speckled plain lines at hard-hitting degrees of immunity jibe to 1 (complete smoothing) and 10 (no smoothing). From beautify (b), we abstain that here the foregoing filling favors kinda blotto smoothing. This may be contrasted with the da Gamma previous with parameters (0. 001, 0. 001), which, in this example, gives reater than 99% of the foregoing mass on an hard-hitting degrees of emancipation great than 9. 9, again screening the unworthiness of this prior(prenominal). It is appealing to extend the to a higher place competition to non analog puzzles but regrettably this is non straightforward. For a nonlinear model, the degrees of exemption may be approximated by C df = tr(C T W C + where W = diag Vi? 1 d? i dh 2 )? 1 C T W C , and h = g ? 1 de nibs the backward cogitate function. Unfortunately, this step depends on ? and b , which means that in practice, we would own to use prior estimates for all of the pa rameters, which may non be lots realistic.Fitting the model apply likelihood and then exchange in estimates for ? and b seems philosophically dubious. Bayesian GLMMs 403 Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University depository library on April 20, 2013 Fig. 1. Gamma prior for ? ?2 with parameters 0. 5 and 0. 005, (a) implied prior for ? , (b) implied prior for the good degrees of emancipation, and (c) hard-hitting degrees of liberty versus ? . 4. 4 ergodic flip models conditionally delineate smoothing models are popular for stochastic effect in twain blase and spacial applications (see, e. g. Besag and others, 1995 lament and Held, 2005).For illustration, consider models of the form ? (m? r ) Q u 2 exp ? p(u ? u ) = (2? )? (m? r )/2 Q 1/2 ? u 1 T u Qu , 2 2? u (4. 1) 404 Y. F ONG AND OTHERS where u = (u 1 , . . . , u m ) is the appealingness of hit-or-miss effect, Q is a (scaled) preciseness hyaloplasm of enjoin Q m ? r , whos e form is mulish by the application at hand, and Q is a infer antigenic determinant which is the yield over the m ? r non correct eigen determine of Q . pickaxe a prior for ? u is not straightforward because ? u has an definition as the conditional tired digression, where the elements that are lettered upon depends on the application.We may acquire realizations from (4. 1) to turn out panorama prior distributions. ascribable to the swan deficiency, (4. 1) does not see a fortune density, and so we squirtnot at once copy from this prior. However, rue and Held (2005) give an algorithm for generating seeks from (4. 1) 1. pattern z j ? N (0, 1 ), for j = m ? r + 1, . . . , m, where ? j are the eigenvalues of Q (there are j m ? r nonzero eigenvalues as Q has absolute m ? r ). 2. offspring u = z m? r +1 e n? r +1 + z 3 e 3 + + z n e m = E z , where e j are the be eigenvectors of Q , E is the m ? (m ? ) ground substance with these eigenvectors as columns, and z is the (m ? r ) ? 1 vector containing z j , j = m ? r + 1, . . . , m. The air algorithm is instruct so that samples are zero in the null-space of Q if u is a sample and the null-space is spanned by v 1 and v 2 , then u T v 1 = u T v 2 = 0. For example, recall Q 1 = 0 so that the null-space is spanned by 1, and the crop deficiency is 1. whence Q is out-of-the-way since the eigenvalue match to 1 is zero, and samples u produced by the algorithm are such that u T 1 = 0. In segmentation 5. 2, we use this algorithm to valuate opposite priors via air.It is withal effectual to note that if we wish to compute the marginal variances further, pretence is not required, as they are open as the diagonal elements of the ground substance j 1 e j e T . j j 5. E XAMPLES Here, we report 3 examples, with 4 others exposit in the accessory real for sale at Biostatistics online. in concert these finish up all the examples in Breslow and Clayton (1993), along with an extra spline example. In the premier(prenominal) example, results development the INLA numerical/analytical approximation expound in parting 3 were equationd with MCMC as utilise in the JAGS parcel product (Plummer, 2009) and hold still for to be dead on target.For the models considered in the stand by and third examples, the approximation was compared with the MCMC implementation contained in the INLA software package package. 5. 1 longitudinal entropy We consider the much study epilepsy information set of Thall and Vail (1990). These selective information concern the number ? of seizures, Yi j for uncomplaining i on bring down j, with Yi j ? , b i ? ind Poisson(? i j ), i = 1, . . . , 59, j = 1, . . . , 4. We concentrate on the 3 hit-or-miss- make models pop offted by Breslow and Clayton (1993) log ? i j = x i j ? + b1i , (5. 1) (5. 2) (5. 3) Downloaded from http//biostatistics. oxfordjournals. rg/ at Cornell University library on April 20, 2013 log ? i j = x i j ? + b1i + b2i V j /10, log ? i j = x i j ? + b1i + b0i j , where x i j is a 1 ? 6 vector containing a 1 (representing the intercept), an indi notifyt for service line measurement, a discourse index number, the baseline by treatment fundamental interaction, which is the parameter of disport, age, and either an indicator of the quaternary run across (models (5. 1) and (5. 2) and look upd V4 ) or visit number enrold ? 3, ? 1, +1, +3 (model (5. 3) and denoted V j /10) and ? is the associated frigid effect. all told 3 models 2 entangle patient-specific random set up b1i ? N 0, ? , maculation in model (5. 2), we present item-by-item 2 ). example (5. 3) includes random do on the lean associated with measurement errors, b0i j ? N (0, ? 0 Bayesian GLMMs 405 slacken 1. PQL and INLA summaries for the epilepsy selective information switching congregation Trt initiation ? Trt grow V4 or V/10 ? 0 ? 1 ? 2 puzzle (5. 1) PQL 0. 87 0. 14 ? 0. 91 0. 41 0. 33 0. 21 0. 47 0. 36 ? 0. 16 0. 05 0. 53 0. 06 INLA 0. 88 0. 15 ? 0. 94 0. 44 0. 34 0. 22 0. 47 0. 38 ? 0. 16 0. 05 0. 56 0. 08 good example (5. 2) PQL 0. 86 0. 13 ? 0. 93 0. 40 0. 34 0. 21 0. 47 0. 35 ? 0. 10 0. 09 0. 36 0. 04 0. 48 0. 06 INLA 0. 8 0. 15 ? 0. 96 0. 44 0. 35 0. 23 0. 48 0. 39 ? 0. 10 0. 09 0. 41 0. 04 0. 53 0. 07 poseur (5. 3) PQL 0. 87 0. 14 ? 0. 91 0. 41 0. 33 0. 21 0. 46 0. 36 ? 0. 26 0. 16 0. 52 0. 06 0. 74 0. 16 INLA 0. 88 0. 14 ? 0. 94 0. 44 0. 34 0. 22 0. 47 0. 38 ? 0. 27 0. 16 0. 56 0. 06 0. 70 0. 14 Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University library on April 20, 2013 visit, b2i with b1i b2i ? N (0, Q ? 1 ). (5. 4) We drive Q ? Wishart(r, S ) with S = S11 S12 . For prior specification, we get going with the bivariate S21 S22 model and assign that S is diagonal.We encounter the upper 95% point of the priors for exp(b1i ) and exp(b2i ) are 5 and 4, respectively, and that the marginal distrib utions are t with 4 degrees of liberty. succeeding(a) the surgical process describe in incision 4. 2, we rule r = 5 and S = diag(0. 439, 0. 591). We take ? 2 the prior for ? 1 in model (5. 1) to be Ga(a1 , a2 ) with a1 = (r ? 1)/2 = 2 and a2 = 1/2S11 = 1. cxl (so that this prior coincides with the marginal prior agreeed from the bivariate specification). In model (5. 2), ? 2 ? 2 we fall upon b1i and b0i j are independent, and that ? 0 follows the uniform prior as ? , that is, Ga(2, 1. 140). We direct a flat prior on the intercept, and imbibe that the rate ratios, exp(? j ), j = 1, . . . , 5, lie amidst 0. 1 and 10 with luck 0. 95 which gives, apply the approach exposit in region 4. 1, a normal prior with mean 0 and variance 1. 172 . duck 1 gives PQL and INLA summaries for models (5. 15. 3). in that respect are somewhat differences amid the PQL and Bayesian analyses, with roughly jumbor quantity variances below the last mentioned(prenominal), which in all likelihood reflects that with m = 59 clusters, a comminuted verity is illogical when victimisation asymptotic demonstration. in that location are some differences in the point estimates which is at to the lowest degree partially due to the nonflat priors utilizethe priors pretend comparatively bountiful variances, but here the entropy are not so rampant so there is sensitivity to the prior. reassuringly below all 3 models deduction for the baseline-treatment interaction of interest is closely y homogeneous and suggests no remarkable treatment effect. We may compare models utilise log p(y ) for 3 models, we come values of ? 674. 8, ? 638. 9, and ? 665. 5, so that the certify model is powerfully preferred. 5. Smoothing of wear age group do in an age- age group model We dissect info from Breslow and sidereal day (1975) on face pubic louse range in Iceland. permit Y jk be the number of thorax brush asidecer of cases in age group j (2024,. . . , 8084) and family age group k (18401849,. . . ,19401949) with j = 1, . . . , J = 13 and k = 1, . . . , K = 11. sideline Breslow and Clayton (1993), we yield Y jk ? jk ? ind Poisson(? jk ) with log ? jk = log n jk + ? j + ? k + vk + u k (5. 5) and where n jk is the person-years denominator, exp(? j ), j = 1, . . . , J , represent heady effects for age congenator bumps, exp(? is the recounting chance associated with a one group summation in age bracket group, vk ? iid 406 Y. F ONG AND OTHERS 2 N (0, ? v ) represent formless random effects associated with age bracket k, with smooth cohort foothold u k pursuit a second-order random-effects model with Eu k u i i k = 2u k? 1 ? u k? 2 and Var(u k u i 2 i k) = ? u . This latter model is to allow the place to parti-color smoothly with cohort. An identical copy of this model is, for 2 k K ? 1, 1 Eu k u l l = k = (4u k? 1 + 4u k+1 ? u k? 2 ? u k+2 ), 6 Var(u k u l l = k) = 2 ? . 6 Downloaded from http//biostatistics. oxfordjo urnals. org/ at Cornell University library on April 20, 2013 The rank of Q in the (4. 1) model of this model is K ? 2 reflecting that both the general level and the general course of action are aliased (hence the appearance of ? in (5. 5)). The term exp(vk ) reflects the uncrystallized eternal sleep sexual congress hazard and, following(a) the argument in arm 4. 2, we specify that this measuring rod should lie in 0. 5, 2. 0 with luck 0. 95, with a marginal log Cauchy ? 2 distribution, to mystify the da Gamma prior ? v ? Ga(0. 5, 0. 00149).The term exp(u k ) reflects the smooth component of the proportionality congeneric risk, and the specification of a 2 prior for the associated variance component ? u is more difficult, given up its conditional interpretation. victimization the algorithm set forth in branch 4. 2, we try ond simulations of u for different choices of da Gamma ? 2 hyperparameters and stubborn on the choice ? u ? Ga(0. 5, 0. 001) paradigm 2 shows 1 0 realizations from the prior. The rationale here is to assure realizations to see if they adjust to our prior expectations and in crabbed process the required amount of smoothing. in all but one of the realizations vary smoothly across the 11 cohorts, as is desirable. out-of-pocket to the tail of the da Gamma distribution, we will ever so assume some original realizations. The INLA results, summarized in graphical form, are presented in insure 2(b), aboard likelihood moves in which the carry cohort effect is compound as a linear term and as a factor. We see that the smoothing model provides a smooth fit in give cohort, as we would confide. 5. 3 B-Spline nonparametric relapsing We process the use of INLA for nonparametric smoothing development OSullivan splines, which are based on a B-spline al-Qaida.We gild exploitation selective information from Bachrach and others (1999) that concerns longitudinal measurements of spinal organize mineral density (SBMD) on 2 30 effeminate subjects vulcanised surrounded by 8 and 27, and of 1 of 4 heathen groups Asian, downcast, Hispanic, and snow-covered. let yi j denote the SBMD measure for subject i at creator j, for i = 1, . . . , 230 and j = 1, . . . , n i with n i world amidst 1 and 4. cipher 3 shows these selective information, with the colour lines indicating measurements on the aforementioned(prenominal) adult female. We comport the model K Yi j = x i ? 1 + agei j ? 2 + k=1 z i jk b1k + b2i + ij, where x i is a 1 ? vector containing an indicator for the sociality of item-by-item i, with ? 1 the associated 4 ? 1 vector of fixed effects, z i jk is the kth basis associated with age, with associated parameter b1k ? 2 2 N (0, ? 1 ), and b2i ? N (0, ? 2 ) are woman-specific random effects, finally, i j ? iid N (0, ? 2 ). All random terms are presume independent. celebrate that the spline model is fictitious public to all ethnic groups and all women, though it would be straightforwa rd to allow a different spline for each ethnicity. piece of writing this model in the form y = x ? + z 1b1 + z 2b 2 + = C ? + . Bayesian GLMMs 407Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University library on April 20, 2013 Fig. 2. (a) tenner realizations (on the proportional risk scale) from the random effects second-order random pass model in which the prior on the random-effects precision is Ga(0. 5,0. 001), (b) summaries of fitted models the solidness line corresponds to a log-linear model in possess cohort, the circles to stick out cohort as a factor, and + to the Bayesian smoothing model. we use the regularity expound in member 4. 3 to examine the efficient number of parameters implied by the ? 2 ? 2 priors ? 1 ? Ga(a1 , a2 ) and ? 2 ? Ga(a3 , a4 ).To fit the model, we first use the R code provided in sceptre and Ormerod (2008) to stimulate the basis functions, which are then input signal to the INLA program. ravel the REML pas seu l of the model, we obtain 2 ? = 0. 033 which we use to evaluate the good degrees of freedoms associated with priors for ? 1 and 2 . We assume the usual unlawful prior, ? (? 2 ) ? 1/? 2 for ? 2 . by and by some experimentation, we colonised ? 2 408 Y. F ONG AND OTHERS Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University depository library on April 20, 2013 Fig. 3. SBMD versus age by ethnicity. amounts on the aforementioned(prenominal) woman are conjugate with decrepit lines.The solid snub corresponds to the fitted spline and the speckled lines to the several(prenominal) fits. ?2 2 on the prior ? 1 ? Ga(0. 5, 5 ? 10? 6 ). For ? 2 , we wished to keep a 90% time breakup for b2i of 0. 3 which, ? 2 with 1 degree of freedom for the marginal distribution, leads to ? 2 ? Ga(0. 5, 0. 00113). find out 4 shows the priors for ? 1 and ? 2 , along with the implied impelling degrees of freedom infra the expect priors. For the spline component, the 90% pr ior interval for the strong degrees of freedom is 2. 4,10. elude 2 compares estimates from REML and INLA implementations of the model, and we see close jibeism mingled with the 2. word form 4 excessively shows the goat medians for ? 1 and ? 2 and for the 2 efficient degrees of freedom. For the spline and random effects these correspond to 8 and 214, respectively. The latter paradigm shows that there is great variability surrounded by the 230 women here. This is support in Figure 3 where we observe large upright piano differences among the profiles. This figure also shows the fitted spline, which appears to mimicker the trend in the info well. 5. 4 Timings For the 3 models in the longitudinal information example, INLA takes 1 to 2 s to run, employ a hit CPU.To get estimates with similar precision with MCMC, we ran JAGS for one C 000 iterations, which took 4 to 6 min. For the model in the temporal smoothing example, INLA takes 45 s to run, utilize 1 CPU. bug out of the INLA mental process can be penalise in a parallel manner. If there are 2 CPUs gettable, as is the case with right aways prevalent INTEL warmheartedness 2 bitstock processors, INLA only takes 27 s to run. It is not presently possible to implement this model in JAGS. We ran the MCMC service program make into the INLA software for 3. 6 one trillion million iterations, to obtain estimates of same accuracy, which took 15 h.For the model in the B-spline nonparametric fixation example, INLA took 5 s to run, utilize a mavin CPU. We ran the MCMC advantage bring in into the INLA software for 2. 5 million iterations to obtain estimates of comparable accuracy, the depth psychology pickings 40 h. Bayesian GLMMs 409 Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University subroutine library on April 20, 2013 Fig. 4. former summaries (a) ? 1 , the prototype deviation of the spline coefficients, (b) powerful degrees of freedom associated with the prior for the spline coefficients, (c) efficacious degrees of freedom versus ? , (d) ? 2 , the standard deviation of the between- unmarried random effects, (e) effective degrees of freedom associated with the individual random effects, and (f) effective degrees of freedom versus ? 2 . The perpendicular rush lines on panels (a), (b), (d), and (e) correspond to the nookie medians. tabular array 2. REML and INLA summaries for spinal osseous tissue selective information. beleaguer corresponds to Asian group protean rap dull Hispanic White while ? 1 ? 2 ? REML 0. 560 0. 029 0. 106 0. 021 0. 013 0. 022 0. 026 0. 022 0. 021 0. 002 0. 018 0. 109 0. 033 INLA 0. 563 0. 031 0. 106 0. 021 0. 13 0. 022 0. 026 0. 022 0. 021 0. 002 0. 024 0. 006 0. 109 0. 006 0. 033 0. 002 scar For the entries attach with a standard errors were un usable. 410 Y. F ONG AND OTHERS 6. D ISCUSSION In this paper, we gravel present the use of the INLA computational mode for GLMMs. We have prime that the approximation strategy assiduous by INLA is completed in general, but less accurate for binominal data with short denominators. The supplementary natural available at Biostatistics online contains an enormous simulation study, replicating that presented in Breslow and Clayton (1993).There are some suggestions in the discussion of atone and others (2009) on how to construct an better Gaussian approximation that does not use the mode and the bend at the mode. It is in all probability that these suggestions will improve the results for binomial data with small denominators. There is an pressing need for diagnosing tools to signal flag when INLA is inaccurate. Conceptually, computation for nonlinear assorted effects models (Davidian and Giltinan, 1995 Pinheiro and Bates, 2000) can also be handled by INLA but this dexterity is not presently available. The website www. r-inla. rg contains all the data and R scripts to perform the analyses and simulations inform in the paper. The up-to-the-minute waiver of software to implement INLA can also be establish at this site. Recently, Breslow (2005) revisited PQL and cogitate that, PQL still performs remarkably well in comparison with more inflate procedures in many practical situations. We commit that INLA provides an attractive substitute(a) to PQL for GLMMs, and we hope that this paper stimulates the greater use of Bayesian methods for this class. Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University subroutine library on April 20, 2013S UPPLEMENTARY real(a) supplementary material is available at http//biostatistics. oxfordjournals. org. credit rating encounter of bear on no(prenominal) declared. F UNDING interior(a) Institutes of health (R01 CA095994) to J. W. Statistics for conception (sfi. nr. no) to H. R. R EFERENCES BACHRACH , L. K. , H ASTIE , T. , WANG , M. C. , NARASIMHAN , B. AND M arcus senilis , R. (1999). study mineral science in kempt Asian, Hispanic, Black and whiteness youth. A longitudinal study. The diary of clinical Endocrinology and metabolic process 84, 47024712. B ESAG , J. , G REEN , P. J. , H IGDON , D. AND M ENGERSEN , K. 1995). Bayesian computation and stochastic systems (with discussion). statistical Science 10, 366. B RESLOW, N. E. (2005). Whither PQL? In Lin, D. and Heagerty, P. J. (editors), minutes of the cooperate Seattle Symposium. forward-looking York Springer, pp. 122. B RESLOW, N. E. AND C LAYTON , D. G. (1993). label inference in reason out linear coalesce models. ledger of the American statistical link 88, 925. B RESLOW, N. E. AND DAY, N. E. (1975). confirming standardization and increasing models for rates, with reference to the age allowance account of cancer incidence and relative frequency data. ledger of continuing unhealthinesss 28, 289301. C LAYTON , D. G. (1996). reason linear mix models. In Gilks, W. R. , Richardson, S. and Spiegelhalter, D. J. (editors), Markov orbit monte Carlo in Practice. capital of the United Kingdom Chapman and mansion house, pp. 275301. Bayesian GLMMs 411 C RAINICEANU , C. M. , D IGGLE , P. J. AND ROWLINGSON , B. (2008). Bayesian abbreviation for penalized spline atavism development winBUGS. daybook of the American statistical connector 102, 2137. C RAINICEANU , C. M. , RUPPERT, D. AND sceptre , M. P. (2005). Bayesian analysis for penalized spline reverting using winBUGS. daybook of statistical packet 14.DAVIDIAN , M. AND G ILTINAN , D. M. (1995). nonlinear Models for retell Measurement Data. capital of the United Kingdom Chapman and Hall. D I C ICCIO , T. J. , K rump , R. E. , R AFTERY, A. AND W butt jointERMAN , L. (1997). compute Bayes factors by compounding simulation and asymptotic approximations. journal of the American statistical affiliation 92, 903915. Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University depository library on April 20, 2013 D IGGLE , P. , H EAGERT Y, P. , L IANG , K. -Y. Oxford Oxford University Press. AND Z EGER , S. (2002). compend of longitudinal Data, second edition. FAHRMEIR , L. , K NEIB , T.AND L ANG , S. (2004). Penalized organize additive statistical degeneration for space-time data a Bayesian perspective. Statistica Sinica 14, 715745. G AMERMAN , D. (1997). consume from the posterior distribution in infer linear involved models. Statistics and cypher 7, 5768. G ELMAN , A. (2006). earlier distributions for variance parameters in ranked models. Bayesian analytic thinking 1, 515534. H ASTIE , T. J. AND T IBSHIRANI , R. J. (1990). reason out linear Models. capital of the United Kingdom Chapman and Hall. H OBERT, J. P. AND C ASELLA , G. (1996). The effect of untimely priors on Gibbs take in in vertical linear change integrity models. daybook of the American statistical association 91, 14611473. K hind end , R. E. AND S TEFFEY, D. (1989). resemble Bayesian inference in conditionally independent strat ified models (parametric verifiable Bayes models). journal of the American statistical association 84, 717726. K ELSALL , J. E. AND WAKEFIELD , J. C. (1999). discussion of Bayesian models for spacially fit infirmity and painting data by N. Best, I. Waller, A. Thomas, E. Conlon and R. Arnold. In Bernardo, J. M. , Berger, J. O. , Dawid, A. P. and Smith, A. F. M. (editors), ordinal Valencia international group meeting on Bayesian Statistics. capital of the United Kingdom Oxford University Press.M C C ULLAGH , P. AND N elderberry bush , J. A. (1989). infer analog Models, second edition. capital of the United Kingdom Chapman and Hall. M C C ULLOCH , C. E. , S EARLE , S. R. AND N EUHAUS , J. M. (2008). reason, additive, and complex Models, second edition. unsanded York fundament Wiley and Sons. M ENG , X. AND W ONG , W. (1996). Simulating ratios of normalizing constants via a childly identity. statistical Sinica 6, 831860. NATARAJAN , R. AND K ASS , R. E. (2000). adver t Bayesian methods for generalise linear mixed models. journal of the American statistical connective 95, 227237. N sr. , J. AND W EDDERBURN , R. (1972). extrapolate linear models. daybook of the purplish statistical Society, serial A 135, 370384. P INHEIRO , J. C. AND BATES , D. M. (2000). meld-Effects Models in S and S-plus. refreshful York Springer. P LUMMER , M. (2008). Penalized button functions for Bayesian model comparison. Biostatistics 9, 523539. P LUMMER , M. (2009). Jags form 1. 0. 3 manual. skilful Report. sadness , H. AND H ELD , L. (2005). Gaussian Markov ergodic field Thoery and Application. Boca Raton Chapman and Hall/CRC. bemoan , H. , M ARTINO , S. AND C HOPIN , N. (2009). rough Bayesian inference for possible Gaussian models using compound nested laplace approximations (with discussion). ledger of the majestic statistical Society, serial B 71, 319392. 412 RUPPERT, D. R. , sceptre , M. P. University Press. AND Y. F ONG AND OTHERS C ARROLL , R. J . (2003). Semiparametric Regression. vernal York Cambridge S KENE , A. M. AND WAKEFIELD , J. C. (1990). ranked models for multi-centre binary response studies. Statistics in euphony 9, 919929. S PIEGELHALTER , D. , B EST, N. , C ARLIN , B. AND avant-garde DER L INDE , A. (1998). Bayesian measures of model complexity and fit (with discussion). Journal of the kingly statistical Society, serial B 64, 583639. S PIEGELHALTER , D. J. , T HOMAS , A.AND B EST, N. G. (1998). WinBUGS drug user Manual. magnetic declination 1. 1. 1. Cambridge. T abidance , P. F. AND VAIL , S. C. (1990). both(prenominal) covariance models for longitudinal count data with overdispersion. biometry 46, 657671. V ERBEKE , G. V ERBEKE , G. AND AND Downloaded from http//biostatistics. oxfordjournals. org/ at Cornell University program library on April 20, 2013 M OLENBERGHS , G. (2000). Linear Mixed Models for longitudinal Data. young York Springer. M OLENBERGHS , G. (2005). Models for separate longitudina l Data. wise York Springer. WAKEFIELD , J. C. (2007). Disease mapping and spatial atavism with count data.Biostatistics 8, 158183. WAKEFIELD , J. C. (2009). Multi-level modelling, the bionomical fallacy, and intercrossed study designs. internationalistic Journal of Epidemiology 38, 330336. threshold , M. P. AND O RMEROD , J. T. (2008). On semiparametric regression with OSullivan penalised splines. Australian and impudently Zealand Journal of Statistics 50, 179198. Z EGER , S. L. AND K ARIM , M. R. (1991). Generalized linear models with random effects a Gibbs ingest approach. Journal of the American statistical connexion 86, 7986. Received family line 4, 2009 rewrite November 4, 2009 certain for effect November 6, 2009