Skip to contents

Changelog

Source: NEWS.md

iop 0.1.0

First release. iop estimates ordered probit and ordered logit models whose outcome has an inflated category, together with the plain ordered models, on one engine with one set of methods.

Estimators

  • oprobit(), ologit(): ordered probit / logit, with survey weights, offsets, partial proportional-odds (non-parallel) effects via parallel =, and analytic, robust, or cluster-robust standard errors.
  • iop(): inflated ordered probit for any single inflated category (inflate = "bottom" | "middle" | "top" or a category label), the zero-/middle-/top-inflated ordered probit of Harris and Zhao (2007), Bagozzi and Mukherjee (2012), and Bagozzi, Joo and Mukherjee (2024), with optional correlated errors (correlated = TRUE).
  • iol(): the inflated ordered logit counterpart.
  • All four share the formula interface y ~ x1 + x2 | z1 + z2, a C++ likelihood with analytic gradients (a deterministic Gauss–Legendre bivariate-normal algorithm for the correlated form), column-scaled quasi-Newton optimization with an exact-Hessian Newton polish, and one S3 class ("iord") for every method.

Second referee panel (2026-08-22)

  • vuong() and the Vuong block of inflation_test() are now consistent with the package’s frequency-weight semantics: the statistic equals the one computed on duplicated rows (weight total as the sample size, weighted mean and variance of the per-row log-likelihood difference); previously the unweighted row count and the unweighted standard deviation of the weighted contributions were used, which gave a different statistic under non-constant weights. The BIC (in summary(), glance(), texreg, inflation_test()) uses the weight total as n for the same reason.
  • classification()’s confusion table is weighted (sums of weights per cell), so it agrees with the weighted share correctly classified printed under it.
  • A cell probability that underflows to a subnormal double no longer sends w / P to Inf (and Inf * 0 to NaN) in the gradient: the divisor is floored at the smallest normal double. The case arose with an extreme offset and, in random-intercept fits with a large unit variance, at the outer quadrature nodes of a high nAGQ, where it could leave the fit at a non-stationary point with a warning.
  • $max_grad is reported in the optimizer’s standardized parameterization, the units the convergence rule uses.
  • A character response is refused with a message naming the alphabetical order it would have received; print() and summary() state the response levels in the order used.
  • predict(newdata = ) returns NA rows, as documented, when an all-NA covariate column arrives as logical (data.frame(x = NA)); previously that case failed with “subscript out of bounds”.
  • A data frame with no complete cases gives a message naming the cause; ?ame points to first_difference(newdata = ); inflation_test(boot = ) reports the expected running time when it is long and documents the seed guarantee, the cost, and the independence of its simulated data from any cluster structure; se = "bootstrap" documents the seed guarantee; ?update.iord notes what a fit restored with readRDS() needs; ?classification cautions about modal accuracy for minority categories; the data pages state the provenance and terms of the bundled data sets.
  • Documentation states that weights are frequency weights and that survey (probability) weights call for se = "robust" or cluster =; inflation_test(boot = ) notes that its parametric bootstrap draws independent observations and does not reproduce a cluster structure.

Final pre-submission additions (2026-08-22)

  • vignette("iop") gains two worked examples outside political science: a zero-inflated ordered probit for the number of extramarital affairs (AER::Affairs) and a middle-inflated ordered probit for self-rated health in the 1988 NMES (AER::NMES1988); AER is in Suggests and the chunks run only when it is installed.
  • $fe_units lists the unit levels actually estimated (a unit that disappears through missing covariates is no longer counted).
  • Tests for the remaining low-level C++ guards (iord_probs_cpp, the random-intercept entry points) and for the two examples.

Referee round 3 (2026-08-22)

  • Unit-dummy columns are identified from the design’s term assignment, not by a name-prefix match: a covariate whose name starts with the fe = variable’s name (unitprice next to fe = "unit") is a covariate – it is printed, screened by the separation diagnostics, and reported by texreg like any other; previously it was folded into the hidden dummies ($fe_index, print(), summary()$block, extract()).
  • glance() gains converged, boundary, and ill_conditioned, and the texreg GOF block gains “Converged” and “Boundary/ill-conditioned” rows, so tables built from a saved fit carry the fit-quality flags.
  • Column scales treat a covariate as constant only when its SD is negligible relative to its magnitude (1e-10), so covariates measured in billionths are standardized like any other (the absolute 1e-8 floor left them unscaled); the column scales are computed once per fit.
  • Documentation notes why degenerate split/outcome standard errors stay visible while a cutpoint at +/-Inf is reported NA.
  • The inflation-equation unit dummies (fe_inflation = TRUE) are exempt from the split-equation quasi-separation signatures, as the outcome-equation dummies already were; a short panel with many inflation dummies no longer gets a “simplify the inflation equation” message when the cause is the dummies (the incidental-parameters warning, the conditioning flag, and the NA standard errors already say so).

Referee round 2 (2026-08-22)

  • Standard errors, the condition number, and split_test()’s score and information (both forms) are now all computed in the optimizer’s scaled parameterization (every covariate column standardized) and mapped back, so none of them depends on the units of the covariates. Previously the Hessian was differenced in raw parameter units: a covariate in cents next to a dummy could return a negative delta-method variance (an NA standard error) for a perfectly identified fit, and split_test(information = "hessian") could report “not positive definite” on a well-behaved null fit with a split covariate in dollars. Regression tests fit the same model at two covariate scales.
  • The hurdle-type boundary flag (ordered stage empties the inflated category) also fires when the ordered-stage share is below 1e-2 and the adjacent cutpoint’s standard error already exceeds 50, so a fit the optimizer stopped just short of the boundary is named rather than left to the generic large-standard-error note; the degenerate-split-SE and large-SE notes are on the standardized scale.
  • The quasi-separation signatures (standardized coefficient above 10, or standardized standard error above 50) are also applied to the outcome equation of every model (unit dummies exempt).
  • update() refuses a formula. with more than one |, and a two-part formula. on a fit without an inflation equation, with a clear message; the estimators refuse formulas with more than one |.
  • The size simulation behind the split_test() documentation (inst/mc/split_test_size.R, with its per-replication results; the log is kept in the repository) ships with the package; the rank-deficiency message names the dependent column and says so; the quasi-separation message says the estimate “may not be finite”.
  • Tests for the low-level C++ guards (wrong-length parameter vector); ame() documents that it has no profile mode by design.

Referee round 1 (2026-08-22)

  • ame() and first_difference(average = TRUE) now honour the fit’s weights when averaging over the estimation data (they equal the duplicated-rows fit exactly); previously the averages were unweighted.
  • Boundary diagnostics are graded and surfaced: $boundary also flags a quasi-separated split (standardized split slope above 10 or intercept above 40, or degenerate split standard errors), not only the all-at-0/1 case; $boundary_messages, $ill_conditioned (information matrix with reciprocal condition number below 1e-12 in the optimizer’s scaled parameterization, so that covariate units play no role, or a generalized inverse needed), and $se_na (non-positive delta-method variances, reported as NA rather than 0) are stored and printed by summary() and print(); the thresholds are listed in ?iop (“Diagnostics and fixed thresholds”).
  • The Newton polish and the covariance use a central-difference Jacobian of the analytic gradient (2p gradient evaluations) instead of numDeriv’s Richardson extrapolation (about 8p), a 3-4x saving that matters for fits with many unit dummies; the documented scaling of fe = is now explicit.
  • split_test(information = "opg" | "hessian"): the outer-product LM of Brown, Harris and Spencer stays the default (correctly sized under the null in simulation, always defined); the observed-information form is available and is reported as NA with a note when the Hessian is not positive definite at the restricted estimates. The documentation now states plainly that the LM is a local screening statistic that can be far from the LR under a strong alternative (267 vs 57 on bp) and that the LR is the one to report.
  • update() method for "iord" fits: two-part formula. arguments (. ~ . + x3 | ., . ~ . | . + z2) update the outcome and inflation equations separately; a one-part formula. changes the outcome equation and keeps the inflation equation (stats::update.formula() alone folds the | into the outcome part).
  • print(inflation_test(...)) notes when the parametric-bootstrap LR was not run; vuong()/lr_test() check that the two fits use the same data rows, not only the same response vector; update() semantics, missing-value handling, and newdata rules are documented on the estimator pages; the jackknife covariance is justified (Dhaene and Jochmans 2015) and noted in summary(); maxit documentation clarified.
  • A boundary case the split-side diagnostics did not name: the ordered stage can empty the inflated category (every such observation attributed to the inflation process, a hurdle-type split), leaving the adjacent cutpoint at +/-Inf with no information; $boundary now flags it (ordered-stage share of the inflated category’s probability mass below 1e-6), names the cutpoint, and summary()/confint() report its standard error as NA instead of the generalized-inverse value. $flat_hessian is now computed from the final Hessian of every fit, not only when the Newton decrement was unavailable.
  • The outcome-equation rank check now includes the constant that the cutpoints absorb: a set of columns that sums to a constant (0 + g, an interaction x:g without the x main effect next to poly(x, 2)) is rejected with a message naming the columns, where previously it fit with a singular information matrix; an explicit 0 + / - 1 in the outcome formula is ignored with a message (factors are coded as with an intercept).
  • Bootstrap relabelling of duplicated units uses a suffix guaranteed absent from the original labels; pbvn_cpp() and friends guard empty inputs; mundlak() no longer uses <<-; parallel listed in Suggests; the CI workflow runs the full suite (NOT_CRAN = true).

Category-specific split equations, tests, scores, bootstrap (2026-08-22)

  • split = "category" in iop() / iol(): every non-inflated category gets its own split (inflation) equation – and its own rho with correlated = TRUE – the generalised GZiOP / GMiOP of Brown, Harris and Spencer (2020), which nests the common-split ZiOP / MiOP / TiOP. The fit starts from its common-split counterpart ($loglik_common); coefficients are infl_<term>:<category> and rho:<category>; regime quantities become one column per non-inflated category. riop() draws from the generalised process when gamma is a matrix.
  • split_test(): Lagrange-multiplier (score) test of the common split from the common-split fit alone, plus the likelihood-ratio test from the category-specific refit.
  • inflation_test(boot = R): a parametric-bootstrap likelihood-ratio test of the inflated against the plain ordered model (simulated from the fitted ordered model), the boundary-aware alternative to the Vuong test; the help page documents the Wilson (2015) / Dale and Sirchenko (2021) objection to the Vuong test for this nested comparison.
  • predict(type = "zeros"): the two components of the inflated-category probability (through the inflation process / through the ordered stage, Harris and Zhao’s two types of zeros), with se.fit; first_difference() and ame() gain decompose = TRUE to report effects on both components. Also predict(type = "mean") (expected category index) and type = "cumulative".
  • classification(): confusion table, share correctly classified, Brier and ranked probability scores, mean log score, and per-category precision, recall, and adjusted noise-to-signal ratio (Dale and Sirchenko’s ziopclassification), on the estimation data or on new data.
  • ame(stage = "outcome" | "inflation"): stage-specific average marginal effects, matching first_difference(stage =).
  • diord(), piord(), qiord(), riord(): the d/p/q/r family of the (inflated) ordered response – probability mass, cumulative probability, quantile, random draws – from explicit parameters (eta, tau, a, k, link, rho, including the category-specific split) or from a fitted model at newdata.
  • se = "bootstrap" (with nboot, cores) on every estimator: nonparametric bootstrap over rows, clusters (cluster =), or random-intercept units, with relabelled duplicates; replicate estimates in $boot; confint(type = "percentile"). Recommended by Dale and Sirchenko for the error correlation in small samples.

Inference and comparison

Quantities of interest and diagnostics

  • ame(): average marginal effects (derivatives for continuous covariates, discrete changes for 0/1 covariates and factor levels) on every category and on the regime probability, with delta-method intervals; plot() methods for ame() and first_difference() tables.

  • predict(se.fit = TRUE): delta-method standard errors for predicted probabilities.

  • parallel_test(): likelihood-ratio tests of the parallel-regression assumption term by term and jointly; parallel = "auto" relaxes terms automatically (forward autofit).

  • fe_correction = "jackknife": split-panel jackknife bias correction of the common parameters in fixed-effects fits (Dhaene and Jochmans 2015).

  • predict() types: category probabilities, modal class, ordered-stage probabilities, regime and inflation probabilities, posterior probability of being an inflated case, linear predictors.

  • first_difference() by category and regime, by stage ("both", "outcome", "inflation"), at a profile or averaged over the data, with delta-method or simulation intervals.

  • simulate() for DHARMa-style simulated-residual diagnostics; riop() draws data from the model’s DGP.

  • Panels: unit random intercepts by adaptive Gauss–Hermite quadrature (re =, optionally also in the inflation equation with re_inflation = TRUE; ranef() for the empirical-Bayes unit effects, marginal and conditional predictions), unit fixed effects as dummies (fe =, fe_inflation = TRUE; units without identifying variation dropped with a message; incidental-parameters warning for short panels), and the mundlak() correlated-random-effects helper (both equations). A Monte Carlo (inst/mc/) documents the trade-off: with a covariate correlated with the unit effect, unit dummies are biased +26/+11/+5/+2.5 percent at T = 4/8/16/32, a random intercept +21/+11/+6/+3 percent, pooled +12 percent, Mundlak within 0.3 percent.

  • broom tidy()/glance()/augment(); texreg extract().

Data

  • bp (Besley and Persson 2009 political-violence panel, as in Bagozzi et al. 2015), pta (Baccini, Dur and Elsig 2015 escape-flexibility data, as in Bagozzi, Joo and Mukherjee 2024), repression (Girod, Stewart and Walters 2018 repression-of-campaigns data, as in the same article’s appendix).

Documentation

  • Four vignettes: vignette("iop") (getting started), vignette("quantities") (predicted probabilities, first differences, average marginal effects, tables, diagnostics), vignette("panels") (random intercepts, fixed effects and the jackknife, the Mundlak device, the Monte Carlo), and vignette("model") (likelihood, identification, estimation, boundary cases, tests, validation).
  • A package overview page (?"iop-package") indexing every function; examples on every method page; @family cross-references between related pages; a pkgdown site configuration (_pkgdown.yml) and GitHub Actions workflows for R CMD check and the site.

Robustness (adversarial pass)

  • With fe =, a covariate that does not vary within units is refused with a message that names it (it is an exact linear combination of the unit dummies); previously the generic rank check blamed an arbitrary dummy column. The same check applies to the inflation equation with fe_inflation = TRUE.

  • The boundary flag distinguishes its two inflation cases in the warning: a split that collapses to the plain ordered model (every regime probability at 1) and a quasi-separated split (a few units at regime probability 0, the rest at 1, coefficients not finite).

  • fe_correction = "jackknife" stores the two half-panel coefficient vectors, log-likelihoods, sizes, and dropped-unit counts in object$jackknife, so the correction can be inspected before it is reported.

  • Formulas with transformed terms (poly(), ns(), scale(), I(), interactions, expression responses) are built from per-equation model frames so that predict() on new data uses the fitted bases (predvars); constant or collinear covariates are rejected with a clear message; unused factor levels are dropped.

  • Inflated fits are multi-start (binary-model starts at several slope scales and signs, flat high-regime starts, and the ordered baseline); the log-likelihood reached from every start is stored in $start_logliks. This finds the second likelihood mode of the Besley–Persson ZiOP/ZiOPC with the full inflation equation (published ZiOPC2: -1,370, rho -0.912).

  • The degenerate-inflation diagnostic is based on the fitted regime probabilities (all at 0/1), not on coefficient magnitudes.

  • lr_test() refuses pairs that differ in link, inflated category, or random-intercept structure.

Validation

  • Bivariate-normal probabilities agree with pbivnorm and mvtnorm to 1e-13; analytic gradients agree with numerical differentiation in every model configuration.
  • oprobit()/ologit() reproduce MASS::polr, ordinal::clm, and VGAM::vglm(cumulative(parallel = FALSE)) (coefficients, cutpoints, log-likelihoods, robust and cluster-robust SEs via sandwich); their random-intercept versions reproduce ordinal::clmm(nAGQ = 15).
  • iop() reproduces Table 1 of Bagozzi, Hill, Moore and Mukherjee (2015) on the bundled bp data (OP, ZiOP, ZiOPC, ZiOPC2 log-likelihoods and rho), Table 1 and the marginal-effects figures of Bagozzi and Mukherjee (2012; MiOP and MiOPC, to the third decimal), and the TiOP/TiOPC log-likelihoods of Bagozzi, Joo and Mukherjee (2024; Table 1 and appendix Table A.5) – the original likelihood code of each paper evaluated at iop’s estimates equals iop’s log-likelihood to machine precision – and matches the Python idcempy implementation (ZiOP, MiOP) where that implementation converges.