State Space Modelling of Dynamic Choice Behavior with Habit Persistence

In this dissertation, I present a new approach to capturing dependence across time in dynamic choice data. To achieve this, I develop a state space dynamic choice model and a novel algorithm to fit the data. Instead of capturing dependence in outcomes through lagged response variables, referred to as state dependence, I introduce a lagged utility term through the latent state equation. The lagged utility term captures habit persistence, which has not been explored directly in earlier models (Heckman, 1981b). The autoregressive nature of the lagged utility provides a significantly richer summary of prior utility than a lagged outcome variable. The fitting algorithm combines a non-linear particle filter with a standard Metropolis-Hastings step to compute Bayesian posterior estimates of the parameters. The model can capture habit persistence (inertia), variety seeking, serial correlation, and unobserved heterogeneity. Through simulation analysis, I demonstrate that while the proposed method is effective in estimating the parameters, both a large sample size and the number of simulated particles are critical. Misspecification in serial correlation in the random component of the utility function is shown to result in biased estimates for certain coefficients, although not the habit persistence term. This method avoids the initial conditions problem common with lagged variables (Wooldridge, 2010). From the perspective of a marketer, the value of the proposed model stems from its ability to distinguish the effects of habit, variety seeking, and heterogeneity.

The algorithm is applied to case studies involving the sales of fast-moving consumer goods, as recorded in scanner data furnished by a major grocery store. The studies demonstrate the wide-ranging variation in purchasing habits and price sensitivity across customers; this variation highlights the value of the individual-level models applied in this study. Specifically, we find the existence of habitual purchasing behavior in utilitarian goods (e.g., cereal and soft drinks). However, in hedonic goods (e.g., beer), we find no evidence of habit persistence, which is in agreement with earlier studies.