Seeking advice on forecasting weekly infectious disease incidence using fpp3

[Liu499-create]

Dear Professor,

I am a practitioner working in infectious disease prevention and control. The univariate time series examples in your book have given me a great deal of inspiration, and I would like to begin by sincerely thanking you for your generous and valuable contribution.

I am attempting to apply some of the methods learned from your book to forecast weekly reported incidence rates of different categories of infectious diseases at the county/district level. The fpp3 framework has greatly facilitated the modeling process. However, my current forecasting accuracy is still not satisfactory.

I believe that modeling weekly county-level time series data (weekly incidence rates over nearly 10 years) involves several practical challenges. As you aptly stated in your book: “Real data often contains missing values, outlying observations, and other messy features. Dealing with them can sometimes be troublesome.” You also noted that “None of the methods we have considered in this book will work well if there are extreme outliers in the data. In this case, we may wish to replace them with missing values, or with an estimate that is more consistent with the majority of the data.”

At the same time, I fully agree with your warning that “Simply replacing outliers without thinking about why they have occurred is a dangerous practice. They may provide useful information about the process that produced the data, which should be taken into account when forecasting.”

My data originate from daily case reports across the entire district. I aggregated the data by date of onset into weekly incidence rates over the past decade. For weeks with no reported cases, I set the incidence rate to zero. Although I do not regard these as missing values in a strict sense, in the context of forecasting weekly incidence rates for the next seven weeks for certain disease categories, they behave somewhat similarly to missing values. However, I did not treat them as missing during modeling, which prevented me from applying logarithmic or similar transformations.

I am uncertain whether setting these weeks to zero is an appropriate choice, and whether there might be better alternatives. For example, would it be more reasonable to replace these zero values using an ARIMA-based imputation approach? In addition, there are some outliers in the data. After carefully studying your book, I decided not to remove or replace them initially, but I am considering using ARIMA-based fitting to replace these outliers in subsequent analyses.

My goal is to forecast incidence rates for the next seven weeks to support surveillance and early warning. I understand that providing advice may take some of your valuable time, and I sincerely appreciate your willingness to consider my question.

The models I have experimented with include:

stl_ets <- decomposition_model(
  STL(rates, robust = TRUE),
  ETS(season_adjust ~ season("N"))
)

stl_arima <- decomposition_model(
  STL(rates, robust = TRUE),
  ARIMA(season_adjust ~ PDQ(0,0,0))
)

dhr <- ARIMA(rates ~ fourier(K = 6) + PDQ(0, 0, 0))

nnetar <- NNETAR(rates, lambda = 0)

prophet <- fable.prophet::prophet(
  rates ~ season(period = 52,
                 order = 6,
                 type = "additive") +
           growth(changepoint_prior_scale = 0.05) +
           xreg(is_holiday_week)
)

I have also tried combining these models for forecasting, but the MAPE has remained in the range of approximately 20–40%, with limited room for further improvement.

I initially expected that setting robust = TRUE in STL would move outliers into the season_adjust component and that forecasting this component with ETS would improve accuracy, but the improvement was minimal.

I would greatly appreciate your advice on possible improvements. In your opinion, are there parameter adjustments I should consider for these models? Are there other models within the framework you propose that might be more suitable for this type of data? Or do you think it is necessary to replace zero-incidence weeks or outlying values using ARIMA-based imputation before forecasting?

Thank you very much for your time and for any guidance you may be able to offer.

Sincerely @robjhyndman