kalmanFilter
Implement a Kalman filter assimilation
Description
The kalmanFilter class provides objects that implement offline ensemble square root Kalman filters. This assimilation method proceeds by determining the differences between observations and a set of ensemble members. The ensemble members are then updated to more closely resemble the observations using a Kalman gain matrix. The mean of the updated ensemble is typically used as the final reconstruction, and the spread of the updated ensemble helps quantify reconstruction uncertainty. A more detailed sketch of the algorithm is provided below.
Kalman filters require an estimate of the cross covariance between the observation records and the reconstructed state vector elements. In a classical Kalman filter, this estimate is calculated using the cross covariance of the observation estimates with the prior ensemble. However, there are a number of methods that attempt to improve assimilated reconstructions by adjusting these covariances. The kalmanFilter class supports several commonly used covariance adjustments, including: inflation, localization, blending, and directly setting covariance. More details on these methods are provided below.
The kalmanFilter class uses an ensemble square root method, which updates the ensemble mean separately from ensemble deviations. The class always returns the updated mean, and allows users to optionally return the deviations. However, updated deviations can quickly overwhelm computer memory when running a Kalman filter for multiple time steps. Thus, the kalmanFilter class also provides several alternatives to returning the full set of updated deviations. Options include returning ensemble variance, percentiles, and returning climate indices calculated from an updated spatial field.
To summarize, the kalmanFilter class provides objects that help implement the Kalman filter algorithm. These objects provide methods that allow users to provide data inputs (such as observations and a prior) to the filter, select algorithm parameters (such as covariance adjustments), request specific outputs (such as ensemble variance or percentiles), and run the Kalman filter algorithm.
Outline
The Following Is A Sketch For Using The Kalmanfilter Class
Use the “kalmanFilter” method to initialize a new kalmanFilter object
Use the “observations”, “estimates”, “prior”, and “uncertainties” methods to provide essential data inputs for the Kalman filter.
Optionally use the “inflate”, “localize”, “blend”, and “setCovariance” methods to implement covariance adjustments.
Optionally use the “deviations”, “variance”, “percentiles”, and “index” methods to return information about the updated ensemble deviations.
Use the “run” method to run the Kalman filter algorithm
Algorithm
The following is a sketch of the Kalman filter algorithm. For an assimilated time step, the method first decomposes the prior ensemble and the observation estimates into their ensemble means and ensemble deviations. The method then uses the cross covariance of the prior with the estimates to estimate the covariance between the observation records and the state vector elements. Next, the method implements any covariance inflation, localization, and blending. The covariance estimate is combined with 1. the observation uncertainties and 2. the covariance of the estimates with one another to give the standard Kalman Gain matrix. Next, the method determines the differences between the observations and the proxy estimates (the innovations). The innovations are propagated through the Kalman gain and used to update the ensemble mean. The method may optionally also computes the adjusted Kalman Gain matrix. The adjusted gain is combined with the deviations of the estimates in order to update the ensemble deviations. Finally, the method extracts any requested information (such as ensemble variance or percentiles) from the updated deviations.
Covariance Adjustments
The following is a brief summary of supported covariance adjustments.
Inflation: Inflation applies a multiplicative constant to the covariance estimate in order to increase the total covariance. This method is best used for online assimilations when the updated ensemble too closely resembles a single ensemble member. The inflation helps maintain variability in the updated ensemble by providing greater weight for the observation records.
Localization: Localization limits the influence of proxy observations on distant state vector elements. This can help reduce the influence of spurious covariances resulting from finite ensemble sizes. The method allows proxy observations to inform nearby state vector elements, but not distant sites.
Blending: Blending combines the covariance estimate with a second covariance estimate. This method is often used for evolving offline assimilations with small ensemble sizes. The second covariance is typically derived from a larger “climatological” ensemble. The method allows covariance estimates to partially evolve over time, while reducing spurious covariances that result from small ensemble sizes.
Directly setting covariance: In some cases, it may be desirable to directly specify the covariance between the observation sites and state vector elements. This is often used for deep time assimilations, when changing continental configurations result in NaN covariances at many different spatial sites.
Troubleshooting Large State Vectors
Large state vector ensembles can overwhelm computer memory and may prevent a Kalman filter assimilation from running. If this occurs, it is useful to note that the update for each state vector element is independent of all other state vector elements. Thus, you can often circumvent memory issues by splitting the state vector into several smaller pieces and then assimilating each piece individually. The built-in “matfile” command can be helpful for saving/loading pieces of large ensembles iteratively.
Key Methods
These methods are the most essential for users.
All User Methods
Create
Data Inputs
Covariance
Output Options
Run
Console Display
Utility Methods
Utility methods that help the class run. They do not implement error checking and are not intended for users.