Visual MODFLOW Flex 2012.2 now supports PEST with pilot point parameterization. This article provides a brief overview of the key concepts of PEST, how they are integrated in Visual MODFLOW Flex groundwater modeling software, and links to valuable resources on parameter estimation, automated model calibration and uncertainty analysis.
See the new PEST Workflow in action in this short video:
Learn how the new release streamlines model calibration:
Introduction
All groundwater modeling is uncertain! There is uncertainty in the data, the hydrogeologic conceptualization, and the simulated results. Groundwater modelers need some method of assessing this uncertainty, so that they can develop the best model calibration, and build defensible models. It is good to assess model calibration manually, as this provides you with a good understanding of model sensitivity/uncertainty. It is also good to have automated tools available for all parts of the calibration process. This is what PEST is for: it is an automated parameter estimation tool, which provides a sensitivity and uncertainty analysis of the model, and much more.
Model Calibration
Model calibration involves finding a set of parameters and boundary conditions, that match historical field-measured values within a pre-established range of error. Groundwater models are most often used to predict future groundwater conditions. If a model is calibrated to a set of historical observations, the modeler hypothesizes that the model can be used to make predictions about future conditions. Therefore, calibration is a necessary part of the modeling process that enables predictive simulation. Model calibration should be done manually, through trial-and-error, but it is much easier to do with the help of PEST.
Manual vs. Automated Calibration
There are some pros and cons for both approaches to model calibration.
Manual calibration is the traditional approach, commonly used by all modelers. The upside is that it makes use of the modeler’s intuition about the conceptual model (distribution and value of input parameters and boundaries). The downside is that it is difficult to eliminate bias and consider all the angles or possibilities.
The upside to using automated calibration methods is that you can consider many more possibilities and give quantitative insights into the model; the powerful numerical algorithms do all the number-crunching and grunt work. The downside, however, is that there is no intuition and no guarantee that the result from PEST are reasonable. This means that it is important to define reasonable bounds for the parameters, and understand what settings, parameter values (and ranges) are appropriate for your problem type.
Calibration Residuals
The calibration residual is the basis of error statistics in model calibration. The figure below show the definition of the residual, for calculated and observed groundwater heads.
The residual is defined as:
PEST works to minimize the residuals in your model. The Objective Function (phi) is defined as follows:
where wi is a weighting factor assigned to each observation
PEST’s goal is to find the minimum value for the objective function. PEST tracks its own progress by following whether the phi is increasing or decreasing.
About PEST
PEST stands for parameter estimation; it is a non-linear inverse modeling code, developed by John Doherty, Watermark Numerical Computing. Generally, it is often easier in hydrogeology to measure model output (heads, fluxes, concentrations) than it is to measure input parameters (hydraulic conductivity, storativity, porosity, etc.). As a result, we try to infer system inputs, or parameters, from system outputs (results). This is called the “inverse problem”. PEST uses a powerful mathematical technique to estimate a new set of parameters by comparing model results to a set of observations, and provides valuable insight into the conceptual model. PEST is model-independent and is a public domain code. It can be used with any numerical simulator. PEST is well-documented and supported. For more details, visit http://www.pesthomepage.org.
When should I use PEST?
There are several important places in the modeling process where PEST is really useful:
- at the beginning of the calibration process, when the model is first developed and a gross calibration to the hydrogeologic conceptualization is needed
- at the end of the calibration process when “tweaking” the model to obtain the “best” set of model parameters
- during sensitivity analysis and uncertainty analysis to understand the implications of model input uncertainty on the predicted results
What does PEST Do?
- PEST provides you with a set of parameter values that result in a minimum objective function – based on the parameters you have chosen, and the objective function that you have defined.
- PEST provides valuable information on model calibration, and insight into the validity of the conceptual model
- sensitivities (composite and individual)
- correlation between parameters
- However, PEST cannot calibrate your model and PEST does not guarantee the reasonableness of the results. The results can have no bearing on physical reality; if this happens, it’s a strong indicator that the conceptual model is incorrect … this is very valuable information!
How does a PEST Run work?
The following flow chart shows the basics of how a PEST run works.
What do I need to define for a PEST Run?
PEST requires several input files, which Visual MODFLOW will help you to prepare:
- Template file(s): contains a list of what parameters will be adjusted (eg. Conductivity or Storativity parameter zones, Boundary condition attributes, etc.), and how to write the input file for the model
- Instruction file(s): contains the observations, and how to collect MODFLOW simulated equivalents (heads or fluxes) from the model
- Control File: is the master file, contains several parts:
- * control data (# iterations, obj function goal, etc..)
- * parameter data, groups (including pilot points)
- * observation data, groups
- * model command line
- * model input/output
- * prior information
Please refer to the PEST Manual for a full description of these files. There are also several intermediate input/output files that are generated through the use of PEST utilities which are not discussed in this article.
PEST Workflow
Visual MODFLOW Flex provides a logical, intuitive interface for PEST that guides you through the automated model calibration steps; this is done through the use of a workflow. The PEST workflow is shown in the figure below:
Each step in the workflow has a corresponding GUI, with which you interact. Similar to a wizard, you just start at the first step and proceed to the next step when you are finished, or go back to the previous step at any time. Once you have gone through the workflow once, you can freely select any specific step from the workflow tree and go directly to that step.
What are Pilot Points?
Pilot points are locations within the model domain where PEST will estimate a new parameter value (eg. Kx). These can be strategically-placed points with a default initial value (soft points), or field measurements (field samples/tests of Kx, so-called “Fixed” points).
PEST estimates a parameter value at each pilot point, then interpolates to the MODFLOW grid cells using Kriging.
“Fixed” point values are not adjusted (act like control points). The figure below shows Fixed pilot points (red symbols) that represent field measurements for Kx. The black markers represent soft pilot points.
Several guidelines for defining pilot points are available on the PEST HomePage.
Kriging and Variograms
Kriging is spatial interpolation method based on geostatistics. A fundamental part of Kriging is the definition of a Variogram. In a nutshell, the variogram describes the spatial continuity or roughness of a data set. PEST treats each parameter zone (eg. Kx Zone1) as a structure. Each structure can have a unique variogram and Kriging parameters, allowing you to account for different geological characteristics and heterogeneity at your site. Prior to the PEST run, a set of Kriging factors (multipliers) are defined for each grid cell; since these factors are independent of the actual cell values, they need to be set only once. It is not necessary to re-calculate new Kriging factors for each new MODFLOW run, and this saves execution time during a PEST run. The figure below shows a distribution of Kx values for the MODFLOW grid cells, using the values adjusted by PEST.
What are the Benefits of Using Pilot Points vs. the Classical Zone Based Approach?
A Zone Based Approach
With the zone-based approach, each zone is treated as a parameter. PEST adjusts the values until good fit is achieved. The results give an indication of where new zones should be added. You adjust the zonation as needed, then run PEST again. This was generally laborious and slow, and subjective as to where new zones should be added.
Pilot Points Approach
With Pilot Points approach, you can include as many points as needed for each parameter zone.
The variogram can be different in each zone,
reflecting differences in the geology
or in the level of heterogeneity, in each zone. Regularization techniques allow for faster, stable solutions. Since you can add more points, this generally yields, better, more representative results from PEST
What is Regularization?
For PEST to be effective, you need to have a lot of pilot points. However, adding more points (ie. PEST parameters), comes at a cost:
- Non-unique solutions
- Numerical instability
- Longer run times (Remember, a pilot point is a parameter, and PEST must run MODFLOW once for each parameter!)
As you add more pilot point parameters, you can easily end up in the scenario where you have more parameters for which you have observations; this breaks one of the “golden rules” of PEST which is that the Number of Parameters should be less than the Number of Observations.
The solution is a technique called regularization. Regularization adds prior information which act like additional observations, and helps to stabilize the solution, and thus find a solution to the “ill-posed” problem. PEST has two types of regularization: Tikhonov and SVD (Singular Value Decomposition) Assist.
Tikhonov Regularization
Tikhonov Regularization works by adding prior information that defines relations between the parameters (and their values). There are two types of Tikhonov Regularization:
- Preferred Homogeneity: The prior information equations relate the pilot points to one another. Pilot points that are close to each other, should be homogenous. PEST will only introduce enough heterogeneity as necessary.
- Preferred Value: The prior information equations relate the pilot points to their initial value. PEST will try to estimate a value that is as close as possible to the initial pilot point value
Typically it is recommended to run Tikhonov Regularization in conjunction with SVD-Assist.
SVD Assist (Singular Value Decomposition)
Singular Value Decomposition is a method of Subspace regularization. It reduces the number of parameters (or parameter combinations) from the PEST problem. PEST will run a sensitivity analysis to determine which parameters should be “estimated”, and combines these into a sub-set of parameters called super parameters. The remaining parameters that are insensitive, (and thus should not be estimated) are dropped from the PEST run. The number of super parameters is user-defined, and is typically much less than total # parameters.
Why should I Use Regularization?
Simply put, without the use of regularization, you will face very long PEST run times, and will not be able to use a reasonable appropriate number of pilot points to adequately represent the heterogeneity in your model.
Consider this example.
- Your MODFLOW model takes 10 minutes to converge
- You have defined 50 pilot points in your model
- You have defined 20 PEST Optimization Iterations.
Each PEST optimization iteration consists of PEST running MODFLOW once for each parameter (pilot point). The scenario above would require 10,000 minutes to complete (almost 7 days!). Introducing prior information will significantly reduce the PEST run times by improving the model stability. (Hopefully you can also see the benefits using PEST’s parallel/distributed processing capabilities, which will be covered in a separate blog post).
Regularization allows for a more natural approach to calibration; the estimated parameters for a given zone will remain as uniform as possible; heterogeneity is introduced “only because it’s necessary”. The PEST adjusted values should more closely match your original model.
Lastly, by using regularization, you can use more pilot points. Your model can have hundreds or thousands of pilot points; the number of super parameters can be much less (tens). This results in a better fit between model outcomes and field observations.
Interpreting PEST Results
PEST generates several output files:
- Run Record (.REC): Objective function history, Parameters history, Statistics for residuals, groups, correlation…
- Residuals (.RES): Calculated and Observed values for observations (including Weighted) and residuals
- Sensitivities – Parameters (.SEN)
- Sensitivities – Observations (.SEO)
There are also several binary files that are generated, which can be converted into readable formats with the help of some PEST utilities.
In Visual MODFLOW Flex, you can have direct access to the PEST output files in the workflow interface. Click on each tab and you can see the corresponding PEST results.
When interpreting the results, there are a few things you want to look for:
- Calculated vs. Observed heads: have you achieved a better fit with the help of PEST?
- Parameter sensitivities: which pilot point parameters had the highest sensitivities? Which ones had the least?
- Estimated parameter values: are the values reasonable? Do they reflect what you know to be true, based on your knowledge of the site? The adjusted parameter values can also give an indication as to areas where you may need to collect more data (for both parameters and observations). If parameter values are unrealistic, they may give an indication as to a problem with the conceptual model.
Export PEST Results to Excel
Using Visual MODFLOW Flex, the various PEST results can be exported to Excel in a chart-ready format. This allows you to create professional, analytical reports with just a few clicks:
Saving PEST Results and Working with Multiple Models
Typically after a PEST run is complete, you will want to save the adjusted parameters as inputs for a new model run, and run the updated simulation and analyze the results. The multi-model environment of Visual MODFLOW Flex streamlines this process by allowing you to generate, view, and run multiple models within the same project. This allows you to compare results and property distributions between several models (original model and updated model with PEST parameters), and keep track of the changes in versions for your model. This also makes it easier to experiment and evaluate several PEST run options (eg. with regularization, SVD-assist, adjusting weights for observations, etc.) in order to find the most suitable settings for your project. The figure below shows the property distribution for two numerical models: the original model (on the left) and the updated PEST-adjusted model on the right.
References and Additional Resources
The following are links to websites and publications on PEST and MODFLOW
- PEST HomePage
- PEST Users Manual and Addendum to PEST Users Manual
- Approaches to Highly Parameterized Inversion: a Guide to Using PEST for Groundwater Model Calibration: http://pubs.usgs.gov/sir/2010/5169/
- Approaches to Highly Parameterized Inversion: a Guide to Using PEST for Model Parameter and Predictive Uncertainty Analysis: http://pubs.usgs.gov/sir/2010/5211/
- Approaches to Highly Parameterized Inversion: Pilot Point Theory, Guidelines and Research Directions http://pubs.usgs.gov/sir/2010/5168/
- Using Pilot Points to Calibrate a MODFLOW/MT3D Model, by John Doherty Watermark Numerical Computing