“Is this model calibrated?” is a question I don’t feel comfortable answering with a yes or no answer, because a yes/no answer is inherently arbitrary.
Mathematician Gorge Box is quoted as saying “All models are wrong, but some are still useful.”
All hydraulic models lie on a spectrum somewhere between perfect and terrible. Even a poorly calibrated model can provide some insights while no model can ever be perfect. There is just too much uncertainty in hydraulic systems.
The real question that should be asked is, “Is this model sufficiently accurate for my purpose?”
The answer to that question depends on the purpose of the model and the quality of the data used to judge the model.
A model may be great for modeling dry weather sewer flow for a new subdivision but terrible for wet weather flow in an old neighborhood with poor pipe joints and basement sump pumps. A model may be great for predicting average pressure in the central business district but terrible for predicting water quality at a remote location.
The problem lies in developing a standard numerical criterion for judging a model. A discrepancy (notice I didn’t say “error”) between the model and field data is that any such number is a somewhat arbitrary judgement decision. You can’t drive a numerical criterion for calibration acceptance from F=ma.
Another issue lies in the personas who are asking and answering the question. Usually the decision maker (i.e. design engineer or system operator) will ask the modeler, “Is this model calibrated?,” and expect the modeler to answer yes or no. What should happen is that the modeler should show the work that was done to calibrate the model to the decision maker and ask, “Do you think the model will meet your needs?”
A good test of the adequacy of calibration of the model occurs when the distribution superintendent, like my old buddy Tony Gangemi, calls me and asks “What does our model say about this?” The fact that he calls is evidence that the model is somewhat useful.
People have been grappling with the question of model acceptance standards for many years and have come up with arbitrary values like, “Model and field pressures must agree to within 2 psi at 90% of nodes tested.” This is like saying “Is a 3 ft string long enough?”
It is all right to have “target” (not standards) to judge the adequacy of a model but these need to be customized for each model use and with the realization that it is somewhat arbitrary.
If the model and field data disagree by 15 psi at a typical node on an average day, the modeler needs to determine why. (Hint: It’s most likely NOT the C-factor.)
What is the real rule that should be used to judge model calibration? It’s this: The model should be considered adequately calibrated for a given purpose when the cost of additional calibration exceeds the benefit from that additional work. Yes, this is also an arbitrary rule, but it gets to the heart of the problem.
The issue is further complicated by the fact that water and wastewater systems are not static. They are constantly evolving on multiple time scales as pumps are taken offline for maintenance or a new large pipe has been installed.
So, if somebody asks you “Is this model calibrated?,” your best answer may be “What do you think?”
If you want to read more about this topic, you may want to check out.
Walski, T., 2019, “Why Global Standards for Calibration of Water Distribution Models Won’t Work,” Journal AWWA, Vol. 115, No. 5, p. 16
Want to learn more from our resident water and wastewater expert? Join the Dr. Tom Walski Newsletter today!
