August 15, 2013
Uncertainty and Unknown Unknowns
At today’s inaugural ‘Catastrophe Risk Management & Modelling Australasia 2013’ in Sydney, the focus is on model uncertainty, unknown unknowns and best practice model usage in the context of these uncertainties.
As I have observed many times, every catastrophe is the “perfect storm” and the one common factor of all catastrophes is they are all unique. Best practice is looking beyond the models and having a strong sense of “plausible impossibilities”.
We must also make sure we do not forget lessons that are learned in the past, for example the importance of completeness and accuracy of data, and making sure that you understand the policy terms, for example sum insured or paying out for replacement costs. In the case of New Zealand, replacement must be to the latest building codes.
One key question today has been whether the Christchurch earthquake could occur under a big Australian city. An earthquake of the same magnitude of the Lyttleton earthquake is certainly possible, but the soil types are quite different.
As described in Robert Muir-Wood’s previous blog on ultra-liquefaction, one of the key characteristics of ultra-liquefiable soils is that they are glacially deposited; fortunately something that Australia, and even other cities in New Zealand such as Wellington do not have to the same extent as Christchurch. However, other potential surprises may occur, such as landslides in Wellington.
The earthquakes of 2011 are clearly an opportunity to learn and improve our models, but we all need to embrace the fact that there will continue to be sources of surprise – ‘unknown unknowns’ are called that for a reason.
Science and knowledge is always evolving. Best practice today will change tomorrow, just like in sports as diverse as rugby union or the Americas cup, where technology, training practices and even clothing was unimaginable 10 years ago. Our best understanding today will certainly change in the future.
However this does not make models irrelevant. My favorite quote is a play on General Eisenhower’s statement that “In preparing for battle, I have always found that plans are useless but planning is indispensable.” I would say, “all models are wrong, but modeling is indispensable”.
Modeling allows users to develop understanding of the models’ strengths and weaknesses, validate with whatever information is available, assess the methodologies and assumptions used, and decide what they are more comfortable with. In addition, users should consider stress tests and scenarios to further increase their intuition and knowledge of the risk potential.
In 2011, I was part of a working group in London, which produced the very useful report “Industry good practice for catastrophe modeling: A guide to managing catastrophe models as part of an internal model under Solvency II“.
Whilst written in Europe, the principles in this paper are applicable globally in all regions subject to all perils. As Australasia’s risks increase, together with regulatory interest in catastrophe modeling, this paper will continue to provide guidance and advice to all those involved in using catastrophe models to understand and manage their risk.…
