Sensitivity analysis and constraint of the parameter space of Earth System configuration of JULES.

Uses global mean from 1994-2013 of several carbon-cycle outputs to rule out parts of input space.

Where did the model fail to run?

There are clear run failure thresholds in the parameters rootd_ft_io and lai_max, and quite strong visual indications that a_wl_io and bio_hum_cn matter.

Marginal histograms of run failures

Histograms of the output at Level 0 (places the model ran and produced output)

Test out some emulators

At the moment, we’ll just keep to the things that we know should work. We’ll use mean NPP as an example

Andy would like to see timeseries of: cVeg, cSoil and nbp, npp in GtC and GtC/yr.

First, how does NPP respond to each parameter? NAs are removed, but zero values are still included.

A clear threshold in the F0 parameter.

It appears that this ensemble is less “clear cut” in having an output that clearly distinguishes between “failed” (or close to it), and “not failed”.

Having said that, having an F0 over a threshold seems to kill the carbon cycle, as before. Here, we’ve set a threshold of 0.9 (on the normalised scale) for F0, and we remove members of the ensemble with a larger F0 than that when we build emulators.

A DiceKriging emulator for NPP

This emulator is a straight kriging modfel (km) trained with the level 0 data - this includes “zero carbon cycle” but not NAs. The leave-one-out cross validation plot clearly indicates that the emulator over-predicts the carbon cycle when it is very low. This might mean that when constraining, more of the input space is retained than strictly justified. It does suggest that such a constraint is conservative (i.e., it is unlikely that a candidate will be rejected without justifiaction.)

This next emulator uses an input dimension reduction technique (glmnet), shrinking regression coefficients of the more unimportant input variables towards zero, before building a kriging emulator with the retained inputs. This doesn’t (on the face of it) deal much better with the zero-output ensemble members.

Remove anything with f0 over a threshold (level 1 constraint) .

The level 1 constraint removes any input with F0 greater than 0.9 (normalised), which removes many of the zero-carbon-cycle members up front. There are 424 ensmble members remaining.

Plot the regular km emulator.

Plot the “twostep” glmnet/km emulator for the level 1 constraint.

A one-at-a-time sensitivity analysis of the “sum” output

Heatmaps of one-at-a-time sensitivity analysis across a number of variables

How good are the emulators for each output?

The emulators appear to be at least capturing the broad response for all of the output variables.

First, plot the straight kriging emulators

Next, plot the twostep glmnet/km emulators

Constrain input space with a small number of observations

We use thresholds of tolerance from Andy on ‘nbp_lnd_sum’, ‘npp_nlim_lnd_sum’, ‘cSoil_lnd_sum’, ‘cVeg_lnd_sum’ as basic initial constraints on the input space.

Ensemble members that comply with constraints in NPP, NBP, soil and vegetation carbon.

Build a two-step glmnet/km emulator for each output that we have constraints for, and find the input space where those constraints are met.

## [1] "NROY space proportion (%) = 10.082"

Pairs plot of input space that is Not Ruled Out Yet when basic constraints are applied to annual data.

Marginal histograms of space that is Not Ruled Out Yet when basic constraints are applied.

Monte Carlo Filtering for sensitivity analysis.

Monte Carlo filtering (MCF) gives another form of sensitivity metric. MCF splits the input sample into two parts - those that do and do not meet some threshold, for example, and then examines the differences of the resulting parameter distributions. A useful guide to MCF can be found in [Pianosi et al (2016)]https://www.sciencedirect.com/science/article/pii/S1364815216300287

We use the emulated input samples that are ruled out and NROY with the initial constraint.