Emulation tests for MetaWards

Model description (parameters etc. ) are here

Based on Code from James Salter.

Code for this notebook is here

Ensemble design

Cumulative hospital deaths at the start of lockdown against each parameter

Cumulative community deaths at the start of lockdown against each parameter

Emulating cumulative deaths in hostpital (cumHD) in the community (cumCD) at the beginning of lockdown

The DiceKriging Emulator fit for log(deaths) looks good. Leave-one-out isn’t a great measure, but it’s a start.

Week 12 cumulative Hospital deaths Leave-one-out plots

Week 12 cumulative community deaths Leave-one-out plots

Normalise input space and re-fit to produce some sensitivity analyses

one-at-a-time sensitivity for cumulative hospital deaths

one-at-a-time sensitivity for cumulative community deaths

FAST99 sensitivity analysis of Saltelli et al (1999)


Call:
fast99(model = NULL, factors = colnames(X_norm), n = 3000, q = "qunif",     q.arg = list(min = 0, max = 1))

Model runs: 42000 

Estimations of the indices:
          first order total order
R0       0.0357894898 0.052647577
TE       0.0094237203 0.015631583
TP       0.0111223776 0.016713101
TI1      0.0042119359 0.007605358
TI2      0.0005845405 0.002777098
nuA      0.0017362075 0.004688977
ns       0.0040062589 0.006783809
alphaTH  0.0108279236 0.012842142
etaTH    0.0167128822 0.019462817
alphaEP  0.0702844368 0.072592994
alphaI1D 0.0018994493 0.004351074
alphaHD  0.6488963359 0.676379829
alphaI1H 0.0509419175 0.053030910
eta      0.0994782982 0.101617848


Call:
fast99(model = NULL, factors = colnames(X_norm), n = 3000, q = "qunif",     q.arg = list(min = 0, max = 1))

Model runs: 42000 

Estimations of the indices:
          first order  total order
R0       3.707727e-02 0.0488105154
TE       1.159102e-02 0.0140520108
TP       1.114574e-02 0.0133480137
TI1      4.404289e-03 0.0053919290
TI2      7.830393e-05 0.0008514642
nuA      2.755183e-03 0.0039733774
ns       2.318849e-03 0.0036564024
alphaTH  1.607502e-05 0.0005314890
etaTH    1.100540e-05 0.0004947353
alphaEP  5.548157e-02 0.0565196581
alphaI1D 8.110668e-01 0.8287016527
alphaHD  8.497626e-05 0.0011878537
alphaI1H 4.147824e-04 0.0033935812
eta      4.080266e-02 0.0419101180

LS0tCnRpdGxlOiAiTWV0YVdhcmRzIHdhdmUgMDMgcHJlLWxvY2tkb3duIGVtdWxhdGlvbiB0ZXN0IgpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sKLS0tCgoKIyMgRW11bGF0aW9uIHRlc3RzIGZvciBNZXRhV2FyZHMKCk1vZGVsIGRlc2NyaXB0aW9uIChwYXJhbWV0ZXJzIGV0Yy4gKSBhcmUgW2hlcmVdKGh0dHBzOi8vdXE0Y292aWQuZ2l0aHViLmlvL3ZpZ25ldHRlcy9kYXRhL01ldGFXYXJkcy92aWduZXR0ZS9Nb2RlbERlc2NyaXB0aW9uKSAgCgpCYXNlZCBvbiBDb2RlIGZyb20gSmFtZXMgU2FsdGVyLgoKQ29kZSBmb3IgdGhpcyBub3RlYm9vayBpcyBbaGVyZV0oaHR0cHM6Ly9naXRodWIuY29tL2RvdWdtY25lYWxsL2NvdmlkL2Jsb2IvbWFzdGVyL2phbWVzL3dhdmVfMDNfdGVzdC5SbWQpCgpgYGB7ciBzZXR1cCwgaW5jbHVkZT1GQUxTRX0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KGVjaG8gPSBGQUxTRSwgbWVzc2FnZSA9IEZBTFNFLCBlcnJvciA9IEZBTFNFKQpgYGAKCmBgYHtyLCBlY2hvID0gRkFMU0UsIHdhcm5pbmcgPSBGQUxTRSwgbWVzc2FnZSA9IEZBTFNFfQpsaWJyYXJ5KFJTUUxpdGUpCmxpYnJhcnkoZmllbGRzKQpsaWJyYXJ5KHZpcmlkaXMpCmxpYnJhcnkocmdkYWwpCmxpYnJhcnkoZHBseXIpCmxpYnJhcnkocmVzaGFwZTIpCmxpYnJhcnkoZ2dwbG90MikKbGlicmFyeShjb3dwbG90KQpsaWJyYXJ5KERpY2VLcmlnaW5nKQpsaWJyYXJ5KGVtdG9vbHMpCmxpYnJhcnkoaW1wdG9vbHMpCmxpYnJhcnkodml6dG9vbHMpCgpgYGAKCgpgYGB7cn0KIyMjIyBMb2FkIGluIGRhdGFiYXNlICMjIyMKZmlsZW5hbWUgPC0gImRhdGEvd2F2ZTAzX3N1bW1hcmllc19EVy5kYiIgIyBsaWtlbHkganVzdCBjYWxsZWQgd2F2ZV9zdW1tYXJpZXNfRFcuZGIgd2hlbiBkb3dubG9hZGVkLCBJJ3ZlIHJlbmFtZWQKc3FsaXRlLmRyaXZlciA8LSBkYkRyaXZlcigiU1FMaXRlIikKZGIgPC0gZGJDb25uZWN0KHNxbGl0ZS5kcml2ZXIsCiAgICAgICAgICAgICAgICBkYm5hbWUgPSBmaWxlbmFtZSkKbmV3X3dhdmUxX3NlZWRzIDwtIGRiUmVhZFRhYmxlKGRiLCJjb21wYWN0IikKCiMgSWYgaW5zdGVhZCBoYXZlIHdhcmQtbGV2ZWwgc3VtbWFyaWVzCiNmaWxlbmFtZSA8LSAiZGF0YS9zdW1tYXJpZXNfZGVmYXVsdC5kYiIKI3NxbGl0ZS5kcml2ZXIgPC0gZGJEcml2ZXIoIlNRTGl0ZSIpCiNkYiA8LSBkYkNvbm5lY3Qoc3FsaXRlLmRyaXZlciwKIyAgICAgICAgICAgICAgICBkYm5hbWUgPSBmaWxlbmFtZSkKI2RiTGlzdFRhYmxlcyhkYikKCmBgYAoKYGBge3IsIGV2YWwgPSBGQUxTRX0KCiMgbmV3dGFibGUgPC0gZGJHZXRRdWVyeShkYiwgIlNFTEVDVCAqIEZST00gY29tcGFjdCBXSEVSRSB3ZWVrID0gMTIgQU5EIGFnZSA9IDgiKQojIAojICMgRG9uJ3QgY2FyZSBhYm91dCB0aGlzIGNvbHVtbgojIG5ld193YXZlMV9zZWVkcyRIcHJldl9tbiA8LSBOVUxMCiMgCiMgIyBTdW0gZGVhdGhzCiMgbmV3X3dhdmUxX3NlZWRzJGRlYXRocyA8LSBuZXdfd2F2ZTFfc2VlZHMkY3VtSEQgKyBuZXdfd2F2ZTFfc2VlZHMkY3VtQ0QKIyAKIyAjIyMjIEFnZ3JlZ2F0ZSBmb3IgZWFjaCB3ZWVrICMjIyMKIyBFeHRyYWN0V2VlayA8LSBmdW5jdGlvbihhbGxfZGF0YSwgd2Vla3MgPSBjKDEyOjYpKXsKIyAgIGV4dHJhY3RlZF9kYXRhIDwtIE5VTEwKIyAgIGZvciAodyBpbiAxOmxlbmd0aCh3ZWVrcykpewojICAgICAjIEZpcnN0IGV4dHJhY3QgYWxsIHRob3NlIHdpdGggcmVwbGljYXRlcyAoTkEgZWxzZSkKIyAgICAgdG1wX2RhdGEgPC0gc3Vic2V0KGFsbF9kYXRhLCB3ZWVrID09IHdlZWtzW3ddICYgIShpcy5uYShyZXBsaWNhdGUpKSkKIyAgICAgdG1wX2RhdGEgPC0gYWdncmVnYXRlKGNiaW5kKGN1bUgsY3VtSEQsY3VtQ0QsZGVhdGhzKSB+IG91dHB1dCArIHJlcGxpY2F0ZSwgZGF0YSA9IHRtcF9kYXRhLCBGVU4gPSBzdW0pICMgYWdncmVnYXRlIGFjcm9zcyBhZ2UgYW5kIHRydXN0CiMgICAgIHRtcF9kYXRhX3JlcHMgPC0gYWdncmVnYXRlKGNiaW5kKGN1bUgsY3VtSEQsY3VtQ0QsZGVhdGhzKSB+IG91dHB1dCwgZGF0YSA9IHRtcF9kYXRhLCBGVU4gPSBtZWRpYW4pICMgdGFrZSBtZWRpYW4gYWNyb3NzIHJlcGxpY2F0ZXMKIyAgICAgCiMgICAgICMgUmVwZWF0IGZvciBzaW5nbGUgcnVucyAoTkFzKQojICAgICB0bXBfZGF0YSA8LSBzdWJzZXQoYWxsX2RhdGEsIHdlZWsgPT0gd2Vla3Nbd10gJiAoaXMubmEocmVwbGljYXRlKSkpCiMgICAgIHRtcF9kYXRhX3VuaXF1ZSA8LSBhZ2dyZWdhdGUoY2JpbmQoY3VtSCxjdW1IRCxjdW1DRCxkZWF0aHMpIH4gb3V0cHV0LCBkYXRhID0gdG1wX2RhdGEsIEZVTiA9IHN1bSkgIyBhZ2dyZWdhdGUgYWNyb3NzIHRydXN0cwojICAgICAKIyAgICAgIyBDb21iaW5lCiMgICAgIHRtcF9kYXRhIDwtIHJiaW5kKHRtcF9kYXRhX3JlcHMsIHRtcF9kYXRhX3VuaXF1ZSkKIyAgICAgdG1wX2RhdGEkd2VlayA8LSB3ZWVrc1t3XQojICAgICAKIyAgICAgZXh0cmFjdGVkX2RhdGEgPC0gcmJpbmQoZXh0cmFjdGVkX2RhdGEsIHRtcF9kYXRhKQojICAgfQojICAgCiMgICByZXR1cm4oZXh0cmFjdGVkX2RhdGEpCiMgfQojIAojIGFsbF93ZWVrX3NlZWRzIDwtIEV4dHJhY3RXZWVrKG5ld193YXZlMV9zZWVkcywgd2Vla3MgPSBjKDEyOjYpKQojIHNhdmUoYWxsX3dlZWtfc2VlZHMsIGZpbGUgPSAnZGF0YS9hbGxfd2Vla19zZWVkcy5SRGF0YScpCiMgCiMgCiMgIyMjIyBBZ2dyZWdhdGUgZmluYWwgd2VlayBieSBhZ2UgIyMjIwojIEV4dHJhY3RBZ2UgPC0gZnVuY3Rpb24oYWxsX2RhdGEpewojICAgIyBGaXJzdCBleHRyYWN0IGFsbCB0aG9zZSB3aXRoIHJlcGxpY2F0ZXMgKE5BIGVsc2UpCiMgICB0bXBfZGF0YSA8LSBzdWJzZXQoYWxsX2RhdGEsIHdlZWsgPT0gMTIgJiAhKGlzLm5hKHJlcGxpY2F0ZSkpKQojICAgdG1wX2RhdGEgPC0gYWdncmVnYXRlKGNiaW5kKGN1bUgsY3VtSEQsY3VtQ0QsZGVhdGhzKSB+IG91dHB1dCArIGFnZSArIHJlcGxpY2F0ZSwgZGF0YSA9IHRtcF9kYXRhLCBGVU4gPSBzdW0pICMgYWdncmVnYXRlIGFjcm9zcyB0cnVzdAojICAgdG1wX2RhdGFfcmVwcyA8LSBhZ2dyZWdhdGUoY2JpbmQoY3VtSCxjdW1IRCxjdW1DRCxkZWF0aHMpIH4gb3V0cHV0ICsgYWdlLCBkYXRhID0gdG1wX2RhdGEsIEZVTiA9IG1lZGlhbikgIyB0YWtlIG1lZGlhbiBhY3Jvc3MgcmVwbGljYXRlcyAgIAojICAgIyBSZXBlYXQgZm9yIHNpbmdsZSBydW5zIChOQXMpCiMgICB0bXBfZGF0YSA8LSBzdWJzZXQoYWxsX2RhdGEsIHdlZWsgPT0gMTIgJiAoaXMubmEocmVwbGljYXRlKSkpCiMgICB0bXBfZGF0YV91bmlxdWUgPC0gYWdncmVnYXRlKGNiaW5kKGN1bUgsY3VtSEQsY3VtQ0QsZGVhdGhzKSB+IG91dHB1dCArIGFnZSwgZGF0YSA9IHRtcF9kYXRhLCBGVU4gPSBzdW0pICMgYWdncmVnYXRlIGFjcm9zcyB0cnVzdHMKIyAgIAojICAgIyBDb21iaW5lCiMgICB0bXBfZGF0YSA8LSByYmluZCh0bXBfZGF0YV9yZXBzLCB0bXBfZGF0YV91bmlxdWUpCiMgICByZXR1cm4odG1wX2RhdGEpCiMgfQojIAojIHdlZWsxMmFnZV9zZWVkcyA8LSBFeHRyYWN0QWdlKG5ld193YXZlMV9zZWVkcykKIyBzYXZlKHdlZWsxMmFnZV9zZWVkcywgZmlsZSA9ICdkYXRhL3dlZWsxMmFnZV9zZWVkcy5SRGF0YScpCmBgYAoKYGBge3J9CgoKbG9hZCgnZGF0YS9hbGxfd2Vla19zZWVkcy5SRGF0YScpICMgZGF0YWZyYW1lIG9mIHdlZWtseSBjdW11bGF0aXZlIHRvdGFscwpsb2FkKCdkYXRhL3dlZWsxMmFnZV9zZWVkcy5SRGF0YScpICMgZGF0YWZyYW1lIG9mIHdlZWsgMTIgdG90YWxzIGJ5IGFnZQphbGxfd2VlazEyIDwtIHN1YnNldChhbGxfd2Vla19zZWVkcywgd2VlayA9PSAxMikgIyB3ZWVrIDEyIGN1bXVsYXRpdmUgdG90YWxzIG9ubHkKCiMjIyMgTG9hZCBpbiBkZXNpZ24gIyMjIwpOZXdEZXNpZ24gPC0gcmVhZC5jc3YoJ2RhdGEvd2F2ZTAzaW5wdXRzLmNzdicpICMgYWdhaW4sIG1pZ2h0IGJlIGNhbGxlZCBzb21ldGhpbmcgZWxzZQoKIyBNZXJnZSBkZXNpZ24gYW5kIG1vZGVsIG91dHB1dApDb21iaW5lZERhdGEgPC0gYWxsX3dlZWtfc2VlZHMgJT4lIGxlZnRfam9pbihOZXdEZXNpZ24sIGJ5ID0gYygnb3V0cHV0JyA9ICdvdXRwdXQnKSkKCiMgTG9hZCBjc3YgZmlsZSBvZiBMQUQgZGVhdGhzLCB0cnVzdCBhZG1pc3Npb25zIChub3QgcHVibGljKQojb2JzX2RlYXRocyA8LSByZWFkLmNzdignZGF0YS9mb3JUSi1sYWQtZGVhdGhzLWJ5LWFnZS0yMDIxLTAzLTIyLmNzdicpICMgY29uc2lzdGVudCwgZGVwZW5kaW5nIG9uIDI4IGRheSB2cyBjZXJ0aWZpY2F0ZSBkZWZpbml0aW9uCiN0cnVzdF9hZG1pc3Npb25zIDwtIHJlYWQuY3N2KCdkYXRhL2ZvclRKLXRydXN0LWFkbWlzc2lvbnMtYnktYWdlLTIwMjEtMDMtMjIuY3N2JykKCiMgRG93bmxvYWQgdmFyaW91cyBQSEUgZGF0YXNldHMKcGhlX2RlYXRocyA8LSByZWFkLmNzdignaHR0cHM6Ly9hcGkuY29yb25hdmlydXMuZGF0YS5nb3YudWsvdjIvZGF0YT9hcmVhVHlwZT1uYXRpb24mbWV0cmljPWN1bURlYXRoczI4RGF5c0J5RGVhdGhEYXRlJm1ldHJpYz1uZXdEZWF0aHMyOERheXNCeURlYXRoRGF0ZSZmb3JtYXQ9Y3N2JykKcGhlX2RlYXRocyA8LSBzdWJzZXQocGhlX2RlYXRocywgYXJlYU5hbWUgPT0gJ0VuZ2xhbmQnKSAjIHByb2JibHkgd2FudCBXYWxlcyBhcyB3ZWxsIHJlYWxseQoKcGhlX2RlYXRoc19jZXJ0IDwtIHJlYWQuY3N2KCdodHRwczovL2FwaS5jb3JvbmF2aXJ1cy5kYXRhLmdvdi51ay92Mi9kYXRhP2FyZWFUeXBlPW5hdGlvbiZtZXRyaWM9bmV3RGFpbHlOc29EZWF0aHNCeURlYXRoRGF0ZSZtZXRyaWM9Y3VtRGFpbHlOc29EZWF0aHNCeURlYXRoRGF0ZSZmb3JtYXQ9Y3N2JykKcGhlX2RlYXRoc19jZXJ0IDwtIHN1YnNldChwaGVfZGVhdGhzX2NlcnQsIGFyZWFOYW1lID09ICdFbmdsYW5kJykgIyBhZ2FpbiwgbmVlZCBXYWxlcyBhcyB3ZWxsIHdoZW4gdGhpcyBpc24ndCBzdWNoIGEgc21hbGwgbnVtYmVyCgpwaGVfZGF0YSA8LSByZWFkLmNzdignaHR0cHM6Ly9hcGkuY29yb25hdmlydXMuZGF0YS5nb3YudWsvdjIvZGF0YT9hcmVhVHlwZT1uYXRpb24mbWV0cmljPW5ld0FkbWlzc2lvbnMmbWV0cmljPWN1bUFkbWlzc2lvbnMmZm9ybWF0PWNzdicpCnBoZV9kYXRhIDwtIHN1YnNldChwaGVfZGF0YSwgYXJlYU5hbWUgPT0gJ0VuZ2xhbmQnKQoKcGhlX2FkbWlzc2lvbnMgPC0gcmVhZC5jc3YoJ2h0dHBzOi8vYXBpLmNvcm9uYXZpcnVzLmRhdGEuZ292LnVrL3YyL2RhdGE/YXJlYVR5cGU9bmF0aW9uJmFyZWFDb2RlPUU5MjAwMDAwMSZtZXRyaWM9bmV3QWRtaXNzaW9ucyZmb3JtYXQ9Y3N2JykKCmBgYAoKCgpgYGB7ciwgZXZhbCA9IEZBTFNFfQojIyMjIFN1YnNldCBhbGwgZGF0YXNldHMgc28gbWF0Y2hlcyBNVyBvdXRwdXQgIyMjIwpFbmREYXRlIDwtIGFzLkRhdGUoJzIwMjAtMDMtMjEnKQoKI3RydXN0X2FkbWlzc2lvbnMgPC0gc3Vic2V0KHRydXN0X2FkbWlzc2lvbnMsIGRhdGUgPD0gRW5kRGF0ZSkKI3RydXN0X2FkbWlzc2lvbnMgPC0gYWdncmVnYXRlKHZhbHVlIH4gZGF0ZSwgdHJ1c3RfYWRtaXNzaW9ucywgc3VtKSAjIHN1bW1pbmcgYWNyb3NzIHRydXN0cwoKIyBieURheURlYXRocyA8LSBhZ2dyZWdhdGUodmFsdWUgfiBkYXRlLCBkYXRhID0gb2JzX2RlYXRocywgc3VtLCBuYS5ybT1UUlVFKSAjIHN1bW1pbmcgYWNyb3NzIExBRHMKCiMgRXh0cmFjdCB3ZWVrbHkgbnVtYmVycyBmcm9tIG9ic2VydmF0aW9ucywgb24gZGF0ZXMgdGhhdCBtYXRjaCB3aXRoIE1XICh3ZWVrIDEyIGlzIHNob3J0ZXIpCmRhdGVfc2VxIDwtIGMoc2VxKGZyb20gPSBhcy5EYXRlKCcyMDIwLTAzLTE3JyksIGJ5ID0gLTcsIGxlbiA9IDYpWzY6MV0sIEVuZERhdGUpCnRvdGFsRGVhdGhzIDwtIG51bWVyaWMoNykKZm9yIChpIGluIDE6Nyl7CiAgdG90YWxEZWF0aHNbaV0gPC0gc3VtKHN1YnNldChieURheURlYXRocywgYXMuY2hhcmFjdGVyKGRhdGUpIDw9IGRhdGVfc2VxW2ldKSR2YWx1ZSkKfQoKIyBUaGUgUEhFIGRhdGFzZXRzIHN0YXJ0IGF0IGRpZmZlcmVudCB0aW1lcywgc28gdGhpcyBpcyBzbGlnaHRseSBtZXNzeSBhbmQgbm90IGdlbmVyYWwgLSBiZSBjYXJlZnVsIHdpdGggemVyb3MhISEKb2JzX2RhdGEgPC0gZGF0YS5mcmFtZSh3ZWVrID0gNjoxMiwgCiAgICAgICAgICAgICAgICAgICAgICAgbWVkaWFuID0gbG9nKHN1YnNldChwaGVfZGVhdGhzX2NlcnQsIGFzLkRhdGUoZGF0ZSkgJWluJSBkYXRlX3NlcSkkY3VtRGFpbHlOc29EZWF0aHNCeURlYXRoRGF0ZVs3OjFdKzEpLCAKICAgICAgICAgICAgICAgICAgICAgICBsb3dlciA9IGMoMCwwLDAsMCxsb2coc3Vic2V0KHBoZV9kZWF0aHMsIGFzLkRhdGUoZGF0ZSkgJWluJSBkYXRlX3NlcSkkY3VtRGVhdGhzMjhEYXlzQnlEZWF0aERhdGVbMzoxXSsxKSksIAogICAgICAgICAgICAgICAgICAgICAgIHVwcGVyID0gbG9nKHRvdGFsRGVhdGhzKzEpKQpgYGAKCgpgYGB7ciwgZXZhbCA9IEZBTFNFLCBmaWcuaGVpZ2h0PTYsIG91dC53aWR0aD0nNTAlJywgZmlnLnNob3c9J2hvbGQnfQpnZ3Bsb3Qoc3Vic2V0KENvbWJpbmVkRGF0YSwgd2VlayA9PSAxMiksIGFlcyh4ID0gbG9nKGN1bUggKyAxKSwgeSA9IGxvZyhkZWF0aHMgKyAxKSwgY29sID0gbnMpKSArCiAgZ2VvbV9wb2ludCgpICsKICBnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSBjKGxvZygyNTArMSksIGxvZygyNTAwKzEpKSkgKwogIGdlb21fdmxpbmUoeGludGVyY2VwdCA9IGMobG9nKDI1MDArMSksIGxvZygxMDAwMCsxKSkpICsKICBzY2FsZV9jb2xvcl92aXJpZGlzKCkgKwogIGxhYnMoY29sID0gJ1NlZWRzJywgeCA9ICdsb2coaG9zcHMpJywgeSA9ICdsb2coZGVhdGhzKScpCgpnZ3Bsb3Qoc3Vic2V0KENvbWJpbmVkRGF0YSwgd2VlayA9PSAxMiksIGFlcyh4ID0gbG9nKGN1bUggKyAxKSwgeSA9IGxvZyhkZWF0aHMgKyAxKSwgY29sID0gUjApKSArCiAgZ2VvbV9wb2ludCgpICsKICBnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSBjKGxvZygyNTArMSksIGxvZygyNTAwKzEpKSkgKwogIGdlb21fdmxpbmUoeGludGVyY2VwdCA9IGMobG9nKDI1MDArMSksIGxvZygxMDAwMCsxKSkpICsKICBzY2FsZV9jb2xvcl92aXJpZGlzKCkgKwogIGxhYnMoeCA9ICdsb2coaG9zcHMpJywgeSA9ICdsb2coZGVhdGhzKScpCmBgYAoKCiMjIEVuc2VtYmxlIGRlc2lnbgoKYGBge3IsIGZpZy53aWR0aCA9IDEyLCBmaWcuaGVpZ2h0ID0gMTJ9CnBhaXJzKE5ld0Rlc2lnblssMToxNF0pCgpgYGAKCgojIyMgQ3VtdWxhdGl2ZSBob3NwaXRhbCBkZWF0aHMgYXQgdGhlIHN0YXJ0IG9mIGxvY2tkb3duIGFnYWluc3QgZWFjaCBwYXJhbWV0ZXIKYGBge3IsIGZpZy53aWR0aCA9IDEyLCBmaWcuaGVpZ2h0ID0gOH0KCndlZWsxMiA8LSBzdWJzZXQoQ29tYmluZWREYXRhLCB3ZWVrID09IDEyKQoKcGFyKG1mcm93ID0gYygzLDUpKQoKZm9yIChpIGluIDE6MTQgKXsKICBwbG90KE5ld0Rlc2lnblssaV0sIHdlZWsxMiRjdW1IRCwgeGxhYiA9IGNvbG5hbWVzKE5ld0Rlc2lnbilbaV0sIHlsYWIgPSAnY3VtdWxhdGl2ZSBob3NwaXRhbCBkZWF0aHMnKQp9CgpgYGAKCgoKIyMjIEN1bXVsYXRpdmUgY29tbXVuaXR5IGRlYXRocyBhdCB0aGUgc3RhcnQgb2YgbG9ja2Rvd24gYWdhaW5zdCBlYWNoIHBhcmFtZXRlcgpgYGB7ciwgZmlnLndpZHRoID0gMTIsIGZpZy5oZWlnaHQgPSA4fQoKcGFyKG1mcm93ID0gYygzLDUpKQoKZm9yIChpIGluIDE6MTQgKXsKICBwbG90KE5ld0Rlc2lnblssaV0sIHdlZWsxMiRjdW1DRCwgeGxhYiA9IGNvbG5hbWVzKE5ld0Rlc2lnbilbaV0sIHlsYWIgPSAnY3VtdWxhdGl2ZSBjb21tdW5pdHkgZGVhdGhzJykKfQoKYGBgCgojIyBFbXVsYXRpbmcgY3VtdWxhdGl2ZSBkZWF0aHMgaW4gaG9zdHBpdGFsIChjdW1IRCkgaW4gdGhlIGNvbW11bml0eSAoY3VtQ0QpIGF0IHRoZSBiZWdpbm5pbmcgb2YgbG9ja2Rvd24KClRoZSBEaWNlS3JpZ2luZyBFbXVsYXRvciBmaXQgZm9yIGxvZyhkZWF0aHMpIGxvb2tzIGdvb2QuIExlYXZlLW9uZS1vdXQgaXNuJ3QgYSBncmVhdCBtZWFzdXJlLCBidXQgaXQncyBhIHN0YXJ0LgoKYGBge3IsIGZpZy53aWR0aCA9IDUsIGZpZy5oZWlnaHQgPSA4LCAgaW5jbHVkZSA9IEZBTFNFfQoKZml0X2N1bUhEIDwtIGttKH4uLCBkZXNpZ24gPSBOZXdEZXNpZ25bLCAxOjE0XSwgcmVzcG9uc2UgPSBsb2coMSsgd2VlazEyJGN1bUhEKSkKZml0X2N1bUNEIDwtIGttKH4uLCBkZXNpZ24gPSBOZXdEZXNpZ25bLCAxOjE0XSwgcmVzcG9uc2UgPSBsb2coMSsgd2VlazEyJGN1bUNEKSkKCmBgYAoKYGBge3J9Cmxvb19jdW1IRCA8LSBsZWF2ZU9uZU91dC5rbShmaXRfY3VtSEQsIHR5cGUgPSAnVUsnLCB0cmVuZC5yZWVzdGltID0gVFJVRSkKbG9vX2N1bUNEIDwtIGxlYXZlT25lT3V0LmttKGZpdF9jdW1DRCwgdHlwZSA9ICdVSycsIHRyZW5kLnJlZXN0aW0gPSBUUlVFKQoKYGBgCgojIyMgV2VlayAxMiBjdW11bGF0aXZlIEhvc3BpdGFsIGRlYXRocyBMZWF2ZS1vbmUtb3V0IHBsb3RzCmBgYHtyLCBmaWcud2lkdGggPSAxMiwgZmlnLmhlaWdodCA9IDZ9CgoKcGFyKG1mcm93ID0gYygxLDIpKQpwbG90KGxvZygxKyB3ZWVrMTIkY3VtSEQpLCBsb29fY3VtSEQkbWVhbiwgbWFpbiA9ICdMT08gbG9nKDErIEN1bXVsYXRpdmUgSG9zcGl0YWwgRGVhdGhzKScpCnNlZ21lbnRzKHgwID1sb2coMSsgd2VlazEyJGN1bUhEKSwgeTAgPSBsb29fY3VtSEQkbWVhbiAtICgyKiBsb29fY3VtSEQkc2QpLCAKICAgICAgICB4MSA9bG9nKDErIHdlZWsxMiRjdW1IRCksIHkxID0gbG9vX2N1bUhEJG1lYW4gKyAoMiogbG9vX2N1bUhEJHNkKQopCmFibGluZSgwLDEpCgoKeTAgPSBleHAobG9vX2N1bUhEJG1lYW4pIC0gKGV4cChsb29fY3VtSEQkbWVhbikgKiAoZXhwKGxvb19jdW1IRCRzZCkgLSAxKSkKeTEgPSBleHAobG9vX2N1bUhEJG1lYW4pICsgKGV4cChsb29fY3VtSEQkbWVhbikgKiAoZXhwKGxvb19jdW1IRCRzZCkgLSAxKSkKCnBsb3Qod2VlazEyJGN1bUhELCBleHAobG9vX2N1bUhEJG1lYW4pLCBtYWluID0gJ0xPTyBDdW11bGF0aXZlIEhvc3BpdGFsIERlYXRocycsIHlsaW0gPSByYW5nZSh5MCwgeTEpIAogICAgICkKc2VnbWVudHMoeDAgPSB3ZWVrMTIkY3VtSEQsIHkwID0geTAsCiAgICAgICAgIHgxID0gd2VlazEyJGN1bUhELCB5MSA9IHkxCikKCmFibGluZSgwLDEpCgoKYGBgCgoKIyMjIFdlZWsgMTIgY3VtdWxhdGl2ZSBjb21tdW5pdHkgZGVhdGhzIExlYXZlLW9uZS1vdXQgcGxvdHMKYGBge3IsIGZpZy53aWR0aCA9IDEyLCBmaWcuaGVpZ2h0ID0gNn0KCnBhcihtZnJvdyA9IGMoMSwyKSkKcGxvdChsb2coMSsgd2VlazEyJGN1bUNEKSwgbG9vX2N1bUNEJG1lYW4sIG1haW4gPSAnTE9PIGxvZygxKyBDdW11bGF0aXZlIGNvbW11bml0eSBEZWF0aHMpJykKc2VnbWVudHMoeDAgPWxvZygxKyB3ZWVrMTIkY3VtQ0QpLCB5MCA9IGxvb19jdW1DRCRtZWFuIC0gKDIqIGxvb19jdW1DRCRzZCksIAogICAgICAgIHgxID1sb2coMSsgd2VlazEyJGN1bUNEKSwgeTEgPSBsb29fY3VtQ0QkbWVhbiArICgyKiBsb29fY3VtQ0Qkc2QpCikKYWJsaW5lKDAsMSkKCgp5MCA9IGV4cChsb29fY3VtQ0QkbWVhbikgLSAoZXhwKGxvb19jdW1DRCRtZWFuKSAqIChleHAobG9vX2N1bUNEJHNkKSAtIDEpKQp5MSA9IGV4cChsb29fY3VtQ0QkbWVhbikgKyAoZXhwKGxvb19jdW1DRCRtZWFuKSAqIChleHAobG9vX2N1bUNEJHNkKSAtIDEpKQoKcGxvdCh3ZWVrMTIkY3VtQ0QsIGV4cChsb29fY3VtQ0QkbWVhbiksIG1haW4gPSAnTE9PIEN1bXVsYXRpdmUgY29tbXVuaXR5IERlYXRocycsIHlsaW0gPSByYW5nZSh5MCwgeTEpIAogICAgICkKc2VnbWVudHMoeDAgPSB3ZWVrMTIkY3VtQ0QsIHkwID0geTAsCiAgICAgICAgIHgxID0gd2VlazEyJGN1bUNELCB5MSA9IHkxCikKCmFibGluZSgwLDEpCgoKYGBgCgoKIyMgTm9ybWFsaXNlIGlucHV0IHNwYWNlIGFuZCByZS1maXQgdG8gcHJvZHVjZSBzb21lIHNlbnNpdGl2aXR5IGFuYWx5c2VzCgpgYGB7ciwgaW5jbHVkZSA9IEZBTFNFfQoKWCA8LSBOZXdEZXNpZ25bLCAxOjE0XQpYX25vcm0gPC0gbm9ybWFsaXplKFgpCgpmaXRfY3VtSERfbm9ybSA8LSBrbSh+LiwgZGVzaWduID0gWF9ub3JtLCByZXNwb25zZSA9IGxvZygxKyB3ZWVrMTIkY3VtSEQpKQpmaXRfY3VtQ0Rfbm9ybSA8LSBrbSh+LiwgZGVzaWduID0gWF9ub3JtLCByZXNwb25zZSA9IGxvZygxKyB3ZWVrMTIkY3VtQ0QpKQoKWF9vYWF0IDwtIG9hYXRfZGVzaWduKFhfbm9ybSwgbj0yMSwgbWVkID0gVFJVRSkKY29sbmFtZXMoWF9vYWF0KSA9IGNvbG5hbWVzKFgpCgpgYGAKCgoKIyMjIG9uZS1hdC1hLXRpbWUgc2Vuc2l0aXZpdHkgZm9yIGN1bXVsYXRpdmUgaG9zcGl0YWwgZGVhdGhzIAoKCmBgYHtyLCBmaWcud2lkdGggPSAxMCwgZmlnLmhlaWdodCA9IDh9CgoKCnlfb2FhdCA8LSBwcmVkaWN0LmttKGZpdF9jdW1IRF9ub3JtLCBuZXdkYXRhID0gWF9vYWF0LCB0eXBlID0gJ1VLJykKCm9hYXRMaW5lUGxvdChYX29hYXQgPSBYX29hYXQsIHlfb2FhdF9tZWFuID0geV9vYWF0JG1lYW4sIHlfb2FhdF9zZCA9IHlfb2FhdCRzZCwgCiAgICAgICAgICAgICBuX29hYXQgPSAyMSxuciA9IDMsIG5jID0gNSkgCmBgYAoKIyMjIG9uZS1hdC1hLXRpbWUgc2Vuc2l0aXZpdHkgZm9yIGN1bXVsYXRpdmUgY29tbXVuaXR5IGRlYXRocyAKCmBgYHtyLCBmaWcud2lkdGggPSAxMCwgZmlnLmhlaWdodCA9IDh9Cgp5X29hYXQgPC0gcHJlZGljdC5rbShmaXRfY3VtQ0Rfbm9ybSwgbmV3ZGF0YSA9IFhfb2FhdCwgdHlwZSA9ICdVSycpCgpvYWF0TGluZVBsb3QoWF9vYWF0ID0gWF9vYWF0LCB5X29hYXRfbWVhbiA9IHlfb2FhdCRtZWFuLCB5X29hYXRfc2QgPSB5X29hYXQkc2QsIAogICAgICAgICAgICAgbl9vYWF0ID0gMjEsbnIgPSAzLCBuYyA9IDUpIApgYGAKCgojIyBGQVNUOTkgc2Vuc2l0aXZpdHkgYW5hbHlzaXMgb2YgU2FsdGVsbGkgZXQgYWwgKDE5OTkpCmBgYHtyfQoKbGlicmFyeShzZW5zaXRpdml0eSkKIyBHZW5lcmF0ZSBhIGRlc2lnbiBmb3IgdGhlIEZBU1Q5OSBhbmFseXNpcwpYX2Zhc3QgPC0gZmFzdDk5KG1vZGVsID0gTlVMTCwgZmFjdG9ycyA9IGNvbG5hbWVzKFhfbm9ybSksIG4gPSAzMDAwLAogICAgICAgICAgICAgICAgIHEgPSAicXVuaWYiLCBxLmFyZyA9IGxpc3QobWluID0gMCwgbWF4ID0gMSkpCiMgUHJlZGljdCB0aGUgcmVzcG9uc2UgYXQgdGhlIEZBU1Q5OSBkZXNpZ24gcG9pbnRzIHVzaW5nIHRoZSBlbXVsYXRvcgoKCgpwcmVkX2Zhc3RfY3VtSEQgPSBwcmVkaWN0KGZpdF9jdW1IRF9ub3JtLCBuZXdkYXRhID0gWF9mYXN0JFgsIHR5cGUgPSAnVUsnKQpwcmVkX2Zhc3RfY3VtQ0QgPSBwcmVkaWN0KGZpdF9jdW1DRF9ub3JtLCBuZXdkYXRhID0gWF9mYXN0JFgsIHR5cGUgPSAnVUsnKQojIENhbGN1bGF0ZSB0aGUgc2Vuc2l0aXZpdHkgaW5kaWNlcwpmYXN0X3RlbGxfY3VtSEQgPC0gdGVsbChYX2Zhc3QsIHByZWRfZmFzdF9jdW1IRCRtZWFuKQpmYXN0X3RlbGxfY3VtQ0QgPC0gdGVsbChYX2Zhc3QsIHByZWRfZmFzdF9jdW1DRCRtZWFuKQoKCmJwQ29udmVydCA8LSBmdW5jdGlvbihmYXN0bW9kZWwpewogICMgZ2V0IHRoZSBGQVNUIHN1bW1hcnkgaW50byBhbiBlYXNpZXIgZm9ybWF0IGZvciBiYXJwbG90CiAgZmFzdF9zdW1tIDwtIHByaW50KGZhc3Rtb2RlbCkKICBmYXN0X2RpZmYgPC0gZmFzdF9zdW1tWyAsMl0gLSBmYXN0X3N1bW1bICwxXQogIGZhc3RfYnAgPC0gdChjYmluZChmYXN0X3N1bW1bICwxXSwgZmFzdF9kaWZmKSkKICBmYXN0X2JwCn0KCgoKYGBgCgpgYGB7cn0KCnBhcihsYXMgPSAyLCBtYXIgPSBjKDksNSwzLDIpKQpiYXJwbG90KGJwQ29udmVydChmYXN0X3RlbGxfY3VtSEQpLCBjb2wgPSBjKCdza3libHVlJywgJ2dyZXknKSwgeWxhYiA9ICdyZWxhdGl2ZSBzZW5zaXRpdml0eScsIG1haW4gPSAnY3VtSEQgRkFTVDk5IFNlbnNpdGl2aXR5JykKbGVnZW5kKCd0b3BsZWZ0JyxsZWdlbmQgPSBjKCdNYWluIGVmZmVjdCcsICdJbnRlcmFjdGlvbnMnKSwgZmlsbCA9IGMoJ3NreWJsdWUnLCAnZ3JleScpICkKCnBhcihsYXMgPSAyLCBtYXIgPSBjKDksNSwzLDIpKQpiYXJwbG90KGJwQ29udmVydChmYXN0X3RlbGxfY3VtQ0QpLCBjb2wgPSBjKCdza3libHVlJywgJ2dyZXknKSwgeWxhYiA9ICdyZWxhdGl2ZSBzZW5zaXRpdml0eScsIG1haW4gPSAnY3VtQ0QgRkFTVDk5IFNlbnNpdGl2aXR5JykKbGVnZW5kKCd0b3BsZWZ0JyxsZWdlbmQgPSBjKCdNYWluIGVmZmVjdCcsICdJbnRlcmFjdGlvbnMnKSwgZmlsbCA9IGMoJ3NreWJsdWUnLCAnZ3JleScpICkKCmBgYAoKCg==