Dealing with errors after a run
Contents
Dealing with errors after a run#
In this example, we run the model on a list of three glaciers: two of them will end with errors: one because it already failed at preprocessing (i.e. prior to this run), and one during the run. We show how to analyze theses erros and solve (some) of them, as described in the OGGM documentation under troubleshooting.
Run with cfg.PARAMS['continue_on_error'] = True#
# Locals
import oggm.cfg as cfg
from oggm import utils, workflow, tasks
# Libs
import os
import xarray as xr
import pandas as pd
# Initialize OGGM and set up the default run parameters
cfg.initialize(logging_level='WARNING')
# Here we override some of the default parameters
# How many grid points around the glacier?
# We make it small because we want the model to error because
# of flowing out of the domain
cfg.PARAMS['border'] = 80
# This is useful since we have three glaciers
cfg.PARAMS['use_multiprocessing'] = True
# This is the important bit!
# We tell OGGM to continue despite of errors
cfg.PARAMS['continue_on_error'] = True
# Local working directory (where OGGM will write its output)
WORKING_DIR = utils.gettempdir('OGGM_Errors', reset=True)
cfg.PATHS['working_dir'] = WORKING_DIR
rgi_ids = ['RGI60-11.00897', 'RGI60-11.01450', 'RGI60-11.03295']
# Go - get the pre-processed glacier directories
gdirs = workflow.init_glacier_directories(rgi_ids, from_prepro_level=4)
# We can step directly to the experiment!
# Random climate representative for the recent climate (1985-2015)
# with a negative bias added to the random temperature series
workflow.execute_entity_task(tasks.run_random_climate, gdirs,
nyears=150, seed=0,
temperature_bias=-1)
2023-03-07 12:24:43: oggm.cfg: Reading default parameters from the OGGM `params.cfg` configuration file.
2023-03-07 12:24:43: oggm.cfg: Multiprocessing switched OFF according to the parameter file.
2023-03-07 12:24:43: oggm.cfg: Multiprocessing: using all available processors (N=2)
2023-03-07 12:24:43: oggm.cfg: PARAMS['border'] changed from `40` to `80`.
2023-03-07 12:24:43: oggm.cfg: Multiprocessing switched ON after user settings.
2023-03-07 12:24:43: oggm.cfg: PARAMS['continue_on_error'] changed from `False` to `True`.
2023-03-07 12:24:43: oggm.workflow: init_glacier_directories from prepro level 4 on 3 glaciers.
2023-03-07 12:24:43: oggm.workflow: Execute entity tasks [gdir_from_prepro] on 3 glaciers
---------------------------------------------------------------------------
RemoteTraceback Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/pyenv/versions/3.10.10/lib/python3.10/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/usr/local/pyenv/versions/3.10.10/lib/python3.10/multiprocessing/pool.py", line 48, in mapstar
return list(map(*args))
File "/usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/workflow.py", line 108, in __call__
res = self._call_internal(func, arg, kwargs)
File "/usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/workflow.py", line 102, in _call_internal
return call_func(gdir, **kwargs)
File "/usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/workflow.py", line 251, in gdir_from_prepro
return oggm.GlacierDirectory(entity, from_tar=from_tar)
File "/usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/utils/_workflow.py", line 2529, in __init__
self.name = filter_rgi_name(name)
File "/usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/utils/_funcs.py", line 734, in filter_rgi_name
if name is None or len(name) == 0:
TypeError: object of type 'numpy.float64' has no len()
"""
The above exception was the direct cause of the following exception:
TypeError Traceback (most recent call last)
Cell In[1], line 33
30 rgi_ids = ['RGI60-11.00897', 'RGI60-11.01450', 'RGI60-11.03295']
32 # Go - get the pre-processed glacier directories
---> 33 gdirs = workflow.init_glacier_directories(rgi_ids, from_prepro_level=4)
35 # We can step directly to the experiment!
36 # Random climate representative for the recent climate (1985-2015)
37 # with a negative bias added to the random temperature series
38 workflow.execute_entity_task(tasks.run_random_climate, gdirs,
39 nyears=150, seed=0,
40 temperature_bias=-1)
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/workflow.py:533, in init_glacier_directories(rgidf, reset, force, from_prepro_level, prepro_border, prepro_rgi_version, prepro_base_url, from_tar, delete_tar)
531 if cfg.PARAMS['dl_verify']:
532 utils.get_dl_verify_data('cluster.klima.uni-bremen.de')
--> 533 gdirs = execute_entity_task(gdir_from_prepro, entities,
534 from_prepro_level=from_prepro_level,
535 prepro_border=prepro_border,
536 prepro_rgi_version=prepro_rgi_version,
537 base_url=prepro_base_url)
538 else:
539 # We can set the intersects file automatically here
540 if (cfg.PARAMS['use_intersects'] and
541 len(cfg.PARAMS['intersects_gdf']) == 0 and
542 not from_tar):
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/workflow.py:185, in execute_entity_task(task, gdirs, **kwargs)
183 if cfg.PARAMS['use_multiprocessing'] and ng > 1:
184 mppool = init_mp_pool(cfg.CONFIG_MODIFIED)
--> 185 out = mppool.map(pc, gdirs, chunksize=1)
186 else:
187 if ng > 3:
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/multiprocessing/pool.py:367, in Pool.map(self, func, iterable, chunksize)
362 def map(self, func, iterable, chunksize=None):
363 '''
364 Apply `func` to each element in `iterable`, collecting the results
365 in a list that is returned.
366 '''
--> 367 return self._map_async(func, iterable, mapstar, chunksize).get()
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/multiprocessing/pool.py:774, in ApplyResult.get(self, timeout)
772 return self._value
773 else:
--> 774 raise self._value
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/multiprocessing/pool.py:125, in worker()
123 job, i, func, args, kwds = task
124 try:
--> 125 result = (True, func(*args, **kwds))
126 except Exception as e:
127 if wrap_exception and func is not _helper_reraises_exception:
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/multiprocessing/pool.py:48, in mapstar()
47 def mapstar(args):
---> 48 return list(map(*args))
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/workflow.py:108, in __call__()
106 for func in self.call_func:
107 func, kwargs = func
--> 108 res = self._call_internal(func, arg, kwargs)
109 return res
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/workflow.py:102, in _call_internal()
99 gdir, gdir_kwargs = gdir
100 kwargs.update(gdir_kwargs)
--> 102 return call_func(gdir, **kwargs)
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/workflow.py:251, in gdir_from_prepro()
248 tar_base = utils.get_prepro_gdir(prepro_rgi_version, rid, prepro_border,
249 from_prepro_level, base_url=base_url)
250 from_tar = os.path.join(tar_base.replace('.tar', ''), rid + '.tar.gz')
--> 251 return oggm.GlacierDirectory(entity, from_tar=from_tar)
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/utils/_workflow.py:2529, in __init__()
2525 raise RuntimeError('RGI Version not supported: '
2526 '{}'.format(self.rgi_version))
2528 # remove spurious characters and trailing blanks
-> 2529 self.name = filter_rgi_name(name)
2531 # region
2532 reg_names, subreg_names = parse_rgi_meta(version=self.rgi_version[0])
File /usr/local/pyenv/versions/3.10.10/lib/python3.10/site-packages/oggm/utils/_funcs.py:734, in filter_rgi_name()
728 def filter_rgi_name(name):
729 """Remove spurious characters and trailing blanks from RGI glacier name.
730
731 This seems to be unnecessary with RGI V6
732 """
--> 734 if name is None or len(name) == 0:
735 return ''
737 if name[-1] in ['À', 'È', 'è', '\x9c', '3', 'Ð', '°', '¾',
738 '\r', '\x93', '¤', '0', '`', '/', 'C', '@',
739 'Å', '\x06', '\x10', '^', 'å', ';']:
TypeError: object of type 'numpy.float64' has no len()
Error diagnostics#
# Write the compiled output
utils.compile_glacier_statistics(gdirs); # saved as glacier_statistics.csv in the WORKING_DIR folder
utils.compile_run_output(gdirs); # saved as run_output.nc in the WORKING_DIR folder
# Read it
with xr.open_dataset(os.path.join(WORKING_DIR, 'run_output.nc')) as ds:
ds = ds.load()
df_stats = pd.read_csv(os.path.join(WORKING_DIR, 'glacier_statistics.csv'), index_col=0)
# all possible statistics about the glaciers
df_stats
in the column error_task, we can see whether an error occurred, and if yes during which task
error_msg describes the actual error message
df_stats[['error_task', 'error_msg']]
We can also check which glacier failed at which task by using compile_task_log.
# also saved as task_log.csv in the WORKING_DIR folder - "append=False" replaces the existing one
utils.compile_task_log(gdirs, task_names=['glacier_masks', 'compute_centerlines', 'flowline_model_run'], append=False)
Error solving#
RuntimeError: Glacier exceeds domain boundaries, at year: 98.08333333333333#
To remove this error just increase the domain boundary before running init_glacier_directories ! Attention, this means that more data has to be downloaded and the run takes more time. The available options for cfg.PARAMS['border'] are 10, 40, 80 or 160 at the moment; the unit is number of grid points outside the glacier boundaries. More about that in the OGGM documentation under preprocessed files.
# reset to recompute statistics
# Beware! If you use `reset=True` in `utils.mkdir`, ALL DATA in this folder will be deleted!
utils.mkdir(WORKING_DIR, reset=True)
# increase the amount of gridpoints outside the glacier
cfg.PARAMS['border'] = 160
gdirs = workflow.init_glacier_directories(rgi_ids, from_prepro_level=4)
workflow.execute_entity_task(tasks.run_random_climate, gdirs,
nyears=150, seed=0,
temperature_bias=-1);
# recompute the output
# we can also get the run output directly from the methods
df_stats = utils.compile_glacier_statistics(gdirs)
ds = utils.compile_run_output(gdirs)
# check again
df_stats[['error_task', 'error_msg']]
Now RGI60-11.00897 runs without errors!
Error: Need a valid model_flowlines file.#
This error message in the log is misleading: it does not really describe the source of the error, which happened earlier in the processing chain. Therefore we can look instead into the glacier_statistics via compile_glacier_statistics or into the log output via compile_task_log:
print('error_task: {}, error_msg: {}'.format(df_stats.loc['RGI60-11.03295']['error_task'],
df_stats.loc['RGI60-11.03295']['error_msg']))
Now we have a better understanding of the error:
OGGM can not work with this geometry of this glacier and could therefore not make a gridded mask of the glacier outlines.
there is no way to prevent this except you find a better way to pre-process the geometry of this glacier
these glaciers have to be ignored! Less than 0.5% of glacier area globally have errors during the geometry processing or failures in computing certain topographical properties by e.g. invalid DEM, see Sect. 4.2 Invalid Glaciers of the OGGM paper (Maussion et al., 2019) and this tutorial for more up-to-date numbers
Ignoring those glaciers with errors that we can’t solve#
In the run_output, you can for example just use *.dropna to remove these. For other applications (e.g. quantitative mass change evaluation), more will be needed (not available yet in the OGGM codebase):
ds.dropna(dim='rgi_id') # here we can e.g. find the volume evolution
What’s next?#
read about preprocessing errors
return to the OGGM documentation
back to the table of contents