Note
Click here to download the full example code
Usage of the model-versus-data code verification tool on a Finite-Element (Abaqus) model¶
Check an Abaqus cantilever beam model versus a reference dataset with the 'CodeVerificationAgainstData' tool.
from __future__ import annotations
import logging
from gemseo.datasets.io_dataset import IODataset
from gemseo.utils.directory_creator import DirectoryNamingMethod
from vimseo import EXAMPLE_RUNS_DIR
from vimseo.api import activate_logger
from vimseo.api import create_model
from vimseo.core.model_settings import IntegratedModelSettings
from vimseo.tools.verification.verification_vs_data import CodeVerificationAgainstData
We first define the logger level:
activate_logger(level=logging.INFO)
Then we create the model to verify:
model_name = "BendingTestAnalytical"
load_case = "Cantilever"
model = create_model(
model_name,
load_case,
model_options=IntegratedModelSettings(
directory_archive_root=EXAMPLE_RUNS_DIR / "archive/verification_vs_data",
directory_scratch_root=EXAMPLE_RUNS_DIR / "scratch/verification_vs_data",
cache_file_path=EXAMPLE_RUNS_DIR
/ f"caches/verification_vs_data/{model_name}_{load_case}_cache.hdf",
),
)
Out:
INFO - 19:44:54: Found 2 entries in the cache file : C:\Users\sebastien.bocquet\Dev\vimseo\docs\runnable_examples\model_runs\caches\verification_vs_data\BendingTestAnalytical_Cantilever_cache.hdf node : node
We also need a reference dataset. Here we do it programmatically, but we can also create it from a csv file: ``
reference_data = IODataset().from_array(
data=[[10.0, 10.0, -4.0, -12.0], [15.0, 10.0, -6.0, -40.0]],
variable_names=["height", "width", "maximum_dplt", "reaction_forces"],
variable_names_to_group_names={
"height": "inputs",
"width": "inputs",
"maximum_dplt": "outputs",
"reaction_forces": "outputs",
},
)
All inputs to the verification are now available. We create the verification tool we are interested in.
verificator = CodeVerificationAgainstData(
directory_naming_method=DirectoryNamingMethod.NUMBERED,
working_directory="CodeVerificationAgainstData_results",
)
# The options can be modified.
# Alternatively, options can be passed as keyword arguments to
# ``CodeVerificationAgainstModelFromParameterSpace()`` constructor.
verificator.options["metric_names"] = [
"SquaredErrorMetric",
"RelativeErrorMetric",
"AbsoluteErrorMetric",
]
verificator.execute(
model=model,
reference_data=reference_data,
output_names=["maximum_dplt", "reaction_forces"],
description={
"title": "Verification of a cantilever analytic beam for a variation of beam height.",
"element_wise": ["Small height value", "High height value"],
},
)
Out:
INFO - 19:44:54: Working directory is C:\Users\sebastien.bocquet\Dev\vimseo\docs\runnable_examples\03_verification_vs_data\CodeVerificationAgainstData_results
INFO - 19:44:54: Working directory is C:\Users\sebastien.bocquet\Dev\vimseo\docs\runnable_examples\03_verification_vs_data\CodeVerificationAgainstData_results\CustomDOETool
INFO - 19:44:54: Working directory is C:\Users\sebastien.bocquet\Dev\vimseo\docs\runnable_examples\03_verification_vs_data\CodeVerificationAgainstData_results\CustomDOETool
INFO - 19:44:55:
INFO - 19:44:55: *** Start DOEScenario execution ***
INFO - 19:44:55: DOEScenario
INFO - 19:44:55: Disciplines: Model BendingTestAnalytical: An analytical model for the bending of a parallelepipedic beam
INFO - 19:44:55:
INFO - 19:44:55: Load case:
INFO - 19:44:55: Load case Cantilever: A cantilever load case.
INFO - 19:44:55:
INFO - 19:44:55: Boundary condition variables:
INFO - 19:44:55: ['imposed_dplt', 'relative_dplt_location']
INFO - 19:44:55:
INFO - 19:44:55: Plot parameters:
INFO - 19:44:55: {
INFO - 19:44:55: "curves": []
INFO - 19:44:55: }
INFO - 19:44:55: Load:
INFO - 19:44:55: Load(direction='', sign='', type='')
INFO - 19:44:55:
INFO - 19:44:55: Default values:
INFO - 19:44:55:
INFO - 19:44:55: Default geometrical variables:
INFO - 19:44:55: {"height": [40.0], "length": [600.0], "width": [30.0]}
INFO - 19:44:55:
INFO - 19:44:55: Default numerical variables:
INFO - 19:44:55: {}
INFO - 19:44:55:
INFO - 19:44:55: Default boundary conditions variables:
INFO - 19:44:55: {"imposed_dplt": [-5.0], "relative_dplt_location": [1.0]}
INFO - 19:44:55:
INFO - 19:44:55: Default material variables:
INFO - 19:44:55: {"nu_p": [0.3], "young_modulus": [210000.0]}
INFO - 19:44:55: model_inputs:
INFO - 19:44:55: [
INFO - 19:44:55: "length",
INFO - 19:44:55: "width",
INFO - 19:44:55: "height",
INFO - 19:44:55: "imposed_dplt",
INFO - 19:44:55: "relative_dplt_location",
INFO - 19:44:55: "young_modulus",
INFO - 19:44:55: "nu_p"
INFO - 19:44:55: ]
INFO - 19:44:55: model_outputs:
INFO - 19:44:55: [
INFO - 19:44:55: "reaction_forces",
INFO - 19:44:55: "maximum_dplt",
INFO - 19:44:55: "dplt_grid",
INFO - 19:44:55: "location_max_dplt",
INFO - 19:44:55: "dplt",
INFO - 19:44:55: "moment",
INFO - 19:44:55: "moment_grid",
INFO - 19:44:55: "dplt_at_force_location",
INFO - 19:44:55: "error_code",
INFO - 19:44:55: "model",
INFO - 19:44:55: "load_case",
INFO - 19:44:55: "description",
INFO - 19:44:55: "job_name",
INFO - 19:44:55: "persistent_result_files",
INFO - 19:44:55: "n_cpus",
INFO - 19:44:55: "date",
INFO - 19:44:55: "cpu_time",
INFO - 19:44:55: "user",
INFO - 19:44:55: "machine",
INFO - 19:44:55: "vims_git_version",
INFO - 19:44:55: "directory_archive_root",
INFO - 19:44:55: "directory_archive_job",
INFO - 19:44:55: "directory_scratch_root",
INFO - 19:44:55: "directory_scratch_job"
INFO - 19:44:55: ]
INFO - 19:44:55: MDO formulation: DisciplinaryOpt
INFO - 19:44:55: Optimization problem:
INFO - 19:44:55: minimize maximum_dplt(height, width)
INFO - 19:44:55: with respect to height, width
INFO - 19:44:55: over the design space:
INFO - 19:44:55: +--------+-------------+-------+-------------+-------+
INFO - 19:44:55: | Name | Lower bound | Value | Upper bound | Type |
INFO - 19:44:55: +--------+-------------+-------+-------------+-------+
INFO - 19:44:55: | height | -inf | None | inf | float |
INFO - 19:44:55: | width | -inf | None | inf | float |
INFO - 19:44:55: +--------+-------------+-------+-------------+-------+
INFO - 19:44:55: Solving optimization problem with algorithm CustomDOE:
INFO - 19:44:55: 50%|█████ | 1/2 [00:00<00:00, 9.98 it/sec, obj=-5]
INFO - 19:44:55: 100%|██████████| 2/2 [00:00<00:00, 12.81 it/sec, obj=-5]
INFO - 19:44:55: Optimization result:
INFO - 19:44:55: Optimizer info:
INFO - 19:44:55: Status: None
INFO - 19:44:55: Message: None
INFO - 19:44:55: Number of calls to the objective function by the optimizer: 2
INFO - 19:44:55: Solution:
INFO - 19:44:55: Objective: -4.999999999999999
INFO - 19:44:55: Design space:
INFO - 19:44:55: +--------+-------------+-------+-------------+-------+
INFO - 19:44:55: | Name | Lower bound | Value | Upper bound | Type |
INFO - 19:44:55: +--------+-------------+-------+-------------+-------+
INFO - 19:44:55: | height | -inf | 10 | inf | float |
INFO - 19:44:55: | width | -inf | 10 | inf | float |
INFO - 19:44:55: +--------+-------------+-------+-------------+-------+
INFO - 19:44:55: *** End DOEScenario execution (time: 0:00:00.172999) ***
VerificationResult(metadata=ToolResultMetadata(generic={'datetime': '16-06-2026_19-44-54', 'version': '0.1.7.dev4+g13b1eb78b.d20260616'}, misc={}, settings={'input_names': [], 'output_names': ['maximum_dplt', 'reaction_forces'], 'metric_names': ['SquaredErrorMetric', 'RelativeErrorMetric', 'AbsoluteErrorMetric'], 'description': {'title': 'Verification of a cantilever analytic beam for a variation of beam height.', 'element_wise': ['Small height value', 'High height value']}}, report={}, model=ModelDescription(name='BendingTestAnalytical', summary=' An analytical model for the bending of a parallelepipedic beam', load_case=Beam_Cantilever(name='Cantilever', domain='Beam', summary='A cantilever load case.', plot_parameters=PlotParameters(curves=[]), bc_variable_names=['imposed_dplt', 'relative_dplt_location'], load=Load(direction='', sign='', type='')), dataflow={'model_inputs': ['length', 'width', 'height', 'imposed_dplt', 'relative_dplt_location', 'young_modulus', 'nu_p'], 'model_outputs': ['reaction_forces', 'maximum_dplt', 'dplt_grid', 'location_max_dplt', 'dplt', 'moment', 'moment_grid', 'dplt_at_force_location', 'error_code', 'model', 'load_case', 'description', 'job_name', 'persistent_result_files', 'n_cpus', 'date', 'cpu_time', 'user', 'machine', 'vims_git_version', 'directory_archive_root', 'directory_archive_job', 'directory_scratch_root', 'directory_scratch_job'], 'PreBendingTestAnalytical_Cantilever': {'inputs': ['length', 'width', 'height', 'imposed_dplt', 'relative_dplt_location', 'young_modulus', 'nu_p'], 'outputs': ['imposed_dplt_location', 'quadratic_moment', 'reaction_forces', 'moment', 'moment_grid', 'solver', 'boundary']}, 'RunBendingTestAnalytical': {'inputs': ['imposed_dplt_location', 'quadratic_moment', 'reaction_forces', 'moment', 'moment_grid', 'solver', 'boundary', 'length', 'width', 'height', 'imposed_dplt', 'relative_dplt_location', 'young_modulus', 'nu_p'], 'outputs': ['dplt', 'dplt_grid', 'moment', 'moment_grid', 'imposed_dplt_location', 'reaction_forces']}, 'PostBendingTestAnalytical_Cantilever': {'inputs': ['dplt', 'dplt_grid', 'moment', 'moment_grid', 'imposed_dplt_location', 'reaction_forces'], 'outputs': ['reaction_forces', 'maximum_dplt', 'dplt_grid', 'location_max_dplt', 'dplt', 'moment', 'moment_grid', 'dplt_at_force_location', 'error_code']}, 'subroutine_names': []}, default_inputs={<InputGroupNames.NUMERICAL_VARS: 'numerical variables'>: {}, <InputGroupNames.BC_VARS: 'boundary conditions variables'>: {'imposed_dplt': [-5.0], 'relative_dplt_location': [1.0]}, <InputGroupNames.GEOMETRICAL_VARS: 'geometrical variables'>: {'length': [600.0], 'width': [30.0], 'height': [40.0]}, <InputGroupNames.MATERIAL_VARS: 'material variables'>: {'young_modulus': [210000.0], 'nu_p': [0.3]}}, curves=[('dplt_grid', 'dplt'), ('moment_grid', 'moment')], verbose=False)), simulation_and_reference=GROUP inputs outputs Reference
VARIABLE height width cpu_time maximum_dplt reaction_forces maximum_dplt reaction_forces
COMPONENT 0 0 0 0 0 0 0
0 10.0 10.0 0.033884 -5.0 -12.152778 -4.0 -12.0
1 15.0 10.0 0.010976 -5.0 -41.015625 -6.0 -40.0, element_wise_metrics=GROUP SquaredErrorMetric RelativeErrorMetric ... AbsoluteErrorMetric inputs
VARIABLE maximum_dplt reaction_forces maximum_dplt ... reaction_forces height width
COMPONENT 0 0 0 ... 0 0 0
0 1.0 0.023341 0.250000 ... 0.152778 10.0 10.0
1 1.0 1.031494 0.166667 ... 1.015625 15.0 10.0
[2 rows x 8 columns], integrated_metrics={'SquaredErrorMetric': {'maximum_dplt': 1.0000000000000009, 'reaction_forces': 0.5274175950038581}, 'RelativeErrorMetric': {'maximum_dplt': 0.20833333328819448, 'reaction_forces': 0.019061053239892913}, 'AbsoluteErrorMetric': {'maximum_dplt': 1.0000000000000004, 'reaction_forces': 0.5842013888888893}}, description={'title': 'Verification of a cantilever analytic beam for a variation of beam height.', 'element_wise': ['Small height value', 'High height value']})
The result contains the error metrics:
verificator.result.integrated_metrics
Out:
{'SquaredErrorMetric': {'maximum_dplt': 1.0000000000000009, 'reaction_forces': 0.5274175950038581}, 'RelativeErrorMetric': {'maximum_dplt': 0.20833333328819448, 'reaction_forces': 0.019061053239892913}, 'AbsoluteErrorMetric': {'maximum_dplt': 1.0000000000000004, 'reaction_forces': 0.5842013888888893}}
And saved on disk, together with its metadata:
verificator.save_results()
Out:
INFO - 19:44:55: Saving result to C:\Users\sebastien.bocquet\Dev\vimseo\docs\runnable_examples\03_verification_vs_data\CodeVerificationAgainstData_results\CodeVerificationAgainstData_result.hdf5
INFO - 19:44:56: Saving result to C:\Users\sebastien.bocquet\Dev\vimseo\docs\runnable_examples\03_verification_vs_data\CodeVerificationAgainstData_results\CustomDOETool\CustomDOETool_result.hdf5
The saved results can be loaded in a dedicated dashboard to be explored.
The dashboard is opened by typing dashboard_verification in a terminal,
and selecting the tab Comparison case.
The results can also be plotted from the Python API. It shows the scatter matrix of the inputs:
figures = verificator.plot_results(
verificator.result,
"RelativeErrorMetric",
"reaction_forces",
save=False,
show=True,
directory_path=verificator.working_directory,
)
Out:
C:\Users\sebastien.bocquet\Dev\vimseo\.tox\doc\Lib\site-packages\pandas\plotting\_matplotlib\misc.py:100: UserWarning:
Attempting to set identical low and high xlims makes transformation singular; automatically expanding.
C:\Users\sebastien.bocquet\Dev\vimseo\.tox\doc\Lib\site-packages\pandas\plotting\_matplotlib\misc.py:101: UserWarning:
Attempting to set identical low and high ylims makes transformation singular; automatically expanding.
C:\Users\sebastien.bocquet\Dev\vimseo\.tox\doc\Lib\site-packages\pandas\plotting\_matplotlib\misc.py:91: UserWarning:
Attempting to set identical low and high xlims makes transformation singular; automatically expanding.
C:\Users\sebastien.bocquet\Dev\vimseo\.tox\doc\Lib\site-packages\gemseo\utils\matplotlib_figure.py:59: UserWarning:
FigureCanvasAgg is non-interactive, and thus cannot be shown
INFO - 19:44:56: Working directory is C:\Users\sebastien.bocquet\Dev\vimseo\docs\runnable_examples\03_verification_vs_data\CodeVerificationAgainstData_results
and an histogram of the errors:
figures["error_metric_histogram"]
Total running time of the script: ( 0 minutes 2.790 seconds)
Download Python source code: plot_bending_test_vs_data.py