Note
Click here to download the full example code
Deterministic validation case on the bending test analytical beam model against an analytical solution¶
from __future__ import annotations
import logging
from vimseo import EXAMPLE_RUNS_DIR
from vimseo.api import activate_logger
from vimseo.api import create_model
from vimseo.core.model_settings import IntegratedModelSettings
from vimseo.problems.beam_analytic.reference_dataset_builder import (
bending_test_analytical_reference_dataset,
)
from vimseo.tools.validation_case.validation_case import DeterministicValidationCase
from vimseo.tools.validation_case.validation_case import (
DeterministicValidationCaseInputs,
)
from vimseo.tools.validation_case.validation_case import (
DeterministicValidationCaseSettings,
)
activate_logger(level=logging.INFO)
First a model is created:
model_name = "BendingTestAnalytical"
load_case = "Cantilever"
model = create_model(
model_name,
load_case,
model_options=IntegratedModelSettings(
directory_archive_root=EXAMPLE_RUNS_DIR / "archive/validation_case",
directory_scratch_root=EXAMPLE_RUNS_DIR / "scratch/validation_case",
cache_file_path=EXAMPLE_RUNS_DIR
/ f"caches/validation_case/{model_name}_{load_case}.hdf",
),
)
The samples are set from synthetic reference data already generated for this model, to which a bias is added to obtain non-zero error metrics.
reference_data = bending_test_analytical_reference_dataset(shift=10.0)["Cantilever"]
print("The measured data: ", reference_data)
Out:
The measured data: GROUP inputs ... outputs
VARIABLE height imposed_dplt length relative_dplt_location width ... moment moment_grid reaction_forces
COMPONENT 0 0 0 0 0 ... 2 0 1 2 0
0 10.0 5.0 1000.0 0.5 10.0 ... 0.0 0.0 500.0 1000.0 30.0
1 15.0 5.0 1000.0 0.5 10.0 ... 0.0 0.0 500.0 1000.0 77.5
2 20.0 5.0 1000.0 0.5 10.0 ... 0.0 0.0 500.0 1000.0 170.0
3 15.0 10.0 1000.0 0.5 10.0 ... 0.0 0.0 500.0 1000.0 145.0
4 20.0 10.0 1000.0 0.5 10.0 ... 0.0 0.0 500.0 1000.0 330.0
[5 rows x 218 columns]
Then, the validation case tool is created and executed:
validation_tool = DeterministicValidationCase(
working_directory="deterministic_validation_case"
)
results = validation_tool.execute(
inputs=DeterministicValidationCaseInputs(
model=model,
reference_data=reference_data,
),
settings=DeterministicValidationCaseSettings(
metric_names=[
"RelativeErrorMetric",
"AbsoluteErrorMetric",
],
output_names=["reaction_forces"],
),
)
validation_tool.save_results()
Out:
INFO - 16:52:07: Working directory is /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/07_validation_case/deterministic_validation_case
INFO - 16:52:07: Working directory is /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/07_validation_case/deterministic_validation_case/CustomDOETool
INFO - 16:52:07: Found 5 entries in the cache file : /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/model_runs/caches/validation_case/BendingTestAnalytical_Cantilever_.hdf node : node
WARNING - 16:52:07: The cache policy is already set to HDF5Cache with the file path '/home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/model_runs/caches/validation_case/BendingTestAnalytical_Cantilever_.hdf' and node name 'node'; call discipline.cache.clear() to clear the cache.
WARNING - 16:52:07: The cache policy is already set to HDF5Cache with the file path '/home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/model_runs/caches/validation_case/BendingTestAnalytical_Cantilever_.hdf' and node name 'node'; call discipline.cache.clear() to clear the cache.
WARNING - 16:52:07: The cache policy is already set to HDF5Cache with the file path '/home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/model_runs/caches/validation_case/BendingTestAnalytical_Cantilever_.hdf' and node name 'node'; call discipline.cache.clear() to clear the cache.
WARNING - 16:52:07: The cache policy is already set to HDF5Cache with the file path '/home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/model_runs/caches/validation_case/BendingTestAnalytical_Cantilever_.hdf' and node name 'node'; call discipline.cache.clear() to clear the cache.
INFO - 16:52:07: Saving result to /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/07_validation_case/deterministic_validation_case/DeterministicValidationCase_result.hdf5
INFO - 16:52:07: Saving result to /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/07_validation_case/deterministic_validation_case/CustomDOETool/CustomDOETool_result.hdf5
Post-processing the results.
The validation result contains:
- a dictionary mapping the error metric names, output names and the statistical metric values (by default, the mean is used).
- a dataset containing the element-wise error metrics, together with the
- the simulated data, reference data and the input samples.
print("The validation result: ", validation_tool.result)
print(validation_tool.result.element_wise_metrics)
print(validation_tool.result.integrated_metrics)
Out:
The validation result: ValidationCaseResult(metadata=ToolResultMetadata(generic={'datetime': '11-05-2026_16-52-07', 'version': '0.1.7.dev11+g45528c259'}, misc={}, settings={'input_names': [], 'output_names': ['reaction_forces'], 'metric_names': ['RelativeErrorMetric', 'AbsoluteErrorMetric'], 'description': None}, report={'title': 'Validation of BendingTestAnalytical Cantilever'}, model=ModelDescription(name='BendingTestAnalytical', summary=' An analytical model for the bending of a parallelepipedic beam', load_case=Beam_Cantilever(name='Cantilever', domain='Beam', summary='A cantilever load case.', plot_parameters=PlotParameters(curves=[]), bc_variable_names=['imposed_dplt', 'relative_dplt_location'], load=Load(direction='', sign='', type='')), dataflow={'model_inputs': ['length', 'width', 'height', 'imposed_dplt', 'relative_dplt_location', 'young_modulus', 'nu_p'], 'model_outputs': ['reaction_forces', 'maximum_dplt', 'dplt_grid', 'location_max_dplt', 'dplt', 'moment', 'moment_grid', 'dplt_at_force_location', 'error_code', 'model', 'load_case', 'description', 'job_name', 'persistent_result_files', 'n_cpus', 'date', 'cpu_time', 'user', 'machine', 'vims_git_version', 'directory_archive_root', 'directory_archive_job', 'directory_scratch_root', 'directory_scratch_job'], 'PreBendingTestAnalytical_Cantilever': {'inputs': ['length', 'width', 'height', 'imposed_dplt', 'relative_dplt_location', 'young_modulus', 'nu_p'], 'outputs': ['imposed_dplt_location', 'quadratic_moment', 'reaction_forces', 'moment', 'moment_grid', 'solver', 'boundary']}, 'RunBendingTestAnalytical': {'inputs': ['imposed_dplt_location', 'quadratic_moment', 'reaction_forces', 'moment', 'moment_grid', 'solver', 'boundary', 'length', 'width', 'height', 'imposed_dplt', 'relative_dplt_location', 'young_modulus', 'nu_p'], 'outputs': ['dplt', 'dplt_grid', 'moment', 'moment_grid', 'imposed_dplt_location', 'reaction_forces']}, 'PostBendingTestAnalytical_Cantilever': {'inputs': ['dplt', 'dplt_grid', 'moment', 'moment_grid', 'imposed_dplt_location', 'reaction_forces'], 'outputs': ['reaction_forces', 'maximum_dplt', 'dplt_grid', 'location_max_dplt', 'dplt', 'moment', 'moment_grid', 'dplt_at_force_location', 'error_code']}, 'subroutine_names': []}, default_inputs={<InputGroupNames.NUMERICAL_VARS: 'numerical variables'>: {}, <InputGroupNames.BC_VARS: 'boundary conditions variables'>: {'imposed_dplt': [-5.0], 'relative_dplt_location': [1.0]}, <InputGroupNames.GEOMETRICAL_VARS: 'geometrical variables'>: {'length': [600.0], 'width': [30.0], 'height': [40.0]}, <InputGroupNames.MATERIAL_VARS: 'material variables'>: {'young_modulus': [210000.0], 'nu_p': [0.3]}}, curves=[('dplt_grid', 'dplt'), ('moment_grid', 'moment')], verbose=False)), element_wise_metrics=GROUP inputs ... AbsoluteErrorMetric outputs ReferenceOutputs
VARIABLE height imposed_dplt length ... reaction_forces reaction_forces reaction_forces
COMPONENT 0 0 0 ... 0 0 0
0 10.0 5.0 1000.0 ... 10.0 20.0 30.0
1 15.0 5.0 1000.0 ... 10.0 67.5 77.5
2 20.0 5.0 1000.0 ... 10.0 160.0 170.0
3 15.0 10.0 1000.0 ... 10.0 135.0 145.0
4 20.0 10.0 1000.0 ... 10.0 320.0 330.0
[5 rows x 10 columns], integrated_metrics=defaultdict(<class 'dict'>, {'RelativeErrorMetric': {'reaction_forces': 0.12409153366806684}, 'AbsoluteErrorMetric': {'reaction_forces': 10.0}}), stochastic_point_results=())
GROUP inputs ... AbsoluteErrorMetric outputs ReferenceOutputs
VARIABLE height imposed_dplt length ... reaction_forces reaction_forces reaction_forces
COMPONENT 0 0 0 ... 0 0 0
0 10.0 5.0 1000.0 ... 10.0 20.0 30.0
1 15.0 5.0 1000.0 ... 10.0 67.5 77.5
2 20.0 5.0 1000.0 ... 10.0 160.0 170.0
3 15.0 10.0 1000.0 ... 10.0 135.0 145.0
4 20.0 10.0 1000.0 ... 10.0 320.0 330.0
[5 rows x 10 columns]
defaultdict(<class 'dict'>, {'RelativeErrorMetric': {'reaction_forces': 0.12409153366806684}, 'AbsoluteErrorMetric': {'reaction_forces': 10.0}})
Validation results can be visualized as:
figs = validation_tool.plot_results(
validation_tool.result,
"RelativeErrorMetric",
"reaction_forces",
save=False,
show=True,
)
Out:
INFO - 16:52:07: Working directory is /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/07_validation_case/deterministic_validation_case
INFO - 16:52:08: Working directory is /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/07_validation_case/deterministic_validation_case
/home/sebastien.bocquet/PycharmProjects/vimseo/src/vimseo/utilities/datasets.py:376: FutureWarning:
Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
/home/sebastien.bocquet/PycharmProjects/vimseo/src/vimseo/utilities/datasets.py:376: FutureWarning:
Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
/home/sebastien.bocquet/PycharmProjects/vimseo/src/vimseo/utilities/datasets.py:376: FutureWarning:
Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
/home/sebastien.bocquet/PycharmProjects/vimseo/src/vimseo/utilities/datasets.py:376: FutureWarning:
Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
/home/sebastien.bocquet/PycharmProjects/vimseo/src/vimseo/utilities/datasets.py:376: FutureWarning:
Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
/home/sebastien.bocquet/PycharmProjects/vimseo/src/vimseo/utilities/datasets.py:376: FutureWarning:
Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
/home/sebastien.bocquet/PycharmProjects/vimseo/src/vimseo/utilities/datasets.py:376: FutureWarning:
Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
/home/sebastien.bocquet/PycharmProjects/vimseo/src/vimseo/utilities/datasets.py:376: FutureWarning:
Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
/home/sebastien.bocquet/PycharmProjects/vimseo/src/vimseo/utilities/datasets.py:376: FutureWarning:
Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
/home/sebastien.bocquet/PycharmProjects/vimseo/src/vimseo/utilities/datasets.py:376: FutureWarning:
Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
/home/sebastien.bocquet/PycharmProjects/vimseo/.tox/doc/lib/python3.11/site-packages/plotly/basedatatypes.py:2314: DeprecationWarning:
The append_trace method is deprecated and will be removed in a future version.
Please use the add_trace method with the row and col parameters.
INFO - 16:52:09: Working directory is /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/07_validation_case/deterministic_validation_case
INFO - 16:52:12: Working directory is /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/07_validation_case/deterministic_validation_case
a parallel coordinates plot:
figs["parallel_coordinates"]
an error scatter matrix:
figs["error_scatter_matrix"]
a predict-versus-true plot:
figs["predict_vs_true"]
a bar plot of the integrated metrics:
figs["integrated_metric_bars"]
Total running time of the script: ( 0 minutes 8.676 seconds)
Download Python source code: plot_deterministic_validation_case_straightbeam.py
Download Jupyter notebook: plot_deterministic_validation_case_straightbeam.ipynb