Accelerating in-transit co-processing for scientific simulations using region-based data-driven analysis

Tools

Walldén, Marcus and Okita, Masao and Ino, Fumihiko and Drikakis, Dimitris and Kokkinakis, Ioannis (2021) Accelerating in-transit co-processing for scientific simulations using region-based data-driven analysis. Algorithms, 14 (5). 154. ISSN 1999-4893 (https://doi.org/10.3390/a14050154)

[thumbnail of Wallden-etal-algorithms2021-accelerating-in-transit-co-processing-for-scientific-simulations]
Preview
 Text. Filename: Wallden_etal_algorithms2021_accelerating_in_transit_co_processing_for_scientific_simulations.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo
Download (6MB)| Preview

Abstract

Increasing processing capabilities and input/output constraints of supercomputers have increased the use of co-processing approaches, i.e., visualizing and analyzing data sets of simulations on the fly. We present a method that evaluates the importance of different regions of simulation data and a data-driven approach that uses the proposed method to accelerate in-transit co-processing of large-scale simulations. We use the importance metrics to simultaneously employ multiple compression methods on different data regions to accelerate the in-transit co-processing. Our approach strives to adaptively compress data on the fly and uses load balancing to counteract memory imbalances. We demonstrate the method’s efficiency through a fluid mechanics application, a Richtmyer–Meshkov instability simulation, showing how to accelerate the in-transit co-processing of simulations. The results show that the proposed method expeditiously can identify regions of interest, even when using multiple metrics. Our approach achieved a speedup of 1.29× in a lossless scenario. The data decompression time was sped up by 2× compared to using a single compression method uniformly.

ORCID iDs

Walldén, Marcus, Okita, Masao, Ino, Fumihiko, Drikakis, Dimitris and Kokkinakis, Ioannis ORCID logoORCID: https://orcid.org/0000-0001-9694-3764;
CORE (COnnecting REpositories)