Redefining the Concept of Large-Scale Computational Dynamics: Billion-Degree of Freedom Terramechanics Simulation on Affordable Hardware
Abstract
This instrumentation request is made to sustain an ARO research project that seeks to enable the analysis on a workstation of granular dynamics, fluid-solid interaction, and terramechanics problems that currently require dedicated, HPC-grade hardware. In granular dynamics, where currently a large simulation rarely exceeds 500,000 elements, Japanese researchers have run, in what is the largest experiment of practical relevance to date, a simulation with 2.4 billion elements using 16,384 CPUs (131,072 cores) of JapanÕs Ksupercomputer (the 2012 fastest supercomputer in the world). Using hardware procured through this request, we will deliver solutions that run billion-body dynamics problems on $35K hardware from the convenience of a Python script. To that end, we will (i) concentrate the technical effort on multi-GPU workstations; and (ii) draw on ongoing research that guides the development of an open source software infrastructure called Chrono. As part of an ARO project, we recently succeeded in running on one GPU simulations of practical relevance with 420 million bodies [1]. This is an almost 1000 increase compared to what is typically run on a workstation. The commodity hardware and fast interconnect solicited will help by: (a) increasing the problem size to more than 1 billion elements ($130K budget item), and (b) increasing Ònode-to-file serverÓ data movement network speed ($70K budget item). For (a), multiple GPUs working in tandem will increase the amount of fast memory available to the simulation engine. For (b), data movement overhead incurred when running billion element dynamics simulation will be considerably reduced. This procurement will upgrade a heterogeneous cluster (see ÒEquipment & FacilitiesÓ) used in initiating more than 100 students per year in advanced computing through two classes created and taught by the PI. The hardware will also enable the development of Chrono, which has been used by more than 1,000 collaborators and partners in the areas of granular material, terradynamics, geotechnics, fluid-solid interaction, robotics, planetary exploration, soft matter physics, etc. Chrono has been adopted by the DoD HPC Modernization Program (HPCMP) ÒComputational Research and Engineering Acquisition Tools and Environments Ground VehicleÓ (CREATE-GV) as the tool for simulation of wheeled and tracked vehicles [2]. Chrono has been tested in NATO benchmarking both for wheeled & tracked vehicle mobility analysis for off-road scenarios [3, 4, 5, 6, 7]. It has been used by JPL, National Higher French Institute of Aeronautics and Space, Riken (Japan); at universities, such as UT-Austin, MIT, Michigan, Georgia Tech, University of Maryland, Northwestern University, JHU, Mississippi State University, UC-San Diego, University of Maine, University of Akron, Politecnico di Milano, University of Parma, Imperial College London, University of Edinburgh, Technical University Berlin, Harbin University, Tsinghua University, Beijing Institute of Technology, Shanghai Jiao Tong University, Kobe University, etc.; and, companies such as Caterpillar, Oshkosh Corporation, Mitsubishi Heavy Industries, British Aerospace Engineering, Kinetix, SimlabSoft, Intuitive Machines, SRI Robotics, VT MAK, Goldwind, Dynamic Dimension Technologies, Danfoss Power Solutions, etc. Over the past three years, outside groups and organizations have used Chrono in extraterrestrial applications [8, 9, 10], machine learning in robotics [11], image processing, pattern recognition and computer vision [12, 13], architectural studies [14], autonomous vehicles [15, 16, 17], tracked-vehicle design [18], fluid-solid interaction applications [19], bridge suspension [20], hardware-in-the-loop for machine-ground interaction [21], and wind turbine dynamics [22, 23]. As of April 2020, there are 326 user-forum members [24] and the PyChrono module [25] is downloaded on average six times per day.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 25, 2021
- Source ID
- W911NF2110029
Entities
People
- Dan Negrut
Organizations
- Army Contracting Command
- United States Army
- University of Wisconsin–Madison