Old research pages

Scientific Code Frameworks

Generally some type of modular design is required. Such a design treats the component solvers as separate units that communicate with each other and the controlling program through interfaces -- a collection of methods that translate the internal data structures of the components into a common set of data structures (and vice versa). A monolithic design, by contrast, tightly couples its components, making changes difficult. To the extent that interfaces and modularity always degrade performance somewhat, a design tradeoff exists between extensibility and performance: a completely flexible, high-level architecture usually performs poorly, while a fully inlined monolithic code can often achieve very good performance but is very difficult to maintain.

Modern object-oriented languages provide several features useful for building architectures, including encapsulation, inheritance, and polymorphism. In the context of simulations, encapsulation allows interchanging solvers that may need conflicting internal data structures. Inheritance permits the abstraction of common features of different types of solver, increasing the reuse of code. Polymorphism allows the substitution of different solvers that handle the same type of physics. However, object-oriented architectures can be massive and unwieldy, requiring application developers to adopt a framework that is far too general for their needs. Also, poorly conceived inheritance structures can be too rigid, defeating the intent to build reusable code.

Component-based architectures address these issues by providing the means for software components to describe themselves to each other in a standard way. These architectures are already in common use in the business world (e.g., CORBA, COM), but they have not seen wide application in the high-performance world because business-oriented component models impose unacceptable overhead on component interactions and do not include many features needed for high-performance applications. While we envision some type of component-based architecture as ultimately meeting our application needs, an appropriate standard for component interactions (such as that being developed by the Common Component Architecture (CCA) Forum; Armstrong et al. 1999) is still several years away.

The FLASH code framework

Figure 1. Architecture of FLASH 1.x.

An abstract representation of the first-generation FLASH code architecture appears in Figure 1. In this figure, solid lines with triangles denote inheritance relationships, while dashed lines indicate instantiation relationships. Each box represents a class, which supplies data structures to its subclasses and methods to its calling classes. Italicized method names represent virtual functions, which are satisfied by subclasses. As this figure shows, the driver module controls the principal data structures used by the code, making them available to solver objects that it instantiates from the various classes of physics solvers. For time-dependent problems, the driver uses time-splitting techniques to compose the different solvers. The solvers are divided into different classes on the basis of their ability to be composed in this fashion and upon natural differences in solution method (e.g., hyperbolic solvers for hydrodynamics, elliptic solvers for radiation and gravity, ODE solvers for source terms, etc.). The adaptive mesh refinement (AMR) library is treated in the same way as the solvers. The means by which the driver shares data with the solver objects is the primary way in which the architecture affects the overall performance of the code. Choices here, in order of decreasing performance and increasing flexibility, include global memory, argument-passing, and messaging. For historical reasons, and for the sake of performance, the first generation of FLASH uses global memory to manage many of these interfaces.

The codes we have brought together in FLASH -- including the PROMETHEUS hydrodynamical solver, specialized equation-of-state and nuclear burning codes, and the PARAMESH adaptive mesh refinement library -- were written primarily in FORTRAN 77. This language, while efficient, does not by itself support objects or many of the other features needed to construct a modular architecture. However, Fortran 90 does offer some object capability, and it provides a straightforward migration path from FORTRAN 77, since the latter is a subset of the former. Thus we chose Fortran 90 as the integration language for FLASH 1.x. Using the module capability of Fortran 90, we have replaced many of the formerly global data structures in FLASH (e.g., runtime parameters) with container classes that provide externally visible accessor methods. This allows solvers to access global information on their own terms without having to declare common variables. The global memory associated with the mesh data structure has been retained in FLASH 1.x, but FLASH 2.0, currently being tested, replaces this too with an appropriate container class.

While Fortran 90 provides some object-oriented features (Decyk, Norton, & Szymanski 1997), it does not support true inheritance or polymorphism. This limits its runtime flexibility and requires us to set class relationships at compilation instead -- a procedure that is feasible only if compilation is relatively fast, as it is for FLASH. In any case, configuring FLASH at compilation time makes it much easier for the compiler to produce aggressively optimized code, since it does not have to anticipate all possible class interactions.

Figure 2. Setup process for FLASH 1.x.

To provide inheritance and polymorphism at compilation we have created a configuration system consisting of a configuration script and several plaintext configuration files (one for each class) that describe the relationships between classes and subclasses -- for example, which of the other classes a given class requires in order to function. The source code and configuration files for the classes are stored in a directory structure that corresponds directly to the class structure. The user specifies the classes to include in the code via a plaintext file that is taken as input by the configuration script. The script then parses the appropriate configuration files, constructing a build directory containing the requested source code, checking along the way for illegal combinations or missing requirements. The setup script also constructs makefiles for the different classes by concatenating makefile fragments supplied by each subclass. It then generates a master makefile that calls the various class makefiles. After running the setup script, the user builds the code by invoking `make.' A schematic representation of this procedure appears in Figure 2.

Publications and presentations

	FLASH: An Adaptive-Mesh Hydrodynamics Code for Modeling Astrophysical Thermonuclear Flashes Fryxell, B., Olson, K., Ricker, P., Timmes, F. X., Zingale, M., Lamb, D. Q., MacNeice, P., Rosner, R., Truran, J. W., and Tufo, H. Ap. J. S. 131 273 (2000)
	Architecture of the FLASH Code (HTML, Postscript) Ricker, P. M., Fryxell, B., Olson, K., Timmes, F. X., Calder, A., Siegel, A., Tufo, H., Weirs, G., Zingale, M., and Dursi, L. J. Presented at the DOE/ASCI ASAP Frameworks Workshop, Feb. 10-11, 2000, Salt Lake City, UT

References

Armstrong, R., Gannon, D., Geist, A., Keahey, K., Kohn, S., McInnes, L., Parker, S., and Smolinski, B. Argonne Natl. Lab. MCS preprint P759-0699 (1999)

Decyk, V. K., Norton, C. D., and Szymanski, B. K. ACM Fortran Forum 16 (1997)