# openfoam

## Summary

+ **Support Level:** Secondary Support
+ **Software Access Level:** Open Access
+ **Home Page:** [https://www.openfoam.com](https://www.openfoam.com)

## Software Description

OpenFOAM is an open source software package which is designed to scale
well on large distributed memory systems. While originally written for
CFD, with it\'s open source strategy and a large user base OpenFOAM has
grown to be a general purpose finite element toolbox.
## General Linux

MSI currently provide OpenFOAM outside of the module system.

Several working examples are posted, which illustrate how to run interactive with graphics using ParaFOAM,
and how to run non-interactively in the Slurm batch queue.

**Run Interactively with Graphics**

In a terminal window on an MSI OnDemand desktop

	ssh -X agate
	/home/dhp/public/sing/openfoam6/run

This will start  an instance of the openfoam6-paraview54 singularity container.
At the prompt in this container, enter you can  get and run backward
facing step with simpleFoam with:

	cp -r $FOAM_TUTORIALS/incompressible/simpleFoam/pitzDaily .
	cd pitzDaily
	blockMesh
	simpleFoam

View results with Paraview

	export HOME=$myhome
	paraFoam

Exit container

	exit

**Run in Slurm batch**

From a login shell on Agate, get a copy of the  OpenFOAM v6 tutorial pitzDaily example, Slurm
run script, and submit it to the queue:

	cp -r /common/software/install/migrated/openfoam/6.0/pitzDaily_v6 .
	cd pitzDaily_v6 cp /common/software/install/migrated/openfoam/6.0/slurm_run .
	sbatch slurm_run

**NOTES**

**Problem size**

This is a tiny example which runs for a very short time. The Slurm
resource request only asks for 2 cores and 10 min of run time. In
fact, it's running OpenFOAM single-threaded, and only asks for 2 cores
to ensure it has enough memory. To run larger problems, request all available cores
on a node and a longer wall clock time. For example, to run for as
long as 6 hours and have sufficient memory, use a resource request like:

	#SBATCH --nodes=1 --ntasks-per-node=24 --mem-per-cpu=2580M --time=06:00:00

**Customize problem**

This example runs blockMesh, simpleFoam, and foamToVTK non-interactively
in a Slurm job, using a singularity container. It does so by first
creating a bash script (sing\_input.sh) and then running the container on
this script with

	singularity exec $container_file $wrkdir/sing_input.sh

You can modify run_slurm to run your own OpenFOAM
commands for your own use case.

**Working directory**

This example is setup
to work as-is for any user, so you do not need to make any modifications
to see it run.  It has you copy the OpenFOAM v6 tutorial pitzDaily
directory to a location anywhere under your home directory.  You then
use this directory as a working directory (i.e.,. where you run your
job, and OpenFOAM does its work). The Slurm script (see the file slurm_run
copied below) generates a path to the working directory in the shell
variable wrkdir with

	wrkdir=$HOME/$(echo $PWD | cut -d'/' -f8-)

You can leave this as is if you want to have your Slurm run script in the
OpenFOAM project directory.

**Example Slurm Batch File (slurm_run)**

	#!/bin/bash
	#SBATCH --nodes=1 --ntasks-per-node=2
	#SBATCH --mem-per-cpu=2580M
	#SBATCH --time=00:10:00
	#SBATCH --output=%j.out

	cd $SLURM_SUBMIT_DIR

	# Construct working directory path that will be valid in container
	wrkdir=$HOME/$(echo $PWD | cut -d'/' -f8-)

	# Create bash shell script with commands you want to run in container
	echo "#!/bin/bash

	# Set OpenFOAM6 environment, and working directory, inside of container
	source /opt/openfoam6/etc/bashrc
	cd $wrkdir

	# Your OpenFOAM commands go here (must be non-interactive)
	blockMesh
	simpleFoam
	foamToVTK
	" > sing_input.sh
	chmod 755 sing_input.sh

	module load singularity
	container_file=/common/software/install/migrated/openfoam/6.0/openfoam6-paraview54.simg
	singularity exec $container_file $wrkdir/sing_input.sh