Skip Nav

Computational Science Environment (CSE)

Table of Contents

1. Introduction

CSE Logo In efforts to provide a common platform for utilizing and developing data analysis and assessment applications in heterogeneous High Performance Computing (HPC) environments, the Computational Science Environment (CSE) has been created. CSE is a module based software release structure. CSE provides a stable suite of data analysis and assessment tools, applications and libraries that are common across most HPC environments and developers workstations. See section 4 for a list of installed packages.

CSE includes the eXtensible Data Model and Format (XDMF), a common data hub where HPC codes and tools can efficiently exchange data values and meaning. CSE provides a common platform that enables the development of full featured portable HPC applications.

1.1. Systems with CSE Installed

Systems with CSE Installed
LocationSystem(s)Install Directory
AFRL Mustang (GNU) /app/cse/CSE
Mustang (Intel) /app/cse/CSEIntel
Thunder (Intel) /apps/cse/CSE
ARL Centennial & Excalibur (GNU) /app/CSE/CSE
Centennial & Excalibur (Intel) /app/CSE/CSEIntel
ERDC Onyx & Topaz (GNU) /app/unsupported/CSE/CSE
Onyx (Intel) /app/unsupported/CSE/CSEIntel
Navy Gaffney & Koehr (GNU) /app/CSE/CSE
Gaffney & Koehr (Intel) /app/CSE/CSEIntel
Conrad & Gordon (GNU) /app/CSE/CSE
Conrad & Gordon (Intel) /app/CSE/CSEIntel

1.2. Contact Information

Support E-Mail: usarmy.apg.arl.list.cse-support@mail.mil.

2. Using CSE

2.1. Sourcing Module

If your system does not have the module program sourced, you may source the CSE version.

~$ source [CSE_INSTALL_DIR]/modules/init/bash

You can replace bash with whatever linux shell you are using. Alternatives currently provided include csh, ksh, perl, python, sh, tcsh, and zsh. Once you have sourced this directory, you should have access to the command 'modulecmd' and an alias for 'module' in your environment. Also, your modules path should be set up to point to the CSE installation you sourced.

2.2. Module Use

The first step to using CSE is to tell module where to find the cseinit files.

~$ module use [CSE_INSTALL_DIR]/modules/COTS

2.3. Checking Available Modules

Use the "module avail" command to see the COTS directory.

~$ module avail

You should see output similar to below.

------------------------ /home/username/CSE/modules/COTS/ ------------------------- 
cseinit       cseinit-devel

2.4. Module Loading and Unloading

Once CSE is installed and modules are sourced, you need to load cseinit. The cseinit module will set up your environment to see all of the available packages in CSE.

~$ module load cseinit

Loading the cseinit module will allow you to use the full extent of CSE. Call the avail command again to see all of the loadable modules.

~$ module avail cse
---------------------- /usr/cta/CSE/Datamodels/modules/data-tools ----------------------

cse/boost/1.69.0             cse/metis/latest             cse/pypcap/1.1
cse/boost/latest             cse/netcdf/4.4.1.1           cse/pypcap/latest
cse/dpkt/1.9.2               cse/netcdf/latest            cse/xdmf/2011.11.16
cse/dpkt/latest              cse/netcdf_fortran/4.4.4     cse/xdmf/latest
cse/hdf5/1.8.13              cse/netcdf_fortran/latest    cse/xdmf2/2014.09.18
cse/hdf5/latest              cse/netdmf2/2016.02.01       cse/xdmf2/latest
cse/libpcap/1.9.0            cse/netdmf2/latest           cse/xdmf3/3.3.4
cse/libpcap/latest           cse/pybindgen/0.19.0         cse/xdmf3/latest
cse/metis/5.1.0              cse/pybindgen/latest

----------------------- /usr/cta/CSE/Analysis/modules/rel-tools ------------------------

cse/R/3.2.5                  cse/matplotlib/2.2.3         cse/pyblosc/1.7.0
cse/R/latest                 cse/matplotlib/latest        cse/pyblosc/latest
cse/cblosc/1.15.1            cse/mgen/5.02c               cse/pygal/2.4.0
cse/cblosc/latest            cse/mgen/latest              cse/pygal/latest
cse/chorus/2017r2            cse/mock/2.0.0               cse/pytables/3.4.4
cse/chorus/latest            cse/mock/latest              cse/pytables/latest
cse/cloudpickle/0.6.1        cse/networkx/2.2             cse/pytecplot/0.8.2
cse/cloudpickle/latest       cse/networkx/latest          cse/pytecplot/latest
cse/cython/0.29.2            cse/nose/1.3.7               cse/pytz/2018.9
cse/cython/latest            cse/nose/latest              cse/pytz/latest
cse/dask/1.0.0               cse/numexpr/2.6.9            cse/s3fs/0.2.0
cse/dask/latest              cse/numexpr/latest           cse/s3fs/latest
cse/dateutil/2.7.5           cse/numpy/1.12.1             cse/scipy/1.2.0
cse/dateutil/latest          cse/numpy/latest             cse/scipy/latest
cse/distributed/1.25.2       cse/pandas/0.23.4            cse/skimage/0.14.1
cse/distributed/latest       cse/pandas/latest            cse/skimage/latest
cse/edat/2.3b                cse/partd/0.3.9              cse/sklearn/0.20.2
cse/edat/latest              cse/partd/latest             cse/sklearn/latest
cse/gnuplot/5.2.6            cse/pbr/5.1.1                cse/snappy/1.1.5
cse/gnuplot/latest           cse/pbr/latest               cse/snappy/latest
cse/h5py/2.9.0               cse/petsc4py_complex/3.10.0  cse/tecplot/2017r2
cse/h5py/latest              cse/petsc4py_complex/latest  cse/tecplot/latest
cse/lapack/3.8.0             cse/petsc4py_real/3.10.0     cse/theano/1.0.3
cse/lapack/latest            cse/petsc4py_real/latest     cse/theano/latest
cse/locket/0.2.0             cse/petsc_complex/3.10.0     cse/toolz/0.9.0
cse/locket/latest            cse/petsc_complex/latest     cse/toolz/latest
cse/lxml/4.3.0               cse/petsc_real/3.10.0        cse/tornado/5.1.1
cse/lxml/latest              cse/petsc_real/latest        cse/tornado/latest
cse/lzo/2.08                 cse/pkgconfig/1.4.0          cse/xv/3.10a
cse/lzo/latest               cse/pkgconfig/latest         cse/xv/latest


---------------------------- /usr/cta/CSE/modules/rel-tools ----------------------------

cse/Pillow/5.4.1             cse/logilab/latest          cse/scons/3.0.2
cse/Pillow/latest            cse/mercurial/4.8.1         cse/scons/latest
cse/Pygments/2.3.1           cse/mercurial/latest        cse/simplegeneric/0.8.1
cse/Pygments/latest          cse/mesa/7.7.1              cse/simplegeneric/latest
cse/anaconda2/2018.12        cse/mesa/latest             cse/sip/4.19.13
cse/anaconda2/latest         cse/mpi4py/3.0.0            cse/sip/latest
cse/anaconda3/2018.12        cse/mpi4py/latest           cse/six/1.12.0
cse/anaconda3/latest         cse/openssl/1.1.1a          cse/six/latest
cse/astroid/1.6.5            cse/openssl/latest          cse/sqlite/3.26.0
cse/astroid/latest           cse/patch/2.7.6             cse/sqlite/latest
cse/cmake/2.8.12.2           cse/patch/latest            cse/statprof/0.1.2
cse/cmake/3.9.6              cse/pexpect/4.6.0           cse/statprof/latest
cse/cmake/latest             cse/pexpect/latest          cse/swig/3.0.12
cse/decorator/4.3.0          cse/pickleshare/0.7.5       cse/swig/latest
cse/decorator/latest         cse/pickleshare/latest      cse/tcltk/8.5.19
cse/docutils/0.14            cse/ptyprocess/0.6.0        cse/tcltk/latest
cse/docutils/latest          cse/ptyprocess/latest       cse/traitlets/4.2.1
cse/flex/2.6.4               cse/pygccxml/1.9.1          cse/traitlets/latest
cse/flex/latest              cse/pygccxml/latest         cse/vtk/5.6.1
cse/gccxml/latest            cse/pylint/1.9.4            cse/vtk/6.3.0
cse/gccxml/master            cse/pylint/latest           cse/vtk/7.1.1
cse/git/2.20.1               cse/pyparsing/2.3.0         cse/vtk/latest
cse/git/latest               cse/pyparsing/latest        cse/vtk_mesa/5.6.1
cse/ipython/5.8.0            cse/pyqt/4.12.3             cse/vtk_mesa/6.3.0
cse/ipython/latest           cse/pyqt/latest             cse/vtk_mesa/7.1.1
cse/ipython_genutils/0.2.0   cse/pyro/4.74               cse/vtk_mesa/latest
cse/ipython_genutils/latest  cse/pyro/latest             cse/winpdb/1.4.8
cse/libxml2/2.9.9            cse/python/2.7.15           cse/winpdb/latest
cse/libxml2/latest           cse/python/latest           cse/wxpython/2.8.12.1
cse/libxslt/1.1.33           cse/python3/latest          cse/wxpython/3.0.2.0 
cse/libxslt/latest           cse/pyzmq/17.1.2            cse/wxpython/latest
cse/lineprofiler/2.1.2       cse/pyzmq/latest            cse/zeromq/4.2.5
cse/lineprofiler/latest      cse/qt/4.8.7                cse/zeromq/latest
cse/logilab/1.4.2            cse/qt/latest

---------------------------- /usr/cta/CSE/modules/rel-apps -----------------------------

cse/doxygen/1.8.15           cse/graphviz/2.40.1         cse/openmpi/1.10.7  
cse/openmpi/3.1.3            cse-info                    cse/doxygen/latest  
cse/graphviz/latest          cse/openmpi/2.1.5           cse/openmpi/latest  cse-tools

------------------------------ /usr/cta/CSE/modules/COTS/ ------------------------------

cseinit                      cseinit-devel

When you are done using a module, you may unload it using the module unload command.

3. CSE Structure

CSE is structured into four parts, each with its own repository.

3.1. Stable

Information: Stable contains many development and debugging tools needed for software developers, as well as some of the base components to allow CSE to work on many different systems.

Packages: cmake, curl, doxygen, gccxml, graphviz, ipython, libxml2, libxslt, lineprofiler, mesa, modules, mpi4py, openmpi, patch, pygccxml, pygments, pylint, pyqt4, pyro 4, python, qt4, readline, scons, setuptools, sip, sqlite, statprof, swig, tcltk, vtk, winpdb, wxpython, mercurial, six, zeromq, GIT, Pillow, Anaconda3, logilab, asteroid, pexpect, decorator, pickleshare, docutils, traitlets, ptyprocess, flex, pyparsing, pyzmq

3.2. Datamodels

Information: Datamodels contains packages for handling various data formats.

Packages: boost, dpkt, hdf5, libpcap, netcdf, netcdf fortran, netdmf2, pybindgen, pypcap, xdmf, xdmf2, xdmf3, metis

3.3. Analysis

Information: Analysis contains scientific and visualization tools to help analyze different types of data.

Packages: cython, dateutil, gnuplot, lapack, lzo, matplotlib, nose, numexpr, numpy, pygal, pytables, scipy, snappy, tornado, R, skimage, sklearn, theano, pandas, petsc, pytz, pyblosc, cblosc, mock, Anaconda2, mgen, TecPlot, Chorus, pytecplot, cloudpickle, networkx, dask, s3fs, distributed, edat, partd, pbr, h5py, locket, toolz, lxml, pkgconfig, xv

4. Installing Python3 Packages

Most python packages will come with a setup.py file as part of their source code. This makes the build and install process fairly simple and will be the preferred method of this tutorial as we need to be able to set a specific location to install the library instead of installing directly into the python3 directories.

4.1. Load CSE

In order to install a new python package for python3 you need to load the python3 module from CSE. If you do not have module installed to your system you can source it from CSE.

~$ source [CSE_INSTALL_DIRECTORY]/Modules/init/[bash, csh, ksh, perl, python, sh, tcsh, zsh]

Once sourced you can use it to load the CSE init module.

~$ module use [CSE_INSTALL_DIRECTORY]/Modules/COTS
~$ module load cseinit

4.2. Load Python3 Module

First you need to load the CSE Python3 module.

~$ module load cse/python3/latest

Remember the Python3 binary is named "python3" by default and not simply "python".

4.3. Build

Some packages require you to call the build command first.

~$ python3 setup.py build

4.4. Install

Next you need to call the install command. The prefix should be set to a directory you have write access too such as your home directory.

~$ python3 setup.py install --prefix=[INSTALL_DIRECTORY]

Add Install Directory to the PYTHONPATH

~$ PYTHONPATH=[INSTALL_DIRECTORY]/lib/python3.5/site-packages