Python packages are required to be installed on your system in a directory listed in your sys.path, and the other packages/libraries are required to be in standard directories listed in your system path (e.g., /usr/bin, /sw/include, etc.).
Make sure the executable for Python can be called at the Unix command line by typing both python. You might need to define a Unix alias that maps python2.4 (or whichever version of Python you are using) to python.
In this manual, the path to <<1070>>qtcm-0.1.2 will be called the ``qtcm build path'' and be given as /buildpath. When you see /buildpath, please substitute the actual temporary directory you created for building purposes.
| make clean
make _qtcm_full_365.so make _qtcm_parts_365.so |
| python test_all.py |
At some point, I will automate the installation using Python's distutils utilities.
You must have a Fortran compiler installed on your system in order to compile qtcm. The compiler must be able to interface with a pre-processor, as QTCM1 makes copious use of pre-processor directives. qtcm is known to work with the following Fortran compilers on the following platforms:
| Compiler | Compiler Web Site | Platform(s) |
|---|---|---|
| g95 | http://www.g95.org/ | Mac OS X |
It will probably work with other platforms, but I haven't been able to test platforms besides those listed above. Note that g95is not GNU Fortran (gfortran), the Fortran 95 compiler included with the more recent versions of GCC.
The following Python packages are required to be installed on your system in a directory listed in your sys.path:
One other required Python package, f2py, is now a part of the NumPy package, and so installation of NumPy is sufficient to give you both.
The package SciPy, which includes several Python-accessible scientific libraries, also includes NumPy (and thus f2py), so if you install SciPy, you don't have to install NumPy again. Note that SciPy is not the same as Scientific Python; the names are confusing.
A few non-Python packages are also required:
For most Unix installations, the easiest way to install all the above is via a package manager, for instance apt-get in Debian GNU/Linux, aptitude or synaptic in Ubuntu GNU/Linux, and fink in Mac OS X. Of course, you can also download a package's source code and build direct and/or install using Python's distutils utilities.
Two extension modules are created: _qtcm_full_365.so and _qtcm_parts_365.so. Both modules define QTCM1 models where:
(All other makefile macros not listed are also turned off.) The only difference between these two extension modules is that the ``full'' module is used by Qtcm instances where compiled_form is set to 'full', and the ``parts'' module is used by Qtcm instances where compiled_formis set to 'parts'. See Section 4.4 for details about the compiled_form attribute.
The extension modules are created through the following steps:
| NCINC=-I/yourpath/netcdf/include
NCLIB=-L/yourpath/netcdf/lib -lnetcdf |
| make clean && make _qtcm_full_365.so && make _qtcm_parts_365.so |
The makefile will automatically move the shared object files into ../lib, overwriting any pre-existing files of the same name. A detailed description of the makefile and using f2py is given in Section 6.6, if you wish to create a different extension module.
NB: For these tests to work, both python and python2.4 must refer to the executable for the Python installation on your system that you are using for running qtcm.
The tests require a set of benchmark output files in the test/benchmarks directory in the <<1757>>qtcm-0.1.2 directory (the output will be indirectories whose names begin with ``aquaplanet'' or ``landon''). These output files are not included with the qtcm distribution, and must be created, by doing the following:
| NCINC=-I/yourpath/netcdf/include
NCLIB=-L/yourpath/netcdf/lib -lnetcdf |
The created benchmarks will be located in test/benchmarks, in directories with names related to the run that was done, as described earlier. The benchmarks are created using the pure-Fortran QTCM1 model code, version 2.3 (August 2002), with an altered makefile (described above) and the following code change: In all .F90 files, occurrences of:
| Character(len=130) |
| Character(len=305) |
Once the benchmarks are created, you can test the qtcm package by doing the following:
If at the end of the test runs you see this message (or something similar):
|
-----------------------------------
Ran 93 tests in 1244.205s OK |
The wall-clock time values below give the mean over three
separate 365 day aquaplanet runs,
using climatological sea surface temperature for lower boundary forcing.
NetCDF output is written daily, for both instantaneous and mean values.
The time step is 1200 sec, and the version of qtcm used
is 0.1.1.
The horizontal grid spacing of all model versions is
longitude by 
latitude.
Values are in seconds:
| System | Pure | Full | Parts |
|---|---|---|---|
| Mac OS X: MacBook 1.83 GHz Intel Core Duo running Mac OS X 10.4.10 with 1 GB RAM (Python 2.4.3, NumPy 1.0.3, f2py 2_3816). | 152.59 | 153.63 | 158.94 |
| Ubuntu GNU/Linux: Dell PowerEdge 860 with 2.66 GHz Quad Core Intel Xeon processors (64 bit) running Ubuntu 8.04.1 LTS (Python 2.5.2, NumPy 1.1.0, f2py 2_5237). | 43.73 | 44.79 | 47.45 |
``Pure'' refers to the pure-Fortran version of QTCM1. ``Full'' refers to a qtcm run session with compiled_formset to 'full'. ``Parts'' refers to a qtcm run session with compiled_form set to 'parts'. (Section 4.4 has details about the difference between compiled forms.)
The 'parts' version of qtcm gives Python the maximum flexibility in accessing compiled QTCM1 model subroutines and variables. The price of that flexibility is an increase in run time of approximately 4-9% over the pure-Fortran version. The difference in performance between the 'full' version of qtcm and the pure-Fortran version of QTCM1 is between negligible and 3% longer.
To make a timing for the pure-Fortran model, go to test/benchmarks/timing/work in /buildpath and run the timing_365.sh script in that directory. That script runs the QTCM1 model using /usr/bin/time, which at the end of the script will output the amount of time it took to make the model run. Run the timing script three times and average the values to obtain a time comparable to the above.
To make a timing for the qtcm model, type python timing_365.py while in the test directory in /buildpath. Three run sessions will be made for compiled_form equal to 'full' and 'parts', the times are averaged, and the value are output at the end of the script.
This section describes issues and a summary of the installation steps I followed to install qtcm on a Mac running OS X. It is a specific realization of the general installation instructions found in Sections 2.1-2.5. I first worked through these installation steps during June-July 2007, with updates during July 2008. The best way to go through this section is to go through the summary of the installation steps in Section 2.7.6, and looking back to other sections as needed.
This work was done on a MacBook 1.83 GHz Intel Core Duo running Mac OS X 10.4.11. My machine has 1 GB RAM and 64 GB of disk in its main partition.
I recommend you turn-off your antivirus software before you do the installs. Problems have been reported by Fink users using the Fink package manager with antivirus software enabled.
There are a variety of dependencies that are required to get your Mac up-and-running as a scientific computing platform. The most basic is installing Apple's XCode developer tools.2.4This set of tools contains compilers and libraries needed to do anything further. You have to be a member of Apple's Developer Connection, but registration is free.
Besides XCode, there are a variety of Unix libraries and utilities that you need. I first tried installing them by myself, from scratch, into /usr/local, but it was hard to keep track of all the dependencies. A few that did work, and that I installed from their disk images, are: MacTeX, MAMP, and Tcl/Tk Aqua BI (Batteries Included).2.5
For everything else, thankfully, there's the Fink Project which uses a package manager built upon Debian tools to install ports of Unix programs onto a Mac. I just downloaded a binary version of the Fink 0.8.1 installer for Intel Macs, installed Fink, and used its package management tools to install (almost) everything else I needed.2.6
Although you do not need anything besides a Fortran compiler and
the netCDF libraries to run QTCM1 in its pure-Fortran form, in order to
manipulate the model and use this Python version qtcm, you
need to have Python installed. The default Python that comes
with the Mac is a little old, so I used Fink to also install
Python 2.5 and related packages, including
matplotlib,
ScientificPython,
and
SciPy
(see Section ![[*]](crossref.png)
for details).
There are a variety of high-quality, commercial Fortran compilers. Unfortunately, because I do not have a research budget, I am not able to use those compilers. The GNU Compiler Collection (GCC) provides a suite of open-source compilers, some of which are the standards of their language. Most of the GCC compilers are installed on your Mac when you install XCode.
GNU Fortran
(gfortran), is the Fortran 95 compiler included with the more
recent versions of GCC.
Unfortunately, I was not able to get it to compile QTCM1.
There is a second open-source Fortran compiler,
G95
(g95),
which some feel is farther along in its development than gfortran.
I was able to successfully compile QTCM1 with g95 on my Mac.
I used Fink to install G95
(see Section for details).
For some reason, the netCDF libraries and include files installed by Fink didn't correspond to the files needed by the calling routines in qtcm. To solve this, I compiled my own set of netCDF 3.6.2 libraries using the tarball downloaded from UCAR.
Once I uncompressed and untarred the package, and went into the top-level directory of the package, I built the package by typing the following at the Unix prompt:
| ./configure -prefix=/Users/jlin/extra/netcdf
make check make install |
This installed the netCDF binaries, libraries, and include files into sub-directories bin, lib, and include in the directory specified by -prefix. If you want to install the netCDF libraries in the default (usually /usr/local), just leave out the -prefixoption.
Note: When you build netCDF, make sure the build directory is not in the directory tree of -prefix(or the default directory /usr/local).
In the src directory in the qtcm distribution, there is a sub-directory Makefiles that contains the makefiles for a variety of platforms. Edit the file makefile.osx_g95 so that the lines specifying the environment variables for the netCDF libraries and include files:
| NCINC=-I/Users/jlin/extra/netcdf/include
NCLIB=-L/Users/jlin/extra/netcdf/lib -lnetcdf |
are changed to the path where your manually compiled netCDF libraries and include files are.
Copy makefile.osx_g95 from the Makefiles sub-directory in src into src. In other words, from the qtcm distribution directory (i.e., /buildpath), at the Unix prompt execute:
| cp src/Makefiles/makefile.osx_g95 src/makefile |
Compilers in the GNU Compiler Collection (GCC) search libraries and object files in the order they are listed in the command-line, from left-to-right. Thus, if routines in b.o call routines in a.o, you must list the files in the order a.o b.o.
For some reason, that isn't the case for g95. Thus, you will find g95 makefile rules structured like the following (below is part of the rule to create an executable (qtcm) for benchmark runs):
qtcm: main.o
$(FC) -O $(NCINC) -o
$@ main.o $(QTCMLIB) $(NCLIB)
The following summarizes all the steps I took to install qtcm in Mac OS X:
Note that for many of the packages needed to run qtcm, you need to configure Fink to download packages from the unstable trees. To do that, add unstable/main and unstable/crypto to the Trees: line in /sw/etc/fink.conf, and run:
| fink selfupdate
fink index fink scanpackages fink update-all |
When selfupdate runs, choose rsync for the self update method. If you do not, Fink will not look in the unstable trees for packages.
fink - -use-binary-dist install g95 |
| fink --use-binary-dist install python25
fink - -use-binary-dist install scipy-core-py25 |
(Numpy used to be called SciPy Core.) If you want to install Python 2.4 instead, just change the ``25'' and ``py25'' above (and in later occurrences) to ``24'' and ``py24'', respectively. Note that Fink does not have a version of epydoc for Python 2.4, so if you wish to create documentation using epydoc, you will need to install Python 2.5.
| fink --use-binary-dist install tetex
fink - -use-binary-dist install latex2html |
When prompted, choose ghostscript and ghostscript-fonts to satistfy the dependency (which should be the default options). I tried choosing system-ghostscript8, but Fink looks for ghostscript 8.51 and didn't recognize ghostscript 8.57 that was already installed in /usr/local (via my MacTeX install). LATEX2HTML has a package required by the qtcm manual LATEX file.
| fink --use-binary-dist install scientificpython-py25
fink - -use-binary-dist install matplotlib-py25
fink - -use-binary-dist install matplotlib-basemap-py25
fink - -use-binary-dist install matplotlib-basemap-data-py25
fink - -use-binary-dist install xaw3d
fink - -use-binary-dist install fftw fftw3
fink - -use-binary-dist install epydoc-py25
fink - -use-binary-dist install graphviz
fink - -use-binary-dist install scipy-py25 |
This section describes installation issues I followed to install qtcm on my Dell PowerEdge 860 running Ubuntu GNU/Linux 8.04.1 LTS (Hardy). The machine has 2.66 GHz Quad Core Intel Xeon processors (64 bit), 4 GB RAM, and 677 GB of disk in its main partition. This section is a specific realization of the general installation instructions found in Sections 2.1-2.5. I worked through these installation steps during July 2008. The best way to go through this section is to go through the summary of the installation steps in Section 2.8.5, and looking back to other sections as needed.
The easiest Fortran compiler to install in Ubuntu 8.04.1 is GNU Fortran (gfortran), the Fortran 95 compiler included with the more recent versions of the GNU Compiler Collection (GCC); you can use any package manager (e.g., apt-get, aptitude) to install it. Unfortunately, I was not able to get it to compile QTCM1. I was, however, able to successfully compile QTCM1 using the second open-source Fortran compiler, G95 (g95), which some feel is farther along in its development than gfortran. G95, however, is not supported as an Ubuntu package, and so I had to manually install it.
I downloaded the binary version of G95 v0.91 (the Linux x86_64/EMT64 with 32 bit default integers) using the following curl command:2.9
| curl -o g95.tgz http://ftp.g95.org/v0.91/g95-x86_64-32-linux.tgz |
which saves the .tgz file as the local file g95.tgz. After that, I followed the G95 project's standard installation instructions to finish the install.2.11The regular Linux x86 version of G95 (in g95-x86-linux.tgz from the G95 website) did not work on my machine.
For some reason, the netCDF libraries and include files installed from the Ubuntu packages do not correspond to the files needed by the calling routines in qtcm. To solve this, I compiled my own set of netCDF 3.6.2 libraries using the tarball downloaded from UCAR.
Once I uncompressed and untarred the package, and went into the top-level directory of the package, I built the package by typing the following at the Unix prompt:
| export FC=g95
export FFLAGS="-O -fPIC" export FFLAGS="-fPIC" export F90FLAGS="-fPIC" export CFLAGS="-fPIC" export CXXFLAGS="-fPIC" ./configure make check sudo make install |
(The export commands set environment variables for the Fortran compiler and Fortran and other compiler flags. The -fPIC flag enables the compilers to create position independent code, needed for shared libraries in Ubuntu on a 64 bit Intel processor.)
The above installs the netCDF binaries, libraries, and include files into sub-directories bin, lib, and include in /usr/local, the default. The include files for this netCDF installation are thus located in /usr/local/include, and the libraries for this netCDF installation are location in /usr/local/lib. (If you want to specify a different installation location, use the -prefix option in configure.) While you don't have to have root privileges during the configuration and check steps, you do during the installation step if you're installing into /usr/local (thus the sudo in the last step).2.13
In the src directory in the qtcm distribution, there is a sub-directory Makefiles that contains the makefiles for a variety of platforms. Edit the file makefile.ubuntu_64_g95 so that the lines specifying the environment variables for the netCDF libraries and include files:
| NCINC=-I/usr/local/include
NCLIB=-L/usr/local/lib -lnetcdf |
are changed to the path where your manually compiled netCDF libraries and include files are.
Copy makefile.ubuntu_64_g95 from the Makefiles sub-directory in src into src. In other words, from the qtcm distribution directory (i.e., /buildpath), at the Unix prompt execute:
| cp src/Makefiles/makefile.ubuntu_64_g95 src/makefile |
Compilers in the GNU Compiler Collection (GCC) search libraries and object files in the order they are listed in the command-line, from left-to-right. Thus, if routines in b.o call routines in a.o, you must list the files in the order a.o b.o.
For some reason, that isn't the case for g95. Thus, you will find g95 makefile rules structured like the following (below is part of the rule to create an executable (qtcm) for benchmark runs):
qtcm: main.o
$(FC) -O $(NCINC) -o
$@ main.o $(QTCMLIB) $(NCLIB)
In order to compile the model in Ubuntu on a 64 bit Intel processor, the model and the netCDF library it is linked to needs to be compiled to be position independent code (PIC). This is accomplished with the -fPIC flag.
In the qtcm makefiles, the -fPIC flag is introduced in the macro FFLAGSM, for instance:
| FFLAGSM = -O -fPIC |
| -f90flags="-fPIC" -f77flags="-fPIC" |
The -fPIC flag must also be used when compiling the netCDF libraries, as described in Section 2.8.3. Failure to create PIC libraries in 64 bit Ubuntu can result in errors like the following when creating the qtcm extension modules:
| ld: /usr/local/lib/libnetcdf.a(fort-attio.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /usr/local/lib/libnetcdf.a: could not read symbols: Bad value |
The following summarizes all the steps I took to install qtcm in Ubuntu 8.04.1 LTS (Hardy) running on a Quad Core Intel Xeon (64 bit) machine. Note that while I use the aptitude package manager, you are free to use any manager of your choice (e.g., apt-get, synaptic, etc.):
| sudo aptitude update
sudo aptitude install curl sudo aptitude install python-epydoc sudo aptitude install python-matplotlib sudo aptitude install python-netcdf sudo aptitude install python-scientific sudo aptitude install python-scipy sudo aptitude install texlive |
Installing python-scipy will also install NumPy and f2py, so you don't have to install the python-numpy package separately.
Early-on as I debugged my qtcm install on Ubuntu, I encountered errors that I thought came from an old version of NumPy, and thus I replaced Ubuntu's packaged NumPy with NumPy 1.1.0 built directly from source. (Note, you shouldn't install your new NumPy in the default location, which may cause problems later-on with Ubuntu's package manager.) Later on, I concluded the errors I had encountered were not because of the NumPy version, but by then I didn't want to try to reinstall NumPy again. So strictly speaking, the version of Numpy I used is not the one bundled with python-scipy, but that shouldn't be a problem.
curl -o basemap.tar.gz 
http://voxel.dl.sourceforge.net/sourceforge/matplotlib/basemap-0.9.9.1.tar.gz |
The README file in the basemap-0.9.9.1 directory has detailed installation instructions. Note that you have to install the GEOS library first (README has detailed directions on how to do that too). To be on the safe-side, I would set the FC environment variable to the G95 compiler (e.g., with export FC=g95 in Bash).