Android Fortran Step-by-Step Part 3: Bringing it All Together

***UPDATED For NDK-r7***
Now that we have a working Fortran compiler for our Android hardware, it's time to show the NDK tools how to use it to build native applications.  Be warned:  I am not pretending to be a master of the GNU make system and I have done the minimum amount of hacking required so this will work for my own use... YMMV

If you haven't spent any time using the Android NDK then this will probably not be very informative.  Still, I am guessing that anyone who has gotten this far probably has at least tried to build the example applications that come with the development kit.  Specifically, I am referring to the Native Activity example, which is the only one I've seen so far that uses 0% Java and 100% native C code.  My original plan was to gut most of the C and replace it with calls to Fortran routines packaged as a library with interfaces bound to the skeleton C code via ISO_C_BINDING.  As I'm planning to demonstrate in the near future, this works well within reason.

I've also decided to avoid using Eclipse for the time being and stick to doing everything with text editors (gvim) and the command line.  The main reasons for this is that the Fortran plugins for Eclipse are not great and the apk package refuses to rebuild when changes are made to the native source files... I've really tried to like Eclipse, but I just can't shake the opinion that it's difficult to install, bloated, buggy and really only meant for Java programming.  I suppose if I was a computer science major in college then I would probably be hooked on some IDE or other, but I wasn't, so I'm not.  Give me a text editor with decent syntax highlighting and a compiler that produces reasonable debugging information and I'm happy.

On with the show...

Step 1:  Prep the new toolchain for the ndk-build script

The GNU Make based native build system needs a couple of things before it will recognize the new compiler.  The first thing to do is to copy two files from the old toolchain directory into the new one:

Starting in the NDK toolchains directory; if you are following the previous instructions, that will be here:

/opt/Android/android-ndk-r7/toolchains

cp arm-linux-androideabi-4.4.3/config.mk arm-linux-androideabi-4.7.0/

cp arm-linux-androideabi-4.4.3/setup.mk arm-linux-androideabi-4.7.0/

Now, if you built the 64-bit versions of the compiler executables, you will need to fake out the build script by making a symbolic link from the (non-existant) 32-bit tools to the 64-bit ones.

cd arm-linux-androideabi-4.7.0/prebuilt

ln -s linux-x86_64 linux-x86

Step 2: Add gfortran support to ndk-build

This was by far the hardest part for me since I am not a shell scripting master and definitely not a Make system master, but the result works well enough, although it could be much better I'm certain.  I would just make some diff patches, but I don't have a reliable way to post them here, so I'll just put the text of the changes in their entirety with instructions as to where to put them.

The first thing to do is to set an environment variable that tells the NDK which toolchain to use.  Add the following to your ~/.bashrc file:

export NDK_TOOLCHAIN=arm-linux-androideabi-4.7.0

(remember that you need to start a new terminal for the variable to be used)

UPDATE: If you followed the new instructions in the previous post then skip the next section and go straight to the bottom.. you're done!


Edit this file:

/opt/Android/android-ndk-r6b/build/core/build-binary.mk

Change line 195 to this:

all_source_extensions := .c .s .S $(LOCAL_CPP_EXTENSION) .f .f90

Add these lines starting at line 223:

# handle free-form Fortran (.f90)
$(foreach src,$(filter %.f90,$(LOCAL_SRC_FILES)), $(call compile-f90-source,$(src)))
# handle fixed form Fortran (.f)
$(foreach src,$(filter %.f,$(LOCAL_SRC_FILES)), $(call compile-fc-source,$(src)))

Edit this file:

/opt/Android/android-ndk-r6b/build/core/default-build-commands.mk

Add these lines starting at line 93:

TARGET_FC       = $(TOOLCHAIN_PREFIX)gfortran
TARGET_FFLAGS   =

Now for the big one...


Edit this file:

/opt/Android/android-ndk-r6b/build/core/definitions.mk

Add this chunk starting at line 1040:


# slightly modified version for Fortran source files
define ev-build-fc-file
$$(_OBJ): PRIVATE_SRC      := $$(_SRC)
$$(_OBJ): PRIVATE_OBJ      := $$(_OBJ)
$$(_OBJ): PRIVATE_DEPS     := $$(call host-path,$$(_OBJ).d)
$$(_OBJ): PRIVATE_MODULE   := $$(LOCAL_MODULE)
$$(_OBJ): PRIVATE_TEXT     := "$$(_TEXT)"
$$(_OBJ): PRIVATE_CC       := $$(_CC)
$$(_OBJ): PRIVATE_CFLAGS   := $$(_FLAGS)
$$(_OBJ): $$(_SRC) $$(LOCAL_MAKEFILE) $$(NDK_APP_APPLICATION_MK)
    @mkdir -p $$(dir $$(PRIVATE_OBJ))
    @echo "$$(PRIVATE_TEXT)  : $$(PRIVATE_MODULE) <= $$(notdir $$(PRIVATE_SRC))"
    $(hide) $$(PRIVATE_CC) $$(PRIVATE_CFLAGS) $$(call host-path,$$(PRIVATE_SRC)) -o $$(call host-path,$$(PRIVATE_OBJ))
endef


# This assumes the same things than ev-build-fc-file, but will handle
# the definition of LOCAL_FILTER_ASM as well.
define ev-build-fc-source-file
LOCAL_DEPENDENCY_DIRS += $$(dir $$(_OBJ))
ifndef LOCAL_FILTER_ASM
  # Trivial case: Directly generate an object file
  $$(eval $$(call ev-build-fc-file))
else
  # This is where things get hairy, we first transform
  # the source into an assembler file, send it to the
  # filter, then generate a final object file from it.
  #


  # First, remember the original settings and compute
  # the location of our temporary files.
  #
  _ORG_SRC := $$(_SRC)
  _ORG_OBJ := $$(_OBJ)
  _ORG_FLAGS := $$(_FLAGS)
  _ORG_TEXT  := $$(_TEXT)


  _OBJ_ASM_ORIGINAL := $$(patsubst %.o,%.s,$$(_ORG_OBJ))
  _OBJ_ASM_FILTERED := $$(patsubst %.o,%.filtered.s,$$(_ORG_OBJ))


  # If the source file is a plain assembler file, we're going to
  # use it directly in our filter.
  ifneq (,$$(filter %.s,$$(_SRC)))
    _OBJ_ASM_ORIGINAL := $$(_SRC)
  endif


  #$$(info SRC=$$(_SRC) OBJ=$$(_OBJ) OBJ_ORIGINAL=$$(_OBJ_ASM_ORIGINAL) OBJ_FILTERED=$$(_OBJ_ASM_FILTERED))


  # We need to transform the source into an assembly file, instead of
  # an object. The proper way to do that depends on the file extension.
  #
  # For C and C++ source files, simply replace the -c by an -S in the
  # compilation command (this forces the compiler to generate an
  # assembly file).
  #
  # For assembler templates (which end in .S), replace the -c with -E
  # to send it to the preprocessor instead.
  #
  # Don't do anything for plain assembly files (which end in .s)
  #
  ifeq (,$$(filter %.s,$$(_SRC)))
    _OBJ   := $$(_OBJ_ASM_ORIGINAL)
    ifneq (,$$(filter %.S,$$(_SRC)))
      _FLAGS := $$(patsubst -c,-E,$$(_ORG_FLAGS))
    else
      _FLAGS := $$(patsubst -c,-S,$$(_ORG_FLAGS))
    endif
    $$(eval $$(call ev-build-fc-file))
  endif


  # Next, process the assembly file with the filter
  $$(_OBJ_ASM_FILTERED): PRIVATE_SRC    := $$(_OBJ_ASM_ORIGINAL)
  $$(_OBJ_ASM_FILTERED): PRIVATE_DST    := $$(_OBJ_ASM_FILTERED)
  $$(_OBJ_ASM_FILTERED): PRIVATE_FILTER := $$(LOCAL_FILTER_ASM)
  $$(_OBJ_ASM_FILTERED): PRIVATE_MODULE := $$(LOCAL_MODULE)
  $$(_OBJ_ASM_FILTERED): $$(_OBJ_ASM_ORIGINAL)
    @echo "AsmFilter      : $$(PRIVATE_MODULE) <= $$(notdir $$(PRIVATE_SRC))"
    $(hide) $$(PRIVATE_FILTER) $$(PRIVATE_SRC) $$(PRIVATE_DST)


  # Then, generate the final object, we need to keep assembler-specific
  # flags which look like -Wa,:
  _SRC   := $$(_OBJ_ASM_FILTERED)
  _OBJ   := $$(_ORG_OBJ)
  _FLAGS := $$(filter -Wa%,$$(_ORG_FLAGS)) -c
  _TEXT  := "Assembly     "
  $$(eval $$(call ev-build-fc-file))
endif
endef


...And this chunk at line 1213:


# -----------------------------------------------------------------------------
# Template  : ev-compile-fc-source
# Arguments : 1: single Fortran source file name (relative to LOCAL_PATH)
#             2: target object file (without path)
# Returns   : None
# Usage     : $(eval $(call ev-compile-fc-source,,)
# Rationale : Internal template evaluated by compile-fc-source
# -----------------------------------------------------------------------------


define  ev-compile-fc-source
_SRC:=$$(LOCAL_PATH)/$(1)
_OBJ:=$$(LOCAL_OBJS_DIR)/$(2)
_FLAGS := $$($$(my)FFLAGS) \
          $$(call get-src-file-target-cflags,$(1)) \
          $$(call host-c-includes, $$(LOCAL_C_INCLUDES) $$(LOCAL_PATH)) \
                     $$(LOCAL_CFLAGS) \
                    $$(NDK_APP_CFLAGS) \
          $$(LOCAL_FFLAGS) \
          $$(NDK_APP_FFLAGS) \
          $$(call host-c-includes,$$($(my)C_INCLUDES)) \
          -c \


_CC   := $$(TARGET_FC)
_TEXT := "Compile-fortran $$(call get-src-file-text,$1)"


$$(eval $$(call ev-build-fc-source-file))
endef


# -----------------------------------------------------------------------------
# Function  : compile-fc-source
# Arguments : 1: single Fortran source file name (relative to LOCAL_PATH)
# Returns   : None
# Usage     : $(call compile-fc-source,)
# Rationale : Setup everything required to build a single Fortran source file
# -----------------------------------------------------------------------------
compile-fc-source = $(eval $(call ev-compile-fc-source,$1,$(1:%.f=%.o)))
compile-f90-source = $(eval $(call ev-compile-fc-source,$1,$(1:%.f90=%.o)))

These are just slightly modified versions of the functions that handle C/C++ source code.  Once this is done, you should be able to put fixed form (.f) and free-form (.f90) Fortran source files in the jni folder of your native application, add them to LOCAL_SRC_FILES in ./jni/Android.mk, and then run ndk-build like normal to compile your Fortran into the shared library for your Android application.  If you are using Eclipse you should be able to build and deploy directly to your device, but remember that it will not regenerate the apk if you make changes to the source code (at least that's what happened to me).  If you are on the command line then you can use ant to build the apk and deploy it from within the project folder:

ndk-build -B V=1

ant release

ant install

When I find some more time, I'll put up an example of a working native application that demonstrates how to package some assets, move them to the device SD card and then use them to do something fun in Fortran using OpenGLES and the device sensors.

Android Fortran Step-by-Step Part 2: Building a Custom GCC Toolchain

***Updated for NDK-r7*** 
 This part is relatively tricky.  I would never have gotten this far without the help of another clever person who needed a newer GCC toolchain for Android, but for a completely different set of reasons.

Thank you: http://glandium.org/blog/?p=2146

There are a lot of steps here, so I'll try not to clutter it up with jabber.  If you have completed Part 1, then you should be ready to dive right in and start cutting and pasting the following commands.  Maybe if I find some time, I'll make this into a shell or Python script that will automate the process, maybe not.

Step 0:  Get a few more required packages

sudo yum install subversion ncurses-devel flex bison texinfo

Step 1:  Get the NDK build package

This would normally be installed by the ./build/tools/download-toolchain-sources.sh script included with the NDK, but since kernel.org was hacked a couple of months ago that doesn't seem to be working.  Luckily it is available from the glandium.org post, so get it here:


https://bugzilla.mozilla.org/attachment.cgi?id=549787

Copy the ndk-build-r6.tar.gz2 file into /opt/Android/android-ndk-r6b and extract it there:

 tar jxvf ndk-build-r6.tar.bz2

You should now have a directory called:

 /opt/Android/android-ndk-r6b/src/build

UPDATE: The google source repositories are back, so you don't need to use the download-toolchain-sources.sh script.  Simply use git (yum install git if you don't have it already):

mkdir src
git clone https://android.googlesource.com/toolchain/build src/build

Step 2:  Put the GCC sources into your Android NDK install

cd /opt/Android/android-ndk-r7

mkdir -p src/gcc

svn export -r182917 svn://gcc.gnu.org/svn/gcc/trunk src/gcc/gcc-4.7.0

This will give you the exact set of sources that I used.  Depending on your connection speed this could also take some time to finish.  If you want to be adventurous, you can get the bleeding edge sources by omitting the -r182917 from the command above.  You can also use checkout instead of export if you want to be able to keep your sources up-to-date via 'svn update'.  Of course, as of the date of this post, this version IS the bleeding edge ;)

Step 3: Patch the GCC sources


There is an anoying 'bug' in the GCC gfortran configure process when building a cross-compiler (at least this one anyway).  Fortunately, this is much easier with the latest sources than it was the first time I tried it, but it still is a bit of a hack.


Open this file:

/opt/Android/android-ndk-r6b/src/gcc/gcc-4.7.0/libgfortran/configure

Change 'exit $1' on lines 279 and 26306 to 'echo $1'

This will keep the configure script from crashing because of a link test that can't be done for the cross-compiler.  If anyone has a more elegant solution to this then please post it in the comments!


Now open this file:

/opt/Android/android-ndk-r6b/src/gcc/gcc-4.7.0/libgfortran/io/unix.c

Add the following starting on line 1113:

#if !defined(S_IREAD) && defined(S_IRUSR)

#define S_IREAD  S_IRUSR

#define S_IWRITE S_IRUSR

#endif

This fixes a file permission issue when linking against the Android Bionic C library instead of the normal glibc.

Step 4: Get all of the extra sources you will need

There are a number of extra packages that are required in order to build a fully functional GCC cross-compiler.  These also need to be put in the right location for the Android helper script to use.  Note that the sources for gcc binutils and gdb need to be extracted, the others can stay packed up.

cd src

mkdir binutils

cd binutils
wget http://sourceware.mirrors.tds.net/pub/sourceware.org/binutils/snapshots/binutils-2.22.51.tar.bz2

tar -jxvf binutils-2.22.51.tar.bz2

cd ..

mkdir gmp

cd gmp

wget http://ftp.gnu.org/gnu/gmp/gmp-5.0.2.tar.bz2

cd ..

mkdir mpfr

cd mpfr

wget http://ftp.gnu.org/gnu/mpfr/mpfr-3.0.1.tar.bz2

cd ..

mkdir mpc

cd mpc

wget http://www.multiprecision.org/mpc/download/mpc-0.9.tar.gz

cd ..

mkdir gdb

cd gdb

wget http://ftp.gnu.org/gnu/gdb/gdb-7.3a.tar.bz2

tar -jxvf gdb-7.3a.tar.bz2

cd ..

!!!IMPORTANT!!!

Use the exact versions of each of the above that are shown.  There is a problem building with mpfr-3.1 and it will make it a lot harder for me to help you if you are using other versions!

You should now have a directory structure like this:

/opt/Android/android-ndk-r7/src:

ls *

src/binutils:
binutils-2.22.51  binutils-2.22.51.tar.bz2

src/build:
build-sysroot.sh  config.guess  configure     COPYING      dejagnu     Makefile.in
clear_header.sh   config.sub    configure.ac  COPYING.LIB  install-sh  README

src/gcc:
gcc-4.7.0

src/gdb:
gdb-7.3  gdb-7.3a.tar.bz2

src/gmp:
gmp-5.0.2.tar.bz2

src/mpc:
mpc-0.9.tar.gz

src/mpfr:
mpfr-3.0.1.tar.bz2


Step 5: Setting up the NDK helper script

The Android NDK comes with a nice script that takes care of most of the build process, as long as it knows where to find what it needs.  That requires a little bit of editing...

Open up:

/opt/Android/android-ndk-r6b/build/tools/build-gcc.sh

and make the following changes:

GDB_VERSION=7.3

BINUTILS_VERSION=2.22.51

GMP_VERSION=5.0.2

MPFR_VERSION=3.0.1

Add a new definition for MPC just below the one for GDB:

MPC_VERSION=0.9
register_var_option "--mpc-version=<version>" MPC_VERSION "Specify mpc version"

Add a new option to control the languages that are to be built.  This can be placed anywhere in the upper portion of the script.  I placed it right under the line that starts: register_var_option "--keep-libstdcxx"

ENABLE_LANGUAGES="c,c++,fortran"

register_var_option "--enable-languages=<name>" ENABLE_LANGUAGES "Experimental: specify which languages to build"

Now add the new definitions to the configure command about 2/3rds down in the script:

mkdir -p $BUILD_OUT && cd $BUILD_OUT && run \

$BUILD_SRCDIR/configure --target=$ABI_CONFIGURE_TARGET \

                        --enable-initfini-array \

                        --host=$ABI_CONFIGURE_HOST \

                        --build=$ABI_CONFIGURE_BUILD \

                        --disable-nls \

                        --prefix=$TOOLCHAIN_PATH \

                        --with-sysroot=$TOOLCHAIN_SYSROOT \

                        --with-binutils-version=$BINUTILS_VERSION \

                        --with-mpfr-version=$MPFR_VERSION \

                        --with-gmp-version=$GMP_VERSION \

                        --with-mpc-version=$MPC_VERSION \

                        --with-gcc-version=$GCC_VERSION \

                        --with-gdb-version=$GDB_VERSION \

                        --enable-languages=$ENABLE_LANGUAGES \

                        $ABI_CONFIGURE_EXTRA_FLAGS

UPDATE:
Since makefiles are notorious for crapping out due to invisible typos, I've decided to post a patch that should get your ndk-r7 directory all fixed up.

This patch for ndk-r7
or
This patch for ndk-r7b

Copy the above patch into your android-ndk-r7 directory and then patch as follows:

patch -p1 -i ndk-r7-fortran.patch

If it patches five files successfully then you should be ready to start the build...

Make sure you are in the NDK base directory:

cd /opt/Android/android-ndk-r7

Start the script and go to bed.. :)

./build/tools/build-gcc.sh --try-64 $PWD/src $PWD arm-linux-androideabi-4.7.0

If everything goes ok, this will take quite a while, possibly an hour or more. When it finishes, your shiny new Fortran enabled compiler suite will be here:

/opt/Android/android-ndk-r7/toolchains/arm-linux-androideabi-4.7.0/prebuilt/linux-x86_64/bin

Now you can remove the old compiler from your ~/.bashrc file and replace it with the new one (see the last post).

[mike@fedorocks bin]$ arm-linux-androideabi-gfortran --version

GNU Fortran (GCC) 4.7.0 20111019 (experimental)
Copyright (C) 2011 Free Software Foundation, Inc.

GNU Fortran comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of GNU Fortran
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING

If you've made it this far then congratulations! You can now use your shiny new Arm enabled Fortran compiler just as you would your regular Fortran compiler.  I've already tested this with the Linpack benchmark as well as several other codes, so I am confident that it works.  While that is all well and good, our ultimate goal is to fix up the Android build system so we can put Fortran native code in our Android applications and pack them up for distribution on the Android Market.

Stay tuned...

Android Fortran Step-by-Step Part 1: Preparing Your Build Machine

***Updated for NDK-r7***
I'm going to keep these next few posts light on the humor if that's alright.. being funny takes too much time and effort.. :(

In case this is your first visit to this blog, you might want to have a look at some background..

here..
and here..
This is nice too if your into that sort of thing.

So, if you're still with me, lets start by assuming that you are using a Windows computer.  If not, then good for you, and I'll try to keep this as neutral as possible.  As much as I like to endorse a Microsoft free lifestyle (well, they do make good mice), I am not so much of a Linux snob that I believe there aren't people out there who have a good reason to run Windows.  So, if that is you, go ahead and log out of Call of Duty, stand up, stretch, focus on an object across the room for a moment, then head over to www.virtualbox.org and install the latest version.  Let's just pretend that VirtualBox isn't owned by Oracle for a little while.

I'm going to let someone else show you how to get Fedora 15 installed.. I'm sure that those of you who prefer Ubuntu or another Linux version could make most of this work as well, but lets stick to Fedora for the moment, OK?

Come back when you have something that looks like this:

Now here's a little short-cut that should help you get all of the necessary 32-bit junk installed to support the Android development tools.  Open a terminal and type this:

sudo yum install wine wine-devel wget

If you are running 64 bit Fedora then this is the quickest way I know to setup the machine for 32 bit applications, plus you'll be able to run quite a few Windows programs with wine if you feel the need to.  There are actually a lot of Windows 95/98 games that run better under wine than under Windows XP (haven't tried any in Win7, but can't imagine it would be better).  Notice I also included the wget package, we'll need that soon enough.

Now install Java (the horror!)

sudo yum install java java-devel

This should install openjdk by default, which has been working fine for me.

Now you should have everything you need to install the Android SDK and NDK.  I usually put this kind of thing under /opt and I recomment you do the same for now...

sudo mkdir /opt/Android

sudo chmod 777 /opt/Android

Those commands should be all you need to make a place for your Android stuff to live that will have the right permissions for you to work without needing root privileges. So, without further ado:

cd /opt/Android

wget http://dl.google.com/android/android-sdk_r16-linux.tgz

wget http://dl.google.com/android/ndk/android-ndk-r7-linux-x86.tar.bz2

Isn't that wget handy?

Now unpack the SDK and NDK:

tar -zxvf android-sdk_r16-linux.tgz

tar -jxvf android-ndk-r7-linux-x86.tar.bz2

Notice the -z and -j above.. Google appears to have warring internal factions over the preferred compression scheme for tar archives.  I'll let history decide which is better, bzip or gzip.. nah, just kidding, it's bzip ;)

 Almost ready for the fun stuff.. just have to add a couple of things to our environment.. open .bashrc in your home directory and add these two lines:

UPDATE: I forgot to include the SDK platform-tools directory, where the android debug bridge (adb) lives.. can't get far without that!

PATH=/opt/Android/android-sdk-linux/tools:/opt/Android/android-sdk-linux/platform-tools:/opt/Android/android-ndk-r7:$PATH

export PATH

As an alternative to monkeying around with .bashrc, I highly recommend using environment modules, especially if you use a lot of different development tools like I do on a daily basis.  For now, let's stick with the simplest approach.

So, you should now be ready to fire up the Android SDK.. just open a new terminal and type:

android

If your environment is configured correctly then you should see this:

Now you need to install the android packages that are required for a basic development platform.  I've been working with 2.3, so I recommend starting there as well.  I think there is a good chance that the newer platforms will work also (maybe even better), but my phone is 2.3, so that's what I'm targeting.

I have to admit I like Windows 7's snip tool, and no, my handwriting isn't much better when I'm not trying to write with a mouse!  When the install is done, you should see something like this:

DO NOT push that innocent looking Update All button... unless you want every available package installed!  This is also a good time to set up an AVD (virtual device) if you don't have a real device to develop on.  Let me know in the comments if you need help with that.

We're almost done with this part... In order to build our fancy new Fortran enabled tool chain, we need to temporarily add the crusty old one to the shell environment path.  So, open your .bashrc file again and add this line above the export PATH line from before:

PATH=/opt/Android/android-ndk-r7/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin:$PATH

Now if you open a new terminal, you should have the native Android compilers available:

At this point, you have everything you need to start working with native code in C and C++.  In the next post, I'll get to the good stuff... adding gfortran to the mix and telling the Android build system how to use it!

Almost there...

Fortran pining for the Fjords? Not even close!

Since I've started down this rabbit hole, I may as well keep digging until I reach bedrock.  It's been interesting to me that I've gotten more feedback on this subject than any of the rocket related stuff.  Well, when I say feedback, I don't just mean comments on the posts, but include e-mails that I've received from people with questions about how to get it to work for them.  Not that I mind fielding these questions, but I think in the future I'd like to keep as much of these discussions public as possible.  By that I mean, if you have a question or comment, don't be shy... post it in the comments and I'll do my best to respond there quickly.

It would be a shame if something you or I learned together wasn't available for other people to benefit from!

So, where to begin... One of the reasons I haven't put anything out here for so long is that I've been making such steady progress that I couldn't find a good stopping point.  I would get a little further along and then say 'hmm.. it would be so cool if I could just get that one thing working and then people would REALLY be impressed'.  Well, as a German flight test engineer friend of mine once said, it's time for a 'Full Stop'.  Pack up the equipment, go home, forget about it for a long weekend and start sorting through the data with a clear head on Monday.

That is the spirit that I've decided to adopt with this project.  Since I'm calling it a 'project' now, there probably should be a set of goals:

• Help Fortran become a viable companion to C and C++ for developing native applications for the Android OS.
• Make it easier for other people to get in on the fun.
• Convince Google to abandon the Dalvik virtual machine altogether.  

That last point deserves a little expansion.  If Android is to ever really compete with iOS in responsiveness and stability, they are going to have to drop the extra layer.  Android has all the right ingredients: a Linux kernel, a decent C runtime, OpenGL, OpenSL and, let's not forget, the backing of a giant corporation with almost limitless resources.  Unfortunately, that last item carries with it a level of institutional inertia that gives it the maneuverability of an ocean liner sitting on top of an aircraft carrier.  Still, what do I know?  As someone who has no formal computer science education and no real programming credentials to speak of, I accept that my opinion probably has approximately zero weight, but I can dream, can't I?

That's about it.  I wish I could come up with something funny to say here, but there really is nothing funny about working with the Android development tools, so I'll just fall back on my old standby.

JAVA SUCKS™

Really.

No, seriously.

I should devote an entire post to the time that I've spent trying to do anything useful with Java and C++ and contrast that with the one long weekend it took me to go from zero to comfortable with Python.  Unfortunately, if I did that, I would be wasting even more of my time, so it's not too likely.  Not to worry though, Java is now owned by the one company in the world most likely to drive it toward extinction.. and, amazingly enough, that's NOT Microsoft.  Seriously, though, time will tell, but I'm just getting this out there so I can link back to this post 20 years from now and say CALLED IT!

Sadly, I believe we are stuck with C++.  In the giant ball of mess that the language is, there are a handful of things that make it useful for people who are already comfortable with C.  That's not me, but I can envision a future where it is.

Before anyone jumps on a high horse and starts quoting the tiobe index or saying that Fortran is dying, dead, buried and/or in some other way not relevant today, I say..

1. Why are you reading this in the first place?
2. If you Fortran, you know.
3. See item #1

 There is a saying that is part of Fortran lore that I've seen a number of variations on over the years and it goes something like this:

"Letter O considered harmful"

Oh wait, it was this one:

"I don't know what the syntax or features of the preferred programming language of scientists and engineers will be in the future, but I know that it will be called Fortran"

Next up.. getting your Android on.. or.. Learning Linux for fun and profit!

Short Update.. Android Fortran Linpack

Sorry I haven't had much time for blogging lately, hopefully I will be able to make some soon.  Until then, I wanted to put some more information on the Fortran compiler that I hacked together for android and the modified Linpack benchmark.  I've gotten several e-mails from people with comments about the procedure and questions about how to make it work for themselves.  Some of the comments might even allow me to make the compiler work correctly without the ugly hacks, but alas, it works ok right now, so my motivation is low.

Anyway, a couple of days ago, I dug all of it back out and verified that it still worked.  Then I made a couple of small changes to the code to make it easier to install and pushed it back out to my phone and tried it out.  It was a bit frustrating since my phone had lost it's ability to give me superuser access after the last update from Verizon.  Technically, it wasn't entirely necessary for me to root the phone for this, but I decided I wanted to re-root it before I installed the new version of the benchmark, just in case I ran into any trouble.  Eventually, I'd like to pack all this up nicely in a signed apk and drop it for free on the android market, but that is really low on my priority list right now.

So, I've made the source code and the pre-compiled arm-android executables available here:

http://aeromonkey.homeip.net/public/android

There isn't too much to say about the code that you won't learn by looking at it.  The makefile is probably the most useful thing, since it handles the linking step that gave me so much trouble before.  Installing the executable is easy enough if you have the adb bridge set up.  You just need to put it where it can be executed.. in my case:

adb push linpack_d /data/local/

Then it can be run using a terminal app (or the adb shell).  Once you have a terminal open, just go to where you put it and run it.

cd /data/local
./linpack_d {Matrix-Size} {Results-File}

or more specifically,

./linpack_d 500 /mnt/sdcard/tmp/linpack.results

There is a definite bug in the compiler right now that makes it fail when trying to overwrite an existing results file, so it's best to remove the file manually before running the benchmark if you want to save the results.

I've also included the python script I wrote that lets the benchmark be run in a more interactive way.  This requires the Scripting Layer for Android and is left as an exercise for the reader. (Hint: put it in /mnt/sdcard/sl4a/scripts)

I hope some industrious people out there can get this to work on a few other phones.. my Droid 1 seems to top out at just under 25 MFlops using the best compiler options I could find for the cortex-a8.  I ran it on a friend's Droid 2 a while ago and got about 40 (if I remember right).

It would be fun to hear how it runs on some newer hardware, so let me know in the comments section below!

DISCLAIMER:  if you break your phone it ain't my fault.. and besides, you really shouldn't be installing random apps you downloaded from some guy's blog anyway.

My Android speaks Fortran.. yours can too!

UPDATE: This is inferior technology... see here for the new way!

Ok.. this is WAY off-topic, even more so than my last couple of posts.  I promise that I'll get back to the rocket business eventually, but I just think this is too cool not to share.  Plus I wanted to record what I did somewhere so I don't forget about it.  Not to mention the amount of nerd street cred that something like this is worth...

In a nutshell, I've managed, through sheer bullheadedness, to get something that is completely useless to 99.44% of android phone owners.  Something that seemed perfectly reasonable to me that was just absent... a Fortran compiler.

A little backgroud...

When I got my shiny new Verizon Droid back in June of last year, I was really excited by the possibilities that the platform presented to me with its open source Linux based Android goodness under the hood.  Unfortunately, I quickly discovered that while google has indeed released the source for the Android platform, that platform is lacking some of the things that make life on planet Unix so fun and portable.  Most notable of these are the X11 windowing system and the full GNU Compiler Collection, lovingly referred to as GCC.  Those two major omissions, along with their decision to use a non-standard c library, mean that it is far from trivial to get applications ported over.  Ironically, this is exactly what makes the far less open Apple mobile iOS much easier to develop for (from what I've read anyway), since most of the facilities from the desktop (OSX) are available in some fashion or other and everything is standardized and collected neatly together under the Xcode IDE.  The google answer to this was to add a bunch of features to the venerable Eclipse IDE, declare that Java will be the language of choice for the platform and called it good.

Now as someone who has sworn off Java for life on the grounds (small pun intended) that it is a language with all the complexities of C++ and none of the speed, I just refuse to stand by while the company that will one day bring us Skynet attempts to keep me from using my phone the way I want.  I actually had high hopes for the so-called scripting layer for android, which makes it possible to run Python (my second favorite computer language) on the phone.  But once again, google has seen fit to hamstring developers by only providing bindings to a handful of the phone's low level functions, relegating Python and friends to second class citizens in the Android kingdom.

This is where my Odyssey began, a simple desire to do what I wanted on my phone in a language that I understand.. that language being Fortran.  Now you might ask me, "Mike, what place does the oldest surviving computer language have on my shiny new personal communication device?".  Well, my answer to that is maybe it doesn't, but I think it should get a chance, don't you?  Fortran is still the undisputed heavyweight number crunching champion of the world, so why not try to take advantage of that to write some smoking fast applications for the little black phone that could.  You might also say, "What's the point since the android API is not accessible from Fortran at all so it's even less useful than Python?".  To that, my answer is the magical Fortran fairy dust called "iso_c_binding".  Since the Android NDK provides interfaces to most of the phone's functions via C, then I should be able to, at least in theory, access those functions from Fortran by wrapping them in C.

So, with this knowledge in hand, I proceeded to download the latest GCC 4.6 pre-release source code and started looking online for information that I needed to build a cross compiler for the ARM CPU in my phone... two weeks and a few gray hairs later, I marveled at the wonder of 'Hello World' in three lines of Fortran running in a terminal on my droid.

Here's how I did it... but be warned, what follows is full of dirty hacks, workarounds and generally poor execution, so don't expect it to work for you the first time if you attempt to duplicate it.  I will, however, try to re-create the process as faithfully as possible.

The first thing you will need is a computer running Linux, in my case I am using Fedora 13, but most other distributions should work without major changes.  Don't even think about asking me how to do this under Windows unless you want to be taunted mercilessly until you have no choice but to abandon this blog altogether... seriously.  It won't work under Windows so don't try it.

The next thing you need is a functional Android SDK and NDK environment with the system ABI resources for your phone.  I leave installation of that as an exercise for the reader.  Once you have that, there is a handy script in the NDK under {ndk-root}/build/tools called make-standalone-toolchain.sh that does just what its title implies, which is to create a "standalone" compiler for the desired Android ABI level, in my case, level 8, which corresponds to Android 2.2, yummy, yummy, Froyo.  After running the script, add the new compilers to your path, and then congratulate yourself on having a fully functional Android C compiler and a somewhat less functional C++ compiler. 

The remaining steps I'll just lay out by example because that will hopefully make it easier to follow and it's getting late and I want to go to bed.

Make one small change to the linux headers in the Android NDK. In the file {path-to}/android-ndk-r5/platforms/android-8/arch-arm/usr/include/linux/stat.h
Add the following two lines:
#define S_IREAD S_IRUSR
#define S_IWRITE S_IWUSR

just below the line that says:
#define S_IWUSR 00200

This fixes a dependency that gfortran has on a couple of deprecated symbols that have been removed in the NDK header files.

Make a directory to build all of the pieces:
(you can skip building gmp, mpfr and mpc if you have the devel packages for your platform installed already)
mkdir /opt/compilers/gcc/build
cd /opt/compilers/gcc/build

#Download and build GMP 
wget http://ftp.gnu.org/gnu/gmp/gmp-4.3.2.tar.gz
tar zxvf gmp-4.3.2.tar.gz
mkdir gmp-build
cd gmp-build
../gmp-4.3.2/configure --prefix=/opt/compilers/gcc/android-8 --enable-cxx
make && make install

#Download and build MPFR

wget http://ftp.gnu.org/gnu/mpfr/mpfr-2.4.2.tar.gz
tar zxvf mpfr-2.4.2.tar.gz
mkdir mpfr-build
cd mpfr-build
../mpfr-2.4.2/configure --prefix=/opt/compilers/gcc/android-8 --with-gmp=/opt/compilers/gcc/android-8
make && make install

#Download and build MPC
wget http://www.multiprecision.org/mpc/download/mpc-0.8.1.tar.gz
tar zxvf mpc-0.8.1.tar.gz
mkdir mpc-build
cd mpc-build
../mpc-0.8.1/configure --prefix=/opt/compilers/gcc/android-8 --with-gmp=/opt/compilers/gcc/android-8 --with-mpfr=/opt/compilers/gcc/android-8
make && make install

#Download and build GNU Binutils 2.19.1 for the android target machine (newer versions will probably work just fine, maybe better)
wget http://ftp.gnu.org/gnu/binutils/binutils-2.19.1.tar.gz
tar zxvf binutils-2.19.1.tar.gz
mkdir binutils-build
cd binutils-build
../binutils-2.19.1/configure --prefix=/opt/compilers/gcc/android-8 --with-gmp=/opt/compilers/gcc/android-8 --with-mpfr=/opt/compilers/gcc/android-8 --with-mpc=/opt/compilers/gcc/android-8 --target=arm-android-eabi --with-sysroot={path-to}/android-ndk-r5/platforms/android-8/arch-arm
make && make install

Download the EXACT version of the GCC source that I used:

svn co svn://gcc.gnu.org/svn/gcc/trunk@168097 gcc-trunk

Download this patch and apply it to the gcc-trunk...

http://aeromonkey.homeip.net/public/ugly_gfortran_hacks.patch

cp ugly_gfortan_hacks.patch gcc-trunk/.
cd gcc-trunk
patch -p0 < ugly_gfortran_hacks.patch

With any luck this should get your GCC source tree into the same sad state that mine is.. sad, but functional!

Now move back out of the GCC source directory and create a new directory in which to build the compilers.  If you've ever built GCC before, you know it is a very bad idea to build it in it's own source tree.

cd ..
mkdir build-android-8
cd build-android-8

Now you need to configure and build the compiler suite.  This is the part that really took a lot of head scratching and googling since I had never built a 'cross' compiler before.  I'll save you the pain of trying to explain what all of this means, so just copy and paste these lines, fix the {path-to} your Android NDK and you should be off to the races.

../gcc-trunk/configure --prefix=/opt/compilers/gcc/android-8 --disable-libquadmath --target=arm-android-eabi --with-gnu-as --with-gnu-ld --enable-languages=c,fortran --with-mpfr=/opt/compilers/gcc/android-8 --with-gmp=/opt/compilers/gcc/android-8 --with-mpc=/opt/compilers/gcc/android-8 --disable-libssp --enable-threads --disable-nls --disable-libgomp --disable-shared --disable-tls --with-float=soft --with-fpu=vfp --with-arch=armv5te --enable-target-optspace --disable-nls --with-sysroot={path-to}/android-ndk-r5/platforms/android-8/arch-arm

make && make install

This could take an hour or more to complete, but you can speed things along in the typical 'make' fashion by using a parallel build (assuming you are on a multicore box).. i.e.. on a quad machine..

make -j 4 && make install

Assuming you get this far, you can now compile and assemble code for Android, but sadly it will not run on the phone quite yet.  This is where I nearly threw in the towel since I could get Fortran code to build, but it would segfault immediately if I tried to run it.. and when I say immediately, I mean before executing even a single line.  After a great deal of effort I was ready to accept that I'm more of a code plumber than a trained programmer and that some tiny piece of the puzzle was beyond my grasp.  That's when the engineer in me spoke up and said.. hold on, just take a step back, leave it alone for a day and try again tomorrow.

That next day I decided to try to get a better handle on what was going on behind the scenes with the compiler.  To do this, I found a simple c program on the internet (btw.. it's amazing how many ways you can calculate the value of pi) and proceeded to build it with both the Android supplied and my newly minted c compilers and push it to the phone.

NDK compiler = success.  New compiler = squat.

Now I broke things down a little further.  I'm by no means an authority on compilers, but I did know that there are a number of stages that code goes through on it's journey from human readable (some languages less than others) to machine readable.  These steps being:

1. Compile
2. Assemble
3. Link

I also knew that while all of these steps often appear to be taken by the single compiler executable (gcc, g++, gfortran, etc.), in reality, the compiler hands off the latter two steps to the 'assembler' and the 'linker'... sneaky little basterd.

This is where things get interesting... Not really expecting to get anywhere, I decided to start mixing and matching the components from the NDK with my new compiler...

• NDK compiler + NDK assembler + NDK Linker ==> works.
• New compiler + New assembler + New Linker ==> no good.
• NDK compiler + NDK assembler + New Linker ==> no good.
• NDK compiler + New assembler + New Linker ==> no good.
• New compiler + New assembler + NDK Linker ==> WORKS!

Ok.. so that's the key.  For some reason, the new linker is the weak.. uh.  link.

After spending some time trying to reverse engineer the linker script used by the NDK linker, I just said forget it, I have something that works.  Luckily, GNU 'makes' (wow, the puns just keep coming..) it really easy to use any linker you want when building code.. so, without further ado...

No, I didn't do the interface bits in Fortran, that's one thing Python IS good for on Android, at least for simple things like this... that and making it talk.  FYI.. the Linpack for Android app available on the market only gets about 8 million floating point operations per second on my phone, which is close to what I got with the default compiler options.  The 25 MFLOPS shown in the video was after optimizing for the Neon floating point unit in the Droid's Cortex-A8 processor.  Makes me wonder how many other applications out there are not making very good use of the hardware.

Well. There's that then.

Longest post ever.

Meet Elmer, Part 2 .. or .. Everything's a Noodle

Ok, so if you read my last post, you know already that I did something a little off of the beaten path for me by dabbling in a little thermodynamics with the Elmer multiphysics package.  Now I'm going to get out the machette and really start hacking a new analytical trail for myself.

One of the most basic things that you learn in engineering school is that every solid thing that exists in the universe is flexible and that there are specifically two things that determine the amount of flexibility that a given thing will have.  The first is the mechanical properties of the material that the item is made out of.  The second and far more interesting is the cross sectional shape of said item.  This is the reason I-Beams exist.  It's certainly easier to manufacture and transport a hunk of steel as a solid bar, but you definitely don't want to build a bridge or sky scraper out of it.  Even though the tensile strength of the bar and the beam are the same if they have the same cross-sectional area, the beam is much stiffer in bending.  The engineering term is 'second moment of area' and I can tell you that calculating the stiffness of an arbitrary beam by manually summing moment areas is certainly fun and easy for simple shapes, but becomes not much fun quickly in the real world.

Even less fun than calculating the stiffness of a beam, but arguably even more important, is determining the vibrational modes of the beam.  This is something that is nearly impossible to do by hand except for very basic shapes (like musical instrument strings), but is critical to understanding the ways in which structural deflections can become destructively coupled to external forces, like say, aerodynamic loads.  One of the most famous examples of this, which most every engineering sophmore has probably seen, is the collapse of the Tacoma Narrows Bridge in 1940.

 

The engineers in the 1940's didn't have multi-gigaflop personal computers, but they were smart enough to design the replacement bridge well enough that it is still in use today.  

Now that the history lesson is over, let's take my multi-gigaflop (about six I think) personal computer and see what we can do with it.  As a first attempt to do this kind of thing with Elmer, I sketched up a 3D model of a simple wing or maybe a rotor blade depending on how you hold it.  After a couple of attempts I got what looks to me to be a decent grid on the thing and put it through the linear elastic solver in Elmer to get the mode shapes, which took less than an hour of compute time on two processor cores.

The first four mode shapes for the structure when held rigidly at one end are shown with the surfaces colored by the displacement magnitude to help highlight where the biggest deformations occur.  Fortunately, Elmer makes it fairly easy to animate the modes, making it much easier to see what is happening...

 

As you can see, the first mode is simple bending, while the higher modes start to represent more complicated deformations.  It's funny to think that a seemingly solid object can take on such strange shapes, but it's very real.. and very scary...

 

Kind of looks like mode #3 to me, what do you think?