Building OpenCV for Android with libjpeg-turbo

Nowadays, OpenCV offers a way to install a common shared library on an Android device and use it in multiple apps without the need to build every single app with OpenCV inside it: this is called OpenCV Manager. In this way the total size of the apps is definitely smaller. Anyway, you might need to tweak OpenCV or, like I’m showing here, work with libjpeg-turbo instead of the standard libjpeg used by OpenCV, so you might need to build your own copy of OpenCV (useful also for debugging purposes).

As you know, building your own copy means having multiple choices on compilation options (for example the target Android platform) and multiple choices mean a lot of headaches while trying to setup everything correctly and finding the right options that will work for us. This process will probably be no exception, especially if you’ll also need to build libjpeg-turbo, but the good news is that you can do it! You’re a software developer for a reason, so you’re used to all of this stuff and sure this won’t scare you right? It’s time to start now!

In this tutorial I’m going to work with the following environment:

You first need to have all the necessary tools installed (Android SDK and NDK, CMake, Apache Ant, Python, Java JDK) paying attention to the versions of some of them: the NDK must be 32bit otherwise you’re going to have compilation problems with some ASM files inside OpenCV and your build will fail; Python must be 2.x and not 3.x due to compatibility problems with the 3.x version.

You need to have libjpeg-turbo compiled for Android and you can find out how to do it following my other tutorial Building libjpeg-turbo for Android.

Once you’re ready with all the tools, you can download the Linux version of OpenCV (we need the Linux version because we have to compile it), then decompress the opencv-2.4.5.tar.gz file to have the root of OpenCV in C:\OpenCV. Read more

Building libjpeg-turbo for Android

In order to build libjpeg-turbo for Android, here are the tools you’ll need:

  • Linux (Ubuntu Linux works fine)
  • Android NDK
  • libjpeg-turbo source code

The compilation process won’t work with Cygwin since the Android NDK doesn’t officially support it so the make will fail. No worries, you can easily work with a Linux distribution. If you don’t have it already installed in your system, you can setup a VMWare virtual machine with Ubuntu and everything will work just fine: download the free VMWare player and get an Ubuntu ISO to create the virtual machine.

Before starting the build process, we must have all the needed tools ready. In this tutorial I’m going to work with the following environment:

  • Windows 8 Pro 64bit
  • VMWare Player 5.0.1 with an Ubuntu Desktop 12.10 64bit virtual machine
  • Android NDK Revision 8e (for Linux 64bit)
  • libjpeg-turbo 1.2.90

Setup the Android NDK and create a standalone toolchain

Download the Android NDK from http://developer.android.com/tools/sdk/ndk/index.html. Make sure you get the one for the Linux 64-bit (x86) platform (or the 32bit version if you’re using a 32bit Linux distribution).

Place the downloaded file in your home directory of the Linux distribution (in this tutorial my home directory will be /home/andrea), open a Terminal window and from your home directory extract the file content typing

tar -xf android-ndk-r8e-linux-x86_64.tar.bz2

This will create the android-ndk-r8e folder with the whole NDK. NOTE: it’s important that you use the tar command from the commandline and don’t extract the compressed archive through the Linux GUI as it might mess up with the files permissions causing you some troubles later.

Now it’s time to create a standalone toolchain from the NDK so we’ll be able to use it later to compile libjpeg-turbo. If you need information about the standalone toolchains, you can read the related document at android-ndk-r8e/docs/STANDALONE-TOOLCHAIN.html. Read more

Saving the Android WebView cache on the SD card

Source code

It could be useful to save the cache of an Android WebView on the SD card (or integrated external memory) especially for devices with a limited amount of internal memory, but how can you do that? Well, it’s simple. You know that some Android browsers already do it and here I’m going to tell you a way to do it in your own app as well. This solution is made to work with Android 2.1 and higher, so I’m not going to use the API that was introduced only in a later version of the OS.

The key is the getCacheDir method of the ContextWrapper class. This is what is used by the cache manager to decide where to store the cache files. The behavior is slightly different for Android 2.1 and Android 2.2 and higher, so we’ll also keep this in mind.

I’ll explain everything going through the source code of the example application that you can download through the link on top of this post. Let’s start with the mainfest file:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
	package="com.devahead.androidwebviewcacheonsd"
	android:versionCode="1"
	android:versionName="1.0">

	<uses-sdk android:minSdkVersion="7"/>
	
	<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
	<uses-permission android:name="android.permission.INTERNET"/>
	<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
		
	<application
		android:icon="@drawable/ic_launcher"
		android:label="@string/app_name"
		android:name="com.devahead.androidwebviewcacheonsd.ApplicationExt">
		<activity
			android:name=".MainActivity"
			android:label="@string/app_name">
			<intent-filter>
				<action android:name="android.intent.action.MAIN"/>
				<category android:name="android.intent.category.LAUNCHER"/>
			</intent-filter>
		</activity>
		<activity
			android:name=".SecondActivity"
			android:label="Second activity"/>
	</application>

</manifest>

Here we have the permissions to write to the SD (the external storage) and access the web. We also have a custom Application class extension called ApplicationExt and a couple of activities, MainActivity and SecondActivity.

The ApplicationExt class is where most of the things are done to have the cache on the SD:

package com.devahead.androidwebviewcacheonsd;

import java.io.File;

import android.app.Application;
import android.os.Environment;

public class ApplicationExt extends Application
{
	// NOTE: the content of this path will be deleted
	//       when the application is uninstalled (Android 2.2 and higher)
	protected File extStorageAppBasePath;
	
	protected File extStorageAppCachePath;

	@Override
	public void onCreate()
	{
		super.onCreate();

		// Check if the external storage is writeable
		if (Environment.MEDIA_MOUNTED.equals(Environment.getExternalStorageState()))
		{
			// Retrieve the base path for the application in the external storage
			File externalStorageDir = Environment.getExternalStorageDirectory();
				
			if (externalStorageDir != null)
			{
				// {SD_PATH}/Android/data/com.devahead.androidwebviewcacheonsd
				extStorageAppBasePath = new File(externalStorageDir.getAbsolutePath() +
					File.separator + "Android" + File.separator + "data" +
					File.separator + getPackageName());
			}
	
			if (extStorageAppBasePath != null)
			{
				// {SD_PATH}/Android/data/com.devahead.androidwebviewcacheonsd/cache
				extStorageAppCachePath = new File(extStorageAppBasePath.getAbsolutePath() +
					File.separator + "cache");
	
				boolean isCachePathAvailable = true;
				
				if (!extStorageAppCachePath.exists())
				{
					// Create the cache path on the external storage
					isCachePathAvailable = extStorageAppCachePath.mkdirs();
				}
				
				if (!isCachePathAvailable)
				{
					// Unable to create the cache path
					extStorageAppCachePath = null;
				}
			}
		}
	}
	
	@Override
	public File getCacheDir()
	{
		// NOTE: this method is used in Android 2.2 and higher
		
		if (extStorageAppCachePath != null)
		{
			// Use the external storage for the cache
			return extStorageAppCachePath;
		}
		else
		{
			// /data/data/com.devahead.androidwebviewcacheonsd/cache
			return super.getCacheDir();
		}
	}
}

In the onCreate method we start by checking if the external storage is actually mounted and we can write on it, then we build the base path of the app on the external storage. This path is {SD_PATH}/Android/data/com.devahead.androidwebviewcacheonsd because com.devahead.androidwebviewcacheonsd is the package declared in the manifest file and using this path structure makes sure that the entire path and all its content will be automatically deleted by Android 2.2 and higher in case the app is uninstalled avoiding to have garbage files on the SD. Note that this works only in Android 2.2 and higher, while in Android 2.1 the path will not be deleted by the OS so you’ll have to deal with it on your own. The full path for the cache files will be {SD_PATH}/Android/data/com.devahead.androidwebviewcacheonsd/cache, so the base path plus the cache directory. We must also make sure that the path is actually available before using it, so we create all the directories with the mkdirs method in case they don’t already exist.

Now that we have the cache path on the SD, we’re ready to use it and all we have to do is override the getCacheDir method inside our ApplicationExt class. This method is invoked by the cache manager when the application is started so basically we’re saying that the cache will be stored on the SD if it’s available and writeable, while we use the default path in case we’re allowed to use only the internal memory. Android stores all the cache data for our app in the /data/data/com.devahead.androidwebviewcacheonsd/cache directory by default on the internal memory.

We’re done with the implementation for Android 2.2 and higher, but what about Android 2.1? For that version of the OS the cache manager doesn’t use the getCacheDir method of the Application context, but it uses the one of the Activity context instead. So to make our solution work also with Android 2.1, we must override the getCacheDir method inside our activities.

To immediately understand how it works with Android 2.1, let’s take a look at the activities of the example application. We start with the MainActivity: Read more

Preserving the state of an Android WebView on screen orientation change

Source code

If you’ve tried to use a WebView inside your app, you know that the standard behavior on screen orientation change is not satisfactory in most cases because the full state of the WebView is not preserved. Here I’m going to show you a possible implementation to keep the full state of the WebView every time you rotate the screen. This is the same implementation I used in my own app (FWebLauncher) for the internal web browser.

Standard implementation (the state is not completely preserved)

Let’s start by taking a look at what a standard implementation would look like. This is how you would usually implement the state saving for a WebView according to the Android documentation:

public class StandardImplActivity extends Activity
{
	protected WebView webView;

	@Override
	public void onCreate(Bundle savedInstanceState)
	{
		super.onCreate(savedInstanceState);
		setContentView(R.layout.standard_impl);

		// Retrieve UI elements
		webView = ((WebView)findViewById(R.id.webView));
		
		// Initialize the WebView
		webView.getSettings().setSupportZoom(true);
		webView.getSettings().setBuiltInZoomControls(true);
		webView.setScrollBarStyle(WebView.SCROLLBARS_OUTSIDE_OVERLAY);
		webView.setScrollbarFadingEnabled(true);
		webView.getSettings().setLoadsImagesAutomatically(true);

		// Load the URLs inside the WebView, not in the external web browser
		webView.setWebViewClient(new WebViewClient());

		if (savedInstanceState == null)
		{
			// Load a page
			webView.loadUrl("http://www.google.com");
		}
	}
	
	@Override
	protected void onSaveInstanceState(Bundle outState)
	{
		super.onSaveInstanceState(outState);

		// Save the state of the WebView
		webView.saveState(outState);
	}
	
	@Override
	protected void onRestoreInstanceState(Bundle savedInstanceState)
	{
		super.onRestoreInstanceState(savedInstanceState);

		// Restore the state of the WebView
		webView.restoreState(savedInstanceState);
	}
}

So we call the saveState and the restoreState methods in the onSaveInstanceState and the onRestoreInstanceState methods of the Activity.

The WebView is declared directly in the layout file for the activity:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
	android:layout_width="fill_parent"
	android:layout_height="fill_parent">
	
	<WebView android:id="@+id/webView"
		android:layout_width="fill_parent"
		android:layout_height="fill_parent"/>
	
</LinearLayout>

The main problem with this implementation is that, whenever you rotate the screen, the WebView is created again because the activity is destroyed and its saveState method doesn’t save the full state, but only a part of it like the URL of the page that was loaded and the browsing history. So it happens that for example the zoom and the scroll position are not preserved after the screen orientation change and sometimes the page is reloaded from the web.

State preserving implementation (a possible solution)

I tried many different solutions to preserve the full state of the WebView on screen rotation and the following one proved to be a reliable one that solves our problem. The main point is that the activity must not be destroyed and we must handle the screen orientation change by ourselves. Let’s start by taking a look at the layout file:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
	android:layout_width="fill_parent"
	android:layout_height="fill_parent">
	
	<FrameLayout android:id="@+id/webViewPlaceholder"
		android:layout_width="fill_parent"
		android:layout_height="fill_parent"/>
	
</LinearLayout>

You immediately notice the difference with the standard implementation. Here we don’t declare the WebView inside the layout file, but we declare a placeholder instead. It is the position where our WebView will be placed inside the activity.

Also the code inside the activity will be different of course and here it is: Read more

Recycling objects in Android with an Object Pool to avoid garbage collection

Source code (LICENSE)

If you need to reduce the work of the garbage collector in Android, like in games development for example, all you can do is reduce the amount of objects you create and recycle those you’ve already created. One of the ways to do this is the implementation of the Object Pool design pattern.

An Object Pool could be implemented in many different ways depending on the needed features. What I’m going to show here is an implementation that respects the following requirements:

  • during its execution, the Object Pool doesn’t allocate new objects other than those contained in the pool itself (we prefer plain arrays instead of Java collections);
  • the Object Pool must be efficient in terms of execution speed;
  • the Object Pool is responsible for initializing and finalizing the objects it contains (the objects must be ready to use when retrieved from the pool and they must free any unneeded resource when they are put back in the pool);
  • if there’s no more room for new objects in the pool, then new objects must be created anyway if requested, but they’ll not be stored in the pool when freed;
  • the Object Pool must be able to handle every type of Object (we only ask the objects to implement an interface to make them ready for the pool);
  • sharing the pool between multiple threads doesn’t have to be a problem.

And now, straight to the source code. We start with the definition of the PoolObject interface:

/**
 * Interface that has to be implemented by an object that can be
 * stored in an object pool through the ObjectPool class.
 */
public interface PoolObject
{
	/**
	 * Initialization method. Called when an object is retrieved
	 * from the object pool or has just been created.
	 */
	public void initializePoolObject();
	
	/**
	 * Finalization method. Called when an object is stored in
	 * the object pool to mark it as free.
	 */
	public void finalizePoolObject();
}

This is the interface that an object must implement to let the pool perform two operations: the initialization and the finalization. We use an interface so we can easily extend every other class and make the pool able to handle different objects.

The Object Pool creates the needed objects through a factory class that implements the PoolObjectFactory interface:

/**
 * Interface that has to be implemented by every class that allows
 * the creation of objectes for an object pool through the
 * ObjectPool class.
 */
public interface PoolObjectFactory
{
	/**
	 * Creates a new object for the object pool.
	 * 
	 * @return new object instance for the object pool
	 */
	public PoolObject createPoolObject();
}

The factory class must decide how to create a new PoolObject instance whenever the pool needs it.

All we need to see now is the ObjectPool class, so let’s take a look at it:

/**
 * Object pool implementation.
 */
public class ObjectPool
{
	protected final int MAX_FREE_OBJECT_INDEX;

	protected PoolObjectFactory factory;
	protected PoolObject[] freeObjects;
	protected int freeObjectIndex = -1;

	/**
	 * Constructor.
	 * 
	 * @param factory the object pool factory instance
	 * @param maxSize the maximun number of instances stored in the pool
	 */
	public ObjectPool(PoolObjectFactory factory, int maxSize)
	{
		this.factory = factory;
		this.freeObjects = new PoolObject[maxSize];
		MAX_FREE_OBJECT_INDEX = maxSize - 1;
	}
	
	/**
	 * Creates a new object or returns a free object from the pool.
	 * 
	 * @return a PoolObject instance already initialized
	 */
	public synchronized PoolObject newObject()
	{
		PoolObject obj = null;
		
		if (freeObjectIndex == -1)
		{
			// There are no free objects so I just
			// create a new object that is not in the pool.
			obj = factory.createPoolObject();
		}
		else
		{
			// Get an object from the pool
			obj = freeObjects[freeObjectIndex];
			
			freeObjectIndex--;
		}

		// Initialize the object
		obj.initializePoolObject();
		
		return obj;
	}

	/**
	 * Stores an object instance in the pool to make it available for a subsequent
	 * call to newObject() (the object is considered free).
	 * 
	 * @param obj the object to store in the pool and that will be finalized
	 */
	public synchronized void freeObject(PoolObject obj)
	{
		if (obj != null)
		{
			// Finalize the object
			obj.finalizePoolObject();
			
			// I can put an object in the pool only if there is still room for it
			if (freeObjectIndex < MAX_FREE_OBJECT_INDEX)
			{
				freeObjectIndex++;
				
				// Put the object in the pool
				freeObjects[freeObjectIndex] = obj;
			}
		}
	}
}

The structure of this class is very simple. There are only three methods, the constructor and two other methods to create and free the object instances.

The constructor accepts two parameters, the factory instance and the maximum number of objects that can be stored in the pool.

The newObject method retrieves an object from the pool and returns it to the caller after initializing it. If there are free objects in the pool, then one of those is returned, otherwise a new one is created through the factory. In this way we put a limit on the maximum number of instances in the pool, but only the needed amount is actually created.

The freeObject method puts an object instance back in the pool making it available for subsequent calls to newObject. Before storing the object in the free objects array, its finalization method is called to let it free any unneeded resource if necessary. If there’s no room in the pool for the object, then only its finalization method is called.

Both the newObject and the freeObject methods are synchronized so the interaction between the pool and different threads is managed correctly.

We have all we need, so let’s take a look at an example to see how our new tool can be used and what is its benefit. Let’s say that we need to work with Point instances for a graphic application. We need to have many points, but we want to avoid to keep on creating them and try to reuse the instances if possible. We need to define a PoolObject first: Read more