How to scale images for your Android™ application
Hard to get images scaled correctly for your application? Are your images too large and causing memory problems? Or are they scaled incorrectly with a poor user experience as a result? To find a good medicine for this, we asked Andreas Agvard from the Sony Ericsson software department to help shed some light on this topic.
Andreas Agvard, Sony Ericsson.
Working in the Sony Ericsson software department, I often come across applications where image scaling is needed, for example when handling images from external sources such as content providers or the web. Scaling is needed since the image you wish to present usually doesn’t fit the way you wish to present the image.
This is typically so if you are developing a LiveView™ extension for your application. Most the people developing applications utilising LiveView™ and other second screen devices, most probably ends up with a need to rescale images. Still there is a need to maintain a proper ratio and image quality. This is of course applicable in a lot other cases as well. Rescaling images can be a bit difficult to do in an effective way.
ImageView solves many scaling problems, at least as long as you can set an image source directly without decoding or scaling the image yourself first. But sometimes you need to take control of the decoding yourself, and that is where this tutorial comes in. Along with this tutorial, I’ve written a code sample project. Download the image scaling code example project from Developer World to learn more. The results presented in this text can be achieved by compiling and running that project.
Isolating the problem
I’ve made this tutorial because I’ve implemented a number of useful utility methods for doing scaling in a way that avoids the most common image scaling pitfalls. Pitfalls such as naïve solutions similar to this:
Bitmap unscaledBitmap = BitmapFactory.decodeResource(getResources(), mSourceId);
Bitmap scaledBitmap = Bitmap.createScaledBitmap(unscaledBitmap, wantedWidth, wantedHeight, true);
So what is correct and what is wrong in the code above then? Let’s look at the different lines of code.
Line 1: The entire source image is decoded to a bitmap.
This might cause an out of memory error if the image is too large.
This might result in a decoded image with a higher resolution than required. It might also be unnecessarily slow as smart decoders can scale when decoding at improved performance.
Scaling an image a lot, as when scaling a high resolution bitmap to a low resolution, causes aliasing problems. Using bitmap filtering (for example, passing true as the latter parameter to Bitmap.createScaledBitmap(…)) reduces the aliasing but is not enough when a lot of scaling is applied.
Line 2: The decoded bitmap is scaled to the wanted size.
The aspect ratio of the source image dimensions and the wanted image dimensions may not be the same. This will result in a stretched image.
Left image: Original image. Right image: Image scaled down with a naïve method. Aliasing problems can be seen such as one eye having a sharp highlight and the other having none. Stretching occurs on the height.
Creating a solution
Our solution will have a structure similar to the code above with where one part will replace line 1, where we decode an image in preparation for scaling. Another part will be to replace line 2, and do the final scaling. We’ll start with the part replacing of line 2 as it will introduce two new concepts, crop and fit, which will impact the solution for replacing line 1 as well.
Replacing line 2
In this part we are scaling the bitmap according to our needs. This step is necessary since the decoding line that precedes this will have limited capabilities to scale. Also in this step we might have to adjust the wanted size of our image if we wish to avoid stretching.
To avoid stretching there are two possibilities. Either we adjust the wanted dimensions by making sure they have the same aspect ratio as the source image, i.e. scaling the source image until it fits within the wanted dimensions, or we crop the source image with an area that has the same aspect ratio as the wanted dimensions.
Left image: Image scaled by the fit method. Image has been scaled to fit within the wanted dimensions and as a result the height of the image is smaller than the wanted height. Right image: Image scaled by the crop method. Image has been scaled to fit at least one of the wanted dimensions and as a result the source has been cropped, cutting away the left and right parts of the source image.
In order to scale like this we implement the following method:
public static Bitmap createScaledBitmap(Bitmap unscaledBitmap, int dstWidth, int dstHeight, ScalingLogic scalingLogic) {
Rect srcRect = calculateSrcRect(unscaledBitmap.getWidth(), unscaledBitmap.getHeight(), dstWidth, dstHeight, scalingLogic);
Rect dstRect = calculateDstRect(unscaledBitmap.getWidth(), unscaledBitmap.getHeight(), dstWidth, dstHeight, scalingLogic);
Bitmap scaledBitmap = Bitmap.createBitmap(dstRect.width(), dstRect.height(), Config.ARGB_8888);
Canvas canvas = new Canvas(scaledBitmap);
canvas.drawBitmap(unscaledBitmap, srcRect, dstRect, new Paint(Paint.FILTER_BITMAP_FLAG));
return scaledBitmap;
}
In the code above we use canvas.drawBitmap(…) to do the scaling. This method crops the area specified by the source rectangle from the source image and scales it to an area in the canvas defined by the destination rectangle. In order to avoid stretching these two rectangles need to have the same aspect ratio. We also call two utility methods, one for creating the source rectangle and another for creating the destination rectangle. These are implemented like this:
public static Rect calculateSrcRect(int srcWidth, int srcHeight, int dstWidth, int dstHeight, ScalingLogic scalingLogic) {
if (scalingLogic == ScalingLogic.CROP) {
final float srcAspect = (float)srcWidth / (float)srcHeight;
final float dstAspect = (float)dstWidth / (float)dstHeight;
if (srcAspect > dstAspect) {
final int srcRectWidth = (int)(srcHeight * dstAspect);
final int srcRectLeft = (srcWidth - srcRectWidth) / 2;
return new Rect(srcRectLeft, 0, srcRectLeft + srcRectWidth, srcHeight);
} else {
final int srcRectHeight = (int)(srcWidth / dstAspect);
final int scrRectTop = (int)(srcHeight - srcRectHeight) / 2;
return new Rect(0, scrRectTop, srcWidth, scrRectTop + srcRectHeight);
}} else {
return new Rect(0, 0, srcWidth, srcHeight);
}
}public static Rect calculateDstRect(int srcWidth, int srcHeight, int dstWidth, int dstHeight, ScalingLogic scalingLogic) {
if (scalingLogic == ScalingLogic.FIT) {
final float srcAspect = (float)srcWidth / (float)srcHeight;
final float dstAspect = (float)dstWidth / (float)dstHeight;if (srcAspect > dstAspect) {
return new Rect(0, 0, dstWidth, (int)(dstWidth / srcAspect));
} else {
return new Rect(0, 0, (int)(dstHeight * srcAspect), dstHeight);
}
} else {
return new Rect(0, 0, dstWidth, dstHeight);
}
}
The source rectangle will be the entire source dimension in the fit case. In the crop case it is calculated to have the same aspect ratio as the destination image, resulting either in the width or the height of the source image being cropped. The destination rectangle will be the entire wanted dimension in the crop case. In the fit case it will have the same aspect ratio as the source image, resulting in either the width or the height of the wanted dimensions being adjusted.
Replacing line 1
Decoders are smart, especially the ones used for the JPEG and PNG formats. These decoders can scale the image when decoding, with improved performance. When doing so, aliasing problems are also avoided. Also, since the image is smaller after decoding, less memory will be needed.
Scaling when decoding is as simple as setting the inSampleSize parameter on a BitmapFactory.Options object and passing it to the BitmapFactory when decoding. The sample size specifies a factor of which each side of the image is scaled, for example a factor of 2 on a 640×480 image will result in a 320×240 image being decoded. When setting a sample size, you are not guaranteed the image will be scaled down exactly according to that number, but at least it will never be smaller. For example a factor of 3 on a 640×480 image could result in a 320×240 image since the value 3 might not be supported. Commonly, at least the first powers of 2 are supported [1, 2, 4, 8, …].
Next step is to choose a proper sample size. The proper sample size would be the one resulting in the largest amount of scaling, but still being equal to or larger than the wanted image dimensions. This is implemented like this:
public static Bitmap decodeFile(String pathName, int dstWidth, int dstHeight, ScalingLogic scalingLogic) {
Options options = new Options();
options.inJustDecodeBounds = true;
BitmapFactory.decodeFile(pathName, options);
options.inJustDecodeBounds = false;
options.inSampleSize = calculateSampleSize(options.outWidth, options.outHeight, dstWidth, dstHeight, scalingLogic);
Bitmap unscaledBitmap = BitmapFactory.decodeFile(pathName, options);
return unscaledBitmap;
}public static int calculateSampleSize(int srcWidth, int srcHeight, int dstWidth, int dstHeight, ScalingLogic scalingLogic) {
if (scalingLogic == ScalingLogic.FIT) {
final float srcAspect = (float)srcWidth / (float)srcHeight;
final float dstAspect = (float)dstWidth / (float)dstHeight; if (srcAspect > dstAspect) {
return srcWidth / dstWidth;
} else {
return srcHeight / dstHeight;
} } else {
final float srcAspect = (float)srcWidth / (float)srcHeight;
final float dstAspect = (float)dstWidth / (float)dstHeight;
if (srcAspect > dstAspect) {
return srcHeight / dstHeight;
} else {
return srcWidth / dstWidth;
}
}
}
In the decodeFile(…) method we decode a file optimized for the final downscaling. This is done by first decoding only the dimensions of the source image, then calculating the optimal sample size using calculateSampleSize(…), and finally decoding the image using this sample size. I’ll leave it up to you if you’d like to dig deeper into understanding the calculateSampleSize(…) method. But basically it makes sure the image is scaled as much as possible while still being equal to, or larger, than the source rectangle that was applied before.
Putting it all together
With the help from the utility methods specified above we can now implement the following replacement lines for the initial code presented:
Bitmap unscaledBitmap = decodeFile(pathname, dstWidth, dstHeight, scalingLogic);
Bitmap scaledBitmap = createScaledBitmap(unscaledBitmap, dstWidth, dstHeight, scalingLogic);
Left image: Done by a naïve solution on mdpi device, decoding consumed 6693 kb of memory and took about 1/4 second. The result is stretched and suffers from aliasing artifacts. Middle image: Achived by the fit solution on mdpi device, decoding consumed 418 kb of memory and took about 1/10 second . Right image: Achived by the crop solution on mdpi device, decoding consumed 418 kb of memory and took about 1/10 second.
To learn more, download our code sample project. With this project, you can see the results on your Android phone and follow the flow in the source code.
Feel free to comment and ask questions about this topic! We’ve created a thread on Google groups for any discussions on this topic.
Andreas Agvard
Sony Ericsson Software department
Andreas Agvard, Sony Ericsson.
Working in the Sony Ericsson software department, I often come across applications where image scaling is needed, for example when handling images from external sources such as content providers or the web. Scaling is needed since the image you wish to present usually doesn’t fit the way you wish to present the image.
This is typically so if you are developing a LiveView™ extension for your application. Most the people developing applications utilising LiveView™ and other second screen devices, most probably ends up with a need to rescale images. Still there is a need to maintain a proper ratio and image quality. This is of course applicable in a lot other cases as well. Rescaling images can be a bit difficult to do in an effective way.
ImageView solves many scaling problems, at least as long as you can set an image source directly without decoding or scaling the image yourself first. But sometimes you need to take control of the decoding yourself, and that is where this tutorial comes in. Along with this tutorial, I’ve written a code sample project. Download the image scaling code example project from Developer World to learn more. The results presented in this text can be achieved by compiling and running that project.
Isolating the problem
I’ve made this tutorial because I’ve implemented a number of useful utility methods for doing scaling in a way that avoids the most common image scaling pitfalls. Pitfalls such as naïve solutions similar to this:
Bitmap unscaledBitmap = BitmapFactory.decodeResource(getResources(), mSourceId);
Bitmap scaledBitmap = Bitmap.createScaledBitmap(unscaledBitmap, wantedWidth, wantedHeight, true);
So what is correct and what is wrong in the code above then? Let’s look at the different lines of code.
Line 1: The entire source image is decoded to a bitmap.
This might cause an out of memory error if the image is too large.
This might result in a decoded image with a higher resolution than required. It might also be unnecessarily slow as smart decoders can scale when decoding at improved performance.
Scaling an image a lot, as when scaling a high resolution bitmap to a low resolution, causes aliasing problems. Using bitmap filtering (for example, passing true as the latter parameter to Bitmap.createScaledBitmap(…)) reduces the aliasing but is not enough when a lot of scaling is applied.
Line 2: The decoded bitmap is scaled to the wanted size.
The aspect ratio of the source image dimensions and the wanted image dimensions may not be the same. This will result in a stretched image.
Left image: Original image. Right image: Image scaled down with a naïve method. Aliasing problems can be seen such as one eye having a sharp highlight and the other having none. Stretching occurs on the height.
Creating a solution
Our solution will have a structure similar to the code above with where one part will replace line 1, where we decode an image in preparation for scaling. Another part will be to replace line 2, and do the final scaling. We’ll start with the part replacing of line 2 as it will introduce two new concepts, crop and fit, which will impact the solution for replacing line 1 as well.
Replacing line 2
In this part we are scaling the bitmap according to our needs. This step is necessary since the decoding line that precedes this will have limited capabilities to scale. Also in this step we might have to adjust the wanted size of our image if we wish to avoid stretching.
To avoid stretching there are two possibilities. Either we adjust the wanted dimensions by making sure they have the same aspect ratio as the source image, i.e. scaling the source image until it fits within the wanted dimensions, or we crop the source image with an area that has the same aspect ratio as the wanted dimensions.
Left image: Image scaled by the fit method. Image has been scaled to fit within the wanted dimensions and as a result the height of the image is smaller than the wanted height. Right image: Image scaled by the crop method. Image has been scaled to fit at least one of the wanted dimensions and as a result the source has been cropped, cutting away the left and right parts of the source image.
In order to scale like this we implement the following method:
public static Bitmap createScaledBitmap(Bitmap unscaledBitmap, int dstWidth, int dstHeight, ScalingLogic scalingLogic) {
Rect srcRect = calculateSrcRect(unscaledBitmap.getWidth(), unscaledBitmap.getHeight(), dstWidth, dstHeight, scalingLogic);
Rect dstRect = calculateDstRect(unscaledBitmap.getWidth(), unscaledBitmap.getHeight(), dstWidth, dstHeight, scalingLogic);
Bitmap scaledBitmap = Bitmap.createBitmap(dstRect.width(), dstRect.height(), Config.ARGB_8888);
Canvas canvas = new Canvas(scaledBitmap);
canvas.drawBitmap(unscaledBitmap, srcRect, dstRect, new Paint(Paint.FILTER_BITMAP_FLAG));
return scaledBitmap;
}
In the code above we use canvas.drawBitmap(…) to do the scaling. This method crops the area specified by the source rectangle from the source image and scales it to an area in the canvas defined by the destination rectangle. In order to avoid stretching these two rectangles need to have the same aspect ratio. We also call two utility methods, one for creating the source rectangle and another for creating the destination rectangle. These are implemented like this:
public static Rect calculateSrcRect(int srcWidth, int srcHeight, int dstWidth, int dstHeight, ScalingLogic scalingLogic) {
if (scalingLogic == ScalingLogic.CROP) {
final float srcAspect = (float)srcWidth / (float)srcHeight;
final float dstAspect = (float)dstWidth / (float)dstHeight;
if (srcAspect > dstAspect) {
final int srcRectWidth = (int)(srcHeight * dstAspect);
final int srcRectLeft = (srcWidth - srcRectWidth) / 2;
return new Rect(srcRectLeft, 0, srcRectLeft + srcRectWidth, srcHeight);
} else {
final int srcRectHeight = (int)(srcWidth / dstAspect);
final int scrRectTop = (int)(srcHeight - srcRectHeight) / 2;
return new Rect(0, scrRectTop, srcWidth, scrRectTop + srcRectHeight);
}} else {
return new Rect(0, 0, srcWidth, srcHeight);
}
}public static Rect calculateDstRect(int srcWidth, int srcHeight, int dstWidth, int dstHeight, ScalingLogic scalingLogic) {
if (scalingLogic == ScalingLogic.FIT) {
final float srcAspect = (float)srcWidth / (float)srcHeight;
final float dstAspect = (float)dstWidth / (float)dstHeight;if (srcAspect > dstAspect) {
return new Rect(0, 0, dstWidth, (int)(dstWidth / srcAspect));
} else {
return new Rect(0, 0, (int)(dstHeight * srcAspect), dstHeight);
}
} else {
return new Rect(0, 0, dstWidth, dstHeight);
}
}
The source rectangle will be the entire source dimension in the fit case. In the crop case it is calculated to have the same aspect ratio as the destination image, resulting either in the width or the height of the source image being cropped. The destination rectangle will be the entire wanted dimension in the crop case. In the fit case it will have the same aspect ratio as the source image, resulting in either the width or the height of the wanted dimensions being adjusted.
Replacing line 1
Decoders are smart, especially the ones used for the JPEG and PNG formats. These decoders can scale the image when decoding, with improved performance. When doing so, aliasing problems are also avoided. Also, since the image is smaller after decoding, less memory will be needed.
Scaling when decoding is as simple as setting the inSampleSize parameter on a BitmapFactory.Options object and passing it to the BitmapFactory when decoding. The sample size specifies a factor of which each side of the image is scaled, for example a factor of 2 on a 640×480 image will result in a 320×240 image being decoded. When setting a sample size, you are not guaranteed the image will be scaled down exactly according to that number, but at least it will never be smaller. For example a factor of 3 on a 640×480 image could result in a 320×240 image since the value 3 might not be supported. Commonly, at least the first powers of 2 are supported [1, 2, 4, 8, …].
Next step is to choose a proper sample size. The proper sample size would be the one resulting in the largest amount of scaling, but still being equal to or larger than the wanted image dimensions. This is implemented like this:
public static Bitmap decodeFile(String pathName, int dstWidth, int dstHeight, ScalingLogic scalingLogic) {
Options options = new Options();
options.inJustDecodeBounds = true;
BitmapFactory.decodeFile(pathName, options);
options.inJustDecodeBounds = false;
options.inSampleSize = calculateSampleSize(options.outWidth, options.outHeight, dstWidth, dstHeight, scalingLogic);
Bitmap unscaledBitmap = BitmapFactory.decodeFile(pathName, options);
return unscaledBitmap;
}public static int calculateSampleSize(int srcWidth, int srcHeight, int dstWidth, int dstHeight, ScalingLogic scalingLogic) {
if (scalingLogic == ScalingLogic.FIT) {
final float srcAspect = (float)srcWidth / (float)srcHeight;
final float dstAspect = (float)dstWidth / (float)dstHeight; if (srcAspect > dstAspect) {
return srcWidth / dstWidth;
} else {
return srcHeight / dstHeight;
} } else {
final float srcAspect = (float)srcWidth / (float)srcHeight;
final float dstAspect = (float)dstWidth / (float)dstHeight;
if (srcAspect > dstAspect) {
return srcHeight / dstHeight;
} else {
return srcWidth / dstWidth;
}
}
}
In the decodeFile(…) method we decode a file optimized for the final downscaling. This is done by first decoding only the dimensions of the source image, then calculating the optimal sample size using calculateSampleSize(…), and finally decoding the image using this sample size. I’ll leave it up to you if you’d like to dig deeper into understanding the calculateSampleSize(…) method. But basically it makes sure the image is scaled as much as possible while still being equal to, or larger, than the source rectangle that was applied before.
Putting it all together
With the help from the utility methods specified above we can now implement the following replacement lines for the initial code presented:
Bitmap unscaledBitmap = decodeFile(pathname, dstWidth, dstHeight, scalingLogic);
Bitmap scaledBitmap = createScaledBitmap(unscaledBitmap, dstWidth, dstHeight, scalingLogic);
Left image: Done by a naïve solution on mdpi device, decoding consumed 6693 kb of memory and took about 1/4 second. The result is stretched and suffers from aliasing artifacts. Middle image: Achived by the fit solution on mdpi device, decoding consumed 418 kb of memory and took about 1/10 second . Right image: Achived by the crop solution on mdpi device, decoding consumed 418 kb of memory and took about 1/10 second.
To learn more, download our code sample project. With this project, you can see the results on your Android phone and follow the flow in the source code.
Feel free to comment and ask questions about this topic! We’ve created a thread on Google groups for any discussions on this topic.
Andreas Agvard
Sony Ericsson Software department
Posts
No comments: