Harshdeep 2.0

January 19, 2008

Poisson Image Editing

Filed under: Graphics, Imaging — harshdeep @ 8:42 pm

After spending a totally laidback month in Delhi, I’m now back in Austin for the second semester. I did a couple of interesting projects towards the end of the first semester.

The first one was for Computer Graphics course with Dr. Bill Mark. I implemented an influential SIGGRAPH 2003 paper by Perez et al, Poisson Image Editing (PDF). The technical details are there in the paper. In this post, I’ll just show some interesting results.

The paper provides general machinery for handling several important image processing tasks – seamlessly cloning one part of an image to another, for example. Say we start with the images of Mona Lisa and an angry woman.



Now, say you want to see how Mona Lisa would have looked if she was really angry at Leonardo. You select a part of the angry woman’s face and paste it on top of Mona Lisa’s in, say, Photoshop. Now, you don’t want to spend too much time with the Lasso tool, or something like that, carefully selecting parts of the source image and adjusting its color, contrast etc so that it doesn’t look out of place when you place it over the source image. You have just enough time (or inclination) to make a rough selection like this.

Mask (The selected region is NOT darkened)

When you simply paste the selected region over Mona Lisa’s face, this is what you get.

It looks much better with Poisson Image Editing.


The basic idea is to modify the copied pixels under two constraints

  1. The pixels at the boundary should remain the same as the target image
  2. The pixel gradient in the copied region should be the same as the gradient of the source image

Other effects can be generated by modifying either of these constraints. Let’s say we want to seamlessly clone objects that have transparencies too. For example, say we want to paste an airplane complete with smoke trails over a bunch of clouds. With the original method, this is what we get. Notice the blurring of clouds around the smoke trail.

Essentially we want to preserve important structures in both the source (smoke trail) and the target (clouds). We can do this by modifying the constraint 2 above. The pixel gradient in the copied region should be the same as the stronger of the source and target gradients at each pixel. The paper calls this method mixing gradients. This is what we get now. Notice the clouds showing up between the two smoke trails.

Now, say we have a small image that we want to use as a texture. Such an image is expected to tile seamlessly (so that it doesn’t produce artifacts when wrapped on a 3D surface). But when the original image is tiled, it shows considerably distinct seams between the tiles.

Original Texture

Tiles with distinct seams

This intuitively looks like a problem that Poisson equations can help us with. All we need to do is to treat the entire image as the area to be copied, and modify the first constraint such that the pixels up north are the same as pixels down south and similar with east and west sides. The resulting texture image does not look very different from the original, but we no longer see any seams on tiling it.

Processed Texture

Seamless Tiling

The method generally works well, but it makes some assumptions. One of the strong assumptions is that the neighborhood where we are going to insert an object is similar to its neighborhood in the source image. If this is true, we don’t have to change the pixel values of the object too much to satisfy the boundary condition above (constraint 1). To understand this, let’s try to put the lion’s face in a pool full of people.



And bang, this is what we get. Interesting effect, but not exactly what we wanted. Colors in lion’s face had to be changed too much to satisfy constraint 1 above.


We get a better result when we try to clone the floating boy in the next image. This is because he is already in a bluish background. Smaller modifications are required to be made in the color of his hair and body to satisfy the constraints.



In general, Poisson image editing can be used to selectively operate on a particular object in an image. Maybe you want to adjust the intensity of one object and not the rest of the image. Or maybe you want to make the entire image black and white, but leave one object in full color. You don’t have to tenuously select the object – a rough selection is all that’s needed.

As I mentioned in the beginning, this is a widely cited paper. Here are two follow-up papers, both from Microsoft Research, that are very useful.

  1. Drag-and-Drop Pasting, Jia et al, SIGGRAPH 2006
  2. Poisson Matting, Sun et al, SIGGRAPH 2004

I did another project last semester that I want to write about, but I’ll save that for another post.


Blog at WordPress.com.