Harshdeep 2.0

January 19, 2008

Poisson Image Editing

Filed under: Graphics, Imaging — harshdeep @ 8:42 pm

After spending a totally laidback month in Delhi, I’m now back in Austin for the second semester. I did a couple of interesting projects towards the end of the first semester.

The first one was for Computer Graphics course with Dr. Bill Mark. I implemented an influential SIGGRAPH 2003 paper by Perez et al, Poisson Image Editing (PDF). The technical details are there in the paper. In this post, I’ll just show some interesting results.

The paper provides general machinery for handling several important image processing tasks – seamlessly cloning one part of an image to another, for example. Say we start with the images of Mona Lisa and an angry woman.

Target

Source

Now, say you want to see how Mona Lisa would have looked if she was really angry at Leonardo. You select a part of the angry woman’s face and paste it on top of Mona Lisa’s in, say, Photoshop. Now, you don’t want to spend too much time with the Lasso tool, or something like that, carefully selecting parts of the source image and adjusting its color, contrast etc so that it doesn’t look out of place when you place it over the source image. You have just enough time (or inclination) to make a rough selection like this.

Mask (The selected region is NOT darkened)

When you simply paste the selected region over Mona Lisa’s face, this is what you get.

It looks much better with Poisson Image Editing.

 

The basic idea is to modify the copied pixels under two constraints

  1. The pixels at the boundary should remain the same as the target image
  2. The pixel gradient in the copied region should be the same as the gradient of the source image

Other effects can be generated by modifying either of these constraints. Let’s say we want to seamlessly clone objects that have transparencies too. For example, say we want to paste an airplane complete with smoke trails over a bunch of clouds. With the original method, this is what we get. Notice the blurring of clouds around the smoke trail.

Essentially we want to preserve important structures in both the source (smoke trail) and the target (clouds). We can do this by modifying the constraint 2 above. The pixel gradient in the copied region should be the same as the stronger of the source and target gradients at each pixel. The paper calls this method mixing gradients. This is what we get now. Notice the clouds showing up between the two smoke trails.

Now, say we have a small image that we want to use as a texture. Such an image is expected to tile seamlessly (so that it doesn’t produce artifacts when wrapped on a 3D surface). But when the original image is tiled, it shows considerably distinct seams between the tiles.

Original Texture

Tiles with distinct seams

This intuitively looks like a problem that Poisson equations can help us with. All we need to do is to treat the entire image as the area to be copied, and modify the first constraint such that the pixels up north are the same as pixels down south and similar with east and west sides. The resulting texture image does not look very different from the original, but we no longer see any seams on tiling it.

Processed Texture

Seamless Tiling

The method generally works well, but it makes some assumptions. One of the strong assumptions is that the neighborhood where we are going to insert an object is similar to its neighborhood in the source image. If this is true, we don’t have to change the pixel values of the object too much to satisfy the boundary condition above (constraint 1). To understand this, let’s try to put the lion’s face in a pool full of people.

Source

Target

And bang, this is what we get. Interesting effect, but not exactly what we wanted. Colors in lion’s face had to be changed too much to satisfy constraint 1 above.

Result

We get a better result when we try to clone the floating boy in the next image. This is because he is already in a bluish background. Smaller modifications are required to be made in the color of his hair and body to satisfy the constraints.

Source

Result

In general, Poisson image editing can be used to selectively operate on a particular object in an image. Maybe you want to adjust the intensity of one object and not the rest of the image. Or maybe you want to make the entire image black and white, but leave one object in full color. You don’t have to tenuously select the object – a rough selection is all that’s needed.

As I mentioned in the beginning, this is a widely cited paper. Here are two follow-up papers, both from Microsoft Research, that are very useful.

  1. Drag-and-Drop Pasting, Jia et al, SIGGRAPH 2006
  2. Poisson Matting, Sun et al, SIGGRAPH 2004

I did another project last semester that I want to write about, but I’ll save that for another post.

Advertisements

8 Comments »

  1. Cool!

    Comment by harry — July 27, 2008 @ 7:04 pm

  2. wow.. u’re good at editing…!!!

    Comment by otherside7 — January 28, 2009 @ 5:01 pm

  3. Great work!
    May I use some pictures from your post in my non-commercial educational article?

    Comment by kovlan — February 2, 2009 @ 8:52 am

  4. Hi there. I am too working on this paper but not getting exact results.

    I have some questions:
    Q1 .If you look at Eqn 7 and 8,
    is the variable fp is unknown and rest are known ?

    Q2. the Vpq is given as dot vector of V((p+q)/2). pq,
    Is Here do ‘p’ and ‘q’ represent the coordinates of pixels p and q or their intensities?
    Thanks.

    Comment by cirus — April 1, 2009 @ 10:54 pm

  5. i liked this…….i want to know how this can be done.i want to submit my project on this topic…

    Comment by anki — August 12, 2010 @ 5:56 am

  6. plz help me to get this

    Comment by anki — August 12, 2010 @ 5:58 am

  7. “The pixels at the boundary should remain the same as the target image.”
    Which boundary are you talking about?
    The rectangular boundary of the target image OR the boundary of the patch created by the source image?

    Comment by srijan — December 18, 2010 @ 7:30 pm

  8. If there is a change in the search engine algorithms, it is also up to the service provider to keep a watch on
    such activities. If you encourage your visitors to write
    their original reviews about your products instead of coping reviews from competitive websites to show you have circulation and satisfied customers,
    your website is more likely to earn higher ranking in the major search engines.
    It’s a competitive world out there on the world-wide web, but SEO consulting can help you dramatically improve your odds of being found in the din and getting your message across. This gives your name higher credibility and better search engine rank. Thus, in minimum cost, you get an opportunity to increase your site ranking on Search Engine Result Page (SERP). Meta Name and Meta Description Tags are two of the important ones. Since keyword analysis is needed for both SEO (search engine optimization) and SEM, we often confuse using these terms. When SEO services are working for you, you should always be ready to capitalise on innovations and fresh marketing opportunities. Isn’t that something you and
    your business should be a part of. For those of us who are using
    the internet to make money, we all know that it is absolutely
    essential to get visitors out to our websites.
    Arrange the H1, H2 and H3 tags serially with proper hierarchy.
    The concepts make use of the current social networking sites, which are very popular among the online audience.
    Effective SEO techniques for Bloggers aren’t any different. You can Google maps link for your website, which will be very helpful if any person search in images sections. In this article, I will show you the best Word – Press SEO plugins you need to use in order to boost your search engine rankings and get more traffic to your blog. The sooner a site is presented in the search result, or the higher it “ranks”, the more searchers will visit the site. Online Advertising and SEO Marketing. SEO content is a huge deal in today’s
    online oriented business world. The website serves as a way for customers to find the business and be able to see what services are offered.
    Yahoo style guide.

    Comment by Drusilla — February 14, 2013 @ 3:21 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: