Text & Images By Gary Small a.k.a. Photoshopman

Today’s tip covers a couple of ideas here. First off, the Refine Edge command is not new to Photoshop. It was introduced in Photoshop CS 3. However, in CS 5, this new and greatly improved version of the Refine Edge command also addresses the fact that (in case nobody noticed) the Extract tool mysteriously vanished. Actually, the Extract tool disappeared when CS 4 was released, but as an afterthought, the plug-in was sent along, to be put back in afterwards. Now, with the new Refine Edge command, there is really no need for the Extract tool anymore. Let’s see how it works…

In this example, I have a nice picture of a girl that was taken on location. Mom would have loved a nice studio portrait of her, instead of using the available location, but time constraints did not allow a background to be brought to the job site. You know how it is. Let’s see what we can do to give mom what she wanted.

Step 1. Using the Quick Select tool, I drag around the contours of the young lady. The Quick Select tool is an amazing tool, because it looks for the edge contrast in the area you’re selecting and “locks on” to those edges, kind of like a souped up version of the Magnetic Lasso tool. Depending on the subject and background, there are some selections that can be made with this tool in one brush stroke. One important note about the Quick Select tool, is that, unlike all other selection tools, where you have to hold down the shift key to add to the selection, the Quick Select tool is always in “Add” mode. What that means is, you don’t have to hold down an extra key to add to an existing selection with this tool. You just make another stroke on the area you want to add and it will add it to the selection. Subtracting from the selection is the same as the other selection tools in Photoshop, in that you’d hold down the Alt (Windows) / Opt (Mac) key, and draw across the part of the selection you want to remove. In the first illustration, you can see the original picture with the selection drawn out.

Here is the original image, after selecting the subject using the Quick Select Tool.

Step 2. The Refine Edge command is found on the Options Bar on top, just below Photoshop’s menus. As long as you have any selection tool chosen in the tool box, the “Refine Edge” button will be available in the Options Bar. Click the button to bring up the dialog box. You’ll see it is broken into 4 sections: View Mode, Edge Detection, Adjust Edge, and Output. If you click the View Mode drop down button, you’ll see several choices of how you can view your selection: Marching Ants, Overlay (like a quick mask), On Black, On White, against transparency, as a Black and White mask, On Layers, and Reveal Layer. In the example here, I have the subject against a black background. I usually look at my subjects against black and against white, to see if there’s any contamination or fringing when they get cut out. But that’s where the real power of this new improved tool comes into play. As you can see in the initial preview, the edges are very sharp and jagged. Let’s see how to make it look more natural.

Here is what we see when we first click the “Refine Edge” button. This shows the different view modes available. For this image, I chose “On Black” to show the selected area against a black background.

Step 3. In the Edge Detection section, move the Radius slider to the right. This increases the size of the radius, which is the area of transition. In other words, this is the area that includes some background that’s being removed and some foreground. The greater the radius, the more transition. But it gets to a point where there’s too much transition and it’s not effective, so if you think you can just crank the slider all the way to the right and get great transitions, think again. The next image shows the radius view. If you check the “Show Radius” checkbox in the View Mode section, you’ll see the size of the radius and can use this view as a means to see just how much transition you want in your selection. Anybody who has had experience with the Extract tool in previous versions of Photoshop will know exactly what I’m talking about when I refer to a transition area. With the Extract Tool, that transition area was defined by the green highlight you drew around the subject you wanted to extract from the background. This is almost the same thing. If you check the “Smart Radius” checkbox, it will try to automatically smooth out the transition. This is great for places where there is hair and uneven edges that you are trying to cut out or select. One new tool that was added was the Refine Radius tool, which looks like a paintbrush that was added to the Refine Edge dialog box. This is an amazing feature. You can actually paint across wisps of hair and it will include them in the selection while keeping everything around the hair or whatever Photoshop considers the “non-foreground” areas hidden (or unselected). If you took away too much, you can switch to the Erase Refinements tool, which is the opposite of the Refine Radius tool, and you can paint away wherever you overdid it.

This is what the selection looks like after increasing the radius. As you can see, the edges around the girl smoothed out a bit, but some areas still need to be refined a little (see the girl’s arm), plus we can make the ends of the hair more realistic too.

Step 4. Looking further down the panel, you can see the Adjust Edge controls. These are very similar (almost identical) to the controls in the previous version of the Refine Edge command. Here you can soften (feather) the edges of the selection, as well as smooth, or add contrast to the selection edge. In addition, a slider was added, called “Shift Edge”. Think of this as a replacement for “Expand/Contract”. Moving the slider to the left (negative amount), you contract, or shrink the selection. Moving it to the right (positive amount), you’re increasing or expanding the selection.

When I checked the Show Radius checkbox, we see exactly what the radius looks like that we’re working with. With this view, we can adjust the size of the radius and fine tune the “transition” area, which will help get us a more natural selection, and cutout.

Step 5. The last section of the panel is the Output options. Here’s where you choose how you want the results to appear. Choices are: Selection (if you just want a refined selection), Layer Mask, New Layer, New Layer With Layer Mask, New Document, and New Document With Layer Mask. There’s also a checkbox marked “Decontaminate Colors”. Check this if you are cutting a subject out of a colorful background (think green screen) and you want to get rid of the color fringing that sometimes comes as a result. I’ve had this happen a lot when I cut people out of a white background and had to spend a lot of time retouching the color contamination that I got afterwards. This works really well. It is not always perfect, but it is a huge improvement! The next illustration shown is the result of choosing New Layer as my output choice.

After choosing “New Layer” as my output option, once I click the “OK” button, this is what I got as a result…a new layer with just the selected area from the original image. Note, in order to see this, I turned off the visibility (the eyeballs) of the other two layers on the Layers panel.

Step 6. After I have my cutout, all that is left to do, is to drop a new background in behind my subject, apply a Drop Shadow layer style, to make it look more realistic, then crop as necessary.

All that’s left now is to drop in a new background. I added a drop shadow layer style to make it look more realistic. Otherwise it would have looked like it was just cut and pasted and not like it was actually photographed that way.

The last illustration shows the finished product, with the addition of a corner burn to make it look like a custom printed portrait. And Voila! We turned a so-so image taken in a hurry, into a studio portrait. Believe me, it took longer to describe the process than it took to actually do it. I believe it only took me maybe 5 minutes, tops.

Here is the finished image, after cropping it square and applying a corner burn, to give it that “custom printed” look. By the way, I used the “Refine Edge” command to help create the soft selection that I used to burn (or darken) the edges of the image too.

If you like these new features, just wait till you check out Photoshop CS 5 which began shipping this week! There are over 100 improvements and new features that are sure to get you excited and trust me when I tell you, any one of them is worth the price of the upgrade! For more information about Creative Suite 5, go to www.adobe.com.

• Gary Small a.k.a. Photoshopman is a Professional Photographer, Photoshop Guru and master of color management. Check out his work at www.jsmallphoto.com.