Ever wonder how those simple looking iPhone ads are shot? We recently solved this puzzle to create an iPhone app commercial for a client. The challenge: make iphone screen/hand interaction look fantastic on camera. Let’s start with a look at the finished video, and then jump into how it was shot. Later this week, I’ll post a followup with information about how we cut it.
In preparation for this project, I did some Googling and found much disagreement in the pro video forums on how to approach shooting an iPhone screen. Some recommended filming the screen live, others suggested painting green paint over the phone face, and one person said it couldn’t be done without complicated compositing software like Mocha.
We spent a full day of pre-production testing different approaches, and hit on a relatively simple way to do the job, with post-production handled almost entirely within Final Cut Pro X. Here’s how it works.
First up: overall lighting. Our first pass at lighting the hand holding the phone was a little too dramatic – we discovered it’s important to fill at almost a 1:1 ratio, otherwise the shadows cast by the finger on the screen area will be way too dark to key easily, as will be seen later. I’ll cut to the chase and share our lighting diagram, as well as the fact that we ended up using two setups: a white background to capture the establishing shot with the hand and phone, and green screen for everything else.
First, let’s briefly look at what doesn’t work. We initially tried shooting the app off the phone itself, but I suspected it wouldn’t look good enough to stand up to close inspection. I was right. Not only was the screen subject to moire patterning, but it was difficult to get the color temperature of the screen’s LEDs to match with the lighting in our studio environment. It just didn’t look professional. So we knew we would need to do some kind of masking or keying to show a screen capture of the phone in the display area.
Next, we needed to determine whether the talent could hold the phone still enough. It’s unlikely that anyone has more steady hands than my partner Lisa, but close up with a 100mm lens, it wasn’t possible for her to hold the phone even close to steady. We next tried making a cradle for her hand. But within a few minutes of holding the phone, she was getting really tired and that translated to more shake. Scratch that idea.
A close screening of this Apple ad reveals that after the hand comes into the shot, it settles quickly into complete stillness. That is, a still photo. Or in our case, a freeze video frame The only hint of motion in the talent’s left hand during the remainder of the shoot is a shadow falling on his fingers caused by the movement of the right hand approaching the screen. So I thought: let’s just have the talent move the phone into the shot, freeze the frame, and use that for the duration. Then we’ll composite the right hand doing the manipulations.
We solved this problem by doing a frame grab, but we couldn’t get the hand to come to rest naturally. So we ended up introducing the hand and phone by using and using Apple’s Slide transition to slide it into the frame (slide-push, to be precise: you access these options by control-clicking on the transition in the timeline, and selecting from among the options in the Inspector). I go into more detail about how we cut the ad in Final Cut Pro X in part II of this post, coming later this week.
Ultimately what we realized is that it’s all about the app, not about the hand. On screen, anything that moves gets our attention. So the hand shouldn’t move after it’s introduced. Just doing a freeze frame accomplishes everything that needs to happen, and keeps the attention where it belongs. Left hand with phone, as shot and ready for luma key (discussed below in Editing):
Next problem: how to composite the right hand into the scene. The further problem, of course, is that the talent needs to hit very specific buttons and perform actions like pinching on precise points on the screen. It seemed we would need to keep the phone there for reference, but we weren’t sure how we’d key it out in that case.
We tried displaying a green screen on the phone. But that tended to illuminate the talent’s hand, making it harder to key. I hit on the idea of taking screen shots of the app, converting them to black and white, then colorizing them and shifting them all to green.
Recording screen captures: an app for that?
Around the middle of last year there was an app called Display Recorder available. However, it has been pulled from the App Store for some reason, so there’s currently no app that does this directly.
The good news is that there IS an app for capturing iPhone screens, indirectly, via Airplay on a Mac. You don’t even need an AppleTV to make it work: just a Mac and an iPhone 4s or newer (sorry, it won’t work with older iPhones). It’s called Reflector, and it costs just $15.
If you’re shooting an iPhone ad, you want this app. Not only does it allow you to mirror the contents of your phone’s screen, but it has a robust set of preferences that seem designed with filmmakers in mind.
First exhibit: Built-in video recording capabilities (no need to use Quicktime 8 or third-party screen capture tools) that allow you to record in the highest possible resolution supported by your hardware.
Next: the option to wrap the contents of the screen in an iPhone frame (you can choose white or black).
But wouldn’t be even cooler if you could have the video output against a green screen, so you could just drop a key on it and away you go? No problem. Just select this preference:
And tell it to output in full screen mode:
This will output a file that looks like this:
Armed with this tool, you’re able to capture a screen recording of all the on-screen action you will need for your final piece. But’s only half of the battle. We now need to turn our attention to figuring out how to add finger gestures into the mix.
For that, we need green screen. And a way to ensure that the talent’s fingers would be pointing to the precisely correct location on screen, so that when we composite everything in post, it lines up.
After a lot of testing, we hit on a way to do it. In Photoshop, we created a 2″x3″ file (the dimensions of an iPhone screen) which we filled with bright green (make sure it’s not a yellow-green, but rather pure green or slightly blue-green, because any yellow can start to overlap with skin color making it difficult to key).
We created a file for each set of actions (analogous to a “scene” in film terms). The idea is to create buttons using a darker shade of the same green color, to show the talent where the button is. Each button is then numbered in the order in which they are to be pressed, swiped, or pinched. We used stroked rectangles to indicate swipe regions. But how to know exactly where to create the buttons? We took screen shots of the iPhone app screen (by simultaneously pressing the phone’s home and on-off buttons), and copied these files into our green file, placing them on the bottom layer. We then made the green layer half transparent, so that we could see through it enough to trace the outline of each spot, and make precise marks.
Then, we printed out each screen on regular paper. The first time they came out way too dark, so we had to go back and choose an even lighter shade of green. Once we got the color looking something like a green screen, we cut each out of paper using a ruler and exacto knife, and taped it to a piece of piece of green foam core about 1.75″ wide by a few inches tall.
To put this together requires shooting in front of a green screen, and holding the cardboard screens suspended vertically on a clamp using a c-stand, so that they are positioned comfortably for the hand talent. Here’s how our setup looked:
Some gotchas to watch out for:
Reflections. Our hand talent had received a manicure the morning of the shoot, so his nails were nearly perfect – for reflecting light. This caused us a lot of grief.
We ended up scrubbing his nails with a kitchen sponge, the only thing we could find, until they were dulled down a bit. A good reason to have a makeup person on set!
Above: Fingernail with reflection reduced after roughing up surface of nail with kitchen sponge.
Shadows on the green card. As the talent’s hand comes into contact or near-contact for each button press, shadows will be cast. We had to set up a third fill light just behind and above camera to cut these down enough so that we could get a clean key. This caused the right hand to be lit a little brighter and more directly than the left hand had been, but we found a mid-point that we could live with.
Using this approach involves the following steps:
1. Script it. We created a storyboard for every screen. Know in advance exactly what the voiceover will be, and which graphic will be displayed on screen, and what gestures you will need.
2. Build green
We rented studio space at The House Studios in Queen Anne for this shoot. Their rates are not prohibitive, and the place is spacious, clean, even a bit swanky.
There is a kitchen area, changing rooms, and comfortable waiting area with couches and tables. It’s no good for sound recording, with lots of HVAC white noise. But it’s great for MOS projects like this one. Having the extra space made everything go smoother and faster.
Here’s a list of all the equipment we used to pull off this shoot, starting with the pile my front porch on the morning of, ready to load:
- Tripod: Vinten Vision Blue
- Camera: Canon 5dmkiii
- Lens: Canon 100mm f/2.8L macro.
- SmallHD DP6 field monitor, with Zacuto Zamerican Arm
- ChromaPop green screen
- Tilta Matte Box
- 2 Arri 650 lights
- 2 Smith-Victor 650 open-faced lights, with shoot-through umbrellas
- Lowel Pro-Light 250-watt
- 2 c-stands, 3 light stands
- 5 pelican cases for above; 1 golf club case for transporting above stands.
- 2 extension cords with 4-outlet terminals
- Multi-cart R-12
- 24×36 silk, 24×36 flag, 4×4 foam core with roscoe reflector glued to one side.
- 5×8 sheet of bleached muslin
- 8×10 sheet of blackout cloth
- Milk crate loaded with pair of 15lb Matthews sand bags; handful of 10lb shot bags for anchoring light stands.
- Misc A clamps and background stand clamps.
- Background stand.
- 15″ 2012 Macbook Pro
- Lexar USB3 CF card reader
- 4TB Lacie 2big Thunderbolt hard drive for on-set data backup.
- Studio supplied gear:
- Traveling wall (for white background)
- c-stands, a clamps, grip clamp, heavy duty extension cords
OK, so we’ve got all the footage dialed. How does this all come together in post? Join me later this week for part II of this post, where I’ll explore how to edit an iPhone app commercial.