Hybrid Video Production – Ways to Make AI Part of Your Workflow

Annoying AI slop is all around the Web, and video turbines are a subject of big controversy for a lot of causes. A number of professionals are on the fence. They will’t think about how generative fashions may very well be helpful of their work. Why would I desire a machine to create a random, generic clip for me – artificial, uncanny, questionable, and utterly devoid of my voice? If we put it this manner, I don’t have a solution. But what if it may maintain your voice? And what if there have been certainly workflows for which AI video turbines may very well be of actual use? Seasoned filmmaker Drew Geraci presents a distinct perspective and reveals us his strategy to hybrid video manufacturing with AI.Drew Geraci is a famend cinematographer and photographer with over 20 years of expertise within the {industry}, creating charming time-lapse and hyper-lapse works that you just’ve most likely seen. (As an example, the “Home of Playing cards” opening credit.) Just lately, Drew has launched his newest MZed course, “Directing the Future: Moral AI Video for Filmmakers.” This text makes use of supplies from the course and presents a sneak peek at his workflow for reworking nonetheless pictures into movies.Hybrid video manufacturing and what function AI performs in itMy colleagues and I’ve written quite a few information articles on AI video turbines over the previous few years. Sora, Google Veo, Dream Machine, Veo 3, Sora 2…  A few of them are already tremendous outdated, as a result of the generative tech is growing at a breakneck pace. But, I’ve by no means used AI video turbines in actual life, aside from testing them out to report on their present capabilities. Why? Totally different causes, with the query of ethics on the forefront. But additionally, I simply couldn’t think about how a generated AI video may very well be a part of my filmmaking routine. Particularly, as a result of I like concepting and creating impactful cinematic pictures, working with actual folks, and having full management of the end result. Frankly, none of that applies to AI video. Or am I unsuitable?In his course, Drew Geraci demonstrates varied use instances that supply a distinct strategy to those deep-learning instruments. Let’s name it a hybrid video manufacturing. A manufacturing that seamlessly blends AI-generated components with reside footage and retains your creative voice intact. It may very well be, for example, the creation of property, based mostly in your actual pictures, after which rigorously animated and post-processed in After Results to place collectively a compelling shot:Totally different phases of property creation, as demonstrated within the course. Picture credit score: Directing the Future course / MZed / Drew Geraci One other strategy can be to convey movement to your nonetheless pictures and mix generated clips with live-action footage in post-production. Let’s have a more in-depth take a look at Drew’s workflow right here.Cautious planning earlier than the shootSo, for this demonstration, Drew went on a sunny shoot in a pool with a mannequin and an assistant. More often than not, it was similar to every other skilled photograph and videography gig. Excessive-resolution cameras (Sony a7R V and Sony a1 II, on this case), working with lighting, in search of the very best picture compositions, giving performing instructions, and capturing varied angles.Nevertheless, they knew from the beginning that this might be a hybrid video manufacturing, so in addition they deliberate for it:A number of the frames have been taken as nonetheless photos. Others – as video clips. This was accomplished with the flexibleness facet in thoughts, which we are going to discuss under.Drew’s mannequin, Michaela, wished to regularly flip right into a mermaid on the finish of the video. So, they made certain to search out the very best perspective, which might visually assist the impact. It turned out to be a top-down overhead shot, captured from a drone.Picture credit score: Directing the Future course / MZed / Drew GeraciAfter the shoot, Drew Geraci went by way of his standard choice course of and edited the images in Adobe Lightroom, giving them a tender, nearly ethereal look he was striving for.Tips about turning stills into clips utilizing Google FlowSo, now he had a group of gorgeous nonetheless pictures that he may already ship to the consumer. But aside from that, Drew determined to make use of a few of them for a video edit. For that, he wanted to offer the photographs just a little movement to match them seamlessly with different footage. And that is the place AI video era is available in.In his course demonstrations, Drew Geraci makes use of Google Move. It is a platform for creatives, with an easy-to-understand interface that makes use of Google’s newest generative fashions, together with Veo 3. (Please be aware {that a} trial model is out there, however for additional creations, a subscription is required.) Google Move is likely one of the most generative instruments we all know of, using textual content prompts. Nevertheless, it additionally features a characteristic known as “Frames-to-Video.”Picture credit score: Directing the Future course / MZed / Drew GeraciUsing Frames-to-Video, Drew uploads one among his nonetheless pictures after which writes a textual content description of the movement. For the toes close-up above, the immediate was: “Stationary shot; a lady’s toes stroll ahead utterly right into a pool with each toes, one after one other, in a really delicate method; cinematic look.” As you possibly can see, it’s fairly an elaborate textual content with quite a few actual particulars. The rationale behind that is that you just’ll want a number of iterations to realize the specified end result. After the primary attempt, you probably will get a end result the place one thing is off. This could information you in altering the textual content immediate and what you should specify. Usually, in Drew’s expertise, the extra particular your command, the higher end result you get.So, what was the end result? Google Move took Drew’s photograph, stored the picture composition, the lighting, the colours, and the mannequin’s toes, and solely guessed the proper movement based mostly on the textual content immediate. It didn’t invent something, and it stored the inventive imaginative and prescient intact.Easy VFX transitionAnother tip for utilizing Google Move, as shared by Drew Geraci, is to attempt utilizing totally different AI fashions in your photographs. In a roll-down menu, you possibly can allow older variations of Veo, as proven within the screenshot:Picture credit score: Directing the Future course / MZed / Drew GeraciWhy? Some would say: the newer, the extra superior. But Drew finds Veo 3’s outcomes typically too plastic-looking, as in the event that they attempt to easy out the picture’s particulars. Additionally, this neural community can go a bit astray out of your established colour grading look. Thus, for the mermaid transformation thought, he used Veo 2.AI-generated video scene nonetheless, based mostly on an actual shot. Picture credit score: Directing the Future course / MZed / Drew GeraciIt took a number of iterations to realize a transition from legs to a mermaid tail that appeared respectable. Nevertheless, as a result of the impact had been deliberate prematurely they usually had chosen a very good angle, it wasn’t a troublesome course of.Publish-productionBefore continuing, Drew makes use of Topaz Labs to upscale AI generations to 4K. In any case photographs are accomplished, he combines them right into a single timeline in DaVinci Resolve and edits them right into a single video. His aspect tip for working with Google Move’s outcomes is to mess around with pace. As an example, he prompted the sequences within the water to be in gradual movement. But, when he will increase their pace, say, to 150%, impulsively, they really feel far more life like.Engaged on the ultimate edit. A nonetheless from the course, picture supply: Drew Geraci / MZedThe remaining clip, demonstrated within the course, seems constant, though it incorporates each AI-enhanced footage and live-action photographs. Properly, that’s as a result of each use the identical expertise, model, lighting, colour palette, and imaginative and prescient. As Drew factors out, this may very well be a brief, enjoyable, social media clip. Alternatively, if you wish to have a extra elaborate VFX transition to a mermaid, accomplished by hand by an skilled artist, this clip may function a terrific stylized previz for a pitch.So, this was Drew’s tackle hybrid video manufacturing and the way it can streamline a few of your workflow whereas nonetheless preserving your voice and imaginative and prescient. Whereas a number of what is alleged about AI video turbines is unfavourable (for example, that they’re used for scams and in different dangerous methods), this might change into a distinct, extra moral, and sustainable strategy. Drew is at the moment engaged on further classes, so keep tuned!What else are you able to be taught with MZed Professional?As an MZed Professional member, you possibly can entry a whole bunch of hours of filmmaking training. Moreover, we frequently add new programs (a number of are at the moment in manufacturing).Beginning at simply $29/month (billed at $349 within the first 12 months, $199 from the second 12 months, or billed month-to-month at $49), right here’s what you’ll get:60+ programs, over 800+ high-quality classes.Extremely produced programs from educators with many years of expertise and awards, together with a Pulitzer Prize and an Academy Award.Limitless entry to stream all content material throughout the 12 months.Offline obtain and viewing with the MZed iOS app.Reductions to ARRI Academy on-line programs, solely on MZed.Most of our programs present an industry-recognized certificates upon completion.Buying the programs outright would value over $9,500.Course matters embody cinematography, directing, lighting, cameras and lenses, producing, indie filmmaking, writing, modifying, colour grading, audio, time-lapse, pitch decks, and extra.7-day money-back assure in case you resolve it’s not for you (on annual billing solely).Full disclosure: MZed is owned by CineD.Be a part of MZed Professional now and begin watching right now!What are your ideas on this strategy to AI video turbines? Would you think about a hybrid video manufacturing for a few of your initiatives? How else may it change into part of your real-life workflow? Share your concepts with us within the feedback under!Picture credit score: Directing the Future course / MZed / Drew Geraci.