The frontiers of the use of AI in sectors traditionally linked to human creativity, such as screenwriting, are still to be explored. At the moment, however, we can already hypothesize at least a collaboration between man and machine, that is, a sort of co-creation, in which AI already provides a rich and complex base of elements that screenwriters can use. An example is the computer program Benjamin, the result, not surprisingly, of the collaboration between a director (Oscar Sharp) and a researcher specialized in AI (Ross Goodwin). By providing AI with an huge archive of data relating to science fiction films, the program was able to produce Sunspring, considered to be the first example of a film based on a screenplay created entirely by AI.

This result, which at first sight is extraordinary, must however be evaluated in more specific terms: the program has in fact provided a script very similar, if not identical, to those of the films stored in its memory, and it clearly showed, in addition to narrative inconsistencies, the limited ability of the system to be sensitive, for example, to the definition of the characters, the situations and the emotional states in which they act, and, more generally, to the structure and “style” that a film should possess. At least for the moment, programs like Benjamin do not seem to demonstrate the awareness and critical vision of stories and characters that are crucial features of the human mind.

Other important, although perhaps more specific, functions that AI is already able to perform in the pre-production of a film are linked to the ability of AI to access understanding, processing and, in a near future, creation of natural language (NLP – Natural Language Processing), with interesting developments for the relationship between texts and images. From a text it can already be expected that AI can generate an image, which also opens up interesting uses for the actual production phase of a film.

2. Production

The phase of concrete realization of the film, or in other words of the actual shooting, thus far involving very concrete and specific interventions by numerous professional figures, starting from the director, is perhaps the one that records the interesting results, although still at an experimental at the moment for the time being.

The shooting of a film involves technical choices, translating into stylistic choices, which use the various aspects of cinematographic language, such as shots, camera movements, lighting, and so on. An AI-based program would therefore have to be “trained”, through the usual huge amount of data loaded into it, to “understand”, so to speak, these cinematic conventions, including those that are the product of the choices made by the filmmakers and their concrete actions also from a strictly “technical” point of view. Therefore, as we have already suggested, in this case, too, an interactive collaboration between machine and man is essentially envisaged, which however allows the sometimes very innovative proposals provided by the program to be taken into consideration.

In this way, for example, certain scenes or sequences, loaded into the program, could be replicated, updated or modified without the need for reshoots. This would entail, in addition to obvious savings in time and costs, a significant adherence to the previous aesthetic choices made by the director and his collaborators, and therefore a possibly deeper stylistic coherence. Particular shots, camera movements or lighting conditions, for example, could be automatically provided and executed by the program whenever similar scenes or sequences appear, without proceeding with manual checks and adjustments. In particular, AI is already capable of precisely separating parts of a shot from its background (for example, an actor or a moving object), with the possibility of replacing the human figure or object with others, or to replicate them in other appropriate sequences. It is not just a question of physical movements: as mentioned, the ability of AI to “understand” and process natural language can open up the prospect that from verbal descriptions the program can “create” corresponding images and entire scenes or sequences.

One area in which AI has already contributed quite massively to film production is the use of drones and remote-controlled devices, in which drones are equipped with cameras capable of filming, on the basis of based on instructions loaded into the program, even entire scenes or sequences, also following, in addition to purely technical instructions, cinematographic principles and even aesthetic choices. We must never forget that these devices are “autonomous learning machines”, therefore capable of “learning”, that is, of storing in memory all the previous experiences acquired during the simulated “training” phases, and of creating new associations.

The result of the use of AI in contexts like these can already be appreciated in products such as the film In the robot skies (see YouTube), created with the use of three drones specially “instructed” on the use of cinematic techniques, and filmed from the “point of view” of these drones. The film therefore constitutes, first of all, an exploration of the use of drones both as cultural objects and as specific devices for film making. The film itself does not give up a more “narrative” part, as it at least suggests the story of a young couple in a dystopian urban landscape ( Frohlick Alex 2020. Artificial Intelligence and Contemporary Film Production: A Preliminary Survey, University of London, p. 14). The creators themselves described this experiment as follows on their YouTube channel:

“Directed by speculative architect Liam Young and written by fiction author Tim Maughan, In the Robot Skies is the world’s first narrative shot entirely through autonomous drones. In collaboration with the Embedded and Artificially intelligent Vision Lab in Belgium, the film has evolved in relation to their experiments with specially developed camera drones each programmed with their own cinematic rules and behaviours. In this near future city drones act as agents of state surveillance but also become co-opted as the aerial vehicles through which two teens fall in love.”

3. Post-production

The end of filming does not coincide at all with the final realization of the film – we could on the contrary say that we are only halfway there. A series of operations remain to be carried out, the most important of which is probably editing, but also the addition of special visual and sound effects, the integration of the musical soundtrack, and, as we will see, many other possibilities made available precisely by the use of AI.

3.1. Editing

Editing involves a series of complex operations, which concern in particular the choice of shots, scenes or sequences to include or discard, and their sequencing for the final result. Since these operations normally require a lot of time (months, in some cases even years) and huge human and material resources, the advent of AI has been more than welcome by the film industry, not only for the variety of solutions that it can offer, but also (and perhaps above all) for the drastic reduction of time, energies and resources. What a team of human operators can accomplish over the course of several months can be accomplished by a computer program in a matter of minutes or even seconds.

We have already mentioned, for example, how actors and objects can be removed from shots to be replaced by something else, perhaps even by the same actors and objects who appear however in different shots. This allows you to create new sequences using the best resources available, and also leaving the program with the task of selecting, based on the parameters provided, the shots and/or scenes that are qualitatively best or most suited to the final result you want to achieve. Furthermore, if you want to avoid, for example, abrupt transitions of shots (jump cuts), or if you want to underline sequences by increasing the dramatic or emotional impact in general, you can provide the AI system with these basic indications, and the system it then processes, in sync, the script and the footage taken, so as to match the chosen dialogues with the actual available audio recording. This data is then further processed to take into account other variables (such as the position of the actors and the camera, or the types of shots taken) in order to integrate the new dialogue flow with the available video footage. Thus, for example, if you choose to ask the program to intensify the emotions, perhaps by accelerating the pace, the program will process sequences of shorter duration and with a higher number of close-ups of the actors. On the contrary, by inserting instructions to slow down the pace, the sequences will become longer and their number will increase. Ultimately, AI offers human operators the opportunity to test different approaches using different editing styles in a fraction of the time it would take if these operations were performed manually. Even in this case, it is clear how the potential of AI is at its best when combined with human creativity.

3.2. Special effects

For decades the film industry has been using computer-generated images (CGI) to obtain increasingly effective special effects, both visual and auditory. AI has made it possible to expand and further improve these techniques, being able to count, once again, on a mass of data obtained through filming and subsequently translated into thousands of detailed images, which then allow even the smallest details (e.g. of a face)to be combined in order to obtain the most effective final result. The character of Thanos in Avengers: Infinity War (see YouTube), for example, was obtained with a sophisticated and highly detailed combination of images processed through an AI program.

Other cases demonstrate the endless possibilities offered by AI in providing human operators with a range of choices that would be very laborious to make manually. In the short film The Human Race, a real car was subjected to processing in order to obtain a series of virtual cars, different in terms of models, colors and other details, which could be used in real time by directors and editors to choose the most suitable version depending on each situation. The video below illustrates examples of the possibilities offered by the AI program in the creation of this short film (for further details, read the description of the project by viewing the video on YouTube.)

Designers, and graphic artists in general, can make use of the potential of AI to transform their basic drawing into highly realistic images, even creating virtual worlds depending on the input, i.e. the instructions, provided to the program. Along the same lines, Kirsten Stewart, already an established actress, has also tried her hand at directing, and in particular in the creation of surreal dream images starting from the impressions and sensations inspired by a basic image provided to the program. The short film Come Swim (see YouTube) is one of the results of her research, summarized as follows on YouTube:

“Kristen Stewart’s directorial debut depicts a surrealistic journey through one man’s imagination. Alternating between abstract, artistic representations, and alarmingly realistic scenes, the film takes audiences through the journey of one man’s day coping with anxiety and heartbreak. Come Swim premiered at Sundance Film Festival 2017 and Cannes Film Festival 2017.”

3.3. The “rejuvenation” of actors

One of the simplest ways of demonstrating the effectiveness of AI is certainly to illustrate how the faces of actors can be “rejuvenated” through appropriate data processing processes which, using a huge amount of data from previous films, are able to “retouch” the current images, obtaining a “rejuvenating” effect that does not alter the basic expressions and specific features of the actors’ physiognomy. These techniques, called deepfake (from deep, i.e. “in-depth processing” + fake, i.e. “fiction, counterfeit”) have recently been used in high-budget productions, such as for Robert De Niro in The Irishman or for Harrison Ford in Indiana Jones and the Dial of Destiny.

Beyond the “mere” rejuvenation of the face of an actor/actress, the deepfake technique can be used for more ambitious projects, such as to reuse in new situations the faces of people who are no longer available for work or who have even passed away. Naturally this generates important implications, starting from ethical ones, as we will see in the Conclusion of this Dossier.

3.4. Other uses of AI

The uses of AI tools are almost endless, and perhaps, at this stage of development, largely still to be explored and discovered. We can mention, just to give a few examples, the production of soundtracks that are suitable for the type of film or sequence for which they are meant to be used; the management of important aspects of filming or editing, such as balancing choices of light, colours, filters and sounds; and all applications in which the ability of AI to understand texts, i.e. natural language, can simplify operations such as translation and dubbing into different languages or, more generally, the synchronization of speech with the actors’/actresses’ lip movements. Finally, the possibility of virtually managing filming, without material elements, can help to enormously reduce the risks associated with dangerous situations, both for the cast and for the crew. All these applications of AI can obviously lead to enormous logistical and organizational advantages, with important economic and financial consequences.

4. Conclusion: critical issues in AI use

As we have already mentioned, the use of AI in tasks which until now were (almost) the exclusive prerogative of the human element opens up the central question, which has been present since the technologies made their massive entry into the production of goods and services, of man-machine interaction, i.e. of the possibilities, but also of the risks, associated with the replacement of man with (more or less sophisticated and “intelligent”) machines. The dangers that could derive from the ability of AI to “learn”, and therefore to autonomously acquire new knowledge and skills, have already been pointed out, and this raises disturbing questions about the type of control that the human agent will have to continue to exercise to avoid an, even if for the moment only theoretical, ability of AI to replace man himself.

Ethical issues include a variety of factors, among which it is important to mention first of all the possible violations of copyright, when AI, by capturing an enormous quantity of textual and audiovisual data, can, even in an unintentional way, freely use the product of the creativity of many professionals in the sector. The risk of plagiarism is always present, and raises ethical but also economic questions about the legality of many operations carried out “automatically”. These risks also concern, in addition to the products of human creativity, the physical persons themselves, as we know that it is already possible today, not only to “rejuvenate”, as we have seen, the faces of actors and actresses, but also to reuse the bodies of people to create new films without the physical presence of the people themselves (or even simply their voices). It is understandable how complex issues come into play, such as the very identity of individuals, their personal and artistic integrity and their possible “duplication” without their approval (not to mention the huge archive of actors and actresses of the past that could be used in new productions).

As with any other production sector, even in the film industry the impact of new technologies has direct and significant consequences both for the number of jobs that could be lost and for the type of jobs that require new skills. These are economic, but also social and cultural changes, of enormous importance, for which the need now arises for the (re)training of workers for new tasks, which in part are not even foreseeable as the speed of the progress of technologies is often higher than forecasting and therefore adaptation capabilities. The strike of Hollywood screenwriters, and then also of actors and actresses, in 2023 had as its crucial motivation the dangers inherent in “manipulating” cultural products, exploiting for example the ability of AI to write or rewrite screenplays without writers, or to replace actors/actresses with their virtual equivalents. The strike, which was long lasting, had a devastating impact on film production, and ended with at least a partial and temporary recognition of trade union demands and of the economic rights connected to the professions in the sector – even if the evolution of technologies will require constant monitoring and adjustment over time of the ethical and economic issues already clearly identified.

Another risk, less evident but not to be underestimated, concerns the possible standardization of the results produced by AI tools: as we have seen, at least for the moment, the films produced in this way are scarcely original, and indicate in an unequivocal way that AI is still far from producing, for example, truly original plots, and above all, plots rich in characters and emotions set in significant contexts. For the moment, the exclusive use of AI does not lead to results of great creativity and originality, at least from a narrative point of view, which only confirms how irreplaceable the human mind is in modulating stories, characters, motivations and emotions. (Of course, if AI products of little originality and creativity were placed on the market without prior evaluation, the market itself would reject them…)

To close with some more positive considerations, we have seen on several occasions how at the moment the work of “machines” can productively complement human work, in a sort of man-machine cooperation which, in addition to producing qualitatively excellent results, has a huge impact on the efficiency of the production system, e.g. by freeing man from the laborious and tiring management of tasks that technology can carry out more quickly and efficiently, and thus freeing human energies that can be spent on tasks of greater creativity and originality.

Finally, we must not overlook what we could define as the “democratisation” of film production: new technologies, which are not always necessarily very expensive, can also offer small companies and independent artists a series of tools, which until recently were reserved to big production companies, with which to create projects that do not require large budgets and which can provide opportunities for creative and non-standardized work.