Skip to content

From Prompts to Pipelines

Artists Building Their Own AI Systems

Image Credit: Teratome, Terrance Broad

Generative AI images are typically produced through prompts—the short text instructions used to guide models such as Stable Diffusion. But many artists working with machine learning are operating at a different level of the technology. Rather than focusing on prompts alone, they design the datasets, training processes, and computational pipelines that shape how generative systems produce images. The release of open diffusion models accelerated this shift. Stable Diffusion, introduced in 2022 by Stability AI and collaborators, made it possible to run high-quality generative image models locally rather than through closed platforms. Because the model weights are publicly available, artists can download, modify, and retrain the system using their own image datasets. They can then integrate those models into custom software environments and generation workflows.

As a result, generative AI artists increasingly resemble software developers and research engineers. Studio practice now often includes dataset construction, model training, parameter experimentation, and code development. In this context, the artwork is not simply the image produced by the model—it is the system that generates it.

Image Credit: AI? We hardly knew each other, Helena Sarin

Building Diffusion Workflows Instead of Single Images
Active in the machine learning art community since the late 2010s, Helena Sarin approaches neural networks as creative instruments that can be trained, modified, and integrated into custom production workflows. Her practice often involves assembling curated image datasets, training generative models, and experimenting with inference pipelines that shape how images are produced. Earlier projects used generative adversarial networks (GANs) trained on everyday visual subjects such as food, household objects, and hand-drawn sketches. By controlling the structure of the dataset and the training process, Sarin investigates how neural networks transform familiar visual categories into stylized or abstract representations.

Image Credit: We're not like the others here, my mischievous dear, Helena Sarin

Rather than relying solely on text prompts, Sarin’s workflow frequently involves modifying models and experimenting with different generation parameters to explore how neural networks interpret visual information. This approach treats the generative model not simply as a tool for producing images but as a system whose behavior can be shaped through training data and software configuration. Sarin regularly shares experiments and technical insights through open platforms such as GitHub and social media, contributing to a broader ecosystem of artists working directly with machine learning systems. The practice reflects a growing tendency among generative AI artists to design the computational pipelines behind image generation rather than focusing only on the outputs.

German artist Mario Klingemann represents an earlier phase of this system-oriented approach to machine learning art. Beginning in the mid-2010s, Klingemann trained generative adversarial networks (GANs) on large image datasets, experimenting with how neural networks reinterpret visual culture. Klingemann’s experiments demonstrated how artists could treat neural networks themselves as creative materials. Rather than simply generating images with existing tools, he trained models and explored how generative systems transform visual patterns contained within large image collections. Although GAN-based systems differ technically from diffusion models, the artistic strategy remains closely related. The artist constructs a training environment, defines the dataset, and observes how the model learns visual structures from those inputs.

Image Credit: Memories of Passersby I, Mario Klingemann

Projects such as Memories of Passersby I (2018), a machine learning installation that continuously generates synthetic portraits, demonstrate how generative systems can function as autonomous image-making processes. In this approach, the behavior of the system itself becomes a central artistic outcome.

Dataset Construction as Artistic Practice
The most consequential artistic decision occurs long before image generation begins. It happens during dataset construction. Argentine artist Sofia Crespo, working both independently and through the collective Entangled Others, has developed a machine learning art practice centered on biological imagery. Her work explores how neural networks interpret patterns found in natural forms such as coral structures, microscopic organisms, and plant morphology. Rather than relying exclusively on pre-trained models, Crespo often assembles custom datasets derived from biological imagery and scientific references. These datasets guide how machine learning models learn visual structures.

Image Credit: Artificial Natural History, Sofia Crespo

In several projects, Crespo explores how generative systems can synthesize organisms that appear biologically plausible while remaining entirely artificial. The results often resemble unfamiliar species—creatures that seem drawn from evolutionary processes but exist only within the representational logic of neural networks. In this approach, the dataset functions as a form of artistic material. The images used for training determine what the model can learn, and therefore what kinds of forms it can generate. Dataset curation becomes a central component of the artistic process. This method has become increasingly common in generative AI art, where artists treat training data not as neutral input but as a medium shaped through selection, filtering, and categorization.

Image Credit: Artificial Natural History, Sofia Crespo

Neural Networks as Research Instruments
British artist and researcher Terence Broad approaches generative AI through a slightly different lens. His work often treats neural networks as tools for investigating how machine learning systems interpret visual culture.
Broad completed a doctoral project at Goldsmiths, University of London that examined machine learning models as analytical instruments within media practice. One of his widely discussed works, Autoencoding Blade Runner, involved training an autoencoder neural network on the entire 1982 film Blade Runner. After learning a compressed representation of the film, the system attempted to reconstruct each frame.

Image Credit: Teratome, Terrance Broad

The resulting video reveals how neural networks abstract visual information. Faces blur into geometric shapes, environments dissolve into color fields, and cinematic details collapse into simplified forms. The project functions both as an artwork and as a study of how machine learning systems perceive images. Broad’s later work has engaged with generative models and related machine learning techniques, examining the ways these systems transform visual media during training and inference. By building experimental training pipelines and observing their outputs, he investigates the assumptions embedded within computational vision systems.
This approach highlights another dimension of machine learning art practice. Generative models are not only image-making tools. They can also operate as research instruments that reveal how artificial intelligence systems encode and reconstruct visual information.

Image Credit: (un)stable equilibrium, Terrance Broad

Across these practices, a broader shift becomes visible. Generative AI artists are no longer defined primarily by how they prompt models. Instead, they design the environments in which models operate. Datasets, training pipelines, and inference workflows form the core of the studio. Images become traces of larger computational systems. As open-source tools continue to expand access to diffusion models and machine learning frameworks, the role of artists as system builders is likely to grow. In this emerging landscape, creative practice increasingly involves designing the mechanisms that generate images rather than producing those images directly.

Comments

Latest