Digital art becomes increasingly expressive and humanized. With the emergence and development of computational aesthetics, advanced artificial intelligence technology could help to generate interesting and unique art works.
Levels of automation complexity
Machine intelligence is the key to computer-generated abstract paintings. We may classify computer-generated abstract paintings into four levels based on their computational complexity rather than their visual complexity [47].
Level 1 needs full human participation using an existing painting software or platform. First, software producer prepares various visual components either generated manually or automatically. Users can select visual components or draw them using the digital brush provided by the software. Of course, they can change visual attributes as needed.
The best representative of Level 2 is fractal art, originated in late 1980s [48]. Fractals require users to provide various attributes, styles and mathematical formulas as inputs. Then a programmed computer can generate results automatically. In other words, at Level 2, results are usually generated based on mathematical formulas parameterized with certain degrees of randomness. The next section will discuss fractal art further.
Methods at Level 3 are often heuristics-based using knowledge-based machine intelligence. There are two general approaches in producing abstract paintings at this level: generative and transformational. Using the generative method, one encodes artists’ styles into computational rules or algorithms. One of the pioneering works by Noll [49] makes a subjective comparison of Mondrian’s “Composition with Lines” with computer-generated images. On the other hand, a transformational method attempts to transform digital images into abstract paintings using image processing techniques. For example, a transformational method can mimic brush strokes or textures and apply them on an input image to transform it into an abstract picture [50]. The best representative of transformational methods is the so-called non-photorealistic rendering [51], which is out of the scope for this paper.
Level 4 is an AI-powered and promising direction for approaches in generating highly creative artistic and design forms. For instance, such an approach detects specific styles from existing paintings and give an objective aesthetic evaluation automatically, or the results will be adaptive to audiences’ emotional and cultural background. The current advances in deep learning and artificial intelligence have created tremendous opportunities for breakthroughs at this level.
Fractal art
Fractal geometry, coined by mathematician Benoit Mandelbrot (1924–2010) in 1975, studies the properties and behavior of fractals and describes many situations that cannot be explained easily by classical geometry. Fractals have been applied to computer-generated art and used to model weather, plants, fluid flow, planetary orbits, music, etc. Different from traditional art, fractal art realizes the unity of math and art aesthetics. A curve is the simplest and classical expression in fractal art, which can be generated recursively or iteratively by a computer program.
We can easily discover four characteristics of fractal art works:
-
a)
Self-similar: enlarge the local part of a geometry object, if the local part is similar to the entire object, we call it self-similar.
-
b)
Infinitely fine: It has fine structure at any small scale.
-
c)
Irregularity: one cannot describe many fractal objects using simple geometric figures.
-
d)
Fractional dimension: generated based on fractal theory, fractional dimension is an index for characterizing fractal patterns or sets by quantifying their complexity.
Singh [52] believes that there is a conversation between him and his computer when he creates his images. In other words, when he talks to his computer, the computer functions would be the translator. He builds on elements library and uses types of string fractals as compositional elements but not the main subject in the image. Figure 13 shows an example of combined result used in the Unfractal series.
Seeley uses fractals as the beginning of his art works [53], making them look less like computer-generated. A number of fractal software may be used to create this type of artworks, such as Fractal Studio, Fractal Explorer, Apophysis, ChaosPro, and XaoS. As shown in Fig. 14, named Yellow Dreamer, Sheeley creates the base image using Fractal Studio, and then transforms it with Filter Forge filters, Topaz Adjust 5, and AKVIS Enhancer.
Modeling abstract painting of well-known styles
According to Arnheim [54], abstract art uses a visual language of shape, form, color and line to create a composition which may exist with a degree of independence from visual references in the world. It is thus clear that a large variety of styles of abstract paintings exist. Accordingly, style analysis is an essential step in generative art, which involves analyzing basic components, background color, component colors and their layout. The components may be independent from each other or dependent with certain rules among them. Geometrical components could be easily modeled by computers while interweaved irregular shapes could be modeled using a layered approach.
Style analysis
Abstract paintings may be divided into two classes, i.e. geometric abstraction and lyrical abstraction. Here we begin with the pioneer of abstract paintings, Wassily Kandisky, to analyze the style of his abstract paintings during his Bauhaus years (1922–1933), such as “composition VIII” (1923), “black circles” (1924), “Yellow Red Blue” (1925), “several circles” (1926), “Hard But Soft” (1927) and “thirteen rectangles” (1930).
We take “Composition VIII” shown in Fig. 15 as an example. According to Kandinsky himself, three primary forms occur frequently: sharp angles in yellow, circles in deeper colors, lines and curves in yellow and deeper colors respectively. He also proposed three pairs of contrast forms:
-
1)
The contrast color pair: yellow vs. blue. For example, a yellow circle is always nested inside a blue circle, or vice versa.
-
2)
A straight line is intersected with a curve line. Several straight lines are intersected with a curve line or individual lines and curves. Some lines are in one color, while others are in segmented colors.
-
3)
Circle(s) with triangle(s). One circle overlaps one triangle, multiple circles overlap one triangle, or several abreast half circles.
Piet Mondrian is another well-known abstract artist, whose style is based on geometric and figurative shapes. While his art forms are drastically different from Kandinsky’s, he took black vertical and horizontal lines as the principal elements and used primary colors red, yellow, blue to fill some of the grids, as modeled in Fig. 16.
Russian artist Kazimir Malevich is the originator of avant-garde movement, and his most famous work “Black Square” in 1913 represents the birth of supremacism. He used different types of basic supremacist elements, such as quads, ovals, crosses, triangles, circles, straight lines and semi-crescent shapes. As noted by Tao et al. [55] in Fig. 17, his works are frequently colored boldly and opaque geometric figures above a white or light colored background. In addition, a large quad determines the orientation of other subsidiary shapes.
Prolific artist Joan Miro developed a unique style in 1920s. He arranged isolate and detailed elements in deliberate compositions. During his middle age, his art works were known as organic abstraction, featuring deformed objects as shown in Fig. 18. Xiong and Zhang [56] call these abstract pictorial elements according to their shapes and appearances. It is easy to find that the colors of Miro’s works are always trenchant and bright. He enjoys using a few specific colors, such as red, yellow, blue, black, and white.
Zheng et al. [57] attempt to analyze Jackson Pollock’s style, who is an influential modern American painter. He draws his paintings by dripping and pouring on canvases instead of traditional painting methods, as shown in Fig. 19. His approach is considered revolutionary for creating aesthetics, as analyzed by Taylor et al. [58] for its visual forms characterized by fractals [5]. Carefully analyzed Pollock’s various paintings, Zheng et al. [57] divide Pollock’s dripping style into four independent layers, i.e. background layer, irregular shape layer, line layer and paint drop layer from bottom up. The elements on each layer are positioned randomly.
Rule-based modeling
After analyzing the styles of various types of abstract paintings, researchers use different approaches to generate abstract images that mimic the original artists’ styles. The components of an abstract painting are usually interrelated. In fact, their spatial arrangements on canvas follow certain rules. For instance, in “Composition VIII” by Kandinsky, full circles with contrasting colors are often surrounded by shades with gradual changing colors and grid forms are always filled with interleaving colors.
Rule-based approach usually follows five steps:
-
Step 1: Choose a specific style for automatic generation of the styled images;
-
Step 2: Generate the background;
-
Step 3: Decide the composition and prepare basic components;
-
Step 4: Position the components based on the designed composition following the analytical rules;
-
Step 5: Add texture and decoration, such as worn signs or pepper noise, if necessary.
Zhang and Yu [59] select four abstract paintings of Kandinsky from his Bauhaus period, including “Composition VIII”, “Black and Violet”, “On White II”, and “Several Circles”, to generate the artist’s style of images automatically. Based on their analysis of the paintings and reading of Kandinsky’s abstract art theories, they summarize a set of rules, for example, thin vertical and horizontal lines build the foundations and intersected by angled lines; dark boundaries are filled with light color; red and black always occur together to create a salient effect.
Zhang and Yu [59] parameterize various attributes of the artist’s typical components, such as boundary color, fill color, size, and location. They then use the above analytical rules to color and position the components, while randomizing other attributes. Example abstract images automatically generated using this approach are shown in Fig. 20.
Tao et al. [55] attempt to automatically generate Malevich style of abstract paintings. They first decide the color and decorations of the background, then prepare the basic components with complexities and flexibilities. Different from the generation approach of Zhang and Yu for Kandinsky style [59], they define a bounding box for each component to avoid overlaps among components, and evenly distribute components on canvas. Figure 21 gives three computer-generated results for “Mixed Shape Style”.
Layered approach
With non-geometrical styles, one could observe the artist’s painting process and follow the process with layers of structures and components.
A typical example is Pollock’s drip style that is quite different from those of Kandinsky and Malevich. It is difficult or even impossible to come up with rules or observe regular patterns. Based on careful analysis, Zheng et al. [57] divide the structures of Pollock’s drip paintings into four independent layers, including background layer, irregular layer, line layer and paint drop layer from bottom up as shown in Fig. 22. The background layer covers the entire canvas and sets the fundamental tone of each painting. The irregular shape layer includes ellipses and polygons of random sizes. The line layer is composed of curve lines of varied lengths and widths. The top layer has all the paint drops of varied sizes. Paint drops are filled in different colors and randomly positioned on canvas. The generation order is bottom up as illustrated in Fig. 22.
Also using a layered approach, Xiong and Zhang [56] propose a process modeling approach to generating Miro’s style of abstract paintings, in the following steps:
Figure 23 shows an example of computer modeled image of “Ciphers and Constellations in Love with a Woman” and an example of generated “Poetess”, both of Miro’s well-known “Constellation” series. Of course, one could obtain varied and restructured versions of the same style by resetting or randomizing different parameters and attributes.
In summary, using the aforementioned generative methods, it is entirely feasibly that more diversified, personalized and innovative images could be generated as desired.
Neural nets approaches
To simulate human aesthetics in depth, Gatys et al. [60, 61] proposed an image transformation approach, using a Deep Neural Network (DNN) approach. Briefly, DNN is a network constructed by layers of many small computational units. In each layer, the units are considered image filters which extract certain features from the input image. A DNN processes the visual information in a feed-forward manner hierarchically. Then, the output of the network is a feature map. Such an approach captures the texture information and obtains a multi-scale representation of the input image. Figure 24 shows an example that combines the content of a photo of Andreas Praefcke in the style of painting “The Starry Night” by Vincent van Gogh (1889).