About
This two-day symposium brings together scholars and artists with experience of working at the intersection of disciplines such as digital humanities, digital art history, cultural and media studies, digital visual studies, deep learning and computer vision.
After the initial “hype” associated with the acronym “AI” in various contexts, the reality of conducting interdisciplinary research projects revealed many existing challenges such as cross-domain knowledge gaps, terminological misunderstandings, the AI “black box” problem, lack of transparency and various data-driven biases. In that context, the dialogue between disciplines is becoming a crucial mechanism to address the various emerging challenges of interdisciplinary research. While “interdisciplinarity” is often used as an appealing buzzword, the reality of truly interdisciplinary research is very challenging.
The focus of this symposium is not on showcasing project results, but on using research projects as starting points for reflecting on the various challenges and opportunities of integrating AI methods in the study of art and culture. The symposium aims to bring together diverse perspectives and encourage cross-disciplinary discussions on the globally relevant topic of AI and its far-reaching implications on the future of science, art and culture.
This symposium is organized as part of the “From Hype to Reality: Artificial Intelligence in the Study of Art and Culture” project, which is funded by the UZH Global Strategy and Partnerships Funding Scheme and aims to strengthen the collaboration between UZH (Digital Visual Studies/Dr. Eva Cetinić) and the University of Cambridge (Cambridge Digital Humanities/Dr. Leonardo Impett), with the goal of facilitating AI literacy in the arts and humanities. Organization of this event is supported through UZH Global Strategy and Partnership Funding scheme and the Center for Digital Visual Studies.
Program
Thursday 20.4.2023.
- 9:45 - 10:00 Opening introduction + Eva Cetinic
Eva Cetinić is postdoctoral researcher at the Digital Society Initiative (DSI), at the University of Zurich, where she is conducting her research project "From Text to Image with AI: How Multimodal Deep Learning Impacts Art and Culture". Her scientific interests revolve around the challenges of interdisciplinary research at the intersection of deep learning, explainable AI and digital arts and humanities, with a particular focus on how multimodal foundation models encode various socio-cultural patterns, as well as impact artistic and cultural production and appreciation.
- 10:00 - 11:10 Lecture session I:
- 11:10 - 11:20 Mini-break
- 11:20 - 12:30 Lecture session II:
- 12:30 - 13:30 Lunch break
- 13:30 - 15:00 Presentation session I:
- 15:00 - 15:15 Coffee break
- 15:15 - 16:15 Presentation session II:
- 16:15 - 17:00 Wrap-up discussion
With the objective of contributing to the epistemological turn in the field of digital art history and cultural heritage studies, in a recent paper I have introduced the idea of a techno-concept, which is defined as a co-production between the machine rationale and the human thought/imagination. Mathematical concepts of n-dimensional, vector and latent spaces would constitute examples of techno-concepts that can be reappropriated and reworked for cultural analysis and interpretation. Within this framework of discussion, this presentation will argue that the computational operations and the corresponding information transformation processes that take place in the context of latent spaces, especially multimodal latent spaces, can be understood as a sort of transductive phenomena. Therefore, the concept of transduction, largely used in different scientific fields, becomes a potential theoretical category to interpret the generation of cultural productions in the era of AI.
Nuria Rodríguez-Ortega holds a PhD in Art History, Specialist degree in Digital Humanities and master degree in Cognitive Sciences (UMA). She is Full Professor of Art History and the Head of the Art History Department of the University of Málaga, where she teaches courses in digital humanities, digital cultural heritage and digital art history. She is the founder and director of the iArtHis_Lab Research Group (iarthislab.eu), an international laboratory focused on the study of art history and artistic culture from digital, computational and techno-critical perspectives. She is also the founder and coordinator of ReaArte.Dix (International Network of Digital Studies on Artistic Culture). In June 2021, she was appointed director of the Telefónica-UMA Chair (5G. New Generation Networks and Information Technologies). Since 2017 to 2021, she was the President of the Sociedad Internacional de Humanidades Digitales Hispánicas (HDH). In October 2021, she was named Honorary President of the HDH. She is a member of the Academia Europaea in the Musicology and Art History section. Since 2016, she has been the academic director of the Summer School on Digital Art History (DAHSS). Her research addresses the convergence between computational languages, digital media and art culture, with special emphasis on the application of data analysis and visualization for the study of complex cultural systems, natural language processing for the analysis of art-theoretical texts, computer vision strategies for the production of new visual epistemologies, and the exploration of alternative narratives through immersive technologies.
This talk introduces my current project, for a new history of machine visuality. The stakes are multiple. In the spirit of histories of visuality (e.g. Jonathan Crary), the recent history of electro-optical media (and specifically machine vision) might tell us something about the dominant modes of thinking about vision over the last century or so. We might also want to learn something about the scopic regimes of machine vision systems because of their use in surveillance, automation, scientific research etc. More broadly, however, I want to argue that discourses and practices of visuality (and thus a set of only partially explicit theories about seeing) have been absolutely central to the invention and development of neural networks, and thus to contemporary AI more broadly (including the superficially non-visual, from chat-bots to audio systems). Where might look for evidence of machine visuality? So far, we have largely focused on “training sets” from the past decade or two - following an anthropocentric dichotomy between nature (our common neural wiring) and nurture (the individual visual culture to which we are exposed). This already uncomfortable distinction completely collapses in the case of machine vision, where the wiring is itself a socially and historically embedded technology. I will attempt to partially redress this imbalance by focusing primarily on the wiring. I’ll investigate some basic building-blocks of today’s neural architectures, largely developed from the 1960s to the 1980s.
Dr Leonardo Impett is a University Assistant Professor in Digital Humanities and convenor of the MPhil in Digital Humanities. He was previously Assistant Professor of Computer Science at Durham University. Leonardo has a background in information engineering and machine learning, having worked or studied at the Cambridge Machine Learning Lab, the Cambridge Computer Lab’s Rainbow Group, and Microsoft Research Cairo. His Ph.D., with Sabine Süsstrunk and Franco Moretti at EPFL, was on the use of computer vision for the “distant reading” of the history of art. In 2018 Leonardo was a DH fellow at Villa I Tatti – the Harvard University Center for Italian Renaissance Studies, from 2018-2020 he was Scientific Assistant, then DH Scientist, at the Bibliotheca Hertziana – Max Planck Institute for Art History in Rome.
Over the past decade, the question of identifying and replicating image style has been a major facet of artificial intelligence research, and art image data has been central to much of this research. More recently, text-to-image generation tools, such as DALL-E, Midjourney, and Stable Diffusion, have made the replications of artistic style central to their operation. For example, DALL-E’s opening page suggests the sample prompt, “An Impressionist oil painting of sunflowers in a purple vase.” The art datasets that train such systems, however, typically align with the traditional canon of western art and thus reflect a largely western-centric notion of art history. Over the past fifty years, art historians have questioned this canon both in terms of which artworks are included and the way they are labeled and framed. In art history, style is a highly inconsistent category and the terminology used is understood to be historically situated. In other words, style terms are dependent on context, reflecting the differing purposes of art historians or critics at different points in history. When style terms become data and enter into automated processes through machine learning, these nuances are often lost. This talk addresses the ways that contemporary artificial intelligence methods reify the western concept of style and what that means for the study of visual culture.
Amanda Wasielewski is an Associate Senior Lecturer of Digital Humanities and Associate Professor of Art History at Uppsala University. Her writing and research investigate the use of digital technology in relation to art/visual culture and spatial practice. Her recent focus has been on the use of artificial intelligence techniques for the analysis and creation of art and other visual media. She has a background as a practicing artist, which has informed much of her research. She has exhibited her videos and installations internationally and was a resident artist at De Ateliers in Amsterdam. She holds an MA in Fine Art (Media) from the Slade School of Fine Art at University College London and received her MPhil and PhD in Art History from the Graduate Center at the City University of New York. She is the author of three monographs: Made in Brooklyn: Artists, Hipsters, Makers, Gentrifiers (Zero, 2018), From City Space to Cyberspace: Art, Squatting, and Internet Culture in the Netherlands (Amsterdam University Press, 2021), and Computational Formalism: Art History and Machine Learning (MIT Press, 2023). Her most recently published articles address the historiography of digital humanities/digital art history and the theory and practice of creating virtual environments.
The current consensus in the digital humanities is that AI models are opaque but useful. In the visual digital humanities, for instance, multimodal approaches like CLIP increasingly facilitate previously metadata-dependent tasks like image retrieval. At the same time, the move to always larger, and often proprietary, pre-trained systems amplifies a long-standing lack of theoretical reflection in the digital humanities. If large visual models like CLIP are indeed models of visual culture, how is (visual) culture actually “modeled”, that is, represented functionally? What, in other words, are large visual models models of? In the talk, I will attempt to answer this question by discussing common strands in current (technical and critical) research. More specifically, I will present results from a recent study on the representation of history in large visual models, which exposes several significant limitations in the “speculative” abilities of contemporary generative models in particular.
Fabian Offert (PhD, UCSB 2020) is tenure-track Assistant Professor for the History and Theory of the Digital Humanities at the University of California, Santa Barbara. His research and teaching focuses on the visual digital humanities, with a special interest in the epistemology and aesthetics of computer vision and machine learning. He is principal investigator of the international research project “AI Forensics” (2022-25), funded by the VW foundation.
Computer Vision (CV) models have improved considerably in recent years, reaching all areas of society with open access engines such as Dall-E or its mini version. However, the euphemism Digital Art History (DAH) has sometimes been used as a synonym for Computer Vision tool. This is a catch-all concept that certainly borrows from art history the traditional models, those on which the recognition of objects, shapes or figures and the identification of the subjects depicted were based. However, this self-proclaimed DAH current fails to take into account the multidisciplinarity from which art history studies have been drawing since the second half of the last century: social art history, psychoanalysis, anthropology and semiotics, Marxism, feminism and gender studies, race studies, identity politics, post-colonialism, structuralism and post-structuralism, as well as the more recent ecopolitical turn. CV scholars strive for analyses that relate works of art to similar symbols or styles. Thus crossing semiotic forms of expression and content with enunciation and visual representation. But instead of reflecting theoretically and methodologically on the functioning of these signifying sets, they are systematised without taking into account the practices in which they are included and the symbolic and self-reflexive operations of representation, which always refers to a subject. The pre-iconographic analyses of iconological analysis, –which concentrate on the visual elements in order to extrapolate themes and symbols– once they have proved obsolete for trying to understand the processes of operability and figurability of images, seem to be making a stronger comeback than ever, reclaiming the original figures of this discipline such as Wolfflin, Panofsky or Warburg. This return to the old, which fixes and freezes a certain method, hitherto dynamic, perpetuates a technique that reifies the very reflection of the theory and history of art behind a certain objective scientism of the machine. But the main question we must ask ourselves is whether this metonymy –DAH– of a series of non-theoretical practices (or practices that repress and overshadow theory) has direct consequences on discourses and practices. In this paper we will try to see which ones by presenting two case studies where the values and meaning behind artwork can easily be misread by a purely visual reading.
Nicolás Marín Bayona is a PhD student at the University of Luxembourg in co-tutorship with the Catholic University of Louvain. He joined the Augmented Artwork Analysis project as a PhD student in visual studies. This project aims to produce a digital tool for the interpretation of artistic images within the museum assisted by Artificial Intelligence. After training as an art historian at the University of Zaragoza (2010-2015), He continued his studies in cinema and contemporary media at the Université Libre de Bruxelles (2013-2014) and at the École des Hautes Études en Sciences Sociales in Paris (2015-2017). His main lines of research are the Histories and Theories of the Arts, with a particular attention to the studies of the so-called analytical and critical iconology and the limits of figurability and representation in artworks.
In this presentation, I would like to question the field of Digital Art History (DAH) –and especially DAH using image analysis algorithms–, through two questions. First: what does it mean to do research in DAH? i.e. what is the epistemology of this field: how is knowledge constructed, through which research questions and methods, with which relations to a past historiography and to a specific corpus, and what is considered a valid contribution? Here, the question of the sources used will be essential: is the search for primary sources –which essentially constitutes the art historical research–, and their historical contextualisation (in a system of causalities), replaced by various features extractions –determined by algorithmic experiments– and the search for patterns of interest (in a system of correlations)? What are the potential discoveries of such an apparent solipsistic analysis, shaping structures and searching for knowledge (e.g. formal proximities, historical patterns) without introducing any new historical evidence? Are we witnessing a new tension between formalism and historicity? What is the space for critics, nuances, doubts, relativity, ambiguity –which define any postmodern humanistic approach– in such an epistemology? To answer this first question, I will conduct a short critical literature review of the DAH field. My second question will be: do these algorithm-oriented methods unlock fundamentally new ways of thinking about a corpus –or do they constitute the continuity/revival of certain past theories and methodologies? In particular, are the concepts inherent to the use of vector spaces and to their visualisation –e.g. continuity, interpolation, transformation, proximity– new to art history? Here, the links between DAH and formalism/semiotics/structuralism will be evoked, and the past interdisciplinary exchanges between sciences –e.g. natural sciences, darwinism– and art history will be pointed out, while the essential originalities of these new computational methodologies will be underlined. Finally, I will briefly discuss my own concrete research project, where I try to study a specific –ornemental– corpus, in relation to a precise –formalist– past historiography, while remaining essentially critical and self-reflective concerning the epistemological limits of the methods used.
Tristan Dot is a first year PhD candidate in digital art history at the University of Cambridge. Coming from a double background in machine learning and in art history, he is interested in the links between formalist theories –in art history– and computer vision representations.
Defining a project in digital art history is exciting and full of promises: from the possibility to unveil artistic trends over a large period of time to the discovery of the source of inspiration to a recurring pattern, via the thrill of training a machine that sees what has been unseen by years of art historical practice. Unfortunately, the reality of such interdisciplinary projects is often far from these expectations that large digital corpuses and computational powers seem to offer. Not only the voice of the art historian will quickly object to the fundamental research questions and the misunderstanding of the engineer who is juggling with exotic concepts, but the whole practical aspect is destined to affect the enthusiasm of the first moments. Machine learning models are trained on real images and do not meet the same accuracies on paintings, and the perfect corpus of paintings does not exist, yet. Not all paintings have been digitized, not all collections have implemented the requirements for computational approaches, and these digital corpuses have not been annotated for the purpose of machine learning training. Through a project on the computational and historical understanding of hands in Early Modern time, the talk aims to reflect on the contrast between the first research questions and the readjustments performed along the use of AI and art historical material. In a context at the crossroads of two very diverging and demanding fields, the talk will also question the possibility to really find an in-between among art history and computer science.
Valentine Bernasconi is currently doing her PhD thesis in the Digital Visual Studies research group at the University of Zurich, in collaboration with the Max-Planck Institute for Art History in Rome. Her research interest focuses on the historical and computational analysis of the depicted hand in a corpus of paintings from the Italian early modern time. Valentine graduated from the Swiss Federal Institute of Technology Lausanne (EPFL), with a master of engineering in digital humanities in 2020, after a first master of science in multimedia design and 3D technologies from Brunel University London, United Kingdom. She also holds a bachelor degree in computer science with a minor in art history from the University of Fribourg, Switzerland.
Depiction of space by means of linear perspective has been widely discussed, whereas pictorial light has received scant attention. Art historians have preferred to emphasize perspective as the major Renaissance achievement rather than light because perspective is more easily defined. Due to its abstract nature, and unlike perspective, accident and intention are not easy to distinguish when describing light with language1 . Moreover, visual psychologists have shown that most of us are not particularly good at judging illumination features in a photograph (nor paintings, by extension). The project hypothesizes that computational language may help to construct a renewed epistemology to further analyze and name light features in early modern painting. Dialectical light strives to confront concepts and metrics; the discursive with the computational; the established with the unknown. This confrontation is the source of contradictions and synthesis that allow deep understanding of the (digital) art historical inquiries to address: What are the differences in the study of early modern light between established art historical theories and contemporary computer techniques?
Pepe Ballesteros Zapata is a Ph.D. fellow at Digital Visual Studies (University of Zurich). His background is in telecommunications engineering. He received an M.Sc. degree in ’Signal Processing and Machine Learning for Big Data’ from ETSIT-UPM. He acquired experience as a software engineer working at Cirrus Logic International S.L., where he performed research on voice bio-metrics with AI technologies for his final bachelor thesis, obtaining the maximum qualification. He has experience teaching AI-related topics to non-technical audiences at “Fundación Univ. Empresa (FUE)”, His master thesis proposal was selected winner from the national Spanish contest launched by RTVE called “Impulsa Visión Ayudas a la Investigación III”. He developed a text generation system to automatically write weather forecasts. He was selected to participate at the Mobile World Congress 2021, where he performed TV and radio interviews to talk about his research. At the moment, he is interested in ways in which computers can help describe pictorial light in figurative paintings.
The'RePAIR' project aims to physically reconstruct ancient artifacts such as vases, amphorae, and frescoes that have been found as large collections of fragments in excavation sites. More specifically, robotic hand technologies for handling fragile objects will be used for automatising both the digitisation and the assembling of fractured artefacts. As an initial stage, prior to physically reassembling the fragments with a robot arm, RePAIR project aims to use AI approaches to construct a representation of the spatial organization of all fragments, which should result in a visually coherent archaeological artifact. However, this phase presents numerous challenges that the system must cope with. These include dealing with a large number of fragments that may not belong to a single archaeological artifact, extracting distinctive pictorial features from the deteriorated intact surfaces of fragments, finding the correct neighbors of fragments due to eroded fractured surfaces which can impede the usage of geometrical features, and solving the puzzle even in the case of missing fragments, and most importantly without a visual reference of the original archaeological artifact. In this presentation, the RePAIR project, and the challenges that the RePAIR-solver to be developed needs to overcome will be introduced through an iconic case study from the UNESCO World Heritage site of Pompeii. Specifically, the talk will focus on fractured frescoes from 'The House of Painters at Work' in Pompeii while addressing the aforementioned challenges and discussing potential solutions.
Sinem Aslan received her Ph.D. in Computer Science from Ege University, Turkey, in collaboration with the Electrical and Electronics Engineering Department of Boğaziçi University, Turkey, in 2016. Following her doctorate, she held postdoctoral researcher positions at IVL of the University of Milano-Bicocca, ECLT and DAIS of Ca’ Foscari University of Venice, Italy, and Ege University, Turkey. During this period, she also worked with the EuroMediterranean Center on Climate Change (CMCC) for the VENEZIA 2021 project, focused on water quality assessment, and collaborated with Luc Steels on the semantic interpretation of figurative paintings for the MUHAI project. She is currently an Assistant Professor at the Department of Environmental Sciences, Informatics and Statistics at Ca’ Foscari University of Venice, Italy, where she is involved in the “RePAIR” project, funded by the Horizon 2020 research and innovation program of the European Union. Her recent research has focused on computer vision and machine learning, applied to fine arts and cultural heritage analysis.
This contribution discusses methods and prospects of a distant viewing of architecture. As a testing ground, we suggest formal analysis of the composition of Venetian facades, echoing a long tradition in post-war Italian architectural scholarship [1-3]. In particular, we choose to operationalise the typologies of composition proposed by Paolo Maretto for Venice’s vernacular domestic housing [4]. Our method transforms a city-wide point cloud into a set of orthographic views of facades, and then reduces (by means of object detection neural networks) these “orthophotos” to geometrical representations where only remains the position of openings (doors and windows). It is this abstract representation that is then compared to Maretto’s grammar. The problems most commonly faced by large-scale digital art history – corpus representativeness, metadata completeness, varied digitisation methods – do not arise here: all 8567 buildings of the city centre are captured in an homogenous process , while our operationalisation experiment is purely formalist and does not require metadata. However, more than the validation of the typologies suggested by Maretto, we advocate that the exception – in other words, facades that escape any pre-established category – would be the main outcome of the experiment: First, by encircling the limits of understanding of our prediction model, ie. by gauging the limits of its language: what has been understood statistically to form an opening in a Venetian facade, and how that is different to an architectural historian’s understanding. Secondly, by recognising “algorithmic failure” as a magnifying glass onto marginal, significant, cases [5]. In other words, by considering the exception as justification for qualitative – and thus, truly interdisciplinary – close-reading [6].
[1] Muratori, Saverio. (1960). Studi per una operante storia di Venezia. Roma, Istituto Poligrafico dello Stato.
[2] Cannigia, Gianfranco and Maffei, Gian Luigi. (1979). Composizione architettonica e tipologia edilizia, Vol 1. Lettura dell'edilizia di base. Venezia, Marsilio.
[3] Concina, Ennio. (1982). Structure urbaine et fonction des bâtiments du XVIè au XIXè siècle: Une recherche à Venise. Paris, UNESCO.
[4] Maretto, Paolo. (1992). La casa veneziana nella storia della città. Venezia, Marsilio.
[5] Walker Rettberg, Jill. (2022). “Algorithmic failure as a humanities methodology: Machine learning’s mispredictions identify rich cases for qualitative analysis”. In Big Data & Society 9(2).
[6] Ginzburg, Carlo. (1986). “Spie. Radici di un paradigma indiziario”. In Miti, emblemi, spie. Morfologia e storia, Torino, Einaudi, pp. 158-193.
Paul Guhennec is a physicist by training and completed my Master’s degree in 2020 in Great Britain. Quickly after, he joined EPFL’s Digital Humanities Laboratory to complete a PhD on the intersection between digital methods and urban studies, in which he specifically focuses on Venetian domestic (so-called “minor") architecture – which has been a long interest of his – and the methodological implications of large-scale methods.
The formalisation of knowledge in a system is mediated by an adaptation of the information content to the structure of the system. The specifics of such a transformation have been extensively studied by anthropology, semiotics, linguistics, and other disciplines. However, such a process has been less examined with respect to the datification of information. The problem is particularly relevant in a context of increasing quantification of complex and historical information, such as in the digital humanities domain. In this context, reflections on methods and bias are numerous. What appears to be missing is the investigation of the limitations in the formalisation of knowledge given by tools and data infrastructure. The materiality of the format and infrastructure used to document our past strongly mediates the process of datification. The materiality of the grid defines the granularity of the data we use. The cell is the only actionable unit, and its manipulation is limited by the constraint of the format. The subdivision of data into cells requires us to synthesise complex data into single meaningful units equivalent to one or more cells. The formalisation of knowledge in SQL databases follows a similar mediation. Databases enforce relationality within data, forcing us to think in terms of links and connections, but also enforcing the adoption of single schemas, a data regime that we must follow. We are forced to decompose arguments into relationable units, where the recomposition of such units and their position in the base is determined only by the capabilities of the query language. The centralised model, the difficulty in adaptation to novel data and the categorical framing of the documented information are all structures enforced by the relational database mindset. The article will propose a critical overview of the material constraints of data creation in digital humanities/history/art history and how they affect the analyses and reasoning over the data. In particular, we will look at possible solutions offered by other information systems to address the material limitations of the database.
Nicola Carboni is a Postdoctoral research within the Visual Contagions project at the University of Geneva, where he also teach courses on Digital Image and Knowledge Graph. Previously Fellow at the Swiss Art Research Infrastructure - University of Zurich and Digital Humanities Fellow at the Harvard Center for Italian Renaissance Studies. He completed his PhD in Engineering, on the topic of Knowledge Representation and Visual Heritage, at the CNRS & NTUA where he was also previously appointed Marie Curie Fellow. He works on the intersection between knowledge graph, big visual data and cultural interpretation.
Friday 21.4.2023.
- 10:00 - 11:10 Lecture session III:
- 11:10 - 11:20 Mini-break
- 11:20 - 12:30 Lecture session IV:
- 12:30 - 13:30 Lunch break
- 13:30 - 15:00 Presentation session III:
- 15:00 - 15:15 Coffee break
- 15:15 - 16:15 Presentation session IV:
- 16:15 - 17:00 Wrap-up discussion
In the mid-2010s, AI art gained wide recognition by leveraging the affordance of subsymbolic machine learning architectures. Since then, it has diversified into a range of practices that emerge out of, and respond to, the phenomenological realms and social implications of AI science, technology, and industry. The identity of contemporary AI art is shaped and challenged by the technocentrism and homogenization of creative approaches, the expressive cogency, epistemic value, and social impact of the realized artworks, and the broader questions of monetization, legislation, cultural positioning, and educational support. My talk explores some of these issues through the lens of poetic similitudes that manifest as unacknowledged conceptual, thematic, narrative, procedural, and presentational parallels between AI- and referent artworks across the disciplinary and historical spectrum. These incidental reverberations belong to a corpus of intellectual blunders, methodological miscalculations, and ethical slippages whose unforeseen consequences are usually undesired by the artists but always instructive for their audience. Taking the well-informed autonomy of expression and the socially responsible freedom of creative thinking as the tenets of artmaking, my discussion interrelates selected experimental, tactical, and mainstream AI artworks whose similarities are symptomatic of the field and beyond. Poetic overlaps in the arts occasionally happen because of the spontaneous convergence of ideas or the cognitive requirements of production but frequently have less justifiable causes, such as carelessness, indolence, ignorance, arrogance, egoism, narcissism, or vanity. They place artmaking issues firmly within the contexts of human nature and existence and often point to the related cultural ambiguities and societal tensions. Therefore, they provide an invaluable perspective for studying the strengths and deficiencies of AI art and for articulating the critical discussion of art and creativity in general.
Dejan Grba is an artist, researcher, and scholar who explores the cognitive, technical, poetic, and relational aspects of emerging media arts. He has exhibited in the Americas, Europe, Asia, and Australia, and published papers in journals, conference proceedings, and books worldwide. In 2019/2020, Dejan was a Visiting Associate Professor in the School of Art Design and Media at the NTU in Singapore. He has served as a Co-founding Associate Professor with the Digital Art Program of the Interdisciplinary Graduate Center at the University of the Arts in Belgrade since 2005 and as a Founding Chair and Associate Professor with the New Media Department at the Faculty of Fine Arts in Belgrade from 1998 to 2020. He was a Guest Assistant Professor at the Computer Art Program in the College of Visual and Performing Arts at Syracuse University in 2007.
In the mid-nineties, in front of the new diversity of images (from metaphors, paintings, to electronic simulations), Gottfried Boehm, the father of visual studies, promised to consider the image in its “immense variety” and “diffuse omnipresence”. In the studies to understand what kind of imagery they were dealing with, the attachment to objects, works of art and their reproductions, too often came in the way of getting to the ontological root of the question of the image. Thus, with a philosophical model of inquiry, I have looked into psychology, neurology, quantum physics and history to see beyond representation to understand the genealogy of an image, what is it ‘made of’, where does it live and how does it work. To figure out a logic of images that would be non predicative, non discursive, visual studies needed to imply a performative logic, a logic of showing which unfortunately excluded images that did not show, that did not perform: the very images that, invisible, articulate thoughts even before the act of seeing, of recognizing, because these images, before showing us anything, they make us see. Now that the field has been enhanced in the process of general digitalization and crossed over from humanities to computer sciences, that images have completely dematerialized so that the object of study is exclusively dematerialized information (digital data sets), taking into account these invisibilities is essential. Digital technologies have an impact on our abilities to imagine and our motivation to discover and to create. Relying on early scientific accounts and philosophical articles, I would like to address the roles of intuition (Anschauung) and imagination (Vorstellung, Einbildung) in research processes, that is the question of abstraction and concretion in the processes of visual thinking and image making. In the current hype around new AI models, these mechanisms are raising more questions than they provide answers. What don’t (do) we know that the machine does (doesn’t)? What is knowledge? How do we/it produce/s meaning? Understanding? By revisiting a few key philosophical concepts in light of synthetic image production, I would like to raise our common understanding of the multifarious thing that is an image beyond the invisible and foster a collaborative reflection on the ontological nature of the AI image. And perhaps be better prepared to define, and interact with, the tools that temper with what makes us most human – our imagination.
Anne-Laure Oberson (PhD) is a philosopher, photographer and curator. Her research focuses on the quantic nature of images, the impact of electronic images on cognitive processes and phenomena such as synchronicity. She is currently writing on imagination and undertaking a comparative study of the writings of Bernard Stiegler and Vilém Flusser in regards to generative models for image creation. She is leading an art residency program and teaching contemporary philosophy. She has edited and contributed to books, catalogs and articles in the field of philosophy, contemporary art, photography, and art history. Her book I see. Do you? Thinking Seeing was published by Atropos Press. She studied History of Art at the University College of London and Philosophy, Art and Critical Thought at the European Graduate School.
18 months ago I started working on the AI+Art initiative at the ETH AI Center and set off with the mission described on our website: «The AI+Art initiative aims to close the loop from science to technology, design, and art back to science and therefore to engage in the field of Art and Critical Thinking. We aim to take an essential perspective of critical thinking with art that questions cultural behavior, perception and imagination of world in relation to AI. By expanding technological knowledge the AI+Art initiative contributes towards a more holistic and appropriate AI research and development. We have 4 goals in mind for our AI+Art initiative: Critical Thinking + Ethics, Diversify Knowledge + Intelligence, New Questions + Different Perspectives and Inspiration, and Visions and Fictions for/of the Future. Understanding how artists, activists, and thinkers respond to and leverage current achievements in the field of AI opens inspiring, transversal, and -disciplinary perspectives on the ETH AI Center's research fields and areas of impact. For this reason we want to foreground conversation and dialogue in our AI+Art fields of activities participating and shaping in a discourse of critical thinking driven by the current and urgent questions.» Coming from a seemingly totally different context than AI, science and technology, i.e. Dada and contemporary art, I not only learnt a lot about this ecosystem but also found myself in quite a different context, i.e. in the eye of the storm of an exponentially rapid technological progress of generative AI models. In this talk, I will reflect about my explorations and learnings starting from Dada to AI.
Adrian Notz (*1977 in Zurich) is curator of the AI + Art Initiative at the ETH AI Center, currently he is also curator of the Art Encounters Biennial 2023 in Timisoara. From 2020 - 2022 he was curator at the Tichy Ocean Foundation and from 2012-2019 artistic director of Cabaret Voltaire in Zurich. He worked there first as a curatorial assistant (2004-2006) and as co-director (2006 -2012). From 2010 to 2015 he was head of the Department for Fine Arts at the School of Design in St. Gallen. Notz has organized and curated numerous exhibitions, events, conferences, actions and interventions with international artists, activists and thinkers in Cabaret Voltaire as well as internationally around the globe.
We live in a world of user interfaces, in which more and more autonomy is taken away from the user. To facilitate us with appealing and smoothly constructed interfaces artificial intelligence is increasingly guiding us through environments permeated with interconnected technology. But the modus operandi of these advanced algorithms is not only strategically obscured (Zuboff, 2019), it also demands of us to reflect our position in the postdigital framework, we are living in. In my proposed paper, I want to ask the question of “who is serving whom” in an AI enhanced and technologically interconnected environment. Or to be more precise, I want to ask, what kind of implications a concept like “serving” allows us to grasp, when we are dealing with contemporary AI developments. Building on an analysis of Markus Krajewski, I consequently want to ask the question, how "functions of the subaltern" have nowadays been technologically absorbed or distanced in order to organize concise systems of perception. (Krajewski, 2010) I want to ask, how specifically the idea of serving can help us to understand a living situation, in which human and technological impulses are deeply interlocked, or even heteromated. (Ekbia & Nardi, 2017) And to finally specify my thoughts, I want to discuss those questions above in their relevance for the work of the curator. In a last outlook, I hence plan to discuss potentials and problems arising explicitly for this professional field.
Ekbia, H. R., & Nardi, B. A. (2017). Heteromation, and other stories of computing and capitalism. MIT Press. Krajewski, M. (2010). Der Diener: Mediengeschichte einer Figur zwischen König und Klient. S. Fischer. Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontier of power. Profile Books.
Dr. Heiko Schmid is an art historian, curator and author. He currently is employed as a lecturer in the Master Art Education Curatorial Studies at Zurich University of the Arts as well as a lecturer in the Bachelor in Artificial Intelligence and Machine Learning at Luzern University of applied Sciences and Arts. Furthermore, Heiko Schmid is chairman of the commission KiöR (art in public space) of the city of Zurich. Heiko Schmid lives and works in Zurich (Switzerland).
Recent advances in generative machine learning have highlighted how crucial perspectives on the temporality of technological change are to their reception. While deep learning models are known to be the result of decades of steady progress in engineering circles, both enthusiastic and reactionary critics seem to privilege the narrative of an unprecedented ontological rupture, either extolling an imminent revolution or warning against the dissolution of supposedly fixed human institutions of authorship and creativity. A core concern for the humanities should then be the contextualization of perspectives on human agency and creativity as contingent and culturally-specific, as evidenced in the work of the late anthropologist Alfred Gell. This decentering might help ground debates about generative AI and counter excessive optimism or pessimism surrounding it. Classical Greek antiquity provides an interesting backdrop against which to examine these questions, as its conception of reality is both distant and different from ours, but often invoked as the source of much of our philosophical outlook. Concepts and practices such as divination (mantikê) and diffuse prophetic rumors (kledôn) suggest alternatives to the current dichotomy of randomness and deliberate human agency; the idea of ekphrasis posits a correspondence between graphic and textual art not unlike AI prompts; legends surrounding mythic founders of the artistic craft, such as Zeuxis or Butades' daughter, reveal how the Ancients envisioned the author's ability to imitate and remix experienced reality. Lastly, Aristophanes' comedy The Birds, depicting the sudden formation of a city in the clouds by birds and the resulting upheaval of human and divine institutions, satirizes the tug-of-war between revolutionary and normative discourses in a way that obviates a certain anthropomorphic bias of the AI debate. By borrowing from this conceptual toolbox, we may steer discussion towards more appropriate concepts for understanding generative AI. My proposal examines classical Greek antiquity as a case study for contextualizing perspectives on human agency and creativity with relevance for debates on AI and the future of art and culture.
Bokar N’Diaye is a graduate student in anthropology of religions in the University of Geneva. He was featured in videos of the american journal Vox regarding AI image generation in May 2022, and animated the discussion on the Workshop on Machine Learning for Creativity and Design during the NeurIPS conference (37th Conference on Neural Information Processing Systems) of December 2022. Along with other Genevan students in digital humanities, he contributes to the Visual Contagions project, under the direction of Béatrice Joyeux-Prunel.
Being mimesis a central problem for the arts, cultural production, and epistemics, it appears necessary to address it in light of the large vision-language models (e.g. diffusion models, or contrastive learning based like CLIP or ALIGN), especially when applied to image and text generation. In this article, we inquire about the modes in which such models operate and discuss to what extent they can be said to engage in processes of projective imagination. We propose a computational pipeline for investigating the cultural landscape of a city through the eyes of a machine, and for questioning modes of embedding culture in machine learning models. Using Rome as a pivotal case study, we study the visual features and textual properties extracted by these models. More specifically, we feed 360° equirectangular panoramic images into OpenAi's CLIP and Stable Diffusion, and analyze how mainstream culture might be captured and expressed in these models through the outputs. In this machine-triggered urban experiment, we investigate overlaps between history and machinic interpretation and whether relevant temporal correlations can be captured through urban generic images only. Furthermore, we discuss the process in light of early modern theories of imagination, particularly those of Marsilio Ficino and Giordano Bruno, and articulate whether these new models propose or not a new paradigm. As a way of contextualizing this approach within the analysis of the cultural relevance of recent multimodal machine learning models, we consider whether these models can be said to be acts of societal aesthetic appraisal.
Darío Negueruela del Castillo is the scientific coordinator of the Center for Digital Visual Studies (Max Planck Society – University of Zurich) since January 2020. Between 2017 and 2019, he was Head of Research at ALICE lab in EPFL, where he completed his PhD entitled "The City of Extended Emotions" in 2017. Until 2022 he was a founding member of Architecture Land Initiative (cooperative, CH). Previously, he was a founding partner of Ná architectural office (ES). He received an MSc in Architecture from TU Delft (The Netherlands) and a BA from the University of Westminster (UK). He recently organized the "2021 Deep City International Symposium" as well as the "Scaffolds-Open Encounters" in 2018 in Brussels. His research spans architecture, urbanism, affect, and spatial and visual perception with an emphasis on imagination and spatial agency. Among his current projects are "On the Urbanity of Images" and "Multimodality and Digital Apophenia".
Iacopo Neri’s research lies at the intersection of architecture, computer science, and urban planning. Iacopo has been involved in academia since 2015, researching computational design and geospatial analysis. He has been involved in teaching activities at the University of Florence, The Polytechnic University of Milan, and at IAAC – Institute for Advanced Architecture of Catalonia, where he is currently faculty of computational design and part of the Advanced Architectural Group’s Computational Design Research Team. Additionally, Iacopo has been guest faculty for workshops, lecture and invited critic in several international universities among which the AA – Architectural Association, the New York Institute of Technology, the Universidad Torcuato Di Tella in Buenos Aires, and the Ecole des Ponts – ParisTech. In parallel, Iacopo has been practicing as a computational designer in Italy, Kosovo, and Spain as a co-founder of Studio Spatial Entities and as a collaborator within External Reference Architects studio, Barcelona.
Two approaches to ‘the work’ appear to collide in the confluence of AI and the arts. If art works today are decomposed into features, and generated from that same vector space, what kind of concept of the work is at play in these practices? And how does it compare to conceptions of the work in the arts? Today artefacts (e.g. a music piece) are classified by machine learning models that were trained on a certain canon, which may be a general training data set (e.g. classical music) or a distinct set (e.g. work by one artist). Artists may then use those models as part of their process, for example for generating compositions or even audio files. But how exactly is a work read and abstracted into a machine learning model? The process of abstraction may include classifying not only the artefact, but also data about its context and its style. In following Peli Grietzer’s theory of vibe, style can be understood as ‘an abstractum that cannot be separated from its concreta’. The abstraction of the work in feature space appears to presuppose that the work is a stable, concrete thing. But a work is rarely stable. The current regime of machine learning deploys a superficial conception of the work, that is, it is geared towards the surface. It is a visual regime in the technical sense, as it is designed to classify and generate artefacts based on specific features in a multidimensional space which is bound by its number of inputs. To put it differently, to a machine learning model, the work is a flat artefact. It is processed as is, not read as it might be. To spell out how this view of the work differs from contemporary understandings, this talk will draw from philosopher Lydia Goehr’s writings on the history of the musical work, a notion that emerged at the end of the 18th century. The work concept has consequently been challenged during the last century and today can be said to include much more than the artefact at hand, rendering the work fluid and contingent on context. Does AI then deploy an archaic concept of the work or might the two concepts be compatible?
Arif Kornweitz is a PhD candidate at the KIM research group on critical AI at the Karlsruhe University of Arts and Design, where I conduct research about conflict prediction in the humanitarian sector. He is the head of a new MFA, at the Sandberg Instituut in Amsterdam, titled Artificial Times, which invites students to critically engage with music, sound and AI. Arif is also the co-artistic director of Ja Ja Ja Nee Nee Nee, a curatorial platform for contemporary art and critical music that explores radio as space for artistic practice.
Artificial Intelligence (AI) is becoming increasingly integrated into global society – in its diverse and growing range of creative implementations, AI is often referred to as the latest general-purpose technology, and at other times as inflated “practice-based speculation” (Zeilinger 12). Upon closer examination, an observable gap becomes clear between the status quo of AI’s social integration, and the sensationalised way it is depicted in the media. Deep-blue visuals employed to represent ‘AI’ and other emergent technologies have flooded the mediasphere: the glowing imagery goes hand-in-hand with the sensationalised language employed to write about new tech. In liaison with media outlets, massive image platforms, such as search engines and globalised stock photo providers, form sites of fabrication of the vision that permeates our collective imaginary. I propose the operational concept of the Deep Blue Sublime as an aid for interrogating the stronghold of AI’s narrowly mediated public visuality. The term picks out both an aesthetic category, pointing towards the often-blue, tired, and outdated cultural tropes employed in (mostly stock) public imagery of AI, as well as an epistemic orientation; a knowledge relationship formed by the images, which stands close to the algorithmic sublime (Ames), and describes the affective foundations of awe, fear, and sublimity, that these images seek to cement as they collectively signify into, and throughout the mediasphere. Every visual figuration of AI is inherently an artistic gesture, and further propagates a socio-technical imaginary (Jasanoff and Kim), torquing the collective archive of received meanings, common understandings, and future visions surrounding the technology. Extending the normative imperative of studying the "ethics [and] politics of AI images" (Romele 4), I contend that the agency and social responsibility of public image-makers, is becoming increasingly crucial to the study of the mediation of emergent tech concepts such as AI to the public.
Ames, Morgan G. ‘Deconstructing the Algorithmic Sublime’. Big Data & Society 5, no. 1 (June 2018): 205395171877919. Jasanoff, Sheila, and Sang-Hyun Kim. ‘Containing the Atom: Sociotechnical Imaginaries and Nuclear Power in the United States and South Korea’. Minerva 47, no. 2 (June 2009): 119–46. Jasanoff, Sheila, and Sang-Hyun Kim. ‘Dreamscapes of Modernity: Sociotechnical Imaginaries and the Fabrication of Power’. University of Chicago Press, 2015. Romele, Alberto. ‘Images of Artificial Intelligence: A Blind Spot in AI Ethics’. Philosophy & Technology 35, no. 1 (29 January 2022): 4. Zeilinger, Martin. ‘Generative Adversarial Copy Machines’, 2021, 23.
Dominik Vrabič Dežman (SI) is a media scholar and information designer, based in Amsterdam, NL. Following studies at Design Academy Eindhoven and the Royal Art Academy in The Hague, Dominik is a graduate student in the departments of Philosophy and Media Studies at the University of Amsterdam. His practice revolves around the study of digital images used to mediate the concept of Artificial Intelligence to the public, as their social presence ties into wider questions of social epistemology and ethics of image-making, examining the visual ecosystems and socio-technical imaginaries surrounding emergent technologies.
We propose a multi-disciplinary discourse on a rather poorly explored area in the intersection of AI and Art: the indirect impact of AI on artistic production through content moderation algorithms on social media. Such algorithms tend to censor artistic pieces that display nudity, acting as inhibitors of human creativity. The algorithmic censorship of nudity has been studied by several scholars, highlighting the disproportionate impact of such censorship on feminist artists, and exploring the adopted artistic techniques to circumvent it. In recent years, several movements have emerged to publicly denounce the issue: these initiatives are of crucial importance to raise public awareness and to highlight the anthropological and sociological consequences of artistic censorship on social media. AI-based algorithmic content moderation poses several societal challenges. First, such proprietary algorithms are developed and maintained by private companies with clear economic incentives, hence their unprecedented power on defining our culture is exercised without any guarantee that it reflects the interests of society at large. Second, the automated decisions made by such algorithms are not always explainable and transparent, particularly if based on deep learning models. Third, algorithms are not foolproof and might not only make mistakes but also be fooled. Fourth, while historically controversial artistic content could be publicly discussed and debated, today artists have a limited ability to respond to censorship by social media platforms. Given the lack of transparency, it is hard to engage in a public debate if the reasons why certain content is banned are unknown. As a consequence, algorithmic censorship leaves no space for what is “blurred” or “faint”, drawing more defined -and yet invisible- lines between the acceptable and the unacceptable. In such a binary environment, breaking the rules is becoming harder, if not impossible.
Piera Riccio is an ELLIS PhD student in Artificial Intelligence, supervised by Nuria Oliver (ELLIS Alicante) and Thomas Hofmann (ETH Zürich). She holds a bachelor’s degree in Cinema and Media Engineering, a Master’s degree in ICT for Smart Societies, and a Master’s degree in Data Science and Engineering. In 2020, she was a research affiliate at Metalab (at) Harvard. In 2021, she was a research assistant at the Oslo Metropolitan University. In her PhD, she is interested in exploring the cultural, social, and artistic possibilities (and/or implications) of AI, especially focusing on women. In parallel, she is also an art practitioner, experimenting in the intersection between Art and AI. In 2017, she co-founded the collective no:topia. Her works have been exhibited in galleries, museums and festivals around Europe, including the Ars Electronica Center (2021-2022), World AI Cannes Festival (2022), FUTURIUM (2019), Re:Publica (2018), and Phest (2022).
In a lecture delivered in 1883, William Thomson, 1st Baron Kelvin, noted that “when you can measure what you are speaking about, and express it in numbers, you know something about it”. Visual and linguistic outputs generated by machine learning models are evaluated using both human based and automatic assessment. These methods are employed with the aims of detecting machine generated outputs and measuring the quality of results. While quantitative measurements are applied in scientific articles, research has placed into question the reliability of frequently used metrics when comparing different models. At the same time, the abilities of established and current methods to differentiate machine generated and human generated outputs are tested as the size of models scales up. The costs, time, and quality of human evaluation at the scale required for objective evaluation of automated systems are also recurring considerations. This contribution builds on a review of commended research on machine learning approaches for generating visual and linguistic outputs published at selected AI, computer vision, and computational linguistics conferences in the past year. A special focus will be placed on two questions: the suitability of quantitative measures for evaluating generative methods - and the role of human evaluation in assessing the models and outputs of machine learning approaches.
Jason Armitage’s research on multimodal machine learning and embodied AI is currently conducted at the University of Zurich in the project “Embodied Cognition in Virtual Environments with Diachronic Analysis of Linguistic and Visual Inputs”. Prior to UZH, Jason conducted research on multimodal machine learning at the University of Bonn and worked in digital product development teams at the BBC and Disney. His academic background is in computational cognitive neuroscience (University of London, Birkbeck), data science (University of Stirling), and machine learning (UC San Diego).
The recent hype for text-to-image models (Stable Diffusion, Dall-E, Midjourney) and previous style transfer methods (Pix2Pix), opened up the possibility to include synthetic images in art history datasets and use them to train deep learning models. Such an application is not unproblematic, prompting spurious contamination of the dataset, but yet useful, allowing to partially mitigate the common issue of lack of data. In this respect, I will pose the question of whether there is a possibility of a fruitful adoption of synthetic images in art historical datasets, discussing the case study of art authentication. In our research, we introduced GANs and Stable Diffusion generated data 'in the style of' the artist to be authenticated. The images were added to the set contrasting the authentic works of the artist, which is defined as anything that is not authentic; and as such, it is conceptually quite naturally permitting the presence of non-real data. However, even in this case, the synthetic data could unwittingly introduce new features not present in reality: the usefulness and risks fall into a grey area. To what point can synthetic data contribute to fecundly extending and augmenting the information in our dataset? What knowledge could algorithms acquire from such synthetic generated data? Would it raise questions regarding the integrity and contamination of the dataset? Can this study generalize to a vastity of other applications and fields?
Ludovica Schaerf is a pre-doctoral fellow Digital Visual Studies Center as of March 2023. She holds a Bachelor’s in Informatics from Amsterdam University College and she recently completed a Master’s of Science in Digital Humanities at the Swiss Federal Institute of Technology of Lausanne (EPFL). Before joining DVS, she worked as a Data Scientist for insurance and academic publishing and as an AI developer in the art market. Interested in interdisciplinary research between the Arts and Artificial Intelligence, Ludovica's research focused on AI-based methods for art authentication (at the Zurich-based start-up Art Recognition), and art historical analyses using computer vision methods, including tracing morphological patterns across large art-historical datasets (at DHLab at EPFL).
Registration
Public event with registration
The participation in the symposium is free and open to everyone. The event will take place on-site in Zurich (Switzerland), and will be live streamed via Zoom.
To register for the event (for either on-site or online participation) please go to: registration
For any additional questions, please send an email to eva.cetinic@uzh.ch