Neurocomputational Poetics

Neurocomputational Poetics

Introduction

Arthur Jacobs, Neurocomputational Poetics: How the Brain Processes Verbal Art (New York: Anthem Press, 2023).

---

Summary

"This book introduces a new thrilling field–neurocomputional poetics, the scientific ‘marriage’ between cognitive poetics, data science and neuroscience. Its goal is to uncover the secrets of verbal art reception and to explain how readers come to understand and like literary texts. For centuries verbal art reception was considered too subjective for quantitative scientific studies and still nowadays many scholars in the humanities and neurosciences alike view literary reading as too complex for accurate computational prediction of the neuronal, experiential and behavioural aspects of reader responses to texts. This book sets out for changing this view. It offers state-of-the-art computational models and methods allowing to predict which crucial textual features of prose and poetry, such as syntactic and semantic complexity or emotion potential, interact with reader features, such as empathy or openness to experience, in shaping a literary reading act. It contains hands-on practical examples on how to do computational text analyses of books and poems that can answer questions like: 'Which is Jane Austen’s most beautiful book?' 'Which poet created the most fitting poetic metaphors?' or 'Which author of plays of the nineteenth century was the most literary?'"

"The core of the NCPM [= Neurocomputational Poetics Model]" (55) is illustrated in the following diagram (adapted from page 33). It "describes key elements and processes that occur during the reading of a literary text at the neuronal, affective-cognitive, and behavioral levels" (32). (See below for discussions of the concepts of Immersive vs Aesthetic and Background vs Foreground.)

Outline

1. Introduction: The Two Boons of an Unnatural Daily Activity
2. Models and Methods
3. Text Analysis
4. Reader and Reading Act Analysis
5. Computational Poetics I: Simple Applications
6. Computational Poetics II: Sophisticated Applications
7. Neurocomputational Poetics I: Upper Route Studies
8. Neurocomputational Poetics II: Lower Route Studies
9. Conclusions

Key Concepts

Immersive vs Aesthetic

There are two different "modes of verbal art reception" (15): immersion and aesthetic feeling. The former is typically experienced when reading prose, and the latter is typically experienced when reading poetry.

Description of an immersive reading experience (when reading prose):

"It starts spontaneously, and it keeps on as long as I keep reading... I immediately immerse myself in the reading, and the problems I usually worry about disappear... I feel as if I belonged completely in the situation described in the book... I identify with the characters, and take part in what I am reading..." (1)

Description of an aesthetic reading experience (when reading poetry):

"It is emotion put into measure... It should strike the reader as a working of his own highest thoughts, and appear almost as a remembrance. It lifts the veil from the hidden beauty of the world, and makes familiar objects be as if they were not familiar..." (1)

According to the NCPM (see above), each of these experiences is associated with different neural, affective-cognitive, and behavioral processes.

Backgrounding vs Foregrounding

Immersive experiences are facilitated by the predominance of background elements in a text, whereas aesthetic experiences are facilitated by the predominance of foreground elements.

"Texts mainly renouncing on stylistic devices but offering many familiar and realistic details to draw readers into the story world can be characterized as primarily backgrounded. In contrast, texts that offer only a minimum of common ground – just enough to put forward their stylistic elements – are primarily foregrounded" (13).

The degree of foregrounding is "determined by the number (and type) of stylistic devices in a text that activate Jakobson's poetic function and inhibit the automatic, fluent, non-reflective reading process" (14). Examples of foregrounding features include rhyme, alliteration, metaphor, ellipsis, rare words, etc.

"Texts full of background elements

• are implicitly processed mainly by the brain's left-hemispheric reading network along the dorsal and ventral paths;
• evoke non-aesthetic fiction feelings such as (vicarious) joy or fear; and
• are characterized by fluent reading, that is high words per minute... and more or higher immersion ratings

In turn, texts full of foreground elements

• are explicitly processed, involving more right-hemispheric networks;
• likely produces aesthetic feelings; and
• a slower reading rate with many backward saccades (regressions), and more or higher beauty ratings" (37)

4x4 Matrix

The 4x4 matrix is a tool for identifying and quantifying foreground and background features. The table below is adapted from the table on page 62. The features listed in the cells of the table below are only examples.

Linguistic Aspect / Feature Level	Metrical	Phonological	Morphosyntactic	Semantic
Sublexical	stress	number of syllables	functional shift	morpheme type
Lexical	stress pattern	sonority score	word type	word beauty
Interlexical	rhythm	rhyme	word order	metaphoricity
Supralexical	global swing	global affective meaning	syntactic complexity	semantic complexity

Density fields

A related article that Jacobs coauthored, The Foregrounding Assessment Matrix, introduces the concept of "density fields." These are "'spots' of the text in which different kind of foregrounding (phonological, morpho-syntactic and rhetoric [cf. the 4x4 matrix above]) unite, agglutinate and combine" (262). They "work as milestones along a reading route" (2020, 262) and are "fundamental for the meaning-making process and the aesthetic appreciation of the text" (2020, 266).

Studies

Relating to Prominence

Predicting the Most Likeable Songs and Lines by Bob Dylan

• About 50 undergraduate psychology students ranked 11 Bob Dylan songs according to how much they liked the lyrics. They also marked the most striking lines.
• Jacob's goal was to develop a model that could predict the song rankings and the most striking lines.
• Using a computational tool called SentiArt, Jacobs "computed four features for each line of the 11 songs and then averaged them for each song: (1) the lexical affective semantic feature AAP [= affective-aesthetic potential], (2) the sub lexical phonological feature alliteration, (3) the inter lexical affective semantic feature arousal span and (4) the supralexical surface feature number of words" (163).
• Results: "With this simple four-feature model I could predict the liking ratings for the 11 songs with a 99 per cent accuracy" (163). "Thus, our sample of readers liked song lyrics with not too many words [= number of words], evoking associations with beautiful concepts [= AAP], drawing attention to their initial sounds by alliteration [= alliteration] and by a contrast between positive and negative arousal [= arousal span]" (163).
• With regard to the most striking lines, Jacobs developed a 15-feature model that could predict the most striking lines with 85 percent accuracy. The most important features were alliteration and AAP (also arousal span, surprise, word length).

Predicting the Most Striking Lines in Shakespeare's Sonnets

• Six participants, trained in literature and familiar with Shakespeare's sonnets, marked the most striking line in each sonnet.
• Out of the total 2,155 lines, the majority (1,473) were not chosen by any participant. "482 were chosen at least by one reader (22%), 155 twice (7%), 37 three times (2%), six four times, and only two by five readers" (166). Thus, "682 lines were chosen once or more and thus they must have something the other 1,473 do not: a higher foregrounding potential due to some unknown features" (166).
• (Interesting side-note: "In general, striking lines were more often chosen from the final parts of the poem" [167].)
• Research question: Can we predict the lines that were selected as most striking?
• Jacobs selected 20 features (following the 4x4 matrix above) computed by SentiArt.
• Result: The model could predict the most striking lines for each of the six readers with 94%-97% accuracy.
• The "ideal sonnet line" for this group of readers has
  • "within-line rimes and several sonorous words (rime quotient and sonority score);
  • short nouns that are positive, beautiful, surprising and that have a large fringe (AAP nouns, pos-neg ratio, word length, semantic neighborhood density, surprisal);
  • a more complex syntactic pattern with greater variation in word type order...;
  • a touch of universal poetic topics that speak of love and friendship, time and truth or heart and desires" (173).
• In a later study, measuring eye movement, Jacobs found that "'beautiful' lines attracted longer total reading times than the non-marked lines" (230).

Density Fields in Shakespeare's Sonnets 27, 60, and 66

The following discussion is based on the article The Foregrounding Assessment Matrix.

• The authors chose three of Shakespeare's sonnets: 27, 60, and 66.
• Using the "Foreground Assessment Matrix" (very similar to the 4x4 matrix above), the authors identified foreground features in each sonnet. The following visual represents the analysis.
  • yellow = phonological foreground elements
  • red = morph-syntactic foreground elements
  • green = rhetorical foreground elements

• "As the foregrounding features (FGs) (phonological, morpho-syntactic and rhetoric) identified at sublexical, lexical, interlexical, and supralexical levels overlap..., they are not to be evaluated singularly, but in their giving rise to what we’d like to call 'density fields'" (2020, 264).
• Examples of density fields (the clustering of foreground features):
  • "work my mind" in sonnet 27 (marked in all three categories and at all levels)
  • "minutes haste" in sonnet 60 (number of green and red underlinings)
  • In sonnet 60, the density field in lines 8-9 "builds the semantic climax of the sonnet attesting the main topic: time. It also develops the main topic by linking it directly or indirectly to other semantic FGs in the text and by building the main meaning-making chain throughout the text: “time” appears to be the semantic core (or hub-word) of the whole stylistic texture.
• Predictions:
  • Sonnet 27 (with less evident density fields) will be easier to understand and less aesthetically appreciated.
  • Sonnet 66 (with lots of foreground elements distributed somewhat evenly, also no metaphors) will be the most difficult to understand and the least aesthetically appreciated.
  • Sonnet 60 (with clearly defined density fields) will be difficult to understand but the most aesthetically appreciated.
• Testing predictions: 30 native English speakers read the sonnets, marked the most important words in each sonnet, and answered questions related to how much they understood and liked each sonnet.
• Results (based on averages):
  • Sonnets 27 and 60 were equally appreciated, and sonnet 66 was the least liked.
  • Sonnet 27 was slightly easier to understand than sonnet 60, and sonnet 66 was the most difficult to understand.
• Jacobs, tracking eye movement, also found that "lines with a greater foregrounding potential or poeticity also take longer to read" (2023, 231).

Relating to Sound

Iconicity in Every-day Language

The 'Why PISS is Ruder than PEE' Study. Abstract:

"In a large-scale lexicon analysis, we focused on the affective substrates of words’ meaning (i.e. affective meaning) and words’ sound (i.e. affective sound); both being measured on a two-dimensional space of valence (ranging from pleasant to unpleasant) and arousal (ranging from calm to excited). We tested the hypothesis that the sound of a word possesses affective iconic characteristics that can implicitly influence listeners when evaluating the affective meaning of that word. The results show that a significant portion of the variance in affective meaning ratings of printed words depends on a number of spectral and temporal acoustic features extracted from these words after converting them to their spoken form (study1). In order to test the affective nature of this effect, we independently assessed the affective sound of these words using two different methods: through direct rating (study2a), and through acoustic models that we implemented based on pseudoword materials (study2b). In line with our hypothesis, the estimated contribution of words’ sound to ratings of words’ affective meaning was indeed associated with the affective sound of these words; with a stronger effect for arousal than for valence. Further analyses revealed crucial phonetic features potentially causing the effect of sound on meaning: For instance, words with short vowels, voiceless consonants, and hissing sibilants (as in ‘piss’) feel more arousing and negative. Our findings suggest that the process of meaning making is not solely determined by arbitrary mappings between formal aspects of words and concepts they refer to. Rather, even in silent reading, words’ acoustic profiles provide affective perceptual cues that language users may implicitly use to construct words’ overall meaning."

Iconicity in Poems

For the following study, see Arash Aryani et al., “Measuring the Basic Affective Tone of Poems via Phonological Saliency and Iconicity.,” Psychology of Aesthetics, Creativity, and the Arts 10, no. 2 (2016): 191–204. The results are summarized by Jacobs 2023.

• The German poet Hans Magnus Enzenberger intuitively classified 57 of his own poems as either "friendly", "nasty" or "spiteful," and "sad."
• 252 native German speakers read and rated the poems in terms of valence (positive vs negative), arousal (calming vs arousing), friendliness, sadness, and spitefulness. The rating scores were almost always consistent with the classification of the poet.
• Using EMOPHON, a tool that "can tell you how salient a given phoneme is in a text via computation of its relative frequency" (196), the authors of the study quantified the basic affective tone of each poem. "We hypothesized that to a certain extent the sound or basic affective tone of the poems corresponded with their global affective meaning" (2023, 198).
• Results. "The results confirmed this hypothesis. The sublexical measure of arousal, based on salient phonological units computed by EMOPHON, accounted for 9.5% of the variance in human sadness ratings, 17% for friendliness ratings, and 22% for spitefulness... The highest arousal value was obtained for the spiteful poems (mean = 3.96), followed by the sad (3.68), and the friendly ones (2.1). We interpret these data as clear evidence that the iconic associations of foregrounded phonological units contribute significantly to a poem's emotional and aesthetic perception by the reader and the author" (2023, 198-199).
• The poem claimed in an interview that these sound effects were not intentional.

Tools

• SentiArt
• jmp 16 pro (for predictor screening to help identify relevant features)
• prosodic (for calculating sonority score)
• EMOPHON (for determining the salience of a phoneme)

---