Multimodal integration in the perception of press advertisements within the Dynamic Model of Meaning

New conceptualizations of meaning-making stress the importance of the contextual factors and the world knowledge resources alongside the linguistic input. The way in which these factors contribute to the ultimate understanding of messages depends largely on the type of communication involved. In advertising discourse exploitation of extra-linguistic resources seems to be exceptionally high. The present paper reports on a study in which the way the textual and pictorial channels support each other in information processing was investigated. To this end eight press advertisements have been modified in such a way that the texts were separated from the pictures and presented in this way to a group of respondents, who were asked to interpret either the texts or the illustrations alone, only later to be shown the complete versions and asked once again to report on their understanding of the messages in their original versions. In the discussion reference was made to such theoretical conceptualizations as the Dynamic Model of Meaning (Kecskes 2008), the notions of multiplying meaning and traversals (Lemke 1998; 2001; 2005), Conceptual Integration Theory (Fauconnier 1994; Fauconnier and Turner 1998; 2002), Resource Integration Principle (Baldry and Thibault 2006), GeM Model (Bateman 2008) and Graded Salience Hypothesis (Giora 2003).