Social Semiotics is Cold Comfort

 
 Interpreting the Signs

I don't know about you, but when I was a child (this would have been maybe 3rd grade) I often tried to figure out how my language was created. I have a particularly vivid memory of sitting in a snow cave I had built on a cold carved reclining seat with my flashlight on a little shelf. I was talking out loud and my knitted mittens were soaked through wet, and my puffy blue snow suit zipped up to my chin.  I was saying something to the effect of "I will call this an 'arm'... wait first I need the concept of I, (here I grunt and point to myself) so I will call me 'I', now I need a concept of labeling so I will call this 'call', now I need a concept of the future..." and here I start getting brain freeze and start all over again.  Language, and this includes the interpretation of visual coding, seemed a huge mystery, and it continues to be that way. 

In its absolute most basic form social semiotics is the idea that there is a divide between an actual object and the representation of that object (called a sign), and that we represent objects and ideas through the three modes of symbols, icons, or indexes.   

To have a sign you need a signifier (the representation) and a signified (the object or concept)

Your signifier can be either a:

• symbol - something that represents the signified but does not resemble it for instance the written words "igloo", or the spoken words in the video below  
  
   neither of these are actual snow caves, nor do they look anything like snow caves, but you know approximately what I mean when you see or hear those.
• icon - something that represents the signified and also looks like the signified for instance the cartoon below. 
  
  
  
• Index - If I were to stand on a hill and point to my snow cave, or I was to refer to my snow cave as "that" or I took a picture of it such as the cave picture above I would be using an index.  An index is something that is not arbitrary, but directly connected to my particular snow cave.

These basics were discovered by Ferdinand de Saussure and were refined by Kress and van Leeuwen in their 1996 book Reading images: The grammar of visual design.  Harrison (2003) explains their concepts of representational metafunction, interpersonal metafunction, and compositional metafunction as it relates to interpreting images. 

• representational is all about the people, places, and things that you see in the image.
• interpersonal is about the actions between the people that produced it or are looking at it. 
• compositional looks at how both of the previous two integrate to create meaning.   

This morning my Microsoft Edge showed this picture, which I spent a long moment marveling at for its composition.   

 

The representational elements in the image are the penguins and the ice. They are moving which forms a narrative story about their powerful jumping out of water maneuver and their social nature.  There is a lovely and humorous element of one penguin looking back at the others though I am probably anthropomorphizing him, as if to say, "Come on y'all I'm having fun, what about you?"  We are invited to join in by the camera angle that is level with the waving penguin but lower than the jumper's making us want to join in. These elements are part of the photo's interpersonal charm.  Additionally, you can empathize with the photographer on his cold wait for just the right moment.  The background, the waving penguin, the narrative of swimming and sliding, and motion all form the compositional metafunction to create that sense of community, fun, and connection despite it being a species very different from our own.   

Let's try another one, a photo by Yvette Cardozo

The representational metafunctions of the images are the person sewing, the clothing, the igloo, the sky, the snow, and the tools being utilized, again there is movement so there is narrative.  The interpersonal metafunctions are about the relationship between the photographer and the subjects, as well as how you feel like you are peeking in at someone performing their private work, because the person is looking at their work not you and you are off to the side not directly speaking to them.  The compositional metafunctions of blue sky, the fact that the person is layered with all this clothing and working with thick gloves on, the colors in the photo, and your peeking combines these to give you a sense of loneliness, and coldness, but because of the smile on the person's face you get a sense that this person is quite happy with the situation and no company is needed.

  

Harrison, C. (2003). Visual social semiotics: Understanding how still images make meaning. Technical communication, 50(1), 46-60.