No more moody mornings

Joost van Kasteren



More eloquent alternatives to the harsh tones of the old-fashioned alarm-clock bells abound, including a newsreader’s voice summing up last nights disasters, or a tape of your favourite early morning music. Still, getting out of bed has its difficult moments. All this could well change in the near future, if Ir. Stephan Wensveen, a graduate student at the ID-StudioLab of the Delft Industrial Design faculty, has any say in it. For his doctorate research at the Form Theory Group of Dr. Kees Overbeeke, Wensveen is designing an alarm clock that deduces how you will feel the following morning from the way in which you set it, and adapts its alarm signal accordingly. In fact, it signals the end of early-morning grumpiness.

Products are becoming more and more intelligent, and are increasingly capable of adapting to users’ needs. However, the added intelligence also makes most products more difficult to use, the video cassette recorder being a case in point. One of the reasons for this lack of user-friendliness is the fact that the only way to communicate with these products is in a strictly rational way. It’s a bit like the well-known story about the mathematician who when asked, «Can you pass me the butter», replies in the affirmative, but takes no further action. In other words, communication with appliances lacks something that is a natural ingredient of communication between humans, i.e. emotion, at least on the part of the machine. As we all know, people can become extremely emotional when a Windows computer displays the Blue Screen of Death announcing a ‘fatal error’. Others will speak encouraging words to the printer when they have to print out an important document.

Researchers at the Form Theory Group at Delft University of Technology
(TU Delft) wondered if there was a way to make smart and complicated products — so forget the potato peeler — show some consideration for the feelings of the persons using them, and if so, how. In other words, in addition to rational intelligence, is it possible to impart some measure of emotional intelligence to products in order to improve the communication between man and machine. ‘A far from simple problem,’ says Stephan Wensveen. ‘In the first place, you have to find out which emotional aspects are involved in the way we experience a process, in this case, waking up. Then there is the question of how to make the product recognize these emotional aspects. And finally, the product has to respond correctly to the emotions it observes.’

Job interview
Wensveen chose the alarm clock because it is a fairly straightforward product with a simple function, i.e. to go off at a preset time. At the same time, most people experience a wide range of emotions when they wake up. These emotions can vary widely, not only between different people, but also from day to day.
Wensveen: ‘On the morning before an important exam or a job interview you feel very different from when you wake up to go cycling with some friends.’
The question is, how do you measure the different emotional experiences. You can hardly go and visit a number of test subjects to sit at their bedside and see how they wake up in the morning. You could ask them to come and sleep at your laboratory, but that would affect the results. So, Wensveen decided to use what are known as ‘cultural probes’, i.e. finding out about the emotional aspects of waking up in a roundabout way by asking test subjects to convey their emotions in images, sounds, and speech.
The test group consisted of twelve persons, from as many different categories as possible, including people with steady jobs, freelancers, pensioners, the young and the elderly, male and female.
Wensveen: ‘The purpose was not to form a representative group as in market research, but rather to map the gamut of emotions. I think we’ve succeeded pretty well. We’ve found that different people have very different ideas about getting up in the morning. Some suffer from proverbial grumpiness, while others jump straight out of bed to seize the day.’

Disposable camera
The test subjects were issued a test set that included a disposable camera, a sound recorder, a couple of pens, a diary, and some labels, as well as a number of assignments to carry out at home. To begin with, for a week they kept a diary to record the times at which they got up and how they felt at the time. The latter is done by selecting from a set of faces, rather like ‘smileys’, but with different expressions. In addition, they were asked to put their feelings in writing and to put down their plans for the day. At the end of the day, they were asked to say how their day had been.
Another assignment was to record their feelings towards their alarm clock. Wensveen: ‘As products become smarter, they also gain personality. In order to find out how people assess the personality of their alarms, they were asked to answer the question, if your alarm had had parents, who would they have been? They could make their selection from a set of labels with famous faces including Marilyn Monroe and Jack Nicholson, as well as Mickey Mouse, a crowing cock, and a sun. They were also asked to choose the parents of the ideal alarm-clock.’
The test subjects were also asked to take photographs of themselves, or their bed, their alarm clock, or their favourite product, as well as pictures of something beautiful and something highly annoying. Quite a few of the subjects took pictures of the mountain of dirty dishes in their kitchen sink to illustrate the latter. To conclude the test, the subjects were asked to record the sound of their voice and/or their alarm clock, as well as three of their favourite pieces of music.
Wensveen: ‘All together, these elements provide a pretty thorough picture of the people involved and the way in which they experience waking up. This kind of emotional information is much richer in content than the statistical surveys from a poll or market research. In those, the user or the products are made anonymous, whereas with our — much more time-consuming — approach, the user comes to life as a person. To the designer, this is much more interesting than a series of statistics in tables, as it gives you an idea of how people interact with products.

Big bang
Apart from a sense of the various ways in which people experience their moments of waking up, the test also provided information on aspects that affect this experience. Some of these aspects are to do with sleeping times. When do people go to bed, when do they have to get up, and how long can they sleep?
Wensveen: ‘We have found that the experience of waking up is affected in particular by changes in the normal sleeping time. If you are used to getting your eight hours of sleep, and for some reason or other your night is cut short by two hours, this will affect how you wake up. Another key factor is how urgently you need to get up. There is a lot of difference between setting the alarm to catch a plane, or not really caring about the time because it is your day off, to be spent gardening. The final factor is your mood late in the evening before you go to bed. This mood also affects your mood upon waking.’
The next question is how an alarm clock can recognise the different emotions involved in waking up. The time-related aspects such as sleep time and the time of rising can be entered, and an alarm can easily be programmed to recognise a sleep time that differs from the norm, but what should we do with the other, more emotional, aspects such as urgency of rising, and the subject’s mood before going to sleep? How to measure these emotions?
Wensveen: ‘There are a number of different ways of doing this. At the Boston MIT Medialab, scientists are researching the ubiquitous computer, the way in which intelligence is becoming ubiquitous in appliances, clothing, houses, etc. They are mapping human emotions by looking at physiological parameters such as heart rate, skin conductivity, and blood pressure. We decided to look at behaviour as an expression of emotions. In other words, we are trying to extract information about emotions from the interaction with the product. Angry persons tend to bang doors, or in the case of the alarm clock, to shut it up with a bang of their fist. Our assignment was to see if we could come up with a concept for an alarm clock that would enable us to deduce a whole range of emotions from the way people use it.’
Since the ID-StudioLab sets great store by the linking of research and teaching, the assignment was put to the students, almost 300 in number, who take the subject as part of their second-year course.
Wensveen: ‘That resulted in a number of ideas that caught our interest, one of which we decided to elaborate. The result is a bit like a 20-centimetre diameter flying saucer. A display window on one side shows the current time. On the top, there is another window that shows the period of sleep that has been set. You set the period by means of 12 sliders that slide from the rim inwards towards the centre of the disc. You slide it all the way for a full hour’s sleep, but you don’t have to operate them one by one to set the full period.
Wensveen: ‘You can also operate a few sliders at a time, or use two hands to move all the sliders simultaneously to get the time you want. The number of variations gives the user great freedom of interaction.’

Easy Rider
The assumption is that both the resulting pattern of the sliders and the way in which they are moved provide information about the user’s mood before going to sleep and the urgency of waking up. In order to test this assumption, an experiment was set up in which emotional moods were linked to the way the sliders were operated. Wensveen based the links on the circumplex model of emotions. This model categorises emotions in two dimensions, arousal and valence. The arousal dimension ranges from low to high, and the valence ranges from negative to positive. This provides a clear distinction between four emotional moods, angry, cheerful, despondent, and satisfied.
To create the various moods, the test subjects were shown a short film clip and music prior to being asked to set the alarm. A fragment of Easy Rider showing Dennis Hopper and Peter Fonda riding their motorbikes through the desert was used to create a feeling of satisfaction. Cheerfulness was induced by a swinging scene from the Blues Brothers. As for the sense of urgency, before and during the clip a message was displayed which read either ‘no appointments tomorrow, day off’, or ‘be at the airport in time tomorrow, do not oversleep’.

Action parameters
After viewing the clip and the urgency messages, the test subjects were asked to set the alarm. The test mapped two types of parameters, action parameters and pattern parameters. The pattern parameters relate to the pattern of the sliders after each action. The patterns can be divided according to whether they are vertically and horizontally symmetrical, centred or not, away from or towards the subject, and according to the regularity of the pattern. The action parameters include the speed of sliding, the length of the move, and the waiting time between the actions. Based on action and pattern parameters, the degrees of arousal, valence, and urgency were calculated for each of the subjects. If the calculated value was in the same region as the emotion expressed by the test subject, there was a correlation between the measured parameters and the emotions with which the alarm was set. Along the axis measuring low to high arousal, the correlation was found to be 80 percent, as it was for the axis measuring negative to positive emotion. In 90 percent of the cases, the sense of urgency matched the calculation from the action and pattern parameters. The results led Wensveen to conclude that the alarm is indeed capable of recognising the user’s emotional moods from the way it is being set. Based on this interaction and the time aspects, the alarm-clock can predict what your mood will be when you wake up in the morning, and adapt the sound of its alarm accordingly.
Wensveen: ‘If a person sets the sliders so that they all form a smooth pattern, for instance because he or she has to go to work at the usual time, the alarm might be a regular kind of sound. If the person has to get up to be at the airport in time, on the other hand, you could add a solid beat.’
One thing Wensveen still has to find out is the right sound to go with each type of mood. Rather than using recognisable melodies adapted to the user’s mood, he is thinking of using anonymous sounds constructed from different fragments (samples), the idea being that the alarm clock will gradually adapt its sound to the user.
Wensveen: ‘The way in which you respond when you are woken in the morning also illustrates the way you feel, of course. If you bash your alarm to shut it up, the appliance knows that the alarm sound did not match your mood, and will select a different sound next morning. This lets you train your alarm clock by a system of reward and punishment, just like you would a dog. Whatever developments the future may bring, emotional interaction between man and machine is going to be a major theme for designers.’ n




For further information, please contact Ir. Stephan Wensveen, phone +31 15 278 3775,
e-mail s.a.g.wensveen@io.tudelft.nl, or Dr. Kees Overbeeke, phone +31 40 2475931,
e-mail c.j.overbeeke@tue.nl