Welcome to my Masters thesis overview...

In 2009, I completed a Computer Science Masters by Research at RMIT University. This page summarises key information about the thesis, at the bottom are links to download the actual thesis. Although all of the information on this page can be found in other areas throughout the website, this page brings together all the aspects into one page.

Thesis title: Using Adaptation and Goal Context to Automatically Generate Individual Personalities for Virtual Characters.

Abstract:

Personality is a key component of characters that inhabit immersive virtual environments, such as games and virtual agent applications. In order to be distinguishable from other characters in the environment, each character should have its own personality in the form of different observable behaviour, not solely in its physical appearance or animation. Previous work in this field has mostly relied on time-consuming, handcrafted characters and static, trait-based approaches to personality. Our goal is a method to develop complex, individual personalities without handcrafting every behaviour. Unlike most implemented versions of personality theories, cognitive-social theories of personality address how personality is developed and adapts throughout childhood and over our lifetimes. Cognitive-social theories also emphasise the importance of situations in determining how we behave. From this basis, we believe that personality should be individual, adaptive, and based on context. Characters in current state-of-the-art games and virtual environments do not demonstrate all of these features without extensive handcrafting.

We propose a model where personality influences both decision-making and evaluation of reward. Characters use their past experiences in the form of simple somatic markers, or gut-instinct, to make decisions; and determine rewards based on their own personal goals, rather than via external feedback. We evaluated the model by implementation of a simple game and tested it using quantitative criteria, including a purpose-designed individuality measure. Results indicate that, although characters are given the same initial personality template, it is possible to develop different personalities (in the form of behaviour) based on their unique experiences in the environment and relationships with other characters. This work shows a way forward for more automated development of personalities that are individual, context-aware and adapt to users and the environment.

Thesis abstract on separate page

Design Document:

In order to test the theory a game was built with the assistance of a programmer for the graphics and GUI aspects. The character AI was implemented entirely by myself based on the theory I developed. Note that the game developed was intended to showcase and test the theory developed, rather than be a "fun" game. Although the final game was different to what was specified in this document, the basics are covered and this was what was used to communicate to the game programmer.

Download the PDF design document.

Code:

My thesis work was based on agent-oriented programming, particularly the BDI paradigm. In this paradigm an agent is said to have Beliefs (what they think the world is like), Desires (what they want the world to be like), and Intentions (commitments to achieve specific desires using methods/plans). The aim of my research was to take simple personalities for characters and improve the depth of the characters by allowing them to learn their own preferences for individual plans. The work is inspired by Damasio's somatic marker hypothesis that states that when we (as humans) are faced with a decision with many options we automatically use gut instinct to narrow down our options into "good" options and "bad" options.

In order to implement this, the characters need to use an execution loop that forces them to evaluate the success or failure of plans that they execute, see diagram above. An incoming event from either the agent itself or a request from another agent causes the agent to consider which plans can handle the event. (Note: In the JACK programming language events and goals are interchangeable words.) If there is more than one plan that can handle the event, then the agent must choose between them. It does this using the Appraisal of Choices plan, which considers the agent's preference (somatic marker) for each plan it could choose between and then chooses a plan according to Damasio's somatic marker hypothesis. The somatic marker hypothesis is a psychological theory of how we (as humans) make decisions using our "gut instinct".

After the agent has chosen a plan, it executes it. This may cause the agent to make further choices using the same process as above. After the top-level plan has finished execution, the agent needs to evaluate whether the plans and sub-plans it executed actually helped it achieve its personal desires (called soft goals in my work). This allows it to determine whether the plans it choose were "good" or "bad" and so update its preference (somatic marker value) for the plans chosen.

The code is written for the JACK programming language (which has a Java base) available through AOS. Here are the links to the two main pieces of code that do most of the work:

• Appraisal of Choices Code: This is the code that decides which plan to execute when there are many choices. It uses each plan's somatic marker (preference) as a guide, so that highly preferable plans are more likely to be chosen.
• Evaluate Code: This code looks at all the plans that were executed for an activity and updates their somatic markers using reinforcement learning techniques. It also updates the agent's current emotion.

The code is generic for any domain and acts similarly to AI Middleware. To build a specific domain the characters need to have multiple plans that achieve the same goal (e.g. to make friends I could talk to someone, buy something for someone and so on...). These plans are then specified to use the Appraisal of Choices plan and the character will automatically build up their own preferences (somatic markers) for future actions. To test the theory an example domain was built based on characters trying to be popular at school.

Code information on a separate page

Demo:

The game to test the proposed theory is based on social interactions in the form of building friendships between characters. The game was limited so that the theory about generating personalities could be tested precisely. The characters are a group of school children, such as those shown in the screenshots, on their lunch break who want to interact with each other. Individual children can choose from three different top-level activities: to wait in one place, move towards or away from character's they like or disklike, and tell another child the name of a child they do not like, i.e. insult someone. The children choose actions in real-time. Note that the children are not allowed to say positive things about each other. This forces the characters not to just be nice to everyone, they must decide who their friends are and make sure they don't insult everyone, otherwise they will have no friends. The goal/plan hierarchy of what the children can choose from is shown below:

The characters choose what to do based on their personal history of previous choices in this situation and their personal goals. A character's goals are set in their starting personality template, but their overall personality in the form of observed choices they make is built up based on their unique history of their current personal context. When a character completes an activity, they evaluate whether it was "bad" or "good" based on their personal goals and use this to update the likelihood of doing the same activity in the same context the next time they encounter it.

The following images are screenshots from the game.

Telling the player they suck and waiting for a response In this scenario, the player is told they suck and they are allowed to respond to this statement. If they wish to please that character, they may decide to agree with the character!

Asking the player for their opinion about Fran In this scenario, Anna asks the player for their opinion of one of the other characters, Fran. If the player agrees with Anna's negative opinion, then she may change her opinion of the player and list them as a friend.

Demo information on a separate page

Testing:

The masters thesis aimed to create characters that adapted, used context and were individuals (i.e. different to the other characters even when their initial personality templates were different). In order to test these three areas I looked at the choices the characters made (i.e. what they did), how well they thought they were achieving their personal goals and I developed a quantitative measure for individuality. Eventually the theory that you can develop "different" characters using my techniques would need to be tested using human participants and asking them whether they notice differences. As a preliminary step before this, a quantitative measure determines whether their are any statistically significant differences between the characters. If the characters have no statistical differences, then it is very unlikely that humans would notice the difference. This means that the quantitative test needs to be passed prior to testing with human participants, i.e. there was no human player in the tests performed. The quantitative measure for individuality that I developed is based on paired t-tests that examine the differences in number of times characters choose each possible plan.

Behaviours of two sample characters according to the top-level choices they can make.

Behaviours of two sample characters according to the top-level choices they can make with different starting conditions.

Behaviours of a single character to show that it can learn specific goals such as "move towards a friend".

Characters learned based on their context which came from their perceived achievement of their personal goals. In this image we are looking at the same character in two different contexts and can see that they prefer different actions/plans in different contexts. This difference was learnt automatically by the character and not pre-programmed.

Testing information on a separate page

Thesis Document:

The full thesis can be downloaded via the Australasian Digital Thesis Database or on the specific page for my thesis. Otherwise, you can download the individual chapters (PDF documents) using these links:

• Front matter: Contents, Lists of Figures and Tables, Glossary, Abstract;
• Introduction: Including motivation, research questions and thesis overview;
• Literature Survey: Survey of previous work done in agents, personality, games and adaptation;
• Agent Personality Development Model: The theory behind the thesis. Including components of the model, Somatic Markers, and adaptation process;
• Implementation and Experimental Setup: Information about the demo game developed and the experimental setup and measures used to test the demo game;
• Results and Discussion: Results based on agent behaviour, personal reward and individuality. Discussion of reasons behind results and details of extra results obtained.;
• Conclusion: Addressing the research questions, considering future research and implications for the games industry;
• Appendix A: Sample data for character behaviour over time;
• Appendix B: Sample data for characters learning a functional soft goal;
• Appendix C: Sample domain dependent results;
• References: Full bibliography of sources used in the thesis.