Posts Tagged ‘Decision making’

Sentiment analysis: the internet becomes a focus group

Sunday, October 31st, 2010

How does the world feel right now? In 2005 wefeelfine.org scoured the world’s blogs for the keyword ‘feel’ and, via an artful and compelling interface, presented each author’s snippets of text and images as a beautiful representation of the mood of the world. Jump forwards to April 2010 when 50,000 websites integrated Facebook ‘Like’ buttons in the first week of its availability. Now with the growth of Twitter, that flow of micro-opinions and feelings has become a torrent. Social media clearly likes to express personal opinion.

Mid way between the pure art of wefeelfine.org, and the visceral grunt of Facebook’s ‘Like’ is a grey area of mostly unquantified social thought. That’s where sentiment analysis (also known as opinion mining) resides. As its tools and techniques become more sophisticated it will reap valuable insights from the cacophony – for brands, governments and researchers to understand the feelings of the masses, and then respond to them – without the need for surveys, polling call centres, or focus groups.

The way it works is via natural language processing which uses algorithms to understand the context within which people’s opinions reside. For an in depth theoretical overview of this area, Bo Pang and Lillian Lee’s book Opinion mining and sentiment analysis has plenty to absorb, but in essence, a sentiment analysis service will rate statements as either positive, negative, or neutral, and add up the results. For example, a YouTube comment such as ‘The Social Network sucks’ is clearly negative, whilst a blog saying, ‘The Social Network is not factual, but does capture the essence of the character’, is either neutral or erring towards the positive, and someone tweeting, ‘OMG u mus c JT in #socialnetwork :-) ’ is positive but needs a lot of syntax interpretation. The most sophisticated forms of sentiment analysis are yet to come, but already the technology exists to rate the strength of feeling and weight key influencers amidst many other factors.

Since context is everything, these algorithms will inevitably have a wide reaching impact upon search engines. According to Google’s Matt Cutts, Google Search is already using Sentiment Analysis for reviews, and they filed for a patent back in 2007. Consider, for example, searching on the phrase, ‘antenna problems with the iPhone 4’. This complex question requires a fair bit of individual research and unpicking from the search results, and while the answer is interesting to a member of the public, it’s vital to the manufacturer. You can be sure Apple was listening closely to the noise on the web before they gave away US$100m worth of free iPhone 4 bumper cases.

Financial markets are also interested in turning the fuzzy into facts with companies such as Thomson Reuters exploring sentiment news in the form of Market Sentiment, but it is in the branding sector that sentiment analysis is seeing the most visible evolution. Over the last ten years the concept of customer engagement has grown due to the fragmentation of market segments and customer loyalty. The rise of social media has opened new ways to review global opinion, with Nielsen Buzzmetrics being one of the first to collate and display the user trends.

The principal is that you type in a brand name or subject and the sentiment analysis service will return a dashboard or chart analysis of the positive, negative or neutral opinions from the social web: Twitter, Facebook, Social Networks, RSS feeds, 100 million blogs.

A lot of big brands are already using these services from the larger players. We see Paypal, Nokia, Skype, Xerox and Ogilvy using Alterian; The Economist using Jodange; AT&T, Barnes & Noble and Playstation are using Lithium; Nokia, Roche and HSBC are with Sysomos; Adobe, National Geographic, 3M and Dell with Radian6 ; Wiley, Vodafone and T-Mobile with Attensity; AOL, Nissan and Marriott with Clarabridge; and The Body Shop are with Brand Watch.

Smaller solutions are also available, such as Chatterscope which allows you to monitor sentiment changes via simple alert messages. And Evri has released their Vibology widget which can be embedded in any web page to show a similar rating meter.

These products are already mature enough to use standardised processes for harvesting, cleaning, processing, and delivering results to sophisticated analytics dashboards. The more developed sentiment analysis companies (above) are correctly perceiving this is vital for Customer Relationship Management, and that they need to provide the correct tools to Marketing Managers who will want to tailor their campaigns based on the results they uncover.

However, it is also an emerging technology, one which requires a lot of artificial intelligence inspired computing to glean relevance – not only from languages other than English, but also from emoticons, acronyms and social media grammar – without which a lot of Twitter would be unreadable. Until these issues are fully resolved, the biggest caveat is therefore on accuracy, and the efficacy of the ratings.

Choice modelling and quality of life

Tuesday, September 14th, 2010

Happiness is one of the hottest areas of economic research. But simply asking people how happy they are is inadequate, says Dr Terry Flynn. Understanding how people choose between competing versions of their lives is a far more accurate way of delving into their psyches.

It ought to go without saying that the aim of social policy and government regulation is to increase society’s general level of well being. Good policy has positive impacts, reaching much further than its immediate consequences. A well-planned and well-delivered health policy that supports the families of patients suffering from terminal illness leads to greater aggregate quality of life and more community happiness. Bad policy does the opposite.

For this and other reasons, happiness has become a growth area of research in economics and the social sciences over the past few years. Suddenly, everyone wants to know what lies behind human contentment, and how to measure and improve it. Happiness scales have risen to dizzying heights of popularity of late; however, their aim – to reflect general sentiment and well being – can be easily lost through the use of numeric scales to elicit responses and collate data.

Happiness research usually asks respondents to choose a number between zero and 10 to indicate how happy they are with their life at that moment. The way people measure their levels of happiness, however, can give very different pictures.

Personal point of view

Each respondent will have a different personal frame of reference in giving his or her response, and such personal reasons are often lost in interpreting the data. For example, survivors of traumas such as war may put on a brave face, measuring their personal levels of happiness in peacetime using a different frame of reference than people who have never experienced war. Evidence suggests that simply reaching old age tends to make people rate their happiness highly.

Cultural cues

People may assign meanings and superstitions to numbers – for example, the number eight is significant in Buddhist and Chinese cultures – and decisions to choose or avoid these numbers affect survey results.

Meaninglessness

Happiness scales have no theory behind them. If a person scores seven out of 10, what does that mean? What are the statistical properties of numbers on the scale? Happiness scales offer limited choice to respondents to express what’s important to them. For the respondent scoring seven, what explains the missing three out of 10? Happiness scores don’t tell us what a person values. Not knowing what’s missing from the person’s life, it is impossible to know what improvements might make that person happier.

The scores don’t allowus to predict future demands, trends, or ways to make product and service delivery more efficient and relevant.

Irrelevance

Often, happiness scales lead to ‘ecological fallacies’. Differences in scores between, for example, married and divorced people are meaningful only at high levels of aggregation, typically at an entire-population level. It is an ecological fallacy, therefore, to infer that such differences are relevant to me or my peer group. I want to know what the differences in scores are among people like myself, not for the ‘average’ Australian.

So what is happiness?

Buddhist monk Matthieu Ricard defines a happy person as ‘anyone who enjoys inner peace’. Such a person ‘is no more broken by failure than he is inflated by success’, because he ‘understands that experiences are ephemeral and that it is useless to cling to them’. – Matthieu Ricard, Happiness: A Guide to Developing Life’s Most Important Skill

What’s that got to do with good policy and innovation? British psychologist and author of Affluenza, Oliver James, rejects happiness as a policy goal for other reasons. High average happiness scores have been observed in those countries with the greatest inequality and highest rates of depression. Instead, he advocates policies that improve four aspects of life:

  • connectedness to people: good relationships and feeling part of a community;
  • autonomy: being independent and in control of your own life;
  • experience: feeling you can make a difference and that there is value in what you do; and
  • security: material and emotional.

Just like the trend to ‘think positive’, happiness scales leave out important ingredients of life. They allow people to focus overly on material gain or hedonistic enjoyment, ignoring the
other three key sources of durable wellbeing in a population.

Qualitative and quantitative choice modelling offers a solution to the limitations of happiness scales. Choice modelling empowers people to say what they want and how much they’re willing to contribute, personally and financially, to making it happen. ‘ICEPOP’ and ‘ICECAP’ might sound like they’d be more at home in Antarctica than in Australia (see box copy, next page), but they are revolutionary instruments being employed by policymakers and innovators alike. ICECAP instruments expand on happiness-rating scales to get a much more intimate measure of a respondent’s quality of life. They give a more realistic cross-section of community sentiment and allow us to hone in, with a person-by-person, topic-by-topic degree of accuracy, on what people like and dislike, what they do and don’t find important to their wellbeing.

Results from ICECAP instruments are based on a well-tested theory of decision-making and cannot be manipulated. With the ICEPOP team, I’ve developed both ICECAP instruments over the past decade, building on the choice-experiment and capability-approach work of Nobel Prize for Economics laureates Daniel McFadden (2000) and Professor Amartya Sen (1998).

Working from the vantage point of measuring respondents’ perceptions of their quality of life, ICECAP instruments help us explore how to maintain and improve services into the future. At CenSoC, we have been developing new instruments for measuring Australians’ perceptions of their quality of life.

Our aim is to first develop a picture of survey-respondent types. The holy grail is a robust, longitudinal set of data that allows us to measure how different types of Australians perceive their quality of life, and how their values and norms change over time. This data will allow us to advise governments on how to plan for the future based on what people value.

Companies will also benefit from data about what people want most. We have developed a specialised measurement and valuation instrument that will allow us to build up individual scales for respondents, based on discrete answers designed to give a true sense of their lives. In our current research, we are presenting people with 16 different lives and asking them to imagine living in each and decide which aspects of each life would be best and worst to live with.

Our survey of 2,400 randomly selected Sydney residents, commissioned by the Independent Public Inquiry, Long-Term Public Transport Plan for Sydney (2010), reflected an overwhelming preference for public-transport solutions to the city’s congestion crisis. Almost two-thirds of respondents expressed a willingness to pay more for better public transport and to consider congestion charges to improve it, but were opposed to paying more for the existing public transport, which received a resounding vote of dissatisfaction.

Already, our work at CenSoC is attracting national media attention and being tested in the following public-policy areas:

  • comparing quality of life, nationally and internationally;
  • eliciting preferences for public-transportation systems;
  • valuing patient and citizen preferences for health-care reform;
  • valuing water supplies;
  • measuring quality of life for young people; and
  • developing personality and compatibility scales.

Ultimately, at CenSoC, we’d like to have a personalised scale to help us understand the differences in the norms of quality of life among Australians. Our research aims to yield a longitudinal and quality measure of wellbeing for the nation, to inform policymakers and innovators about where real differences are in the community. Choice modelling can seed new approaches to service delivery for the best quality-of-life outcomes, letting us have our cake and eat it too.

ICEPOP and ICECAP: not frozen treats

The ICECAP measures we use in our happiness research are conceptually linked to Amartya Sen’s capability approach, which defines wellbeing in terms of an individual’s ability to do and be the things that are important in life.

Investigating Choice Experiments for Preferences of Older People (ICEPOP-O)

In the ICEPOP team at the University of Bristol, UK, between 2001 and 2009, I used choice experiments to elicit the preferences of older people in various studies. For the main study, we developed a quality of life instrument: the ICEpop CAPability instrument for Older people (ICECAP-O). It focused on general quality of life since older people typically received a mix of social and health-care interventions.

In our research, we tried to distill the essence of people’s responses, developing five dimensions from which to measure the data:

  • attachment (love and friendship);
  • security;
  • enjoyment;
  • role (doing things that make one feel valued); and
  • control (independence).

The quantitative work for the ICECAP instruments uses a model that effectively measures how often people choose one of these quality of life dimensions over another.

ICEpop CAPability instrument for All adults (ICECAP-A) We repeated the original study to construct an instrument for adults of any age. The quality of life research we are doing at CenSoC is at the world forefront of what’s going on in this area. Sen stressed that it is a person’s capability to achieve key aspects of life, such as being independent, having relationships, that is important, rather than the amount they might choose to engage in those aspects.

In developing ICECAP-A, five key aspects of life were identified from qualitative work with people, which were similar to those in ICECAP-O:

  • Can respondents have close relationships?
  • Can they have their independence?
  • Can they achieve and progress in life?
  • Can they feel secure in life?
  • Can they have enjoyment in life?

We then get each person to indicate how much of each dimension they can have and assess them against their scale to score and value their quality of life. Our approach will offer a completely new way for politicians and medical decision-makers to measure how people are coping on a day-to-day basis.

Lowy Institute discovers cost of stopping climate change – about $10

Monday, May 31st, 2010

The Lowy Institute have just released their Annual Poll. It focuses on Australian Foreign Policy but also continues some long running questions on climate change.

According to the poll, the majority of Australians (53%) said the issue was very important to them – down from 56% last year. An encouraging 72% believed Australia should take unilateral action to combat climate change, ahead of any global agreement being reached. The respondents were then asked how much extra they would be prepared to pay on their electricity bills if it would help prevent climate change. At this point, like the crowd when a street entertainer puts down his unicycle and brings out his hat, people started drifting away.

The majority of respondents were only prepared to pay $10 or less extra per month on their electricity bill. Let’s say the average monthly residential Australian power bill is $150. That’s a 6% increase to prevent climate change.

Earlier this year the NSW Independent Pricing and Regulatory Tribunal (IPART) forecast that electricity prices in NSW will rise by 40% over the next three years. In that context the 6% doesn’t seem like much. Incidentally, that 40% is without the Trading Scheme which shall not speak its name. With the, ahem, ETS, prices are forecast to rise by 60%.

The Lowy Institute did not ask people what they considered to be the cost of doing nothing about climate change.

Edition 3: The Future

Friday, May 28th, 2010

Business21C Weekly is now available through the iTunes Podcast directory,
Alternatively, to subscribe directly via iTunes on your computer, go to the Advanced menu in iTunes and select Subscribe to Podcast. Then paste the following URL: http://www.business21c.com.au/podcasts/feed

I never was, am always to be,
no-one has ever, or will yet meet me,
but I am the confidence of all
who live and breathe on this spinning ball.

This week’s edition starts with a riddle, and continues with an enigma: the future. We talk with professional futurists Craig Rispin and Glenys McLaughlin about looking into the crystal ball for a living. Later in the conversation we are joined by digital artist and designer Ian Gwilt who is working on a project for the UTS campus using Augmented Reality – a future mobile technology-enabled experience.

Novelist William Gibson said: “The future is already here, it’s just unevenly distributed.” Futurists help organisations draw together the threads of today that will be woven to make the fabric of the future. They have a swag of techniques, from scenario planning to environmental scanning. These techniques help companies shape strategy by managing the risk of disruptive change.

“The primary technique of being a futurist is seeing the world with naive eyes,” says Rispin. Together we canvas the issues that are affecting companies and people as technology, globalisation and convergence accelerate.

Ian Gwilt is a digital artist and academic working on a project to create an augmented reality campus for UTS. By developing a database of what’s happening at the university, from lectures and library usage through to carbon emissions and events, and integrating it with geo-spatial technology and the capabilities of the smart phone, Ian’s project will create a multi-dimensional and rich experience of the campus.

Do you really care? The myth of the ethical consumer

Monday, March 15th, 2010

Professor Timothy M Devinney thinks the idea that people will change the world through what they buy, enabling companies to save the world through their actions, is hopelessly naïve. But that doesn’t mean corporations can’t view the world through ethical lenses. In fact, marketers and strategists need to know much more about their customers’ ethical preferences.

Katy Perry, singer of ‘I kissed a girl’, declares herself a ‘Red artist’. Georgio Armani, American Express, Starbucks, Apple, Dell – all these companies consider themselves to be ‘Red’, as do Converse, Bugaboo and Hallmark. They have pledged themselves to the Red initiative, spearheaded by U2’s Bono and politician Bobby Shriver. They have specified that 50 percent of the profits from designated ‘Red’ products and services will go towards buying and distributing antiretroviral medication to AIDS patients in Africa.

Since its inception in 2006, the Red initiative claims it has raised more than $140 million for its partner The Global Fund so that this fund, in turn, can provide medication to nearly 80,000 people in Ghana, Lesotho, Rwanda and Swaziland. As Katy Perry says: ‘It’s not just like they are coming to my concert: when they purchase a ticket, they are helping out someone across the world – someone who needs medication, who doesn’t have resources; someone to help in the fight against HIV and AIDS.’

Inspirational stuff.

With numbers like those, it would be churlish to argue that initiatives such as Red are not worthwhile. Yet such high-profile activities hide the fact that products with ethical or social dimensions have far more limited uptake than many executives and social activists might hope, leaving many in business expressing uncertainty, in private, about the financial efficacy of ethical consumerism and the role their customers play in sharing obligations to social ethics. Despite the hype it has generated, there is strong evidence to suggest that even Project Red hasn’t had the impact on consumers its partners had hoped for.

In fact, our research shows that the ideal of the ‘ethical consumer’ – that person who is guided in their purchasing decisions by broad ethical or moral concerns – is a myth. That doesn’t mean people aren’t influenced by issues other than price and product – they are. But we find that when you look carefully at people’s purchasing behaviour, it does not tally with what those who promote the idea of the ethical consumer would expect. All too often, survey radicals can turn into economic conservatives at the checkout.

Horses for courses

The ethical consumer, as an ideal, fails to match reality in a number of ways. When you examine how people purchase, you find that they do so in a much more utilitarian way than promoters of the idea of the ethical consumer claim. As our coffee-shop experiment shows (see Fair Trade Coffee, Sir, page 27) how ‘ethically’ people decide to consume is a socially determined factor, and a subsidiary one at that.

People may care about a variety of issues that form part of a broad ethical agenda: third-world debt, child labour, pollution, animal welfare and so on. But they tend to be hard-nosed when they trade these things off against matters that are more salient, immediate and mundane: children’s schooling, healthcare, their mortgage – even simply spending less time at the checkout counter.

The question, it seems, is a relative one: ‘How important are social issues when compared with other economic issues?’

Consumer social responsibility

These findings might tempt strategists to side with the financially driven sceptics. If ethical consumers are a myth, why cater to them – particularly if it is going to cost you money? It may be true that the ‘ethical consumer’ is a myth, but consumers are influenced nevertheless by their social concerns, in the same way that they are influenced by other aspects of the consumption landscape: price, branding, taste, positioning and the context within which consumption occurs. It’s just that you can’t rely on a simple conception of ‘ethicism’ in your marketing strategy.

Instead of conceptualising an ethical consumer, it may be more helpful to think about ‘consumer social responsibility’. This differs from the ‘ethical consumer’ construct because it recognises that consumers’ decisions are influenced by many factors and that the social component of a product is just one. What’s more, it doesn’t attribute a broad and generic ‘ethicism’ to individuals, who are likely to have far more subtle social preference delineations than we give them credit for.

Individuals reveal their social preferences through their patterns of consumption; in fact, it would be almost impossible for them not to do so, since social preferences are an inherent aspect of consumerism.

But you can not determine from their consumption patterns whether or not someone is ‘ethical’.

One person may reveal that he cares deeply about animal rights but is indifferent to the ‘right-to-life’ cause. Another may be a passionate right-to-lifer but have no position on the destruction of the Amazon rainforest. Which of these people is ‘ethical’? Both, surely.

People actively decide the extent to which they will support one social cause over another, and these decisions are reflected in their consumption behaviour. But you can’t use these decisions to predict from one social category to another. Ethicism is not generic.

Nor are people’s ethical concerns strongly influenced by their cultural heritage. Europeans do not appear to be significantly more socially aware than Americans. Consumers from developing countries seem to be no less concerned about environmental issues than those hailing from richer countries.

The reality is more complex than that: our research shows that the rationalisation of behaviour and a person’s understanding of the idea of ‘ethical consumerism’ are culturally informed – but behaviour is remarkably similar.

Ethics is a much more granular subject than the proponents of the ethical consumer would have us believe. It does a disservice to the human race to categorise some aspects of consumption as ethical or unethical.

Instead, it’s better to think about individuals revealing their social preferences through their behaviour. In turn, these preferences are useful in understanding how marketers can influence behaviour.

The socially responsible corporation

Understanding that social ethics is a complex and relative concept that differs markedly among individuals and communities sheds much light on the idea of the sociallyresponsible corporation. Many commentators have promoted the idea that companies can ‘do well by doing good’.

But if we understand that ethics are a complex and individually determined phenomenon, we realise that the idea that corporations can determine what is ‘good’ is naïve.

In fact, taking this argument to its logical conclusion reveals that handing over social policy decisions to corporations would lead to what most would consider to be an unacceptable loss of democratic rights.

To see why, it’s only necessary to take a walk down my street. I have neighbours who were born in China, Korea, India, Syria, Japan, the Netherlands, Croatia and 10 or so other countries. The local election ballot has to be printed in more than 20 languages. Yet, in the same street where several wives were imported for arranged marriages lives a lesbian couple and a dozen individuals cohabiting in de facto relationships, some with ‘illegitimate’ children.

Would corporate policies and choices that appeal to one of these families be likely to appeal to all? Certainly not. Any attempt to do well by doing good among my neighbours would get bogged down immediately in an energetic debate about what was and wasn’t ‘good’.

Corporations, by their very nature, have conflicting virtues and vices that ensure they will never truly be socially responsible, even by the broadest of definitions.

Would you want Walmart running society? The answer is, ‘Probably not’, for two reasons: firstly, because you can’t guarantee that what Walmart decides to do for society will in any way match your conception of a social good; and secondly, because there is a strong likelihood that instead of the corporation being forced to act in ways society deems ‘good’ (even if we could define such an idea), society will be forced to act in a way the corporation thinks is good. What’s more, you can’t vote Walmart out of office – there is no governance mechanism limiting the actions of a corporation in the way there is around a democratically elected government.

As Elizabeth Taylor said:

‘The problem with people who have no vices is that generally, you can be pretty sure they’re going to have some pretty annoying virtues.’

Social marketing

From an overall strategic perspective, it is important for corporations to understand that there are social responsibilities within all aspects of their businesses – whatever that means for the societies in which they are operating. Better companies work to enable these responsibilities to be fulfilled, however they arise (not simply through giving to select charities or enabling particular programs favoured by the CEO).

Corporate social responsibility, as it is traditionally defined, implies that there is a responsibility at the level of the corporation as a whole. But the company is not a being; it is simply an entity that allows humans to interact. It is the humans who have social preferences and responsibilities – as individual customers, workers, owners, investors and executives. Companies that put out glossy brochures proclaiming their extensive environmental, social or ethical activities are not being socially responsible – they are engaging in public relations.

More important is understanding the social links that exist across the corporation, understanding their interplay, and being able to understand and act upon the opportunities that arise from them.

This means enabling workers to have fulfilling lives and time with their families, customers to contribute to a variety of social causes through their consumption decisions, executives to create value for shareholders and investors to contribute to economic growth. The most ethical companies enable all of these and more, in concert, and profit over the long term as a result.

Fair trade coffee, sir?

One experiment we undertook in investigating whether or not the ethical consumer exists took place at a coffee shop in central Sydney over a period of several weeks.

This coffee shop displayed a large and prominent sign indicating the products available, their prices and active specials. To this we added, quite obtrusively, another special, indicating: We have Fair Trade coffee! No extra charge. Just ask.

Unprompted, with only the sign to notify them of the availability of the ‘ethical’ alternative, less than one percent of customers bothered to ask for Fair Trade coffee, even though it was free.

When we opted for the McDonald’s strategy – prompting customers with a reminder that the ‘ethical’ alternative was available – the number of customers opting for the Fair Trade option rose to 30 percent.

We then went a step further and took the customer’s privacy away: each time the clerk prompted a customer with the Fair Trade option, we ensured there was someone standing next to that person at the counter. In this situation, the number of ‘ethical consumers’ rose to 70 percent.

Throughout the experiment, we gave different coloured cups to customers who indicated that they wanted the Fair Trade product. We then questioned those remaining in the coffee shop about the meaning of fair trade and what they thought they were doing by purchasing, or not purchasing, Fair Trade coffee. On the whole, we received informed and insightful answers: customers talked about fair trade; they talked about the conditions of Guatemalan farmers; they could cite many reasons why they had opted for Fair Trade coffee.

None of this meant anything, however. When a customer chose the Fair Trade alternative, his or her decision was based entirely on the context we had created; it had nothing to do with that person’s values or preferences.

What this reveals is the degree to which ethical consumerism, like all consumerism, is strongly determined by the social context in which it is served up: something the marketers behind the Red campaign, and Katy Perry, are likely to have taken into account.

Timothy M Devinney is a Professor of Strategy at UTS Business. His research interests include international business, corporate and social responsibility and social consumerism, corporate strategy and technology/knowledge management. He has also published a number of papers regarding ethics and the role of ethics in consumers’ lives.

Timothy M Devinney’s latest book, The Myth of the Ethical Consumer, is available from Cambridge University Press.

Decisions, decisions

Sunday, March 14th, 2010

Life is just one decision after another. How and why people make the choices they do is a secret that contains many truths for businesses and policymakers. Professor Jordan Louviere investigates our preferences.

Life is about trade-offs. Whether you decide to have bacon or cornflakes for breakfast, buy a Motorola or a Samsung phone, go to work or chuck a sickie. If you do one, it makes the other redundant.

If it weren’t like that we would all live in the largest houses, drive the fastest cars and wear the biggest bling we could find, because there would be no cost to doing so. It is not only the direct economic cost, but also the opportunity cost of choosing one thing over another, devoting time to one activity over another, or selecting a particular brand or holiday destination.

How people make these decisions lies at the heart of economic life. If economics is about scarcity, then choice theory is about how people navigate through scarcity. Decision-making maketh the man or woman. Our character is determined by how our preferences play out against the options we are offered.

It is not too much to say that our lives are the sum of the choices we make.

Given that is the case, it is remarkable how little science is applied by most companies and governments to uncover the trade-off preferences of their customers or voters, how they choose between a complex series of options, and which aspects of a decision are more important than others.

Sure, the public is polled endlessly on every aspect of their lives. They are asked who they’d like to run the country, what washing powder they prefer, whether they want to pay for public transport or would rather walk. But the vast majority of these polls are fundamentally flawed because they don’t put decisions into a framework of trade-offs. A survey cannot be right if it does not make people choose one thing over another, if it does not ask people to actually choose between two mutually exclusive options that have consequences beyond the simple choosing. It is only by doing this that we can really unveil people’s usually well-hidden preferences; be able to say that one politician has real appeal over another, or one function of a mobile phone is crucial to the customer experience while others are not; or understand the value one brand has against its peers.

Twenty years ago, the North American food market was thrown into turmoil when grapes from Chile were found to contain cyanide through sabotage. Canadian authorities frantically surveyed their constituents, and asked them whether or not they were concerned about the safety of their food supplies. The results could be summed up as: ‘Why, come to think of it, I guess I am.’ Given the results of the survey, the authorities came to me at the University of Alberta and asked my advice on how best to design a campaign to reassure the public of the safety of the food supply chain. Rather than do that, I suggested we design a survey to find out exactly where on their list of the public’s concerns food safety lay. Turns out it was dead last.

Instead of spending $3 million on a campaign that would have reassured people about something they weren’t really concerned about in the first place, the government spent $30,000 to really reveal what their constituents’ concerns were. They saved money simply by applying a little bit of science.

Paradigm shift

You couldn’t really blame the Canadian authorities, though, because they were simply working under the influence of what was and remains the dominant view: that people’s future preferences are essentially mysterious and you can’t really ever uncover them. Under this view, the only way you’ll really discover what people want is by putting your choices out there, launching your product or business model, and seeing if it takes off or not. We know, however, that many products fail on launch and many government policies fail simply because of a lack of support from the public.

But the science of choice modelling, as we practise it at the Centre for the Study of Choice (CenSoC), comes at the problem from a different direction. We can reliably uncover, through statistical methods and experimentation, people’s preferences, and predict which products will be successful and which won’t; as well as which policies will garner strong support from the general public. Importantly, however, these experiments must be guided by a theory of how people make decisions in any particular situation.

This alternative view builds on a long line of progress in the understanding of choice making. In the late 1960s, building on the work of LL Thurstone – perhaps the most famous quantitative psychologist who ever lived – Daniel McFadden examined how people choose between multiple options and developed a theory of discrete choice – how people choose between more than two things – and statistical methods to apply it. His work was used to design the San Francisco transport system and to guide investments in phone services and housing for the elderly. He was awarded the Nobel Prize for Economics in 2000 for this work.

In the 1980s, we knew how to estimate the means of choices for groups of people. By the mid-1990s we began working on disaggregating these groups so that we could get down to the level of very small groups. And a few years ago at CenSoC we made a breakthrough, where we can now model single people.

Using these techniques, we can now model choices for large groups of people by building them up individual by individual.

This progress has been built on insights from many different disciplines including economics, marketing, psychology and transport as well as major advances in statistical models and computation. Since the late 1990s, our work has been immeasurably bolstered by the use of internet panels, through which we can get access to readily available samples of hundreds of thousands of people and deliver results quickly, accurately and cheaply. We can also provide the ability to implement large and complex experimental designs and control for many different factors.

Why is theory important to business?

Business people are often extremely sceptical of theory. Business school ideas are widely thought of as pie in the sky without any application to the ‘real world’. But without a theory, we do not know why people behave the way they do. You might know that the world behaves in a certain way, but you do not know why, so you cannot predict whether it will still work the same way tomorrow. As Soviet leader Leonid Brezhnev said: ‘There is nothing more practical than a good theory.’

Many people think things are too complex or unknowable to model, and clearly human behaviour and decision-making is one of these. But just because choices and decisions are things that people make inside their heads, are informed by a myriad different factors and are hugely complex, doesn’t mean that we can’t model them. We’ve been working on this for more than 80 years and have a deep well of science and technology to draw from.

Despite that, most of what happens in the business world remains uninformed by this. Marketers in particular remain obsessed by strategy – doing something differently, presenting their products in a way that makes them stand out from the crowd or focusing on price and positioning, rather than actually doing their job, which is to accurately represent customer preferences to their organisations.

Economist Steven Levitt, author of Freakonomics, puts it this way: ‘In talking with an auto executive a few years back, I got some insight into why so many new car models turn out to be disasters. I asked this auto executive how his company decided between the 10 or 15 concept cars that the design teams proposed. His answer: The five most senior executives at the company looked over the possible vehicles and picked the ones they liked the best.’

It’s not just automakers. Various studies I have undertaken have revealed that administrators of MBA programs cannot predict how potential students will choose their programs; sales people in the potash industry – a commodity industry if there ever was one – do not understand how farmers buy potash; wildlife managers do not understand how hunters make choices. The results are always the same: there is no consistent evidence that managers understand or can predict their customers at all.

Yet marketers have an obligation to understand their customers’ preferences. How else will they be able to efficiently and accurately meet their needs – the heart of the marketing question? The same is true of the politicians who purport to represent their constituencies. Rather than dispassionately examining what their public really wants, politicians expose themselves to a barrage of lobbying, at best averaging across the mass of interest groups to grope for some idea of what the public wants, when it could not be more different from reality. In these cases, the science of choice comes to the rescue.

How to do it

The foundation of choice theory is that life is about making trade-offs. You can have a product for a lower price, but it means lower quality. You can have better public transport, but it means spending less on parks and public spaces. You can have a politician with strong convictions, but it means compromising on their ability to compromise. Unless there is a cost to a particular choice, people are not forced to order their preferences. In fact, every product or policy is a bundle of options, with consumers choosing between particular bundles. For instance, an airline ticket is a bundle consisting of price, destination, timing, changeability, service level and refundability. Customers understand this, and also understand that they are trading one off against the other. Mobile phones have a bewildering array of features, price points, design and services, and customers make these trade-off choices all the time.

In addition to understanding the trade-off issue, understanding choices means having a theory about how people make decisions about the issue at question. If you have a theory about how they choose, and you design a survey around that theory, whether or not the theory is correct will be confirmed by the data when it comes in. If you have no theory about how people make decisions, any survey you design will be misinformed and nigh on useless. If, however, your theory accords with reality, and your survey design builds on that, the results are immensely useful and have huge predictive power.

When Daniel McFadden invented the first choice models, he predicted within half a percentage point the number of travellers that would eventuate on the Bay Area Rapid Transit system when it opened some seven years later.

People accuse choice modelling of being a bleak and pessimistic pursuit, that its protagonists see people only as bundles of numbers exercising decisions based on their demographics and pre-determined preferences. Instead, I think it is a hugely optimistic endeavour, one that sees people as free to determine their own fates but rational in their actions. Choice modelling analyses life as a series of understandable moments of truth. More immediately, though, it provides real and practical tools to examine a vast array of real life issues in both the public and private sectors.

Choice theory: solving wicked problems

Importantly, choice modelling is a multi-disciplinary endeavour. At CenSoC we have marketers, mathematicians, statisticians, economists, psychologists, physicists and more, all working on the same essential question – how do people choose, and why. We have learned that one discipline cannot do it alone – all disciplines have their biases. It is simply too complex and multi-faceted a question to leave to thinkers of one stripe.

The Argyle company town in Western Australia was established based on a series of surveys undertaken by us to examine people’s work preferences – they told us they didn’t want to commute. A company investment of over a billion dollars in today’s money has paid back many times over. Over the course of the past few decades, we have applied our methodology to a vast ream of real world questions, from the value of reliable electricity supply, how to prevent another Exxon Valdez-style oil spill, and the implementation of itemised long-distance phone bills.

The Centre for the Study of Choice at UTS Business is applying choice theory to many complex problems that affect our daily life, including:

  • Understanding and measuring preferences for future public transport options in Sydney;
  • The relationship between how children spend their time, how that impacts their cognitive abilities, and the public policy consequences;
  • How expert opinion leaders, trendsetters and influentials arise in social networks;
  • Why and how consumers choose wine; and
  • What makes people happy.