purple motes

Google recently made website optimization freely available to everyone. Perhaps this truly is a key element in Google's plan for world domination. But in any case, it's good news for those times when you want to choose data analysis over whimsy and hope.

A danger of free, fast, and easy testing might be that its users will become complacent about the background structure of testing and optimization. Statistical tests always depend on background assumptions about data-generating processes. For example, when the number of users of a website increases by orders of magnitude, the attributes of its users (age, sex, internet experience, etc.) may change. What had been an optimal content design may no longer be optimal.

Economists have long pondered problems of behavioral estimates. To get a sense of the issues, suppose that you observe that over the past ten years both the price of widgets and aggregate sales of widgets rose strongly. That doesn't mean that raising the widget price will increase widget sales. More likely the opposite is true. Increasing demand for widgets may account for the correlation between higher prices and greater sales.

The problems for human behavioral estimates are far worse than those of omitted variable biases in other physical systems. Humans are good at collecting and sharing information, and humans often change their behavior in response to new information, new rules, and new incentives. In economics, the Lucas critique is a famous recognition of this reality for parameter estimation.

Among empirical economists, a commonly expressed goal is to estimate "structural parameters." Structural parameters are interpretable parameters relatively insensitive to plausible changes in the environment and rules of the game.[1] When you optimize with respect to such parameters, your optimization is more likely to be valid over time. Moreover, since you can interpret the parameters, you can anticipate changes in the environment that are likely to invalidate your optimization.

Economists have produced a huge, complex literature on structural estimation. But probably all you really need to know from it for website optimization are a few points:

• Website optimization is not forever. Changes in circumstances (New Year's Day promotions won't work year-round), changes in user characteristics, changes in the experience of users with other websites, and changes in your business reputation may change the optimal configuration of your website.
• Seek some understanding of what works for you. Testing can help you discover what works. But sound interpretation and knowledge remain valuable. Understanding why a particular configuration works can give you insights into when it might need to be changed (or re-tested). In addition, understanding what works might provide you with more general insights into your users.
• Expect your users to learn and respond to what you do. If an aspect of your optimization involves baiting and screwing your users, they will learn about it and adapt to it. Your optimal screw of your users can turn to screw you.

Note:

[1] To economists, parameters describing preference functions (demand) and cost functions (supply) are structural parameters. That economists consider parameters describing preferences and technology to be relatively stable may seem laughable to persons engaged in viral marketing and rapid web service development.

1 Comment

The largest share of value in communications services is the value of presence. How can communication services providers measure their performance in capturing this value?

Average Revenue Per User's Relation (ARPUR) is a practical measure of presence value. ARPUR is Average Revenue Per User (ARPU) divided by some measure of user's interaction with other users (relations). Such a measure might be the least number of users who account for in total at least 50% of the given user's communication sessions, time, or revenue. The higher the ARPUR, the more the communication service is creating value through presence.

Persons typically value most highly the presence of family and friends. Limitations of time and attention, which good communication services can help to relax, constrain the number of family and friends that a person can sustain in daily interaction. The value of communication with the family and friends that persons do sustain is typically high and enduring. A good business plan for communication service providers is to capture a large share of this value. ARPUR is a metric of success in doing this.

While not often recognized as such, telephone service is a quintessential presence business. A study in the U.S. in the 1970s found that 50% of residential calls go to a set of five numbers. I think this has been roughly true for personal telephone service in most places throughout the history of telephone service. Creating more value in these relations creates value in this kind of communication service. It's a presence business.

For contrast, consider an anti-presence communication service: telemarketing. Telemarketing involves mass distribution of information of interest only to a small number of persons. The telemarketer typically does not know any of the persons whom she contacts and does not typically repeatedly contact them. Moreover, most of her contacts probably wish that they did not know that she existed. A good communication service for telemarketing users might have a high ARPU. But its ARPUR would be near zero. It's not a presence business.

ARPUR might help a new communication service provider steer its business between the imperatives of viral marketing and the long-term value of presence. Viral marketing, like infectious diseases, propagates most rapidly with some highly promiscuous agents. A communication service that wants to succeed virally needs to enable promiscuous agents. On the other hand, promiscuity is inconsistent with large presence value. The business challenge might be to manage change from low initial ARPUR to strongly rising ARPUR.

Suggested analytical exercise: Consider ARPUR for portraiture over the past 500 years. Take the user relation to be the gift of a picture of oneself to another person. What has been the trend in ARPUR? What has been the trend in total portraiture industry revenue? For relevant information on the economic history of the photography business, see Photographs and Telephone Calls in Sense in Communication.

Take-away message for busy communications executives: Get out of the telecom toilet and get your business purring. Stop sniffing ARPU and start making ARPUR!

1 Comment