Ambient apps

We are now in the era of ambient intelligence, a new phase of the information revolution that will transform how we interact with technology, our surroundings, and society.

Ambient intelligence diffuses information technologies into our local environment to the point where the technology seems to disappear into the background.

Ambient apps built on these technologies act to fulfill our needs with little or no attention. The best known ambient apps are skills for voice assistants like Alexa and Google Assistant. Others include smart home systems like Nest and Ecobee, and, somewhat infamously, security and payment applications built on facial recognition platforms in China.

These ambient apps already engage with hundreds of millions of people, and elements of ambient intelligence in apps like Assistant and Siri reach more than a billion on smartphones. Yet so far the amorphous nature of ambient apps has made this era difficult to recognize as a Next Big Thing.

Just what are ambient apps?

While ambient apps seem amorphous they are evolving from the same core process and so share key characteristics across industries and technologies. Identifying these commonalities helps provide a framework for analyzing the opportunities, benefits, risks, and costs of ambient apps in business, politics, and society.

A thermostat is a primitive ambient app. It senses the ambient temperature, compares it to the preset preference, and takes action to bring a room to the target temperature without user intervention. In this core ambient process, the environment adapts to meet a need.

As ambient apps evolve, they weave increasingly powerful and inexpensive digital sensors together with ML decision-making to transform simple local controls such as a thermostat into super-smart systems that change how we interact our surrounding environment.

Ambient apps operate in the background and require little or no attention. Simple natural gestures, gazes, expressions, and commands just work. Even in primitive ambient apps visual interfaces are rare and discreet. There is no learning curve. Just turn the thermostat. Ask Alexa for the weather or Google Home to set a timer. Approach and a smart lock unbolts. Pay by looking into a camera at a Chinese KFC.

More advanced ambient apps anticipate need. A simple thermostat becomes a Nest thermostat that learns to anticipate future settings based on a range of contextual parameters such as time of day and the weather.

Advanced ambient apps start to adapt to an individual. One example of a smartphone app with ambient capability is Waze, which  anticipates my commute and notifies me when my usual route is stalled. Auras is an ambient smartphone app that makes smart thermostats smarter by learning your individual preferences.  Auras then automatically requests adjustments from any nearby smart thermostats registered on the Auras network– just as Waze anticipated my individual commute, the environment will adapt to your individual need.

Evolving ambient apps will collaborate to coordinate actions and optimize use of shared resources. Auras exchanges preferences with other nearby Auras to find a comfortable temperature for everyone in the room. As people enter and leave, the room adjusts in response to the changed collective preferences.

Ambient apps float. They travel. Places recognize you. Waze will route its notifications through Alexa. The Alexa in your hotel room will know you. Auras operates from your watch in any room. KFCs across China will recognize you and offer choices based on your preferences and budget.

Strategy

Ambient apps remove friction from a service to increase the lifetime value of a consumer. In economic terms, an ambient app removes transaction costs by reducing uncertainty at key decision points, or better still, removes the decision points entirely.

When a need is anticipated and fulfilled, there’s no churn or diversion. It is like a branded moat, but with a different type of awareness. Ambient apps deliver vertical integration of consumer decision-making through a new blend of operations, branding, and technology.

Design

The best ambient apps will transform a product or service into an experience, and experiences into something ethereal. When needs are met without effort or thought, all that remains is the senses, which are free to focus on the experience. The ideal experience then defines the consumer’s conception of the need.

The world’s best hotels have been doing this for years, relying on the intelligence, diligence, and training of their employees.  Years ago, at the Mandarin Oriental hotel in Singapore, I slipped out a side door for a jog and on my return an hour later a doorman met me at the base of the drive with a towel and bottle of water. He had noticed, remembered, and anticipated a guest’s need without a word.

What gift do you think a good servant has that separates them from the others? It’s the gift of anticipation. And I’m a good servant; I’m better than good, I’m the best; I’m the perfect servant. I know when they’ll be hungry, and the food is ready. I know when they’ll be tired, and the bed is turned down. I know it before they know it themselves.

Mrs. Wilson, Gosford Park, 2001

Ambient apps reduce the cost of delivering this personalized, anticipatory experience. Disney’s MagicBand and Carnival’s Medallions are early movers, using ambient intelligence to deliver personalized, friction-free experiences to thousands of people a day.

Done well, ambient apps often seem revolutionary, magical even. Yet the magical experience, and the ambient apps, are really outcomes of improving a process of meeting a well-defined consumer need. The magic emerges from the clever application and integration of some amazing new technologies into this process.  

Form

In an ambient app, an array of local sensors sends data to a decision node that recognizes a need and executes actions to fulfill that need. The simple thermostat embodies the core process flow.

A more sophisticated flow will also anticipate needs and collaborate to optimize outcomes. To learn, the process may iterate and solicit input from a person or sensor.

What is new in this process in this era is the scale, scope, and capabilities of the local sensors and the decision node.

The most critical is the rising density of smart, networked sensors communicating local environmental data in real-time at low latency. Some sensors will be simpletons, such as networked thermometers or motion sensors. Others will be ML-enabled, such as cameras and speakers with image and voice recognition, and will provide data at a higher-level of abstraction. The latter trend is revolutionary.

With this data the decision node builds a contextual understanding to anticipate needs and execute actions to fulfill meet the need. Like the sensors, the decision node can be a simple function or ML-enabled. Simple decision rules acting on high-quality, precise, low-latency sensor data will generate highly “intelligent” outcomes.

This decision node may reside in an environmental device, the cloud, smartphones, or wearables. Disney’s MagicBand and Carnival’s coin-like Medallion serve as smart sensors feeding decision nodes in the cloud. In the Auras app, smartphones and watches provide both the highly-capable sensor arrays and the decision nodes.

Decision nodes trigger actions. Some actions may be automated, such as a thermostat controlling an HVAC system, or an Alexa ordering through Amazon. Other actions will require people to reduce friction or shape an experience.

In practical terms, building ambient apps in the near future will integrate various environmental sensors with a cloud- or smartphone-based decision node. Other solutions may leverage the sensors built into smartphones and wearables. Some ambient apps will be 100% digital and automated, like a smart thermostat, completing the entire process of recognizing and fulfilling a need. Other ambient apps will be hybrids that make it easier for employees to provide the personalized “human touch” that defines a consumer’s experience.

This blog post is the first in an ongoing series examining this era of ambient intelligence and the emerging opportunities, challenges, benefits, costs, and risks primarily in the business sphere.

Links and articles

Walt Mossberg’s last column, the disappearing computer

A good overview of ambient intelligence and its future impact

European Commission’s ISTAG and Philips defines ambient intelligence, loosely

A TechCrunch overview of ambient intelligence from 2016

An interesting blog post about why ML will be in tiny devices and sensors

Google’s Edge TPU doing exactly that, adding ML to embedded devices

Wired’s in-depth look at Disney’s MagicBand  

FastCompany’s article about the development of MagicBand, includes interesting information about the corporate tensions that emerged

A quick look from Harvard Business School at MagicBand’s impact on Disney World’s operations

An interesting article from FastCompany about how Carnival’s Medallions aim to go further than MagicBand

 

Apple Watch’s ambient app

Apple announced two significant ambient apps for the Apple Watch this past week, the Fall Detector and Heart Rate monitor. In keeping with the amorphous nature of ambient apps, the word “ambient” never appeared but ambient they are and worth analyzing through that lens. For this post, I’ll focus on on the Fall Detector, though the commentary could apply equally to the Heart Rate Monitor.

This is a next big thing?

In a presentation the Fall Detector was underwhelming– it’s difficult to get excited about people who have fallen and can’t get up. Yet there’s massive value there, which will be clear when an ad one day appears recollecting how the fall detector saved a life. Hidden value is characteristic of many ambient apps because when an ambient app is any good you won’t even be aware of it at all.

As an ambient app

The Fall Detector can be analyzed as a wearable app, a healthcare app. It’s also worth examining the Fall Detector several common elements shared by ambient apps.

  1. Executes an ambient decision cycle
  2. Requires no user awareness or interaction
  3. Transforms a Product → Service → Experience

Executes an ambient decision cycle

The Fall Detector uses advanced sensors to detect a hard fall, determines if you are ok, and if not, sends an emergency call and notifications to people who can help. So it’s definitely an ambient app: it recognizes a need for help, anticipates how to provide that help, and takes action to contact people who can help– all without user awareness or intervention.

Collaboration: an opportunity

In the current app, there’s collaboration with established contacts but none with other devices. It’s possible that in the event of a fall people nearby might notified to serve as good samaritans. They might be other people with the app, the watch, or even other devices in the Apple ecosystem such as the iPhone.

This offers an interesting dynamic. If you wear the watch and engage the app, you automatically become part of a mutual support system. Taking it a step further, the app could also coordinate a response between nearby actors– it could determine who’s best qualified, who’s capable, who’s available, and what further instructions you need to provide real help. This could be a great way for advanced ambient tech to coordinate and foster humanity’s better instincts and take smart wearable tech beyond the narcissistic selfie or FB post. For Apple, it would be another terrific reason to join its ecosystem.

Requires no user awareness

As with other ambient apps, the user interface is not involved in the Fall Detector’s operation– it’s only there for setup and settings. The user is completely unaware of the app’s function until it brings help.

Product → Service → Experience

The Fall Detector app also transforms how a need (help after a fall) is fulfilled (in many instances, help is not provided at all). It shifts the form of need fulfillment from a product or service (e.g. a Life Alert device) to an experience that requires no conscious effort. In the case of a serious fall, a service is engaged on your behalf, yet all you’d know about is the experience of receiving help.

What to watch

With the Fall Detector Apple adds value and revenue streams to the watch in the near-term while continuing its careful steps towards building health care related revenues. From an ambient app perspective, it’s evidence that ambient apps could find early traction in health care applications, where the delivered value is very high and user awareness and intervention may not exist in any current form. It’s also indicative that for decentralized ambient apps, wearables will be a significant platform.  And the Fall Detector hints at a corollary: ambient apps may deliver the consumer value required for wearables to fulfill an early promise and emerge as the next mass consumer device.