Sally Applin: Embedded, Embodied, Location Aware and Communicating

I recently discovered Sally Applin’s work, having been vectored there by Rethink Robotics blog posts. I’m intrigued by her ideas on blended/augmented reality experiences, and want to capture some thoughts on them.

Her papers posit that we’re at the cusp of an inflection point of how our experience of reality is structured, and ask: how will we build ourselves into it, what could it involve, and what might it look like? In some ways it’s the dual of McCullough’s ambient commons ideas. McCullough talks more about the affordances provided (displays, etc.,) while Applin talks more about how things will be enabled.

So what are her ideas/observations? Lets look at some papers in more detail:

Pervasive Computing in Time and Space: The Culture and Context of ‘Place’ Integration ¹

This paper is from 2011, back in the day when Location Awareness was new and hot (in trying to understand why it took so long to come on the scene, my best explanation is that the CDMA iPhone was released that year  — at that point it became clear that enough people would have these devices to allow for interesting things to happen, enough to both get the necessary data and have a market large enough to be profitable)

The article mentions the color app, which enjoyed much early hype in this space (best described by tech crunch here)

But the beauty of Color stems from what it’s doing differently. Unlike Instagram and Path, there isn’t an explicit friend or following system — you don’t browse through lists of contacts and start following their photo stream. Instead, all social connections in the application are dynamic and established on-the-fly depending on whom you’re hanging out with. And your photos are shared with everyone in the vicinity.

It was a time when we knew something big was happening but didn’t have any idea what it would feel like, or what it would do. Color looked like an answer, but no one was asking that question (it’s not particularly relevant that color was a bad app. If the “question” was of interest, something else would have come along, c.f., Friendster → MySpace → Facebook)

The greater the extent to which communications technology is pervasive and people connect to those networks, the more engaged they become in ‘non-place’ and the more ‘solitary contractuality’ they have within the place that they are physically in at that moment.

For clarity – currently (~ mid 2015) I don’t think we know what a useful thing in this space would do, but we are getting a sense of what it will feel like — certainly solitary contractuality is an apt term for becoming shifted/disjoint/askew from our normally spatially embedded and constrained embodiment to an embodiment that is embedded in a different space which, yes, currently is the result of solitary contracts/choices. However that could easily shift to something richer. This seemed to be part of what color was trying to do, give us an opt-in shared network to hook into, and get an enhanced perspective. This is also part of what McCullough is thinking about: giving us an opt-in network that allows sharing into a spatially embedded, generally available ambient space.

(of course, ads seem to be the fist out of the gate with this  — both unsurprising and a bit creepy).

PolySocial Reality — the full title of the paper is PolySocial Reality: A Conceptual Model for Expanding User Capabilities Beyond Mixed, Dual and Blended Reality. ²

Is a taxonomy building upon the prior Pervasive Computing paper: the axes being

• The number of people involved: 1, or 2+
• Physical locality
• People you know/vs familiar/similar

It groups synchronous and asynchronous since they aren’t germane to the analysis. The analysis is from the perspective of a UX designer, so the familiar/similar attribute doesn’t indicate that the users are known to each other, but rather that they are a known type and are of types that are familiar to the system and similar enough so that assumptions will hold.

Thus, PoSR describes the network transaction space that humans are inhabiting themselves and with others in order to maintain their relationships and engage in new activities with collective dependencies via the social mobile web.

PolySocial reality is how all of these things interact: lets say on a block, in a square, or in a park. This was published right around the time Grindr was taking off and a few years after Foursquare hit. Generally, these apps weren’t very successful — only the “dating” ones succeeded, e.g., Foursquare changed focus, while Grindr grew to spin off Blendr and address other verticals. All of these apps address similar users embodied at a particular location with an incentive to use the affordances provided by locality: aka there’s a reason that people close by are of interest, and the reason is motivating.

For applications to occupy this challenging spot in the taxonomy, sufficient incentives for use remain to be identified. The only one that I’m familiar with the government alerting apps that tell you if a severe thunderstorm etc. is coming. Unsurprisingly, since I’m unaware of anything that achieves this is 2015 her examples in 2012 are all limited examples of the concept.

Thing Theory ³

This paper is starting to take the next step: how would such a reality start: what it would do; how would it work? It uses the metaphor of the Thing from the Addams family TV show. Such a system would have an understanding (model) of each resident: what they wanted to do and when. The characterization of a smart environment is especially apt

An environment, to be truly smart, must learn from the cumulative data within its realm to understand and guess what likely choices might be for a given agent and then facilitate or enact these on behalf of that agent.

The concept is that it is a meta-agent controlling other systems based upon all of its knowledge. Interestingly (& accurately) the model requires that the system have a model of the intention of the residents. Without such a model, it can’t be helpful, since it wouldn’t have a good idea of when it might be wrong in its anticipation of the resident’s intentions. With such a model, it can prompt the resident to select from a set of choices, or alternatively, quickly unwind a mistake by selecting the “second choice” when the resident indicates that it has made an error.

Toward a Multiuser Social Augmented Reality Experience: Shared Pathway Experiences via Multichannel Applications ⁴

This is an advocacy paper, pushing for a richer, more social (multiuser) augmented reality experience. The prime mode identified is story-based and of the applications discussed the only one that still appears operative is the Magic Bus Tour which is a Augmented Reality enhanced tour of 1960’s San Francisco. This provides a cautionary tale about story-based efforts:

• They aren’t trivial to pull off well: great storytellers are comparatively rare
• They don’t wear well: even for the best stories, few get “repeat plays” (viewed multiple times by the same user — unless the target market is children)
• The technology becomes obsolete quickly — mostly because it’s on the cutting edge, where standards are the most volatile. This, combined with the previous factors, implies that they don’t last long.

McCullough, on the other hand, presents the dual of this approach: PolySocial reality as slow moving ambient information, the “window looking out on a sunlit stone wall” — we all share it, and it is highly designed. However, it’s ambient, slower moving and slightly in the background, acting as a substrate for our less focused attention.

There are a few ambient environmental video pieces up in the Boston area (Greenway, Convention Center & probably others), but they are programmed rather than reflecting a “real time PolySocial reality.” The distinction being that between “fixed content,” e.g., a movie (even a long Warhol movie) running in the background and something dynamically generated, such as a display of the ships currently traversing the harbor. The promise of PolySocial reality would incorporate ambient (local, current) information from dimensions not normally available (who’s also going here, how many people are at a non-visible location, GPS of the ships), into perceptible networks whether they be ambiently available or opt-in (apps/Oculus Rift displays).

The key here is in finding what an old colleague of mine called the carrot: “why would a person actually want to use it” The greater the effort, the bigger the required carrot. I like the term carrot better than use-case or similar since it not only points to the “what’s in it for the user” aspect of the problem but also highlights the whole use vs. cost issue.

From this standpoint, the best example I can think of for individual users are our phone map apps that show real time traffic data, since the carrot is large, and the cost is low (although I guess the signs that say “exit y in x minutes” are similar, with even lower cost, even if the displays seem so low tech).

If we shift from the standpoint of “the people with the smartphones” being the agents making the choices, to “the people with the smartphones and their eyeballs” as the product being delivered, the tradeoffs become substantially different, and there is a tension between the two that are profound: In the “user as product” model the costs/benefits are focused on a few large players using/distributing the app. These players are easy to identify, talk with and iterate with as necessary. This is a much different scenario from the end user being the beneficiary, a situation in which the costs/unit are likely higher (since their need assessment and iteration cycles are less direct) likely requiring that the user bears significantly more costs, possibly including out of pocket expenses (aka the user buying the app).

A similar “ad” type of cost structure would be available for “ambient public art,” where the cost is borne by a public institution. Although the available $ wouldn’t be has high as the advertising case, they could be sufficient to fund interesting developments. The ephemeral nature of many of these opportunities (event/festival/semester based, etc.), would allow for the projects to be much more ad hoc and experimental, without incurring any of the costs associated with developing a “product” that would be expected to require long term support.