fields

Documentation of Duncan and my workshop / performance / intervention at the DOMe IoT workshop at Ubicomp2012 in Pittsburgh in September. Thanks to all who take part, it was a small part of a small workshop but for Duncan and I proved very successful. Watching the guys listen to the products for what must have been over 30 minutes was surprising on a number of counts:

1. Watch how loyal the people are to listening and obeying the products

2. Each script was part of a series of steps in fulfilling a larger task – organising things into the correct collection for a specific recipe. However the scripts were pretty linear and could easily have gotten muddled up if people scanned a product too many times, but people figured it out!

3. Even when our scripting wasn’t perfect people were still determined to organise the things according to what they requested. In the final act I think we messed up the actual order, but some participants were determined to keep the ‘things’ happy!

Enjoy…

Below is the position paper that Duncan and I wrote that got us in to the workshops. Duncan is the smart one for the agency discussion…

Take Me I’m Yours: Mimicking object agency

Chris Speed and Duncan Shingleton

ABSTRACT

This paper speculates, through a design demonstration, upon a future context in which objects will begin to talk to us and even give us instructions. The purpose of the research is to anticipate a time when correlations between the data sets that are associated with different objects are found and the objects themselves are used to impart this ‘new’ knowledge back to us. Such an occasion may be considered to represent a form of agency.

Located within the technical and cultural context of the Internet of Things, this paper introduces a lineage for our relationship with objects from 1. Read Only, 2. Read and Write and 3. Read, Write and Act. The paper proceeds to establish the conditions for a third generation of Internet of Things by articulating the nature of networks, their structure and their capacity to support the principles of Actor-Network Theory which may lead to a condition in which objects may take on a form of agency.

The paper further introduces an iPhone App entitled Take Me I’m Yours that operates as a working but speculative design project mimicking the conditions in which objects may talk to us. The designers speculate a design fiction in which object databases may begin to identify associations and propose ‘actions’ to a user. The application and demonstration at UbiComp 2012 will offer delegates an opportunity to experience a sense of what it may feel like in the future when objects may begin telling us what to do.

Author KeywordsInternet of Things, design futures, agency.

ACM Classification Keywords

Design, Theory, Experimentation.

INTRODUCTION

Take Me I’m Yours is a third generation Internet of Things (IoT) artwork that evokes ‘actions’.

1^st Generation – Read Only

The first generation of IoT technologies simply recall immaterial data that is associated with an artefact when it is scanned. Barcode scanners in supermarket checkouts, or near-field scanners used to check passports at airports, devices ‘read’ tags and codes and recall data upon that item from a networked database.

2^nd Generation – Read and Write

Second generation IoT technologies allow for the writing back of data onto a database via a tag. Systems such as www.talesofthings.com or www.stickybits.com allow consumers and owners of artefacts to ‘write’ information back to a tag, and others to recall and comment further.

3rd Generation – Read, Write and Act

A third generation IoT developed by the artist and designers of Take Me I’m Yours involves not only the reading and writing of/to tags, but the passage of instructions and actions through objects to facilitate their movement through space/place. The tendency for the first two generations of IoT is that objects are not shared in the actual world – only the immaterial data that they are associated with.

Context

Take Me I’m Yours responds to the adoption of barcode scanning software that has become available for smart phones. Previously restricted to use at supermarket checkouts, and connected to local and closed databases, the traditional barcodes that are attached to consumer products have become useable by smart phone users. Software such as Red-Laser, Food Scanner and ShopSavvy available for Android, iPhone and Blackberry smartphones allows users to access data associated with barcodes. Companies such as Google Merchant and Amazon support connections between the two barcodes models: UPC (US and Canada) and EAN (Europe, Australia, South America, Africa), and make it possible for the public to link to common product information. Scanning a product with a barcode reader allows users to compare prices across a range of stores, check availability in second hand stores and even give location based information about where the nearest store may be in order to purchase the same product.

More recently, applications such as StickyBits (http://www.stickybits.com) have allowed users of smart phones the facility to scan a product code and attach their own media to it. Short text stories, photographs or videos can be posted to the StickBits database and made available for others to read when scanned again using the same software. Turning the barcode into a media channel, mobile phone camera scanners offer companies and individuals a conduit through which marketing materials can be fed, and social data can be attached.

Whilst StickyBits tends toward using codes that are shared across whole product lines, other platforms offer the public a chance to create new codes for unlabelled items, and tag them with memories, stories and media content. The Tales of Things system (http://www.talesofthings.com) allows individuals to pick a single item and attach social data to it through a website that then generates a unique barcode for them, so that others who come across the object can retrieve that data. Aimed at encouraging the public to record personal stories onto objects, the Tales of Things website demonstrates that some objects that are moving through the world are not only containing quantitative data to ensure product integrity and ‘freshness’, but are beginning to contain qualitative data that is intended to affect how users interpret and use physical objects.

Reflecting upon the development of the consumer experience, from ‘reading’ codes to beginning to ‘write’ on codes, participants of the Heritage Inquiries: A Designerly Approach to Human Values workshop at DIS 2010, began to hypothesise about the next user experience: a context in which objects begin to ‘talk back’. Developed as a ‘sketch’, the Take Me I’m Yours idea speculated a scenario in which ‘things’ may begin to gain a level of agency and start to demand actions of us. The cultural and technical phenomenon in which all objects are connected across networks, and branded as the Internet of Things, is largely framed as a relationship between scanners, tags and databases. Take Me I’m Yours anticipates a context in which connections between databases may lead to emergent properties including assumptions to be made about the needs of a user, or even the needs of an artifact itself. A design fiction in some respects, the workshop attendees recognised that whilst the development of a digital system from which emergence develops may require resources beyond the reach of the team, developing a platform that simulates the experience may not be so difficult.

Figure 1. A representation of a future context in which objects will begin telling us what to do.

NETWORK: STRUCTURE AND AGENCY

The Internet of Things is a starting point for rethinking our relationship with the physical world, as we begin to imagine scenarios where the physical and digital spheres collapse onto each other. Extrapolating from the propositions of Anders (2001), Sterling (2005) and Bleecker (2006), objects in an IoT can be viewed as inheriting the following characteristics:

• · they have multiple trajectories through time,
• · they become an interface for data storage and retrieval – they have memory and they can publish,
• · they are locatable in space,
• · they have an ‘awareness’ of the environment they inhabit.

Through adding a whole array of everyday things as nodes of the Internet, that begin to deal with the notions of space, time, memory and agency, new connections arise that lead us to question the composition of a network.

Networks are formally defined as a set of nodes or network members that are tied by one or more specific types of relations. By focusing on the key idea of actors and how they are connected, we gain insight into the structure of social interactions on an individual level as well as on that of the groups and institutions concerned.

Structure

Lopez and Scott (2000) as cited in Stones (2007:4870) argued that there are two primary ways of conceptualising structure, both deriving from Durkheim. The first is the relational notion of structure, referring to networks of social relations that tie people together into groups and social systems. George Simmel similarly emphasised relationships, conceiving of society as a dynamic of complex social forms and interactions. These may involve smaller or larger numbers of people, or specific types of association, which structure the way which agents behave in one another’s presence. Norbet Elias’s figurational sociology likewise emphasised the webs and networks of relationships within which individual agents act.

The second notion of structure, the institutional, refers to the beliefs, values, symbols, ideas, and expectations that make up the mutual knowledge of the members of a society and allow them to communicate with each other. Durkheim (1984) referred to this dimension of structure as a society’s ‘collective representations’.

Both approaches to structure are compatible with another metaphor routinely associated with structure: pattern. The notion of pattern is often included in the very definition of structure. For example, social structure may be seen as ‘a system of patterned relationships of actors in their capacity as playing roles relative to one another’ (Parsons, 1945, cited Stones, 2007:4870). The key notion here is the relationship of the actors, therefore it is important to identify what can be considered an agent in the structure.

Agency

A crucial feature of an agent within a structure is that it can interact; it can pass on informational messages to other agents and act on the basis of what it learns from these messages (Gilbert, 2008:5). Gilbert notes that these messages may represent a dialogue, or a more indirect means of information flow, such as the observation of another agent or the detection of effects on another agent’s actions.

Agents are conventionally described as having the following four important features (Wooldridge & Jennings, 1995, cited Gilbert, 2008:21):

• Autonomy – there is no global controller dictating what an agent does, i.e. it does whatever it is programmed to do in its current situation,

• Social Ability – they are able to interact with other agents,

• reactivity – they able react appropriately to stimuli coming from their environment,

• Proactivity – an agent must have a goal or goals that it pursues on its own initiative.

If we can understand a networked object as capable of possessing agency, as a result of an amalgamation of characteristics outlined by Anders, Sterling and Bleecker, then it is important to further deconstruct these features; as without consciousness then it is unlikely an object will ever posses autonomy or proactivity due to the need of human intervention to govern these two properties. Thus, a more helpful way of describing agency, within this context, is through the following characteristics derived from Gillbert’s (2008:11) agent-based modeling:

• Perception – they can perceive their environment, possibly including the presence of other agents,

• Communication – they can send messages to and receive messages from other agents,

• Memory – they have a memory, which records their previous states and actions,

• Policy – they have a set of rules, heuristics, or strategies that determines, given their present situation and their history, what behaviours they will now carry out.

This model of agency is based on theories relating to Actor‐Network Theory. Actor-Network Theory insists on the agency of non-humans, although it is critiqued that the properties outlined by Wooldridge & Jennings relating to autonomy and proactivity, or in other words intentionality, fundamentally distinguish humans from animals or things. In the context of Actor‐Network Theory, agency is not located in either human “subjects” or in non‐human objects, but in heterogeneous associations of humans and non-humans; it neither presupposes intentionality nor is assigned to non-humans.

Actor-Network Theory (ANT)

The Internet of Things therefore can be most closely linked to Actor Network Theory (ANT), a method for mapping the patterns of techno‐science, where networks become the substance out of which both individual identity and social organisation are constructed. ANT can be seen as a tool for exploring and describing how the social is assembled by way of technologies; objects and artifacts (Latour, 2005), and its import is one of agency, specifically responsibility that is distributed equally across entities, including a host of non-human ones not normally seen as exercising agency at all (Bijker & Law, 1992; Latour, 1999).

In the Actor‐Network Theory’s analytical frame, also known as the ‘Sociology of Translation’, reality is observed through interactions and is considered as the effect of heterogeneous networks. As the actors in the network can be both human and non‐human, actor network theorists sometimes use the term “actant” to refer to such actors. Society, organisations, agents, and machines are all effects of patterned networks generated through the interactions of actor‐networks (Law, 1992). In other words, they are formed by the relationship between intermediaries ‐ “anything passing between actors, which defines the relationship between them” (Callon, 1991:134), actors and translation ‐ the process of simplification and punctualisation that helps us to understand the complexity of the relationships involved.

Latour (2005) argues that the observation of the social can only be achieved by tracking the traces it leaves when an association is being produced between elements and, therefore, ANT offers an appropriate methodological approach for analysing networked objects in the context of the IOT; objects as actors in the network, considered to have a role within it that extends beyond their material form.

Summary

As objects go online they could create a new layer of complex relationships that were previously not visible in our networks. By allowing us to examine the objective pattern of interactions represented by how people to people, people to things, and things themselves are connected to one another, we will possibly gain new insights into the structure of social interactions. The structure of a network, the relations among network members, and the location of a member within a network are critical factors in understanding social behaviour. Complex, dynamic social systems are analysed in terms of stabilising and destabilising mechanisms, and traditionally it is only human agents who play strategic roles in these processes. Institutions and cultural formations of society are carried by, transmitted and reformed through individual and collective actions and interactions. These social structures help to create and recreate themselves in an ongoing developmental process in which collective agents play constructive as well as destructive and transformative roles in the context of complex sociocultural arrangements. These arrangements of social life involve time, space and place as constitutive factors in the construction and reconstruction of what people do and in the way they do things together, as active agents, with their distinctive characteristics, motivations, and powers contributing to the reproduction and transformation of our networks.

The Internet of Things provides a possible framework that allows not only human agents but also object agents to constitute and reconstitute time‐space, place and cultural forms through their interactions.

Demonstration

Take Me I’m Yours is an iPhone app that allows users to read a traditional barcode that is associated with everyday consumer items. Upon scanning a code the user is prompted with an action to do something with the artefact: “Give me to your neighbour”, or “Take me to work with you”.

Through actions that correspond with ‘real world’ contexts, Take Me I’m Yours encourages the movement of things through people, places and circumstances to provoke new histories and question the perceived function and value of artefacts. When the Cornflakes packet is browsed at home by a family and it says “Turn me inside out and design your own packet”, the artefact is given a voice that provokes a self-transformative action.

The artists would like to demonstrate the Take Me I’m Yours system in Pittsburgh during UbiComp 2012. By associating actions and instructions with a series of consumer products, each item will be able to recall an instruction. The demonstration will include a shelf containing a range of popular consumer items that all have their own bar code. Visitors to the demonstration will be invited to use their own smart phone to scan artefacts and listen to and follow the instructions from their phone. The design team will replenish the shelves throughout the demonstration and we can expect to see objects move throughout demonstration event and possibly out into Newcastle. The actions have been written by an established artist/performer and are intended to project a sense of agency on behalf of an object:

Take me to the window so I can see the mountains.

Sit me on the windowsill with the best view north.

How do I look in your house?

Open me when the sun goes down.

Hum your favourite song to me.

Hold me close, I’m scared of falling.

Conclusion

In conclusion the Take Me I’m Yours demonstration is intended to provide a playful insight into a future

relationship with objects in which they begin to gain agency. The value of this working ‘design fiction’ is to frame the history of users relationships with barcodes as they have moved from closed supermarket systems to more open internet based experiences.

ACKNOWLEDGMENTS

Take Me I’m Yours iPhone application was developed by Chris Speed, Duncan Shingleton and Kristin Mojsiewicz. Take Me I’m Yours was conceived by: Rachel Clarke, Christian Dindler, Daniela Petrelli, Rachel Charlotte Smith and Chris Speed during the Heritage Inquiries: A Designerly Approach to Human Values Workshop, DIS 2010 organised by Elisa Giaccardi and Ole Sejer Iversen.

REFERENCES

1. Anders, P., Cybrid Landscape [online] Available: http://www.uoc.edu/caiia-­‐star-­‐ 2001/eng/articles/anders0302/anders0302.html [date accessed: 20 August 2010]. 2001.
2. Bleeker, J., A Manifesto for Networked Objects – Cohabitating with Pigeons, Arphids and Aibos in the Internet of Things. University of Southern California, California, 2006.
3. Sterling, B.,.Shaping Things, The MIT Press, Cambridge, 2005.
4. Bijker,W.E. & and Law, J. Shaping Technology/Building Society: Studies in Sociotechnical Change. MIT Press, Cambridge, 1992.
5. Callon, M. Techno-economic networks and irreversibility. In: J. Law, ed. A Sociology of Monsters: Essays on Power, Technology, and Domination. Routledge, London, 1991, pp. 132-161
6. Gilbert, N. Agent Based Models. Sage Publications Ltd, London, 2008.
7. Latour, B. Reassembling the Social. Oxford University Press, Oxford, 2005.
8. Latour, B. Pandora’s Hope: Essays on the Reality of Science Studies. Harvard University Press, 1999.
9. Law, J. Note on the theory of the actor-network: ordering, strategy, and heterogeneity. Systems Practice, volume 5, number 4, 1992, pp. 379‐393.
10. Lopez, J. & Scott, J. Social Structure. Open University Press, Buckingham and Philadelphia, 2000.
11. Stones, R. Structure and Agency. In: Ritzer, G. ed., 2007. The Blackwell Encyclopaedia of Sociology. Blackwell Publishing, Oxford, 2007.
12. Wooldridge, M. & Jennings, N. R. Agent theories, architectures, and languages. In Wooldridge and Jennings, eds. Intelligent Agents: Springer Verlag, London, 1995. p.1-22.