fields

ThingTank

Way back in September 2013, I joined a week long ideas retreat in Moscow to develop solutions toward the design of new innovation platforms. Funded through the MIT-Skoltech initiative, part of the Skoltech, the week seemed to respond to the following question: “How can an institution or government seed, grow, and maximize technological innovation and entrepreneurship?”

Through collaboration with Elisa Giaccardi, TUDelft and Neil Rubens, University of Electro-Communications, Tokyo, we cooked up the ThingTank project which proposed that data produced by the practical use and reuse of objects is an untapped potential for innovation. We suggested posited the question how can we mine our material interactions with things to seed and produce new cycles of innovation?

From this starting point We are proposed the development of a platform called ThingTank at the intersection of machine learning, ubiquitous computing and social interaction design. Through an integrated methodology that identifies the use and instrumentation of objects within specific social and industrial contexts, a digital platform would be developed that analysed data derived from the human use of objects to identify emergent patterns, which in turn would inform the development of innovative prototypes.

Much more detail can be found on the ThingTank project site and the below is ‘speedy’ summary of the research.

Background
Owners of things constantly invent novel usages for objects, but often do not have needed resources and expertise to develop products with the novel usage in mind.  On the other hand, product designers are often not the users of the objects; so it is hard for them to imagine novel and meaningful usages.  The ThingTank research is interested in how the mining of data can be used to bridge this gap to discover novel usages by consumers (by analysing sensor data); and use discovered product usages to inspire product designers.

We were particularly interested in Richard Wentworth’s long art project Making Do and Getting By through which he ‘observes the ingenuity of humankind in the appropriation and adaption of everyday objects for new uses, new meanings, and new narratives’. These situated tactics reveal many of the uses for objects that on one hand appear anomalous to their designed use, but make perfect sense when recast in particular contexts. One of the questions for ThingTank was: if your could mine all of the anomalies of use across objects within the Internet of Things (such as a wellington boot being used as a door stop) could you share these uses in such a way as to propose novel designed objects or services?

The work could be part of the techniques that anticipate Gartners prediction: “By 2017, a significant disruptive digital business will be launched that was conceived by a computer algorithm.” Gartner Report 2014, www.networkedworld.com, October 2014

Context
In the past many IoT projects have used the network connection of artefacts to identify cost saving and process efficiencies (eg. vehicle manufacturers), or to track goods within large networks (eg. logistics companies), or to monitor the health and safety of systems (eg. aircraft manufacturers). As these systems move from business-to-business, to business-to-consumer there is a significant push to build interoperable systems that will allow different branded ‘things’ to talk to each other across networks (Garun 2013). Within each of these cases the scale of data that is being streamed from devices and objects that are tagged with sensors is enormous. Data about performance including a host of parameters is streamed to databases that are growing at exponential rates:
“about 90% of all the data in the world has been generated in the past two years (a statistic that is holding roughly true even as time passes). There are about 2.7 zettabytes of data in the digital universe, where 1ZB of data is a billion terabytes (a typical computer hard drive these days can hold about 0.5TB, or 500 gigabytes). IBM predicts that will hit 8ZB by 2015.” (Arthur 2013).

As networked objects become more common the amounts of data that they collect will soon outweigh what we know about the physical device. As artefacts share information with the other artefacts around them, code can be written to interrogate their shared use. Machine learning is being used across a wide variety of databases to identify patterns in order to elicit new insights (Bandyopadhyay & Sen 2011). As the databases of objects intermingle with our own data shadows, it won’t be long before the objects around us begin to make suggestions and possibly become more reliable than our friends at telling us what is good or bad for us.

In writing on value and worth, Ng describes how new economic opportunities (within the digital economy) will increasingly capitalise upon social contexts that are becoming visible through the advent of ubiquitous computing (Ng 2012). Described as ‘contextual archetypes’ Ng suggests that within an IoT, objects can become a point of sale within particular activities, for example, when making a cup of tea, the tea bags will be able to sell you milk because they are part of the same context. This radical shift from vertical lines of consumption to horizontal, means that objects with an IoT are elevated to a role of actors within our networks of distribution and sharing.

The ThingTank project identifies that ‘things’ may soon know more about lives than we do and may also be able to make suggestions about what is missing. The purpose of this project is to explore the potential for identifying novel patterns of use within the data that is streamed through the interaction between people and things, and things and things. Our project builds on research and innovation that has been established by the three investigators across the fields of Internet of Things, Social Experience Design and Machine Learning. Through a better understanding of how what data can tell us about how we use objects, new models of use will emerge and reinvigorate the role of things and people within design and manufacturing.

Work to date
The team were notionally awarded 3 years funding, however in January 2015 as the price of global oil fell through the floor, funding from the project was halted and we no longer have the resources to pursue the project through Skoltech. Nevertheless the achievements to date are considerable and the idea sustains interest across the internet.

Introductory movie: Morph Studio

The project was interested in methods of gathering information for a contextual archetype: making tea. It was felt that two approaches would compliment each other, 1. an ethnographic approach using a combination of interviews, studies using a cultural probe and also the gathering of data from autographers, and ii. a machine learning approach that would involve the review of data derived from the instrumentation of three key artefacts within the making of tea: a kettle, a fridge and a cup.

i. Thing Ethnography: Nazli Cila, TUDelft / Melissa Caldwell, University of California, Santa Cruz.
Nazli and Melissa developed a series of methods to capture the groups personal experiences of making hot drinks in the kitchen in order to capture personal perspectives through the use of a series of artefacts, but in particular kettles, fridges and cups. These three objects were chosen because of how their involved the ‘owner’ in different ways, some fixed in location, some tied to electrical points, others more mobile. Through a design pack / probe, the researchers offered up their own experiences as preliminary data including maps of the interaction through each of their own kettles, fridges and cups:

A second part of the Thing Ethnography was to attach Autographers to the same objects in a volunteers household. Autographers are hands-free, portable digital cameras, the cameras uses five different sensors to determine when to automatically take photos and can take up to 2,000 pictures a day- rather like the Microsoft SenseCams. The intention here was to not use an entirely anthropocentric perspective upon the events in the kitchen, but involve at least two of the other actors in the more than human context. So we attached a device to both the Fridge and the Kettle as well as a member of the household. From the video you can see the three video perspectives providing clues to the involvement of each of the three actors. In the window below you can also see the stream of data coming from the second study.

ii. Machine Ethnography: Fionn Tynan-O’Mahony, University of Edinburgh.
Keen to understand activity from entirely non-ocular perspective, and potentially from the machine Fionn instrumented a series of kettles, fridges and cups (KFCs) that used Electric Imps to stream activity data to the ThingTank cloud.

Although we ran out of resource to distribute all of the KFCs in order to amass as much data as we wanted, the early indicators of what the devices would tell us were very interesting. In this image you can see clearly see a pattern of kettle use that would suggest two uses within relatively quick succession: boiling at 6:51 am to 6:52am and again at 7.15am. Although limited in insight, coupled with interviews with participants to review the activity, the data provided interesting material to reflect upon domestic practices.

These preliminary studies are now the subject of a series of papers that explore human, more than human, and machine ethnographic processes.

The work is introduced and explained well in Elisa’s TEDx Delft talk:

Design Fictions
Part of the discussion around what it might be like in a near future to live with connected objects resulted in the commissioning of two design fictions which help extend the ThingTank research.

Knowing how sharp Anab Jain and Superflux are at identifying the tensions within design circumstances, they were an easy choice in asking them to develop a fictional response. Actually Uninvited Guests isn’t so fictional, and is a very perceptive hypothesis about the tactics that people develop to cope with the social expectations of using connected health devices.

Simone Rebaudengo‘s work on the Addicted Toaster made him a perfect choice for a design fiction. Cast further in the future, a likely situation involving the retraining of connected objects is not so far fetched either. For sure software algorithms are in perpetual adaption to recalibrate how they produce value for their clients / stakeholders, and the fiction provides an insight into how objects may require an education in order to stay relevant to their owner.

Next…
The broader application of the research is very interesting and provide insights into IoT 2.0 and beyond. The scale of data that the Internet of Everything and associated technologies will produce is unimaginable from an anthropocentric perspective. But that’s the very point, and it is the unique use of machine learning to identify unfamiliar patterns that the research team consider to be of interest to broader application.

Full team: Chris Speed, University of Edinburgh; Elisa Giaccardi, Delft University of Technology; Neil Rubens, University of Electro-Communications; Alexander Chernyshev, Skolkovo Institute of Science and Technology (Skoltech); Fionn Tynan-O’Mahony, University of Edinburgh; Nazli Cila, Delft University of Technology; Melissa Caldwell, University of California, Santa Cruz; Anna Sorokina, Skolkovo Institute of Science and Technology (Skoltech).

References: Arthur, C. (2013) Tech giants may be huge, but nothing matches big data. Guardian online: http://www.theguardian.com/technology/2013/aug/23/tech-giants-data Bandyopadhyay, D. & Sen, J. (2011) Internet of Things: Applications and Challenges in Technology and Standardization, Wireless Personal Communications: An International Journal archive, Volume 58 Issue 1, May 2011. Pages 49-69 Garun, N. (2013) Staples Connect bridges all your ‘Internet of Things’ into one managing app. Digital Trends: http://bit.ly/1dGBnhJ  Ng, I. (2012) Value & Worth: Creating New Markets in the Digital Economy, Innovorsa Press.