Big data needs big judgement

Wednesday, 23 July 2014

 33 103

Olly Arber  Nesta

Blog Ref http://www.p2pfoundation.net/Transfinancial_Economics

There is an experimental platform imagined in the classic 80s film Tron where all forms of research can be carried out at unparalleled speeds. It was called The Grid, and 30 years on, this futuristic machine has become a reality.
Down in a series of vast underground caverns and tunnels beneath France and Switzerland, the Large Hadron Collider is about to switch on again after two years of fine-tuning and upgrades. When it does, it will be operating at twice the energy and producing an enormous, mind-blowing quantity of data. And all of that data will be processed by the Grid.
The Grid
The numbers involved are huge.
When the switch is flicked in the new year the Grid will start to consume 160,000 gigabytes of information every day. That works out at 70 petabytes a year. To give you an idea of size, one petabyte is big enough to store the entire DNA material of everyone in the USA.
But how does the Grid cope with such a large amount of data?
Big data
The Grid is a global network of computers, originating in CERN, that quickly processes and shares the big data generated by the trillions of collisions in the Large Hadron Collider. Like its futuristic forbear in Tron, the Grid is incredibly quick. It is composed of hundreds of thousands of processors working in parallel, and it also comprised of a series of Tiers that pass the data along the line, refining as it goes. This enables the information from the experiment to be shared almost in real time with scientists anywhere around the globe.
Handling the data
But the key to its success is not archiving all the data, but reducing it to a manageable level. 
CERN’s data expert Pierre Vande Vyvre has spoken recently about the future for big data processing. He says that the role will change to standard algorithms that can do the heavy lifting first and reduce the data to a more manageable size before it gets close to human analysis. 
“The big science workflows are mainly data reduction. Currently just 1% of data from collision events [at the Large Hadron Collider] are selected for analysis. The archiving of raw data is not the standard anymore.”
What can big science teach us about big data?
CERN is well prepared. They can handle their data. Like the best big science projects, they have a clear theory which they are testing against the data, and they have set up clear parameters that can narrow the data. They also have the in-house skills to analyse what they find.
It's an important framework for dealing with big data that companies and governments could learn from, all of which are setting up their own big data initiatives to help them make predictions about the future.
Big predictions
The Obama administration recently launched a $200 million R&D initiative to 'improve the government's ability to extract insights from digital data', Google has been trying to map flu trends by studying search queries, and Amazon has been using customer data to try and automate our choices for years. All of them are chasing the big data dream.
But it's when you start to use big data to try and predict future behaviour that you get stuck. 
The data is based on social systems which are fraught with unpredictable changes. Unless you understand all future variables, big data for prediction can only tell us more about the past or the present than the future. As Juan Mateos-Garcia argues, “Big data is lurking with biases, mirages and self-fulfilling prophesies. Avoiding them requires access to the right skills and organisation”.
Big data needs big judgement
Operations like the Grid at CERN point the way to better management and analysis of huge data sets. Crucially, they not only have the skills to build the sensors and the computing architecture to deal with the massive flow of data, they also have the judgement to reduce the data and to analyse it afterwards. 
Big data is still a judgement call. Despite companies investing heavily in big data processes, most do not have the skills to analyse what is coming in. It's a problem, and an opportunity, that we've set out in our report Model Workers.
The end of theory
Chris Anderson - the idealist TED leader – made the bold call that big data will spell the end of all theory. Like the Master Control Programme in Tron, big data algorithms will work many times faster than a human and will be able to answer our global problems before they arise.
It is misplaced because it is naïve to assume that machines can make decisions about what data to collect and how to interpret the patterns in that data as information for other machines to use. It feels like the techno-utopian belief that brought the world market to its knees in 2008. Here the idea that computers can measure, control and self-stabilise societies was proved dramatically wrong when the complex algorithms that ran financial markets failed to predict the giant debt bubble. 
Adam Curtis carefully dismantled this dream of computers running things in his brilliant TV series ‘All Watched Over by Machines of Loving Grace’ (2011). The danger now is that the predictive power of big data is feeding the same fantasy that leads to the same distortion and simplification of the world around us.
Keeping big data human
When it is switched back on the data processed by the Grid will make the Large Hadron Collider the biggest of the Big Science experiments in the world. 
But it won’t be the machines peering into the underlying structure of the universe. 
That’s up to us.

Image: The Grid's Tier 0 data center courtesy of CERN.

- See more at: http://www.nesta.org.uk/blog/big-data-needs-big-judgement#comments

The road ahead for the collaborative economy

 Blogger Ref http://www.p2pfoundation.net/Transfinancial_Economics

   Helen Goulden  2013/ Nesta http://www.nesta.org.uk/

Last weekend’s FutureFest explored a whole range of possible worlds – and its lingering message was the need to cultivate greater awareness and agency to shape our own future together.
There are a great many things we don’t know about the future, but what we can know, with some degree of certainty, is that when today’s babes in arms are twenty years old, their fundamental needs and desires as biological beings and as social animals are likely to be the same as they are today.
They will still need food, heat, light, shelter, a mode of transport. Still need clothes, furniture, tools and technologies. Some of those things might look very different than they do today, but they will still be needed.
People will still want freedom and resources to explore the things they’re really interested in; music, being with friends, developing skills and talents. They’ll want meaningful work, a sense of purpose.
But the way we get hold of those goods like clothes, kit, cars; the way we earn and borrow money, and who we interact with to meet those needs and desires could be completely reconfigured. The surface result may look similar. But flip the board and the wiring underneath may have changed quite fundamentally.
And the reason we can intuit this change is that this reconfigured future is here right now. The primordial building blocks of the collaborative economy are here, fumbling toward each other.

The collaborative economy today

If you’d made use of a Boris Bike or a ZipCar, you have participated in a new, collaborative economy. You’ve made a decision to access something rather than owning it outright. There is a wave of innovation that is seeking to shift us from a culture of ownership, to a culture of access. We are beginning to perceive more clearly that the value of a thing is the beauty of the function it performs. Not the thing itself.
I’ll take a guess that most of you have converted to some form of access over ownership when it comes to digital entertainment such as film and music. But what else is on offer?
If I need a printer, I can go and buy one. Or I can look on Streetbank and see whether anyone in my area is up for lending or giving away a printer. If I want to make a bit of money by renting out my printer, I can post it on Ecomodo; along with any other rarely used item that might earn me a few pounds.
The internet is awash with hyperlocal platforms dedicated to connecting supply and demand of physical stuff in local communities.
Sign up to Mudjeans and I can lease my jeans and when they lose that certain something (that all jeans do) I can return them and get sent another pair. I can increase the size of my wardrobe by a factor of thousand by accessing my new clothes through 99 dresses. Paying – literally – buttons for them.
I can ‘Shave Time’ and ‘Shave Money’ by hiring razor blades instead of buying and binning them when they’re blunt with the Dollar Shave Club. If I can’t afford – or can’t stomach – to pay for weighty text books, I can rent them from Chegg.com.
One of the defining characteristics of many of these new platforms, is that they are peer to peer, bypassing traditional institutions to get what you want.
So I could borrow money from the bank, or I can go to Zopa and borrow money from an individual at a competitive rate of interest. If I want to raise money, learn a new skill, seek advice, get my meals cooked for me, park my car, lodge my dog while I’m on holiday, find a place to stay, take a ride somewhere, use a courier or buy insurance – all these needs and more, I can now meet by using platforms to connect with other every-day people, bypassing traditional layers of middlemen and sourcing from completely new kinds of providers.
It’s a phenomenon on the rise and the collaborative economy is now worth £330bn globally and £22.4bn in the UK. New kinds of marketplaces are emerging, seeking to connect you to everything you may want or need.

Why now?

Commentators are keen to explain the reasons why the collaborative economy is emerging. Some argue that this wave of innovation is being driven by financial and environmental pressures; that the combination of less money flowing around and the fear of dwindling natural resources and overflowing landfill is the driving force behind it. Others argue that it’s about changing demography.
Some maintain it’s because we have a desire to reconnect with people following a period of materialism and atomisation of society. In other words, it’s come about because at quite a deep and ill-articulated level, we are unsatisfied with our relationship with material things, with big business and with each other.
Some – who don’t look different to any other anarchist that have preceded them in the last hundred years or so – see this as a revolution; an end to capitalism, ownership and government. The rise of a new society that is free, open and stateless.
It is probably being driven by all those things and more. But first and foremost, it’s being driven forward because human beings are powered by and propelled forward by intrigue and imagination. Now that millions of people are able to connect and communicate across the planet in real time, we have begun to exchange and share whatever we can push and pull through those narrow pipes.
Music, pictures, film, stories. We’ve seen an outpouring of creativity; an outpouring of noise that is only really limited by what can be transmitted in a data packet.
Aside: Imagine if there were a teleporter in every home; if the ease of exchanging and sharing physical things were as instant and easy as exchanging and sharing a photo on Facebook. It stops the heart to think of what that would unleash – what it would mean.
In short, we are pushing the possibilities to further exploit networked, digital technologies because as humans we want – desperately – to see what’s possible with the tools that we have in front of us. We’re not substantively different to our ancestors in this regard, looking down at the ground and musing about what might happen if we rubbed a couple of sticks together. It’s why we’re brilliant – and terrifying – creatures.

Dealing with failure

But before we get carried away with tickling our imaginations with what a mature collaborative economy might look like, let’s pause briefly on some of the challenges it will encounter along the way. Because it will falter. Dramatic failed ventures that have been in receipt of generous investment will cause some commentators to herald the collaborative economy as a pointless flash in the pan phenomenon. That’s inevitable.
And there’ll be a number of reasons for those dramatic failures. In some instances the timing will be wrong; the market and people simply won’t be ready for it; in others it will be because the idea (simply) is not any good. Just because you can do something, doesn’t make it a good idea.
But other innovators will learn quickly from these many failures, standing on their shoulders to get a better view of what might work.

How much regulation?

Probably the thorniest issue for the collaborative economy is regulation. There are already some fairly crunchy debates already happening in terms of how these new ventures interact with existing legislation, regulation, licensing and tax systems. And much has been made of AirBnb being made ‘illegal’ in New York, as well as battles between taxi drivers, city councils and the ride share providers Uber and Lyft.
These early battles are the tip of the iceberg and are often fuelled by an existing industry resenting these very new kinds of competitors who are operating on very fundamentally different models that seem to capture the public imagination and seek to confound and evolve existing regulatory frameworks to which they have dutifully complied. There are bound to be some unhappy faces. And they’re likely to get unhappier.
Remember that Napster was the Big Bang behind the total disruption and revolution in the music industry. Publishing, television, film – wherever there has been a possibility to revolutionise business through a digital medium, we have done so.
Where the sea meets the land you get waves. And as the digital world extends beyond itself, to exert its influence further out into the physical world – containing as it does the power to transform and democractise – we can expect those waves to ride high. Some will surf and some will succumb – and the landscape will be re-shaped.

The collaborative economy tomorrow

Let’s cast our imaginations forward twenty years. Imagine all the major battles have been fought and won. Or lost; depending on who you are. Pressures we have with regard to economic instability, depletion of resources and provision for our ageing population will have only increased. But collaborative economy is still here. What will we see?
With the strong caveat that there are clear limitations from always viewing the future from the present we might see:
A rental renaissance with major retailers who traditionally made money through the sale of disposable, short-shelf life products shifting into the rental of highly durable products with readily reusable component parts. We’ll see a radical shift toward the efficient and elegant use of the material resources and products that surround us. And finally, we will focus as much attention and expend as much effort on how to create efficiencies in consumption as we previously invested in creating efficiencies in production.
Big retailers will have made the transition away from product-based organisations to service-based organisations with the most progressive of those evolving into marketplaces serving the rental and exchange of goods and services across peers, small enterprises and communities in an ecosystem of provision.
Some businesses will die. Brands that we know and love today will be long forgotten, unable to transform themselves in the face of so much disruption to their understanding of how the world now meets its needs.
I suspect that most things we own will be tagged, visible and shareable with whoever we choose. This will increase the variety of ways in which I can generate income, changing the way we work or engage in value exchanges – whether that be through lending out physical assets, loaning money, sharing skills, providing care for others etc.

Multiple currencies

This shift will almost certainly lead to us having more than one kind of currency in our pockets. As the collective participants in a collaborative economy begin to recognise different kinds of value, the idea of flat currency will become crude. Access to some of the things I need becomes not based on whether I can pay, but whether I can contribute.
This means we will probably use time as a medium of exchange to access some things we need, engaging in a reciprocal exchange of time in a distributed and networked way. Existing time-banking systems are perhaps the very clunky precursor to this.
We may also use our identity as a form of currency; through demonstrating trust and reliability in one sphere of life, we will use this as an access key to obtain other things. An early precursor to this might be the ability for your Ebay seller rating to be recognised and integrated with other, completely unrelated marketplaces and communities in which you are operating.
Data is likely become a form of currency and local currencies that are peculiar to one geographic location. There are many visions of the future, but in a more dystopian vision, we might well imagine why some communities will want to put a geographical boundary around the sharing of resources. Other currencies will only be used in certain types of collaborative economic activities. And this proliferation of currencies may be as familiar and logical as having a number of different debit or supermarket loyalty cards today.
But the biggest leap from where we are now to where we may be in 20 years’ time is the speed and means by which we give and gain access to those things we need.

The power of the collective community

At the moment, we’re all the equivalent of telephone switchboard operators; making our own connections; inputting our own requests and calls to access the things we need. We are the manual operators of supply and demand. Technology is making the connections possible but it’s not yet (in any meaningful way) sensing or pre-empting the need.
But as networks connect to networks; sensors connect to online systems and the open data that pertains to me, Helen, and everyone else begins to sharpen into more complete and focused picture, the necessity for me and you to be the manual operators of the collaborative economy will cease. The system will learn and know how to connect the wants and needs of the collective community. And use its massive power to try to serve them.
- See more at: http://www.nesta.org.uk/blog/road-ahead-collaborative-economy#sthash.jp82kHjn.dpuf

The Finance Innovation Lab

 

Incubating and accelerating new forms of prosperity. For people and planet.    

 

          

 

The Finance Innovation Lab is an incubator for systems change in finance.

 

 

Jointly hosted by World Wide Fund for Nature) and ICAEW (the Institute of Chartered Accountants in England and Wales), it was launched in 2008 and in 2012, was named one ‘50 New Radicals, changing the face of Britain for the better’ by NESTA and The Observer newspaper.
We aim to incubate and accelerate new forms of prosperity, for people and planet.
We believe we need to innovate across the whole financial system at once, for change to happen.
We do this in three ways;

1. We incubate new business models, innovation in mainstream and new forms of civil society
2. We accelerate the capacity of leaders to create change
3. We create the wider conditions for change by raising awareness, creating supportive communities and advocating for policy change

We work with people from a broad spectrum of backgrounds, including mainstream finance, the creative industries, academia, NGOs and the alternative business and finance world.
Go to our Projects page to find out more about current programmes and to our Forum for the latest updates on activities and thinking.

 

Navigation

• WHAT
• HOW
• WHO
• PROJECTS
• COMMUNITY