Introducing our latest recruit, Mr. Computer Program.

Unless you have been living on Mars for the last couple of months (oh wait…) you’ll know that the UK will soon be voting on whether to stay in or leave the European Union. Different countries, with different social, cultural, economic and political histories, and languages, which came together – despite their differences – for common interests. Without getting political, I think it’s uncontroversial to say that the world has changed enough since it was set up for us to debate whether it remains fit-for-purpose.

Like many I cannot name my Member of the European Parliament. Whoever they are, though, I have lately been feeling some sympathy for them. Interdisciplinary research can often feel like diplomatic work. Different fields don’t always want or see the need to speak to each other; distinct cultural norms and languages have evolved within each field, with funding and publication systems to support them, which can be in conflict with and contradict each other. However, the world has moved on and, like it or not, now faces complex challenges that require expertise from different fields to tackle together. It’s a battle I believe is worth fighting, and even find rewarding to fight, but my goodness interdisciplinary research is a battle, at the end of which, like my MEP, I have no expectation that people will remember my name either. It’s hard to make your name in one field, let alone two or three.

I have recently been crawling through a labyrinth of literature for two papers, like a mole in a rabbit warren, to find anything at all that speaks to me. To add extra irony, I am also currently supervising a postgraduate student on a literature review assignment! Honestly, you couldn’t make it up. Short story: this mole finally found some light last week shining from three articles. HURRAH!!! Success in even the most everyday academic activities is especially rewarding and motivating in interdisciplinary research.

So this got me thinking about what differentiates my research from all the other research out there. Clearly, it’s different if it takes me so long to find anything similar! But what is my academic identity? This is honestly not intended as a sales pitch. If it’s a sales pitch, it’s directed at myself, to guide me towards the end of the iMars project and beyond, and answers to my existential questions. So, I present to you, for your Friday afternoon entertainment, my  Venn Diagram self-portrait:

Venn_Jess

The Venn Diagram of my Research Interests

  1. Geographical Information Systems and spatial thinking.
  2. Human Factors: the design of automated systems to work for us, not against us, and lately with us…
  3. Data-Driven Discovery: a term that I shamelessly steal from the University of Nottingham’s research priority area. I’m talking about the construction of knowledge from the largest, most dynamic datasets. With special interest reserved for images, which I firmly believe always speak a thousand words (with apologies to my sister the English teacher).

As a geographer I am inherently curious about the connections between people and places (however you choose to define “place”), and the field itself helps me to explore them through its unique combination of relevant aspects from other fields of study. It’s a ‘discipline of disciplines’, so that I am, by nature, too curious to study anything within a black box. For my love of Geography I blame some very fine and funny documentaries starring Michael Palin and a couple of beautiful atlases that allowed me to explore the world from my childhood bedroom. And my good fortune to grow up in Dorset.

My interest in Big Data is founded upon a love of numbers, images of the Earth – Picasso isn’t a patch on Mother Nature – and the patterns they hide.Consider the articles by my good friend Raechel Bianchetti and, more recently, Kevin Schawinski on The Conversation. It’s a long time since my thoughts turned to the search for Donald Rumsfeld’s unknown unknowns, but the collaboration of man and machine is already constructing knowledge on projects like Space Warps and the latest version of Galaxy Zoo. I blame my Maths teacher Miss Johnson for turning me into a number nerd. Maths was my comfort blanket at school; it was a pool of clear water within a murky, grey ocean, where answers were right or wrong.

My recent paddling in the waters of User-Centred Design has given me a new, much more personal frame-of-reference and reignited a fascination I have for us as humans. If Geography and Maths are my ‘British’ and ‘English’ identities then User-Centred Design is my European one. It’s the habitat where my other interests have evolved to reach their potential.

Despite the technological advances for capturing images, whether it’s remote sensing imagery, medical imaging, or security scans, humans still hold the key to meaningful use of imagery. At the most basic level, humans have a capacity for experiential learning and flexible, creative thinking that computers cannot match without significant resources; computers, by contrast, are built to follow the rules we define efficiently. But the volume and variety of data now captured and stored is testing our capacity to extract meaningful information and construct knowledge from it.

In reading and thinking about this, I discovered discussions about the future of work, and human-computer collaboration, also known as human computation, in many areas of life. (Autonomous cars, anyone?) Crowd-sourcing, the approach that iMars exemplifies, is one of the more widely-known incarnations of human side of the human-computation equation, but its fame has come at somewhat of a price to its reputation as it’s come under increasing scrutiny, as if it’s purported to be some silver bullet, which I don’t think even its most ardent proponents would actually claim. The focus of the field, like my literature search, has thus shifted to figuring out how we can get machines and humans to work together and try to answer questions like: What role(s) will the crowd play and what role will the algorithm play in the resulting workflow? How will they work together? Is the Fitts List I described before going to evolve? How do we explore a collaborative workflow’s “Fitt“ness for purpose?

For me the answer lies that last question. Collaborative workflows. Founded upon humility and reflection. Between humans and machines. And researchers from different disciplines. If I’m lucky enough to take this interest further, all my middle-child mediation skills I developed in my doctoral work (as an interdisciplinary researcher and a spokesperson for the end-users at the developers’ table) would come to the fore again, to see how computers and humans (and researchers from different disciplines!) can work together.

I would be lying if I declared now that I am where I am because of a master plan. But whichever way I look at it, however crazy the combination of social and technical ingredients I add to my research, I have never felt more at home at work than I do now.

Sweet (Two Thousand and) Sixteen

Is it really only a month or so since I last wrote?! Christmas break aside, so much has happened that I thought it must have been a lot longer! Time is sprinting faster and faster as iMars and Mars in Motion grow. Since I last updated you, the European Geosciences Union has published a blog post I wrote for them that I point you to here for continuity. Some of it might have repeated what I have already said here, but I am happy and grateful for having had the opportunity to introduce myself to a different, but no less potentially interested, audience.

December started with the Geological Remote Sensing Group meeting  at the European Space Agency (ESA)‘s European Space Research Institute in Frascati, Italy. I was especially happy to be reunited with some of my friends from University College London. I was also grateful to present my talk early in the meeting so that I could relax for the rest of my time there and those there recognised me as ‘Miss Mars’ and I didn’t spend the week explaining myself! A superb range of presentations allowed me to play a very successful round of ‘Buzzword Bingo‘ with Change Detection, Big Data and Crowd-sourcing; presentations that struck a chord included those by Alex Gow from DigitalGlobe who recounted the use of Tomnod for voluntary validation and extraction of features on the ground in remote sensing imagery to inform relief efforts in the aftermath of the Nepal Earthquake, and Alex Tewkesbury who talked about his paper on the detection of change of change in remote sensing imagery. It was a far more international affair than I had anticipated, and I made some valuable connections with people I would have otherwise never met.

frascati

iMars at the Geological Remote Sensing Group Meeting in Frascati, Italy. Photo: Dietmar Backes

Immediately on my return I was on iMars duty for the second round of experiments. We’re already getting insights into the challenges we will have to overcome to present a change detection task with the crowd sourcing approach, in a way that makes sense to the crowd and the scientists. Somebody asked me in Frascati how I would assess the success of this project and, although the experiments focus on the design of the user interface and task, my dream is still that the crowd-sourced data will produce new insights into the Martian environment, to enhance our knowledge as well as informing missions, and one day train a computer algorithm sufficiently well to detect surface changes automatically. This is the ambition, so it will be interesting to reflect on that this time next year.

We didn’t break for Christmas until we had showcased the project to school children attending the University of Nottingham’s Engineering Faculty Christmas Lecture on the theme of “Light and Sound.” It proved a surprisingly useful opportunity to observe our designs’ impact on fresh eyes, without the silence and unavoidable pressure that experimental conditions can impose. Our potential planetary scientists were keen to work together, rather than on their own, so we engaged in discussions that would ordinarily take place online on forums. For our table’s visitors it became a game of ‘Change or no change?‘ They asked and raised some fantastic questions, which were far more difficult to answer than those at ESA! The difficulty of distinguishing between changes in lighting or image quality and those actually happening on the surface are becoming increasingly clear. It was all worth jumping the last minute technological hurdles that seem sent not only to try us but amuse us on such occasions. The irony of struggling to connect to a network or WiFi to see images that have come all the way from Mars is not lost on us!

ChristmasLecture

Mars in Motion … finally in motion. At the University of Nottingham’s Christmas Lecture. Photo: James Sprinks.

Full steam ahead for 2016, during which I intend to write papers rather than do any more space travel. I have a few papers outlined in my head based on what we’ve learned so far and I am just grateful that after my thesis I won’t be writing the papers alone. I also take some comfort and confidence from the prospect of contributing to two upcoming special issues of journals, which suggest I am not alone in my excitement about the potential for my area of research.

But for now, on my agenda for the first working week of the new year? Addressing the final amendments requested to Nottingham’s mid-project reports (against which we are evaluated by the European Commission at the end of the project); preliminary perusal of the responses and comments that participants in our first and second experiments provided, and preparation for the presentation I will give to the rest of the iMars team in Lausanne, Switzerland next week. For now I wish you (and myself!) all the very best for 2016. I will leave you with the belated Christmas present of these images that I recently found for the benefit of our experiment participants, simply because they are too beautiful to keep to myself. They exemplify just some of the features in which the iMars project is looking for changes.

Gullies carved by the flow of water or lava into a (comet or asteroid) impact basin created four billion years ago. Photo: NASA/JPL/University of Arizona.

 

A rippled dune front in Herschel Crater. Photo: NASA/JPL-Caltech/University of Arizona/JHUAPL.

Craters. Image: NASA/JPL/University of Arizona

Slope streaks. Image: NASA/JPL/University of Arizona

Where on Mars…?!

I have been incredibly remiss in posting these last months thanks to a blogger’s paradox. When there is plenty you want to write about you struggle to find the time to write it! Finally I have come up for air for sufficiently long to update you on where I’ve been! 1) the European Planetary Science Congress (EPSC) 2015; 2) the Zooniverse; and 3) Planet Experiment.

EPSC was a joy from beginning to end, from my discovery of the European Space Agency (ESA) lego to sharing the shuttle to the airport with a researcher from Oxford interested in creating their own Citizen Science project for analysing radar data. It transpires that planetary scientists are not only an intelligent bunch but humble with it; I met people for the first time and felt I’d known them all my life. I also grabbed the opportunity to sit with others on the project and tackle problems that ultimately only took five minutes to solve but required us to be in the same place at the same time. Dropbox and Skype can only get you so far with a project like ours! This meant that I worked on Mars in Motion right up until my demo, but the hard work paid off. It was wonderful to see what others are doing with Martian data too, from Where on Mars? and MarsSI. The week was made even more memorable on the Monday with news of saline water on Mars and the structure of the comet 67P.

RSL

The famous Recurring Slope Lineae (RSL), which suggest liquid water is active on Mars, on a scarp of the Hellas impact crater basin; RSL are dark streaks that can run hundred of metres down crater walls and canyons when the temperature goes above 23 degrees C, first captured on camera in 2011. The salinity of the liquid has lowered its melting point. Photograph: NASA/Reuters

RSL_3D

RSL in the walls of Garni crater. Photograph: NASA/AFP/Getty Images.

In conjunction with the conference, the most amazing array of outreach activities were on offer to the public; in addition to ESA’s astronaut lego they could explore Martian landscapes in 3D with Google Cardboard, find out how much they would weigh on different planets with different gravitational strengths and…. French school children toured this stellar showcase of the solar system all week long. To add to the buzz of the building, the event coincided with the release of The Martian, which I defy anyone to watch and not be inspired. It certainly inspired me to begin formalising ideas for my own outreach activities later in the project.

This leads me nicely onto my long overdue “official” visit to Oxford to meet the Zooniverse team. I did an impromptu double-act with Meg Schwamb of Planet Four fame, for the lunchtime seminar, so that the lucky lot had a whole hour of Martian magic! I only hope that I impressed them as much as they impressed me with the insights they gave me into imminent developments of their Panoptes platform. I daren’t divulge too much but keep your eye out for some very exciting extensions to the models of crowd sourcing to which we have been accustomed. The only clue I can provide towards what is to come is this paper by Phil Marshall, in which volunteers’ classifications train and update a computer algorithm in real time, so that it continuously gets smarter and more competent in the analysis, the ultimate aim of which is to reach the project’s goal much more quickly and efficiently for all concerned. In the very least I know that their experience will be invaluable when we are ready to launch Mars in Motion.

The week after I presented in Oxford, it was my turn to take to the lectern for the lunchtime seminar at the NGI. I have been here six months now so I was overdue my turn to share what exactly on Mars I am doing with the people I pass every day. The range of research going on around me every day, in the same building, has been a constant source of surprise and joy to me and I have been helping to organise a schedule of lunchtime seminars that reflects it. It provided me with the opportunity to, not only share the video I keep up my sleeve but also, recruit for our experiments, for which we were finally ready to begin.

This brings me nicely to my current location. Planet Experiment. I have deliberately not looked at the data yet because they are ongoing and I want to be partial in my presentation of the task to participants and don’t want any confirmation bias in my analysis. For this first experiment we are playing with the presentation of the images in a Martian version of Spot the Difference. You won’t be surprised to hear that we are already planning our second round of experiments to kick off as soon as possible. It feels great to have got them underway now and I’m looking forward to reporting preliminary results to the rest of the team before Christmas.

MarsInMotion

Spot the difference…on the surface of Mars!

Incidentally I recently spent a lunch hour answering questions from new PhD students about my PhD experience and attended an event to celebrate 40 years of Knowledge Transfer Partnerships. Both events inspired an idea for a future blog “Post-PhD” post, but this will come as soon as my corrections are approved, which will be this side of Christmas if I have anything to do with it! December is going to be a busy month, not least because I will be presenting at GRSG at ESA ESRIN, and getting used to the minor change to my life that is home ownership. As 2015 comes to a screeching halt, it’s giving 2016 quite an act to follow.

 

Fitting Man with Machines

I’m going to start this by saying that you can blame/thank* (*delete as appropriate) a colleague of mine for this post, after a conversation we had last week.

For the studies I carried out for my PhD I sought end users to help me to improve the design of a software, with the noble aim to, at worst, make their working life more productive and, at best, more enjoyable. On more than one occasion, however, I found myself asking them to show me how they carried out tasks so that we could essentially design the system to replicate it. Was this desirable and ethical, given the cost of the software and the amount of time users had taken to acquire skills and tacit knowledge of their workplace? In other words, were we asking them to help us design a product that would put them out of a job? I was often unsure. The software was difficult to learn, which seemed unsurprising and unavoidable given its context of use, but we (implicitly) persisted to seek a universally-usable design. On the surface this appeared to be a win-win objective: more people would be able to use it, which would simultaneously reduce the number of requests expert users receive from their colleagues without the time or inclination to learn it (freeing up their time for their own work), and potentially increase contract renewals. However, in other respects I perceived (rightly or wrongly) that I was indirectly and unwittingly stirring up workplace politics and impacting the credibility and security of the end users’ role in their workplace.

I was relieved to find last week that I am not the first person to question the implications of the trend for computers to replace humans in the workplace and the dynamics of socio-technical systems. By no means can I (or indeed) will I ignore ethical issues now that I am designing a system to be used by a faceless army of volunteers; it is in fact in my interest to tackle them head on, if I want their engagement and optimum performance. By way of this recent article in New Scientist and the aforementioned conversation I had last week, I’ve stumbled across  Functional Allocation Theory. Long before the era of unexpected items in baggage areas, Paul Fitts dared to declare and classify functions that machines perform better than humans, and functions that humans perform better than machines, in 1951. The history of what is known as “Fitts List” is articulately reviewed by de Winter and Dodou (2014), so I don’t intend to detail it here, other than to point out that Bainbridge later developed these ideas into her narrative on the “Ironies of Automation“.

Also known by the memorable acronym MABA-MABA (Men are better at, machines are better at). Published in: Fitts PM (ed) (1951) Human engineering for an effective air navigation and traffic control system. Published by: National Research Council, Washington, DC.

Also known by the memorable acronym MABA-MABA (Men are better at, machines are better at). Published in: Fitts PM (ed) (1951) Human engineering for an effective air navigation and traffic control system. Published by: National Research Council, Washington, DC.

This work directly challenged any assumption that automation is always desirable and that unreliability and inefficiency are grounds upon which to justify the elimination of human operators from systems. The division of responsibility between humans and computers based on performance, in the tradition of Fitts, precludes the possibility for human-computer collaboration and wrongly assumes that the human operator’s performance (manifested in their effectiveness and efficiency) is constant.  Machines follow programmed rules of operation in a way that humans do not; human performance is unpredictable and inextricably tied to their skill and motivation, which evolve over time.  Noteworthy here is the concept of “autonomy”, the level of which a human operator has within a system can also directly feed back into their performance, especially in safety-critical situations. Too much can be stressful, which leads to errors, and too little can result in low job satisfaction and similarly poor health and well-being, and consequently performance (assuming that they are not so demotivated and ill that they do not turn up for work in the first place).

Although iMars adds another ingredient to this cocktail, the crowd, it helps me to work towards a “Goldilocks” level of autonomy for our volunteers, in the same way that we sought to design a product that would keep existing users and attract new ones during my PhD. iMars amalgamates images of Mars taken over 40 years. A lot of images have been taken in that time, amongst which only a selection will show changes in geological features. The change detection algorithm the project produces can never hope to detect these patterns as well as the volunteers, but we can refine the skill with which it sifts through the images to find ones with change, and improve the certainty with which we know the crowd classifies images in which there has been geological change. Since we cannot visit the surface of Mars to ascertain the validity of the crowd’s classifications, this level of confidence will only be relative, but it’s one way in which we can affect crowd-computer collaboration.

Past, future and finally present

This is a very brief post after an intense few weeks, just to keep myself in check more than anything else, and committed to blogging!

Week one, I headed home to revisit my thesis (and family and friends!), and for once I was grateful for some truly awful weather. One week, with all the associated antics that a trip to the south coast entails, got me 90-95% of the way through my corrections, before I had to head back north to speed-write an application for a Fellowship, for which I had to plan my research for the next five years. Whilst I have no expectation of success, I was surprised by how emotional I felt at the end, and not purely from exhaustion. I underestimated how lucky I am to be working with the people I am and in a place that not just welcomes but actively encourages interdisciplinary research and collaboration. When you have weird and whacky research interests like mine, a sense of academic belonging can be difficult to find, and writing the application made me appreciate how lucky and at home I currently am.

Busy times lie ahead. I’m only a month away from demonstrating Mars in Motion at the European Planetary Science Congress and I still have plenty to do to get it ready and up to date. I also have experiments to design, conference deadlines and discussions around Citizen Science continue from the highest level and I’m looking forward to mentoring to a new cohort of PhD students starting next month, not to mention an exciting opportunity to guest blog for the European geoscience community (you can guess their name!).

Regardless of what happens with the Fellowship, I’m pleased to have had a valid excuse to take the time, whilst I could, to consider where I want my research to lead, to acquaint myself with some new literature, and to simply take stock (and appreciate) my current academic home. I also have ideas now for far more interesting blog posts on Fitt’s List, the ironies of automation and MABA-MABA. Anyone expecting posts on exercise regimes, robots with a sense of humour or tribute bands to a certain Swedish pop band might be disappointed, but I remain committed to my promise of pretty pictures at least. After three weeks spent tugging my mind between 2009 and 2021, it feels good to be back in 2015.

ZooCon 2015 (Part 2/2)

Next up was a stellar talk Alissa Bans, from Disk Detectives, which is investigating the formation of planets and planetary systems. Currently we understand that planetary systems form when gravity pulls particles together into a collapsing cloud of dust and gases until a hot core develops to form a star. The remaining particles start to rotate until eventually they form a flattened disk. Particles clump together to form planetesimals, which in turn eventually form planets. Details of this process, however, are sketchy, so astronomers have been hunting for these disks, birthplaces and homes of planets, for decades.

Disk Detectives identify two categories of disks, Young Stellar Objects (YSOs) and debris disks, that represent different stages of planetary formation. YSOs are young stars, generally less than five million years old, found in clusters, which remain surrounded by large amounts of gas, out of which giant planets like Jupiter form; in contrast, debris disks are found around stars older than five million years old, contain little or no gas and are littered with rocky and icy debris that look like asteroid belts. Both contain large amounts of dust created when debris collides, so they both shine brightly in infrared wavelengths. The NASA WISE (Wide-Field Infrared Survey Explorer) satellite mission has just finished mapping the entire sky in mid-infrared wavelengths, which provides the most powerful resource to date for finding dusty disks. Although the WISE images on Disk Detective don’t have detail (rather they will contain disks in a single point of light), radiation from the dust in the disk enhances radiation coming from the star.

Unfortunately, galaxies, asteroids, active galactic nuclei, and interstellar clouds of dust all emit and glow brightly at these wavelengths, which can easily confuse computer algorithms designed to automatically search for disks, so all disk candidates must be examined by eye to make sure they really are stars with disks, and not impostors. Computer programs can only detect what we tell them to measure and preselected, from 745 million objects captured by WISE, only the brightest 539,000 infrared objects, at wavelengths of 22 microns. Each of these will ultimately be examined by at least ten users, who respond to simple questions asking, for example, whether the view contains multiple objects, or whether objects go outside of the circle targeting them. For each image, the project provides a Spectral Emission Distribution (SED), in addition to example SEDs (provided by volunteers themselves!) to help classification of the more difficult images (Figure 1). Astronomers then examine the SED of each disk candidate and do follow-up observations to confirm which candidates are disks.

Figure 1: Example Spectral Energy Distributions of potential candidates provided by a volunteer.

Figure 1: Example Spectral Energy Distributions of potential candidates provided by a volunteer.

The Disk Detective team predict volunteers will find about 375 previously undiscovered disks, a list of which will be made public and allow astronomers to study the statistics of the disk population. In particular, the WISE data will help us learn about disks around stars that have evolved in areas of the sky not previously known to be forming stars. Astronomers will also use the data to inform targets for observation of some nearby disks at high resolution (and maybe even spot planets!) with NASA’s Hubble or the James Webb Space Telescope, which will replace it in 2018, to explore the origins of planets and life, amongst other science topics, and take more spectacular images.

What I really love and admire about this project is its focus on its volunteers and how clear it is about the nature and quality of the data it desires. They appear to have an amazing relationship with their volunteers, even building up a group of super users they consult for opinions and favourite images, following up classifications with ground-based telescopes and cataloguing example SEDs to aid classification (Figure 1). This has paid off, with impressive results that seem to have even discovered a new type of binary debris disk. This project also provides a precedent for using computer algorithms to preselect images for classification, which we intend to investigate with iMars imagery.

Tom Hart then gave us an overview of Penguin Watch, a project in collaboration with the charity Penguin Lifelines, that launched in September 2014 and has had around 2.5 million hits already from volunteers who love to look at pictures of penguins. (Personally I find it hard to believe that there are people who don’t!) Particularly over the last 50-60 years, the world’s sea bird population has declined. Although some threats in some areas of the world (such as fishing, climate change, harvesting and direct disturbance) are well known, the picture is much hazier than one might assume; the nature and magnitude of threats vary considerably between different places and impact different species to a greater or lesser extent. For example, the number of Gentoo Penguins is actually increasing, albeit at a slower rate than others are in decline. Some parts of this picture are unfortunately even hazier than others because of lack of data; some areas are simply more difficult to reach than others to capture the data for a whole host of reasons that I’m not going to even begin to go into here! Suffice to say that Antarctica is one such place. It’s 50 times the size of the UK, twice the size of Europe and twice the size of the US, and much of the continent is a logistical nightmare to reach. Scientists have traditionally been confined to bases in the most easily accessible places to carry out their research in Antarctica; even in these locations, the infrastructure surrounding bases can itself threaten sea birds, and scientific missions require a huge amount of planning a long time in advance. Result: our knowledge of the impact of climate change on the continent (in terms of air and sea temperature) is not as detailed as we now understand that it needs to be for such a large area. Bear in mind that it’s not just the climate that has changed; whilst whaling was a massive problem in the past, it is now krill fisheries in the Scotia Sea that pose the biggest threat. Krill is a significant constituent of a sea bird’s diet in this region. The ice releases a huge plume of life into the sea each summer when it melts, which requires few resources for the local populations to capture.

This is another project that has carefully considered why they need people power. The threats to sea birds are too complex to understand and manage with traditional approaches to research in Antarctica, limited to small areas over short periods of time. It requires a new approach to data capture and analysis, especially to have any impact on policy. For example, population counts for colonies are typically limited to those close to bases, and only capture birth/death rates, which miss details of how hard the penguins are having to work to raise chicks, how stressed they are and their productivity. The team’s challenge was to capture data in as many places as possible all of the time and to be able to analyse the data remotely. They set about taking any form of transport going, including chartering yachts, and being strategic in how they acquired equipment to distribute a network of American hunting cameras at, currently, around 60 sites, in addition to 40 run by the Australian Antarctic Division, which take around 20 images a day. They have adapted the cameras by coating them in resin, extending their life to five years, and hope to install fuel cell batteries that could potentially extend this to ten years. The latest set of equipment even records air pressure data of interest to climatologists, but already they record: individual nests; individual colonies; the laying and hatching of eggs to within a couple of hours; the volume of sea ice; and, the presence/absence of chicks/adults/predators etc.

These images are then classified by the volunteers online. And this is where I get my excuse to post a penguin picture (Figure 2).

PENGUINS!

Figure 2: PENGUINS!

This data has revealed surprising but basic information on how deep into winter the penguins stay in particular areas; krill fishermen have thus far justified their practice on the premise that they are not depriving penguins of an important food source. The timing of the breeding cycle has also been found to vary around the Scotia Sea, which can inform the operation and management of the fisheries. The dates of the arrival and departure of penguins is also noted in addition to huddling behaviour, which can be a proxy for the chicks’ health and level of independence. Computer vision experts have also already used the data to train an algorithm to identify an Adélie penguin! And, uh, they found that the Gentoo Penguin melt ice with their poo so that they can breed.

In the future the team wants to feed the data back to the volunteers, sponsors of the cameras and adopters of colonies through Penguin Lifelines, so that they can interact with the data and do their own analysis (by colony, time lapses etc) and research. This includes policy makers. They are also looking into capturing data with sensors activated by the penguins themselves, and not limited to 20 images a day, for example, which could capture chick survival and the arrival and departure of their parents, how long the parents spend at sea, and how hard they work. Simply put, they want the penguins to power the research. Treaties such as the Antarctic Treaty need data to be enforced and the data from Penguin Watch has just formed the basis of a report recommending trial closure of some fisheries, to see if it aids the recovery of penguin populations in some areas, and points to the potential for fishing practices to respond to the data more quickly in future. The data also played a significant part in the establishment of the world’s largest Marine Protected Area in South Georgia and the South Sandwich Islands.

Brooke Simmons finally gave a presentation of two halves. The first half reflected on the Zooniverse as a whole. She showed a graph demonstrating that the community of volunteers has continued to grow, at the same rate or higher, consistently over its lifetime; with each new project launch, volunteers appear loyal to existing projects whilst embracing new ones. When she saw this graph, Brooke became interested in why does the Zooniverse work? Why do people volunteer? What is their personal motivation and what do they hope to get out of it? What can the Zooniverse do better? These are especially important questions to ask now that researchers can start their own projects, without the help of the team. What experience can the team pass on? First, its community is a major factor in their success, many of whom interact with each other and/or scientists on projects’ discussion pages. Second, survey after survey has found that the main goal of volunteers is to contribute to science. She then narrowed in on the following sentence in Phil Marshall’s recent paper:

“The most productive citizen astronomy projects involve close collaboration between the professionals and the amateurs involved.”

The paper discusses what does and does not work in terms of how this collaboration manifests itself. Each project can and does take different decisions about how this collaboration takes place and for how long it is sustained, be it through discussion boards, blogs, social media. The authors took this a stage further and engaged the community in its publication, listing the names of the volunteers who contributed. But, will the statement above always hold true? What else is needed? How do we measure it? I won’t list them all here because you can read them in the paper she wrote with Joe Cox, which lists, calculates and defines many more metrics of success under the umbrellas of public engagement and contribution to science and combines them to present a success matrix (Figure 3) for Citizen Science, which she is interested in taking further to establish what can we learn from it? Are some methods of engaging volunteers more successful than others?

 Project positioning matrix from [Cox _et al_, 2015](http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7106413). The success matrix appears to show a positive trend relating to the positioning of Zooniverse projects and their size, indicating that projects receiving more classifications tend to be more successful.

Figure 3: Project positioning matrix. The success matrix appears to show a positive trend relating to the positioning of Zooniverse projects and their size, indicating that projects receiving more classifications tend to be more successful. (Cox et al, 2015)

This followed Swanson’s talk nicely and reminded me of a conversation I had the day before ZooCon about whether I am designing a user interface to obtain accurate results or engagement.  I don’t personally see them as being mutually exclusive, even though I do recognise that a retired professor, for example, might require a different user interface to a schoolchild. When you consider, though, that someone engaged with a project can develop proficiency over time, the notion of “expertise” becomes very blurred. This requires another blog post sometime and can probably only be unpicked at the project level. I would love your thoughts below this post.

The second half of her talk focused on the development of project in response to the earthquake in Nepal. It was only the second project after Snapshot Serengeti that was developed from start to finish with the new Panoptes development platform. This project reflects her preference for moving away from the term Citizen Science to talk about People-Powered Research. Planet Labs provided the team with satellite imagery, who consulted with the UK Charity Rescue Global to find out how they could help. Rescue Global were interested to know if there were any towns that were completely cut off and, as a result, had not yet been reached. By combining the established images of the area around Kathmandu with the crisis mapping information that was coming in they identified two towns that had been cut off.  This impressed at the highest levels of NATO whose Director General of Military Staff is quoted as saying:

“I also noted the organisation’s testing of a new approach to the use of people, computers and social media, to identify Search and Rescue targets of interest.”

That someone so high up should recognise the impact of what they did, so explicitly, boggles my mind.

Her take-homes were:

  1. Engage the community for success: if you don’t do this well then, yes, you’ll still get science out, but if you engage the community you’ll get better science out and faster;
  2. Publish papers with the volunteers is a good metric for this, which should happen more;
  3. Talk is powerful and remains a focus of the Zooniverse.

To cap off what was already a super day, Brooke saved me hours of work because she explained to me (in the pub afterwards!) how to implement some functionality in my project that I didn’t think existed yet. THANKS BROOKE. And thanks ZooCon. What I heard humbled me and reassured me that focusing on the science and the user experience is a good thing, and the questions that have been bothering me might not be so stupid after all. I hope we can help each other in our quest for answers.

ZooCon 2015 (Part 1/2)

That’s right. I have finally managed to shackle the first half of my ZooCon thoughts to the metaphorical mast before they fly away. Hold on tight now for the first of two posts. But first I should admit that I attribute a percentage of my excitement about ZooCon to the fact that it took place on Keble Road, Oxford. Yes, Keble Road. The road named after, and adjoining, the college at which I studied as an undergraduate. I don’t need any excuses to return to Keble but, since ZooCon gave me one on a plate, I instantly booked to stay there the night before. I tell you this only because you cannot even begin to imagine my face, a cocktail of disbelief and glee, when the girl on reception presented me with the key to the very same room that I lived in for two years. What a way to start my ZooCon weekend. A little backwards time travel before I was catapulted into the future the next day.

Becky Smethurst kicked us off with a talk of two halves. First she reported a live Citizen Science project from earlier this year on the BBC’s Star Gazing Live (i.e. classification against the clock!) to find supernovae. From what I understand, when a star explodes and forms a supernova, for a very brief period of time it shines with the same brightness before it diminishes; the apparent brightness of a supernova, as a proportion of their full brightness, therefore indicates how far away they are thus how much the universe has expanded, and thus the age of the universe. To capture how bright a supernova is at its birth would require action on a large scale in a limited period of time. For three consecutive nights, Brian Cox asked viewers of Stargazing Live to trawl through 100,000 images taken by the SkyMapper telescope in Australia and classify them before the sun set in Chile, where a telescope would be turned to focus on those found. More than 40,000 volunteers provided almost 2 million classifications for the images over the three days to find four, ultimately confirmed, supernovae, one of which was a Type 1A, which scientists use to age the universe. I was impressed enough that volunteers found one new supernova, but even more impressive is that this one data point resulted in the calculation of the age of the universe to within 200 million years of the agreed answer (14.03 billion years)! That, my friends, is the power of the crowd. Not just the power of Brian Cox. It also raised the question of how we communicate the science that results from classifications and how we can improve this. The iMars team is already concerned about how we will manage and give feedback to volunteers, for example how we are going to administer discussion board posts. Second, she told a story about how she had taken ten photos of Orion’s belt on a windy evening in Hawaii (a word that always grabs the attention of my ears), each ten seconds apart. She then used open source software to stack the photos and produce one image showing the movement of the sky in that time. BBC viewers used the same process and sent in 790 photos which, stacked together, produced the image in Figure 1.

Figure 1: Orion. As captured by BBC Star Gazing Live viewers.

Figure 1: Orion. As captured by BBC Star Gazing Live viewers.

 

I don’t know about you but this image stole my breath. You can read all the details and download it for yourself here.

Second, Alexandra Swanson talked about Snapshot Serengeti, a personal favourite project for which volunteers identified animals in 1.2 million images taken by 225 camera traps across a 1,125 km squared of the Serengeti National Park in Kenya, to improve our understanding of the migration and interaction of its 1.6 million wildebeest and zebra. The project published its first paper recently, which got an unusually high amount of publicity; this is atypical of Citizen Science projects, which tend to receive more publicity for their launch than their results. Their paper swam against this tide to report what happened to the 10.8 million classifications that over 28,000 registered and around 40,000 unregistered volunteers contributed. It was fascinating to hear how she’d used the project’s data to explore the certainty of individual classifications and, more specifically, how many classifications were needed to be certain that any one classification was correct for different species. Because I’m a nerd and we have similar decisions to make on our project, I was interested to note their criteria for removing images from circulation:

  1. five consecutive “nothing here” classifications;
  2. ten non-consecutive “nothing here” classifications;
  3. ten matching (consecutive or non-consecutive) classifications of species or species combination;
  4. 25 total classifications (whether in agreement or not).

For each image, the team applied a plurality algorithm to produce a “consensus dataset”. Next, for each species, the team calculated the level of agreement between classifications, or “evenness score”; a score of 0 indicated total agreement (i.e. a species that was relatively easy to identify) and a score of 1 indicates total disagreement between classifications (i.e. a species that was relatively easy to identify). Finally, five experts produced 4,428 classifications from a subset of 4,149 images, which could be compared against the combined intelligence of the crowd. Remarkably, 97% of the volunteers’ species classifications, and 90% of their species counts, matched those of the experts. This data further allowed them to delve deeper into the 3% that were incorrect and find that there were two main types of errors: 1) false negatives, when volunteers missed an animal that was there, and 2) false positives, the flip side of the same coin, where volunteers classify something that isn’t there (which is relatively common for rare species such as rhinos that people get excited about). The analysis found that species with a high rate for one type of error, they had a low rate for the other. The interesting implication of this is the potential for dynamic removal of images from projects to help manage their increasing volume; researchers might consider some images to be considered correctly classified after only three matching classifications whereas other species may be more difficult to identify and require more matching classifications i.e. how many pairs of eyes do we need to look at an image? As an extension of this, researchers may also require different levels of classification accuracy, depending on the question they are using the data to answer. It’s worth pointing out that Greg Hines has also explored the weighting of classifications, not only by species, but by individual accuracy and quantified the increase in certainty obtained with each individual classification. Figure 2 shows the graph from this paper that illustrates that rarer species require more eyes for the same level of certainty.

Figure 2:  The number of users required to classify a photo for an accurate result increases according to the level of disagreement between their classifications (Hines et al, 2015).

Figure 1: The number of users required to classify a photo for an accurate result increases according to the level of disagreement between their classifications.

It sparked a conversation in the ensuing break between me and my project mate about how we will measure performance in our project and what we can use to assess the accuracy of classifications. As I said in my last post, the team at UCL’s Mullard Space Science Laboratory is developing an algorithm at the same time that we are developing the Citizen Science platform for iMars. Because both techniques are new we start with the hypothesis that the algorithm will successfully filter for images that show geological change, so that volunteers can do the more fun and interesting job of defining the change, which it is more challenging to code into the algorithm. How we will determine the successful performance of either, however, is still to be determined itself!

After the break Victoria Van Hyning presented the wide range of Humanities projects the Zooniverse team is working on. The main challenge that her work addresses is that computers cannot read handwriting and there aren’t enough experts in the world to create the data to train a computer and garner such rich insights into life/the world before print. She gave us a tour of her current and future plans.

  • Operation War Diary: 1.5 million pages of war diaries, which detail daily events on the front line, the decisions that were made and activities that resulted from them. The key learning of this project was not to make the tasks too complex.
  • Science Gossip: This is a collaboration between ConSciCom (an Arts and Humanities Research Council project investigating the role of naturalists and ‘amateur’ science enthusiasts in the making and communication of science in both the Victorian period and today) and the Missouri Botanical Garden who provide material from the Biodiversity Heritage Library, a digital catalogue of millions of pages of printed text between the 1400s and today related to the investigation of the natural world. Since March 2015, 4,413 volunteers have tagged illustrations and added artist and engraver information to 92,660 individual pages through this website; this will help historians identify why, how often, and who made images depicting a whole range of natural sciences in the Victorian period. This project has the potential for the development of graphical search functionality for these catalogues.
  • Ancient Lives: over 1.5 million transcriptions of the Oxyrhynchus Papyri. The neatest thing on this project is the Greek character keyboard (Figure 3), which users can become proficient without any expertise.
Figure 3: The Greek Character keyboard of the Ancient Lives project.

Figure 3: The Greek Character keyboard of the Ancient Lives project.

Future projects include:

  • Diagnosis London: a future project with the Wellcome Trust and University of Leicester, which will again go back to Citizen Science of the 19th Century and look at public health records.
  • Anno.Tate: a project with Tate UK to transcribe artists’ notebooks, diaries etc.
  • Shakespeare’s World: in collaboration with the Folger Shakespeare Library in Washington DC, seeks volunteers to transcribe records of life when Shakespeare was alive.
  • New Bedford Whaling Logs: a spin off from Old Weather that targets climate data and social history.
She is getting excited that, over the next 12 months, the new web-based Zooniverse platform, Panoptes, is going to be adapted for the Humanities to enable a granular approach to classification, which will facilitate: different levels of participation (so that volunteers can dip in an out of the same document without having to abandon it completely); mitigation of fatigue, and; construction of algorithms. Volunteers to will also be able to create what she called crib sheets of letters and/or words as their own personal reference, which has the potential to improve their consistency and proficiency and combine the crowd’s knowledge to alleviate any common misclassifications. Linguists are interested in taking this data further to examine the evolution of spelling and punctuation, which is often inconsistent. Audio and visual classifications are also something the Humanities arena might explore, with applications within domains such as archaeology.
The second half of the day was no less interesting so I intend to write it up separately in due course. (I haven’t got to the penguins yet!) Until then I would love to hear your thoughts, or recommendations for reading, on any of the issues I have highlighted in bold. They all represent elephants that have been stamping around in my head since I started out on this project, so I was very grateful to hear them aired at ZooCon and intend to remain mindful of them as my project progresses.