Loomio
Fri 15 Jun 2018 6:08AM

How would you measure OSM UK?

JN Jez Nicholson Public Seen by 52

I'm collecting ideas on how to measure how successful OSM is in the UK. How would you, personally, measure 'success'?

A number of measures already exist, what are they?

The whole thing can be flawed. 'Success' is a subjective term. No single measure tells the whole story, and many numbers can be misinterpreted. But let's see if it possible to find some figures that can guide us.

Suggest some and i'll collate them into a dot vote or similar.

Criticism of a measure is welcome, but please be gentle.

JN

Jez Nicholson Fri 15 Jun 2018 7:26AM

Brand Recognition: How many people have heard of OSM, split by different groups. I would expect more Software Developers to have heard of OSM than members of the general public have.

To test it out I did a straw poll in a Freelancers Facebook group: 107 saw the poll, 11 knew OSM, 15 different...81 abstained.

RW

Robert Whittaker Fri 15 Jun 2018 10:17AM

Ideally, I'd measure success by the number of instances of people using the map (and map data) in the wild. This would include maps on websites displaying stuff, printed and web "find us"maps, and people emailing map links to each other. It's not obvious to me how to actually measure this though. Globally the number of search engine results for "OpenStreetMap Contributors" might be a useful measure. It would be a bit of work, but perhaps we could annually check all the FTSE 100 companies, central government departments, and top-level local councils, and see how many include maps on their websites, and how many of those are OSM based.

GMD

Gregory Marler (former Director) Fri 15 Jun 2018 11:28AM

In the directors meeting Jez also asked how to measure active mappers around the UK (e.g. is Birmingham more/less successful than Southampton). The website that first srpung to mind was Overview of OSM Contributors which I think might take mapper locations from hdyc the main editing in last 6 months. This might not be ideal for identifying people that edit in multiple places, and I think even if a mapper mainly edits abroad from the UK then it is a success(of one metric) for OSM in the UK.

Got point from Robert on how much people use OSM. There are 2 measures within that too, as increasingly people will use OSM (Pokemon Go, Snapchat, Apple Maps, and website no longer using Google) without knowing about OSM. Even if it has a visible acknowledgement, people might not read that. The map style is big for branding (and previously caused resistance to not using Google), but we cannot rely on that. This isn't helping find a metric!

When I give talks I start with asking for a show of hands: heard of OSM, knowingly used OSM(such as visiting the website), edited OSM. I should've recorded the results over the years, but I've only used it for identifying/confirming the audience I'm about to speak to.

JN

Jez Nicholson Mon 18 Jun 2018 6:43AM

  • Frequency of city/area meetups
  • Number of members of OSMUK
  • Postbox/post-office/FHRS/etc. completion rates
N

Neil Tue 19 Jun 2018 10:34PM

Try routing from Lands End to John O'Groats?

Foot is odd: https://www.openstreetmap.org/directions?engine=graphhopper_foot&route=50.06523%2C-5.71368%3B58.64401%2C-3.07003#map=6/54.038/-4.731
Car ones don't work, bicycle does :-)

What I was really interested was how distances compared with other mapping services -- a test of the overall "system".

BP

Brian Prangle Wed 11 Jul 2018 10:09PM

Depends whether you want to measure the map or the community. One that might do both is based on Notes - insertion rate, geographic spread, clean up rate

JN

Jez Nicholson Thu 12 Jul 2018 7:10PM

I'm still vague on whether i'm measuring the health of the map, or the health of the community. Maybe the two go hand-in-hand. I'm sure that there won't be one single measure that tells the story.

There was the ITO World https://wiki.openstreetmap.org/wiki/OSM_and_OSL_differences_analysis

@brianprangle just reminded me about Pascal Nice's work on an OSM 'Rank' and other measures http://neis-one.org/

Also, in my past work with marketing people they have a number of measures like RFM (Recency, Frequency, Monetary-value) with variations like RFE (Recency, Frequency, Engagement) that might be applicable - i.e. how long is it since you last mapped, how frequently do you map, and how long do you edit for https://en.wikipedia.org/wiki/RFM_(customer_value)

I intend to bring this discussion to a close soon and pick some to start with.

TS

Tony Shield Fri 13 Jul 2018 10:06AM

I would be looking at four areas
* People who collect data and update the database
* users of the database for whatever purpose
* accuracy and relevance and completeness of the database
* leadership, management and administration of the OSM UK community and database

Some of these areas can be measured objectively with tools available, others can be subjective; most are a combination of both. Combining all of these together could be difficult with a degree of subjectivity. A fundamental question is "Is OSM UK growing"

My view about the contributing people is to look at the history of UK based users - where are they contributing, frequency of contribution, if someone signs up are they active or dormant - are there any patterns of their contributions.

Assuming that contributors are individuals are the consumers individuals or organisations or a mixture. How can we measure their satisfaction with the database?, relevance of the data offered, completeness and accuracy? The routing example given previously illustrates the point well.

Accuracy and Relevance and Completeness of the database - questions such as is the data consistent? - recent footpath analysis and questions lead the way here. Assessment of the Post Boxes challenge tells us about the completeness of the data and the health and distribution of active contributors. Analysing previous challenges is also relevant.
Leadership, management and administration - always so difficult to measure or assess, much depends on the assessment tools available and used. Are there targets paths and plans? are they being followed? Are they reasonable and are they dependant on resources? Can resources be increased or organised?

An agreement of methods of data collection and how to analyse them is essential - questionnaires need to be tested before they are issued and the range of possible answers analysed to test if the questions add value.
Assessments such as these are of limited value in isolation, several iterations will give greater insights, size of assessment report is not relevant but the quality is and the report in itself is an indicator of the health of OSM UK.

Do hope this is helpful.

JN

Jez Nicholson Mon 16 Jul 2018 10:33AM

The new https://mym.rtijn.org/ tool pulls 'mappers near you' based on an OSM relation id.

JN

Jez Nicholson Tue 16 Oct 2018 9:38AM

I've been pointed at Osmand http://osmand.net/osm_live as well, for lists of active users

Load More