Loomio
Sat 6 May 2017

Getting the UK map going...

NW
Nick Whitelegg Public Seen by 584

Thread to discuss the development of the UK map, including feature discussion, technical issues and other issues such as obtaining a server, server configuration/software installation etc.

NW

Nick Whitelegg Sat 6 May 2017

Hi,

Adam has invited me to lead the project to develop the UK map, so I'd like to kick things off with a view to actually getting development started as soon as we can. I'd like to discuss both getting a server and development issues/ideas.

The first issue is that we need a server. For my Freemap project I've been using Bytemark for many years and have been very happy with them: they provide a VM with full root access. Price list is at

https://www.bytemark.co.uk/prices/

They have a flexible model where you can start with a basic server and scale up. For Freemap I use the basic £10/month offering; for a production UK map server we'd obviously need something more than that, but for initial development, something like that would probably do.

An alternative is Digital Ocean (https://www.digitalocean.com/products/compute/) who offer a similar product; overall it looks a little cheaper, particularly regarding data transfer, though they're US based and charge in dollars. I have no experience of Digital Ocean... anyone?

What shall we do about actually physically paying, once we've selected a server provider... do we have an account?

The other topic I'd like to discuss is the actual development. I'm happy to do much of it myself but obviously I can't do everything - and I am a developer rather than a web designer, graphic designer or UI/UX expert. So we need volunteers for those aspects in particular.

In terms of server-side implementation language I'd suggest PHP unless there are strong objections - basically it's the server-side language I have most experience in, and I already have a good deal of code I use for Freemap which could be reused for certain features (see below).

So in summary I'm happy to do much of the following (with some help from other contributors, as obviously I can't do it all...)
* Install and configure PostGIS, Mapnik, a tile server etc
* Client- and server- side coding (JavaScript, PHP or equivalent)
* some work on the Mapnik style file, though would need input from cartography specialists too

but the following aspects would be better done by others:
* Web design/UX
* Graphic design, if appropriate

There seems to be a consensus towards Landranger style rendering, is this still the case?

Detailed features are probably left for further down the line, but here are some ideas, based on a walker's perspective admittedly:
* the map should definitely have contours. On Freemap I've always used the OS OpenData height data as it's the best available, I presume there are no legal issues with this if they're in a separate DB and then combined with the OSM data by the renderer?
* make the map highly interactive, e.g. clickable POIs, report footpath problems, add missing tags to ways, allow users to create walk routes etc.

Many of these are already implemented by Freemap in PHP, so with some cleanup of the code could be incorporated into OSM-UK; this is why I suggest PHP as sever side language.

  • users to use their OSM account to login (OAuth)

I still like the idea of client-side, vector rendering using technologies such as Kothic-JS, however not sure if it's quite mature enough for a project as high profile as this.

Any thoughts, or volunteers? As I said, first priority is to get the server.

Thanks,
Nick

R

RobJN Sat 6 May 2017

Hi @nickwhitelegg Thanks for taking this one on :-D

We're in the process of getting the bank account up and running (they have asked for more info from us) and then need to do a membership drive. After that we should have funds available.

My suggestion is to focus on some of the other parts first, perhaps as a demo/pilot. This way we are not waiting on the server.

By all means, please share server options (we're looking at this at the moment so the timing is great). We'd like email with webmail access so if there are options to bundle this with the server cost then that would be great.

NW

Nick Whitelegg Sun 7 May 2017

Hi Rob, no problem :-) Will do my best...

How about I get a low-spec Bytemark server going now and then claim the payments I make in the meantime (which is only £12/month, inc vat, so not a huge bind) back when the account is ready and funds available? Would this be possible? I can always ask Bytemark whether it would be straightforward to change the account details when the OSM UK account is ready - I'd presume it would be.

This way we'd have some dedicated server space to play around with and people could have shell logins - rather than using my own server or people developing stuff at home.

I'm almost certain Bytemark or similar providers would allow you to set up an email server... it's basically full root access so you can do what you like, within reason

R

RobJN Sat 6 May 2017

P.s. If you like, I can send 1 to 2 short paragraphs (4-5 sentences each) to those subscribed to the email newsletter. Let me know.

NW

Nick Whitelegg Sun 7 May 2017

That would be great... how about something like the following (feel free to edit):

Hi, I am leading the project to get the OSM UK map going and am looking for volunteers! We aim to have dedicated server space very soon so would be good to kick off the development of a prototype/demo, focusing on a small part of the UK (say Hampshire and surrounding areas). The aim is to produce a UK "slippy map" and accompanying website, so I'm particularly looking for people with skills in web design, UX and graphic design... and cartographers too.

I'm happy to do much of the server and client side scripting and server software installation, but would still welcome help here too. I'd like to suggest PHP as the server side language unless there are any strong objections - reason being I have some code developed already which could be used to implement some of the more interactive features of the map such as clickable POIs, annotations and walking routes.

So please let me know on Loomio or via email (nickw4426@gmail.com) if you're interested in contributing, or have any ideas for input!

Thanks,
Nick

R

RobJN Sun 7 May 2017

Hi @nickwhitelegg

We will discuss at the next meeting, but don't let this stop you starting any work that does not require money (e.g. cartography decisions).

NW

Nick Whitelegg Mon 8 May 2017

Hi @robjn OK that's fine. Where is the best place to discuss cartography issues... on here? I have a few ideas but these are largely related to the needs of walkers.. so need more input for urban mapping. Are enough people a member of this group? Or is it best done on the mailing list?

R

RobJN Mon 8 May 2017

Up to you @nickwhitelegg. If you think the decision tools here will be useful then stick with Loomio and maybe just post an invite to the mailing list.

AM

Alex McKee Wed 10 May 2017

@nickwhitelegg , saw your post on Talk-GB. I'm a PHP developer and I have some experience with OSM rendering stack as well (Postgres+PostGIS, Mapnik). If you need extra hands for the development side of things I'd be interested in helping. I've been contributing to OSM since 2008.

NW

Nick Whitelegg Thu 11 May 2017

@alexmckee thanks!

AA

Andy Allan Thu 11 May 2017

Unfortunately I don't have the time to help with this, but I hope you don't mind some back-seat driving. My main piece of advice would be to take the time to reduce the scope of the project as much as possible - from experience, I know that creating the server software, developing the cartography, and building a team from scratch to do either are three big challenges. So I would suggest a quick win is to commit to only using off-the-shelf tileserving software, whether that's mod_tile, tessera, tileserver-gl, tilestache or alternatives - but avoid coding another one.

NW

Nick Whitelegg Sat 13 May 2017

@gravitystorm Thanks for that - yes, the plan was to use an existing tileserver - the server-side development I had in mind was more for additional features over and above the rendering e.g. allowing contributors to add missing designation tags, etc, and sync them with the main OSM database. I'd be looking for something which allows a fair degree of control e.g. allowing tiles to be re-rendered on demand, I was thinking that one nice feature would be to allow users to add new pubs (say) and have them re-rendered instantly and synced with the main OSM database. But this is probably all subject for a future discussion.

DB

Dennis Bauszus Thu 11 May 2017

I am happy to help with some coding and the UX. I doing a lot of node.js / express stuff and work mostly with Leaflet but also with Openlayers.

NW

Nick Whitelegg Sat 13 May 2017

@dennisbauszus thanks!

NW

Nick Whitelegg Sat 13 May 2017

Thanks everyone who's joined this group. Would be great if we can have some discussion about the cartography, as I have some ideas and have done my own custom rendering, but am by no means a cartography expert.

Most ideas I do have relate to countryside mapping. Earlier discussions suggest something inspired by Landranger; I would agree as it is the style most familiar to UK walkers. A combination of Landranger style rendering and Explorer style detail? Perhaps something like http://geo.gy/ (Grough Map) but with Landranger style rights of way and, crucially, field boundaries?

Contours should of course be included, are we legally OK using the Ordnance Survey OpenData for this if it's on a separate database and composted with the tiling software (this is another reason why a flexible and scriptable tileserver would be nice)? Hill shading (from the same source) would also be good.

I would also include stiles and gates at higher zoom levels, and maybe signposts too. What would be nice in mountainous areas is to indicate crags in the same way the OS do on their Explorer maps.

Have different types of field boundary rendered differently? e.g. fence, hedge and wall? Render ditches as a "sunken" effect?

Show permissive bridleways and footpaths in a less intense version of the RoW colours, perhaps magenta? Early versions of Freemap did this; OS use orange but this maybe clashes with contours.

Cycle paths in blue?

Pubs, teashops and restaurants with icons?

Anyway that's a few ideas, but obviously we need a good deal more, so would be great to see a discussion on this.

DI

Dudley Ibbett Sat 13 May 2017

Hi

One question I would ask at this stage is what is the overall intended purpose? Is this a “shop window” map that shows what can be done? A map designed for people to use to navigate in rural/urban areas? Is it a map that can be used for QC specifically related to the UK. i.e. PROW tagging? I’m sure it could be a mixture of all three but colour coding to identify tags, whilst useful for people editing the map may be a bit confusing for people that want to use it for just navigation.

I guess one answer to the above is to have more than one rendering.

If the intention is to mark rights of way then this may not be easy if you also want to differentiate the access=private tag on a highway that the main OSM site does. I would also suggest you may need to move away from a single dashed line for a track and adopt the OS double dashed line so the PROW line can be drawn in the middle.

Having just had someone delete a service=driveway tag arguing that the OSM site display footways at a zoom level before this I would suggest taking a careful look at when highways appear so driveways link up to footways at all relevant zoom levels.

If we are looking to OS like but also differentiate ourselves from the standard OS rendering:

Andy’s map differentiates between walls/fences and hedges and renders them at low zoom levels. It is also a useful feature for QC. It would be challenging to differentiate between walls and fences but would show what OSM is capable of. https://map.atownsend.org.uk/maps/map/map.html#zoom=15&lat=53.02012&lon=-1.73005

Rendering, gates, stiles, kissing gates would be good. As you get older it is good to avoid step over stiles!

One big advantage of OSM is the detail you can put in when navigating through farmyards and residential areas (ex farmyards). OS maps lacks this and quite often this isn’t accurate. Ensuring this information is clearly rendered would be good.

I would advocate trying to render sidewalks, particular along main roads in rural areas. It is useful to know about these when walking in the countryside and OS doesn’t do this.

Finding a “soft” way to map the random “natural” tree rows that appear along rivers, streams & railways lines would be good.

Tea shops would be very good and again aren't on an OS map.

Hopefully some of the above comments are of some use.

Kind Regards

Dudley

NW

Nick Whitelegg Mon 15 May 2017

@dudleyibbett Thanks for your input - yes these are all good ideas. I'm wondering whether (as a long term thing) we should aim to actually offer different renderings in different styles? Or maps which look different at different zoom levels? Something Landranger-like at 14, Explorer like at 15 and 16 and "standard OSM" like at higher zoom levels (but with UK conventions for road colours, green for trunk etc) - and something resembling a standard road atlas at lower zoom levels?

Wondering whether to start specific threads for a) rendering and b) proposed features, so we can discuss each more fully.

S

SomeoneElse Tue 30 May 2017

Hi all,

At this stage I'd pick something relatively simple to deliver and concentrate on delivering that. That might be something as "simple" as a transparent PRoW layer that can be overlaid over an existing map style, or a new style of interactive map UI (as mentioned in another thread) or something else entirely, but trying to do "everything new" at the very start is a huge ask.

Dudley's mentioned map.atownsend.org.uk above - if you end up going the osm2pgsql/lua/carto/mapnik route then you may well be able to borrow from that map's lua and stylesheet parts. It's definitely worth taking more time than I did thinking about the lua side though - don't just let it "evolve".

I'd also suggest doing whatever's necessary to help people directly contribute - people definitely struggle with the likes of switch2osm and the ircama tutorials, but it's much more useful to be able to say "here is $idea implemented for one county, see what it looks like" than "here is $idea".

I'd also suggest using existing OSM help resources wherever possible (help.osm.org, IRC, the "tile serving" and "dev" lists etc.). Many of the people able to help with advice won't be from the UK and won't know about a discussion on here, but they may see a discussion elsewhere.

Best Regards,

Andy

AH

Adam Hoyle Wed 31 May 2017

Hi Nick,

There are an overwhelming number of options and I agree with Andy about keeping it simple and going for something deliverable - which can always be built upon.

Loomio has a recently added 'dot poll' option - so maybe it's worth putting all of the options that have been discussed onto one and seeing what gets voted for most and starting there?

Best,

Adam

R

RobJN Fri 2 Jun 2017

Hi @nickwhitelegg

I'd echo the keep it simple philosophy. We are going to draft up some high level plans for the other projects. Perhaps you might want to do the same (even if just to help you keep track of things).

On the subject of servers, we are currently exploring some options here whilst waiting for the bank account to be opened (still awaiting bank response). It was pointed out that tools such as Jawg provide a free service with 50k views per month and 2 custom tile sets. This might help you get started on playing with some cartography styles....?

NW

Nick Whitelegg Sat 23 Sep 2017

An update on this: if you're not aware we now have server space for developing the UK map website and there's now a GitHub organisation (osm-uk) and repo (osmuk), i.e URL = https://github.com/osm-uk/osmuk. Please let me know if you want read/write access to the repo.

S

SomeoneElse Fri 29 Sep 2017

Hi Nick,
Any reason why you appear to have created a new, empty https://github.com/osm-uk organisation? https://github.com/osmuk already exists and someone who I had assumed was you https://github.com/nickw1 is a member.
Regards,
Andy

NW

Nick Whitelegg Sun 1 Oct 2017

Hello Andy, I'd forgotten about that one initially... also I don't have admin rights on the 'website' repository within it so can't invite other people.

I've now created a 'website_real' repository in 'osmuk'. Will use this from now on; someone with admin rights on 'osmuk' please feel free to delete 'website' and rename 'website_real' to website.

Adam, Robert, Jez, James - please note the real repository is now
https://github.com/osmuk/website_real

NW

Nick Whitelegg Sun 24 Sep 2017

Now first commit added to repo. Not much there (just a map and a non-functional login form) but the project is now up and running!

The site is located at
http://47.91.91.133/dev-nick-osm

JN

Jez Nicholson Sun 24 Sep 2017

Hi Nick,

I'd like to join the github team: jnicho02

Regards,
Jez

NW

Nick Whitelegg Tue 26 Sep 2017

Hello Jez, have just sent invitation to you and Robert.

RW

Robert Whittaker Sun 24 Sep 2017

Can I join the github project too? I'm rjw62 on github. Thanks!

NW

Nick Whitelegg Tue 26 Sep 2017

Hello Jez and Robert, have just sent invitations to you both.

NW

Nick Whitelegg Mon 16 Oct 2017

Quick update on this - sorry, there's been some delay getting the permissions I needed on the BiRT server hence no updates for the past couple of weeks. The bad news, for the moment, is that the server space given to us on the BiRT server will only be available until the end of the month, as they are moving to a new server; we should have this server available to us at some point but maybe a month or two after that. Good news though is for the moment I will have superuser access on this server (for the next 2 weeks or so) so can experiment with playing around with the stylesheet - once this is working then I can add a very basic Mapnik stylesheet to the git repository and/or the existing OSM stylesheet which people can then obviously contribute to and/or edit. Also Alan (sys admin of BiRT) is going to recommend some cheap hosting options which we could use for a longer-term solution to hosting the map server.

If absolutely necessary I can also use my Freemap server as an interim solution - if you need an account on that (and you're someone I know/trust) I can always give you access.

NW

Nick Whitelegg Fri 20 Oct 2017

Further update. Got a mod_tile setup going on my own Freemap server using only Hampshire - which in many ways is a good county as it has a mix of urban, rural and coastline so a good place to develop any walkers'/PRoW map features. Until the new BiRT server comes online (or we get alternative hosting - Alan from BiRT is coming back to me on this) I'll be using this as the test environment. I will transfer the basic OSM-UK map page here shortly.. will let you know when this is done.

In terms of developing a rendering, any opinions on whether to start something from scratch or just adapt the default OSM style? At the moment I've got it going with v2.45.1 of the standard osm style (newer versions need mapnik 3 which isn't available by default on my distro.. though if necessary I can install it from source)

RW

Robert Whittaker Fri 20 Oct 2017

There would be advantages in terms of developer time and user familiarity if we just piggybacked on the default OSM style and added our own GB tweaks on top -- such as fixing the road colours, PRoW and stile rendering, welsh language support etc.

NW

Nick Whitelegg Sat 21 Oct 2017

OK, now got the site running on the Freemap server, see http://www.free-map.org.uk/osmuk/. Due to server constraints only Hampshire and West Sussex are covered but this will be enough for a proof-of-concept. Login not working, but will shortly use oauth to authenticate with the main OSM server. Using v2.45.1 of OSM style, as I indicated yesterday - so will stick with this unless there are any objections. GitHub repo at https://github.com/osmuk/website_real, so if anyone wants to hack on the map stylesheets or contribute website code, we're good to go!

NW

Nick Whitelegg Sun 29 Oct 2017

Was having a chat with Jerry yesterday and we came to the decision that it might be better to use Andy Townsend's map style as a basis for this as he's already done a good deal of work on the sort of issues we have to deal with and it will save a lot of effort having to re-invent the wheel, so to speak. Andy - I presume you're happy with this?

NW

Nick Whitelegg Sun 29 Oct 2017

... this is now up on http://www.free-map.org.uk/osmuk. This is Andy's style, very slightly modified to show footpaths and bridleways in Landranger style. Hampshire only.

S

SomeoneElse Sat 2 Dec 2017

Just a quick note - if no further changes have been made other than "red bridleways" it might be worth re-fetching and reapplying those changes, as there have been a number of changes to the parent style as described at http://map.atownsend.org.uk/maps/map/changelog.html . I suspect that the historic stuff is most worth having (see e.g. Stonehenge http://map.atownsend.org.uk/maps/map/map.html#zoom=20&lat=51.1788263&lon=-1.826101 ) and in the light of the current Shell petrol stations work the changes to show operator and/or brand on those is worth having too.