How to use real world data road shapefiles from OpenStreetMap (OSM) in ArmA 3.
In this tutorial we download OSM shapefiles from the web, crop them to our terrain size and reproject them to match terrain builder UTM zone.
If not otherwise mentioned, global mapper is assumed.
Geofabrik site has OpenStreetMap (OSM) shapefiles available for download.
Do not try to download “Asia” or “Africa” as those files are huge, click on Asia or africa instead and from there you can find individual country pages like iraq and afghanistan so your downloads are mere few megabytes instead of gigabytes (not to mention the unpacked data is insanely large).
OSM includes roads but also many other features, they even have buildings and vegetation areas. Although no idea how to use the building information with terrain builder or other tools.
If you cannot find country from geofabrik, try extract.bbbike.org instead.
OSM Roads Using Global Mapper and QGIS
We need BOTH of these software's to achieve OSM roads in terrain builder because global mapper v18 does not shift road shapefiles properly (v15 does apparently). If you know how to do this alone in either software please contact us. :(
You cannot replace OSM road features with arma3 ID/ORDER before you transform them! If you do, QGIS will fail the v.transform process.
DOES NOT WORK IN V18!:
What does not work in v18? Well the coordinates jump to some crazy large number 3999999999999900 something like that, I have no idea why and what I do wrong in that part. Read below how to transform roads in QGIS instead (or use older global mapper like v15 which should work).
Global Mapper ID / ORDER
Enjoy your real world data openstreetmap roads in terrain builder
In global mapper when you have chosen your terrain area by creating user created features grid, you need to export ASCII Grid ASC heightmap file to see the header for coordinates.
Use proper text editor to open this ASC file and check the header for xllcorner and yllcorner values. These values are used to calculate the terrain builder easting 200000 and northing 0 values.
Easting 200000 will be done by calculating 200000 - xllcorner value. And northing is dead simple, you just take yllcorner value as-is.
So now you have these two values, they can me inputted to QGIS to transform your project coordinates into those of arma3 terrain builder.
Here is example heightmap.asc header:
ncols 512 nrows 512 xllcorner 618849.895 yllcorner 3433172.322 cellsize 5
So you would take xllcorner and calculate that like this “200000 - 618849.895” equals to -418849.895 value. Then as terrain builder northing is just 0, we dont have to do any calculations we use it as is.
Result shift / transform would be:
X: -418849.895 Y: -3433172.322
ArmA 3 crashes if you have more than roughly 9000 roads / features in the roads.shp shapefile. This is not precise number as it also depends on your other terrain properties, highest we have had in-game is 9,198 and lowest that caused crash was 4,941 so its not exact science. This is only issue for those large terrains or well, terrains that have large number of roads in the shapefile.
Note: for example plain deserts (heightmap) of iraq easily allows OSM road features of 9000 but the rugged mountains (heightmap) of afghanistan crashes for less then 5000. So use your own judgement with all these gazillion variables when you track down the amount of roads your terrain can handle.
Mexico juarez 81km terrain had 7469 road features which caused virtual blocks crash. After global mapper 6m combine 4056 it still crashed. Finally after I cleanly reduced the features by deleting FCLASS roads to number I didn't write down (damn) and then 6m combine down to 1139 the crash was avoided.
OSM shapefiles come with all kinds of backyard “path” roads etc, you might want if not have to remove some of these.
OSM Road attribute types are described in openstreetmap map features: highway.
Use search → search by attributes. This search vector data dialog you can do all kinds of search & delete operations.
If you click to sort the FCLASS column, you can most likely see “unknown” and “unclassified” types, its up to you to decide which road types to remove (see the above OSM road attribute types list), as you really don't need all in arma3 terrain.
How to delete
Choose attribute/item dropdown for FCLASS, then type your search string to compare value field, then hit new search button.
Now you can click select all and delete selected buttons to delete all these search matches.
Move the search vector data window away from normal global mapper view and orient the road features so you can see them. For those pimps with triple screen setup this is awesome but its doable even with single screen.
You can see the selected roads highlighed in yellow.
While you click to delete roads, monitor the cropped_roads layer for feature count, once it reached 9000 or below you can stop because that is the rough limit arma3 crashes.
Once you have searched roads, you can search * to return all roads, because every search is only searching what is listed currently.
Check the FCLASS or type, sort it and browse to see all listed there. Sometimes there are really odd types like “yes”.
Once you're done with deleting “bad” roads, simply export your roads by file → export → export vector/lidar format, as usual.
ArmA 3 Specific Roads Deletion
Delete these roads as they are mostly too excessive for arma3 terrains:
bridleway steps foot* cycle* path pedestrian track_grade* *link* living_street service unknown unclassified residential tertiary track secondary
track* takes too many roads from ukraine luhansk, from 17k to 380 heh.
tertiary has lot of country roads in ukraine.
residential has those nice city roads, depends on your terrain if you need to delete these.
Anything below 9000 OSM features in roads.shp should work in arma3.
List of important features for arma3 gameplay:
motorway trunk primary
Simplify Road Polylines
Most likely your OSM downloaded road shapefiles are too complex for terrain builder, so you might need to simplify them. Meaning there are way too many vertices in the polylines which can be reduced. Note 2018-03-02 update: arma3 loads any OSM poly line shape file just fine, there is no need to simplify for arma3's sake, PMC has done hundred terrains with various size and quality OSM road poly lines and all of them have been working great.
If you import too complex or “large” road shapefiles to terrain builder it will start to stutter and freezes up.
In global mapper select all your road features, digitizer tool RMB → move / reshape features → simplify reduce vertices of selected line area features.
From enter simplification threshold dialog, use horizontal threshold 10 and vertical threshold elevation (meters) 0.2 for good results.
USA Los Angels 40km terrain openstreetmap.org road polylines even after “bad roads” reduction were over 17,000 features which is like ten thousand over the limit. They need to be reduced. Please note that this was some very odd issue on these specific OSM shapes because normal roads / streets were split up to hundreds of small features when it should have been one feature.
I have not experienced such splitting up of polylines in other terrains in this scale.
Select all roads → RMB in the viewport → crop/combine/split functions → combine. Then use about 6 meter value, should work fine (we used also 20m but that might be too large). In select combine options menu let it be default (combine any overlapping and everything else ticked).
When you are using OSM road shapefiles there are lot of crap info attributes which are not used by terrain builder, you must delete these and replace with ID and ORDER attributes. If you try to bring OSM shapefiles as-is to arma3, it will crash.
Terrain Builder uses ID and ORDER values for road shapefile database. See further details from PMC Ultimate Terrain Tutorial: Roads - Export.
Make road orders for roads and always connect a smaller road to bigger road so that the bigger road has smaller order than the road joining it. This way the joining road should go automatically under the road that it's joining.
If you don't use proper ID value, then you get some placeholder textureless road.
If your roads shapefile is faulty, for example if you forgot to tweak the ID, ORDER etc values when exporting from Global Mapper or QGIS, then arma3 will simply crash when you load terrain using the shapefile.
The .shp and the .osm packages are “nominally” the same. the main visible difference being the shapefile downloads are individual “layers” and the ”.osm“ is “All-In-One”.
However, you'll sometimes find that the ”.osm“ version of an area has more detail than the equivalent shapefiles pack, or there's additional layers not included in the matching .shp pack. So it's always a good idea to grab both sets of data for any area you're seriously interested in, and check them both out thoroughly - cherrypick the best of the useable data from whichever seems best.
You'll find that the .osm's seem to consist of all the shapefiles muddled together on a single layer… However, all the individual shapes are carrying “metadata” - they have ID's, names, types, etc…
Most GIS programs, GM included, have a “metadata search” function which will allow you to search that whole jumbled layer of shapes and select only those labelled “residential road” for example, or “highway”… you can then select only that “class” of shape and isolate them from the rest by copying to a new layer…
Keep doing that and you can extract a whole bunch of useful layers from lakes & rivers, thru roads, vegetation areas, agricultural areas, etc, etc… with OSM it's all down to luck as far as coverage and quality is concerned, but most times you should be able to acquire some useable stuff, and save a considerable amount of time….