User Tools

Site Tools


arma3:terrain:real-world-data-tutorial

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
arma3:terrain:real-world-data-tutorial [2017-02-07 11:01]
snakeman created real-world-data-tutorial initial page.
arma3:terrain:real-world-data-tutorial [2024-03-01 04:59] (current)
snakeman updated.
Line 1: Line 1:
-====== ArmA 3 Terrain ​Real World Data ======+====== ArmA 3 Real World Data Terrain Tutorial ​====== 
 + 
 +**2024-03-01T04:​59:​00Z Update**
  
 **How to create ArmA 3 terrain using real world data tutorial.** **How to create ArmA 3 terrain using real world data tutorial.**
Line 5: Line 7:
 This tutorial assumes you have already experience making arma3 terrains, if you do not then please read [[arma3:​terrain:​ultimate-terrain-tutorial|PMC Ultimate Terrain Tutorial]] first. This tutorial assumes you have already experience making arma3 terrains, if you do not then please read [[arma3:​terrain:​ultimate-terrain-tutorial|PMC Ultimate Terrain Tutorial]] first.
  
-Overview is that you download heightmap from OpenTopographysatellite with Terra Incognitaroads from Geofabrik ​and compile all the files together using Global Mapper and QGIS.+This tutorial uses web browserglobal mapperQGIS, terra incognita ​and [[arma2:​texturing:​imagemagick|Imagemagick]] to get terrain source ​files into terrain builder.
  
-At the time of writing this tutorial we unfortunately have to use both Global Mapper ​and QGIS, there is no single tool method that we knowif you can help please [[:​contact|contact]] us.+Important Note: Global Mapper is commercial software which costs like $500**you cannot make ArmA 3 terrain of decent size with the TRIAL version.** If you do not have full / commercial version of global mapper I suggest you read Ross's QGIS real world data tutorial instead, QGIS is a free open source software.
  
-Individual pages for [[arma3:​terrain:​real-world-data-global-mapper|Real World Data Global Mapper]] and [[arma3:​terrain:​qgis-real-world-data|QGIS Real World Data (WIP)]]. 
  
 +====== Choose Terrain Location ======
  
-====== UTM Projection ======+Use [[https://​www.google.com/​maps/​|google/​maps]] or [[http://​www.arcgis.com/​home/​webmap/​viewer.html?​useExisting=1|ArcGIS]] to find terrain location somewhere on the globe.
  
-ArmA 3 terrain builder uses **UTM zone 31N** with the odd easting 200000 and northing 0 values.+These browser based maps are so much quicker / lightweight to use than global mapper.
  
-If you are downloading real world data from some other UTM zone than 31Nthen you must use that locations zone insteadotherwise the data is all stretched to almost beyond recognition.+In our tutorial example we chose [[https://​www.google.com/​maps/​@47.2590802,14.3896057,7.75z|Austria]].
  
-For example **Iraq WGS 84 UTM zone is 38N** instead of 31N. If you are downloading data from Iraq, use Global Mapper / QGIS WGS 84 UTM zone 38N. And if you are downloading data from Afghanistan,​ use WGS 84 / UTM zone 42N, etc. 
  
-You can search "UTM zones <​countryname>"​ with google, for example time of writing we found [[http://​georepository.com/​crs_32638/​WGS-84-UTM-zone-38N.html|georepository for Iraq]].+====== Create Global Mapper Project ======
  
 +Open global mapper. We use v18.2 in this tutorial but mostly the commands are the same even with v15 or even v13.
  
-====== Finding Location On Globe ======+===== Setup Map View =====
  
-Use your web browser and go to maps.google.com and navigate to the place which interests you. Web browser seems to be much quicker and "light weight"​ to operate than using street ​maps in global mapper.+Choose online sources -> worldwide data -> world street ​map, but dont click connect just yet.
  
-RMB position -> whats here? -> click the link that popped up bottom center of browser. Now the coordinates appear in the navigation bar on left side of the browser with coordinates of this location, ​from there its easy to copy paste them to your text notes.+Global mapper is damned sluggish when you browse ​the maps, thats why we used google/maps to browse our location ​first so we know excatly where we want to go. So you //can// to tweak some detail levelsthis is not necessary but especially for beginner ​its so much nicer to browse with quicker global mapper.
  
-You can also use Global Mapper -> Connect to Online ​data (Online Sources) ​-> World street map, to open street maps which show you locations in countries just like maps.google would. However maps.google is much faster / optimized and has some search plus additional features.+{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Online-Source-To-Download.png }}
  
 +In display options box, use the detail level slider to bring it down / left about 0.25 or even lower if you want. Then click connect.
 +
 +===== Zoom To Target =====
 +
 +Now you see the whole world map. Use Zoom icon (ALT-Z) which should be selected, then drag a very small box around the austria we checked out in google/​maps.
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Zoom-Icon.png }}
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Globe-Map-View.png }}
 +
 +You might have to zoom few times if you dont get it right the first time, its easy to zoom at wrong spot etc.
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Zoomed-In.png }}
 +
 +You can drag the map around with having the hand icon pan (ALT-G) selected and then just LMB hold and drag map around, mouse wheel zooms but be careful as the loading of new data is so damn sluggish (even on lower details).
 +
 +===== Switch to UTM Projection =====
 +
 +Now that you are close enough, its not that important how well aligned the map is it just has to be roughly in the area, we switch into UTM projection.
 +
 +Select Tools -> configure -> projection.
 +
 +Select from the projection dropdown UTM.
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Configuration-Projection.png }}
 +
 +You notice that in austria global mapper offers UTM zone 33, this is fine (yes terrain builder uses UTM 31N but we'll deal with that later).
 +
 +Click apply to start converting the current project into UTM projection.
 +
 +This will take a long time, global mapper doesnt show any progress bar and feels unresponsive,​ pe patient and //wait//.
 +
 +When its done, click OK to close projection dialog.
 +
 +Now your project is on UTM projection.
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-UTM-33N-Projection.png }}
 +
 +
 +===== Create Grid =====
 +
 +Global mapper user created feature grid is the one that will be your terrain area. Click create grid icon on top menu bar (no hotkey, hoh).
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Create-Grid-Icon.png }}
 +
 +Now click somewhere on the map view and it brings up the grid setup dialog.
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Grid-Setup.png }}
 +
 +Type in width / height values for your terrain size, type in 40960 which is 40km size for terrain.
 +
 +Then click OK to finish the grid.
 +
 +===== Moving Grid =====
 +
 +Now that the grid has been selected, you can move it with (CTRL-SHIFT-M),​ RMB -> move area feature or with main menu bar move selected features icon.
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Move-Selected-Features-Icon.png }}
 +
 +Here I have moved the grid to location of my liking, also zoomed the map in a bit and panned it around bit more center.
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-User-Created-Feature-Grid.png }}
 +
 +As the terrain location and grid is finalized, its time to save the project. I use filename "​Tut_Austria_40km"​ when saving.
 +
 +
 +====== Terra Incognita ======
 +
 +Terra incognita is used to download the satellite imagery. You could also download straight from global mapper but it will crash very often, almost every time while downloading large satellites so its not really a good idea.
 +
 +Open terra incognita and select map source -> openstreetmap.
 +
 +Now on global mapper untick world street map layer (we dont need it right now), then click the globe button.
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Connect-To-Online-Data-Icon.png }}
 +
 +Select openstreetmap.org global street maps, make sure select area to download -> current screen bounds is selected, then click connect button. Note; you can again choose detail level but as we already are in pretty close zoom it wont be necessary.
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Connect-To-Online-Data-OpenStreetMaps.png }}
 +
 +Have digitizer tool selected.
 +
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Digitizer-Tool-Icon.png }}
 +
 +Then click somewhere outside the grid to unselect it.
 +
 +Use terra incognita to zoom close enough to the global mapper area. It is very difficult as there is not automated way to align these two, you just have to use mk.1 eyeball to align them.
 +
 +{{ https://​pmc.editing.wiki/​images/​Terra-Incognita-Align-Map-View-To-Global-Mapper-Grid.png }}
 +
 +In terra incognita use selection -> rectangle selection, now you are in the box drawing mode.
 +
 +Drag a box and then resize it by dragging the edges so it will be about 45 x 45 kilometers. Do not click on empty dark map location because that clears the selection. You can LMB drag the selection around.
 +
 +{{ https://​pmc.editing.wiki/​images/​Terra-Incognita-Selected-Rectangle.png }}
 +
 +Then use selection -> rectangle selection again to disable the dragging.
 +
 +Now use selection -> copy to clipboard. This will copy the lat/lon coordinates into clipboard, save these coordinates to some text file. Here is mine (but dont use these to cheat haha):
 +<​code>​
 +47.626772N 13.061371E, 47.626772N 13.666992E, 47.213807N 13.666992E, 47.213807N 13.061371E
 +</​code>​
 +You can also align the map location to these coords by using go to -> paste latitude longitude, the map will be centered to lower left corner of the coordinates.
 +
 +Select map source -> esri arcgis -> esri arcgis, this is the arcgis world imagery data.
 +
 +{{ https://​pmc.editing.wiki/​images/​Terra-Incognita-ESRI-ArcGIS-World-Imagery-Source.png }}
 +
 +Now use zoom levels -> level 17. You can also center the map location as said above, but it doesnt matter, its just nice to see the red selection rectangle there heh.
 +
 +{{ https://​pmc.editing.wiki/​images/​Terra-Incognita-ESRI-ArcGIS-Zoom-Level-17.png }}
 +
 +Now you are ready to download, create directory like Tut_Austria_40km_arcgis_17_level and use file -> save map -> oziexplorer map. Browse to the dir you just created and save under Tut_Austria_40km_arcgis_17_level filename.
 +
 +Terra incognita asks "very large area selected! continue?"​ click OK to start the download.
 +
 +{{ https://​pmc.editing.wiki/​images/​Terra-Incognita-Downloading-Oziexplorer-Map.png }}
 +
 +Download has started, just let terra incognita be, dont use it to make sure it wont crash etc.
  
 ====== Heightmap ====== ====== Heightmap ======
  
-Best elevation data source available ​is SRTM, read all about [[arma3:​terrain:​heightmap-real-world-data|Heightmap Real World Data]] downloading.+Back in global mapper make sure digitizer tool is selected and double click on the grid to bring up modify feature info dialog.
  
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Modify-Feature-Info.png }}
  
-====== Satellite Texture ======+Delete the "​A1"​ string in the name field, the click OK to close.
  
-Best satellite texture option is [[arma3:​terrain:​satellite-texture-terra-incognita|Satellite Texture Terra Incognita]]. That page explains how to download satellite texture in OziExplorer .MAP format which can be opened in Global Mapper and QGIS.+Now you can untick openstreetmap map view too.
  
-Not usable but just in case you have it [[arma3:​terrain:​universal-maps-downloader|Universal Maps Downloader]].+Use online sources and choose STRM worldwide elevation data (1-arc-second resolution, SRTM plus v3), make sure current screen bounds is selected.
  
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Select-Online-Data-Source-To-Download.png }}
  
-====== Roads ======+Click connect to load it up, this takes few moments so be patient and wait.
  
-You can compile all other files together in global mapper, except roads. For roads you need to use some QGIS which is all explained in [[arma3:terrain:​road-shapefiles-real-world-data|Road Shapefiles Real World Data]] page.+{{ https://​pmc.editing.wiki/​images/​Global-Mapper-SRTM-Heightmap-Online-Source.png }}
  
 +Use file -> export -> export elevation grid format
  
-====== Tutorial ======+{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Export-Elevation-Grid-Format.png }}
  
-**Find a spot**+From select export format dialog choose dropdown of Arc ASCII Grid.
  
-Start by finding location in globe as described above, you should probably know it beforehand if you decided to do real life location like lets say Mogadishu, Somalia.+{{ https://pmc.editing.wiki/​images/​Global-Mapper-Select-Export-Format.png }}
  
-**Heightmap**+In arc ascii grid export options dialog type in sample spacing x and y: 10 meters, which will be 40km terrain with 10 meter cell size for us.
  
-Use opentopography to download SRTM data, this data is very lightweight to download so grab a LARGE area just in caseThis way you can move your exact terrain position around a bit if necessary.+{{ https://pmc.editing.wiki/​images/​Global-Mapper-Arc-ASCII-Grid-Export-Options.png }}
  
-Start your terrain by opening the heightmap with global mapper as explained on its page.+Select export bounds tab, then tick crop to selected area features(s) option.
  
-**Satellite**+{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Arc-ASCII-Grid-Export-Options-Export-Bounds.png }}
  
-Use terra incognita ​to download satellite texture and remember ​to leave enough room around your chosen terrain area as mentioned above.+Click OK to start exporting. Browse ​to the tut_austria_40km directory, then give filename "​heightmap"​ (it auto assigns .asc extension). Wait until it has downloaded the SRTM heightmap for you.
  
-Remember to create global mapper map catalog file out of the oziexplorer .map files to ease working with the image tiles.+===== Header 200,000 / 0 =====
  
-When arma3 v1.60 visual update was released it forced everyone to use very dark colored satellite images, so you need to use photoshop / gimp or imagemagick to apply some auto-levels or brightness darkening ​to the satellite texture images.+Terrain builder uses 200,000 / 0 coordinates ​so we must fix heightmap.asc header ​to fit for these coords.
  
-Assuming you are downloading large satellite textureyou need to remember to export in tiles from global mapper. If you are developing tiny terrain ​then you can ignore ​this, but anything above 20480 x 20480 resolution (or maybe even that) requires you to export as tiles for easier management of the imagesI mean, have you ever edited 40960 x 4096 resolution image in photoshop? Ugh!+Open heightmap.asc in proper text editor, then you see the header like this
 +<​code>​ 
 +ncols         ​4096 
 +nrows         ​4096 
 +xllcorner ​    ​356294.301 
 +yllcorner ​    ​5233219.645 
 +cellsize ​     10 
 +nodata_value ​ -9999.0 
 +</​code>​
  
-You should use imagemagick ​to stitch those global mapper exported satellite tiles to something larger more manageable sizeassuming you are exporting crazy large satellite on 6 x 6 tiles. If you only do small 20480 in 2 x 2 tiles or so then you most likely dont have to do any of that.+Copy paste these xllcorner and yllcorner values ​to some text filewe need them lateron when we transform some shapefiles.
  
-**Roads**+Edit the header to read 200000 and 0, like this: 
 +<​code>​ 
 +ncols         ​4096 
 +nrows         ​4096 
 +xllcorner ​    ​200000 
 +yllcorner ​    0 
 +cellsize ​     10 
 +nodata_value ​ -9999.0 
 +</​code>​
  
-Download roads as explained ​in our roads shapefile pagedon't forget ​to simplify them so they are easier ​for terrain builder / arma3 engine.+Get calculator out and type in 200000then subtract the xllcorer value from it, you'll end up with -156294.301 value. The yllcorner is easy because all we need to do is to add minus on front of it. Lets save these values ​for later use.
  
-If you are doing large terrain ​with lot of city coverage ​or just area that has great number of roads, then you most likely need to be heavy handed ​when deleting "bad" roads as explained ​in the roads page.+I keep these values saved on single line, like this: 
 +<​code>​ 
 +-156294.301 -5233219.645 
 +</​code>​ 
 + 
 + 
 +====== OpenStreetMap Shapefiles ====== 
 + 
 +OpenStreetMap (OSM) shapefiles is used for roads and vegetation object placement. 
 + 
 +Goto [[http://​download.geofabrik.de/​|geofabrik.de]] and browse to europe, then download the astria [.shp.zip] link zip file. 
 + 
 +Unpack the zip file into "​Austria"​ directory. 
 + 
 +In global mapper untick SRTM layer so it wont slow us down. Make sure grid is selected. 
 + 
 +Click open icon, browse to the OSM austria dir and open these files: 
 +<​code>​ 
 +gis.osm_landuse_a_free_1.shp 
 +gis.osm_natural_a_free_1.shp 
 +gis.osm_places_free_1.shp 
 +gis.osm_roads_free_1.shp 
 +</​code>​ 
 +Make sure NOT to select "​places_a_free", ​you have to choose it without the _a_ because it designates area, we dont want areas for places. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-OpenStreetMap-Shapefiles-Loaded.png }} 
 + 
 +Next click the crop to selected areas. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Crop-To-Selected-Areas-Icon.png }} 
 + 
 +Now choose only types of features to crop; areas, from dropdown select create new layer for feature option. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Crop-To-Selected-Areas-Setup-Areas.png }} 
 + 
 +Click OK to crop, give it name "​cropped_areas"​ and click OK and OK (or just hint enter twice). 
 + 
 +Now repeat the same thing; crop to selected areas, but this time select only lines tickbox (areas and points ​are unticked). 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Crop-To-Selected-Areas-Setup-Lines.png }} 
 + 
 +Same thing but give it name "​cropped_roads"​ instead. 
 + 
 +One last time, this time tick points and give it name "​cropped_points"​. 
 + 
 +Now in control center SHIFT-LMB select all the "​gis.osm_"​ starting layers, hit DEL key to delete. Note that global mapper is bit bitchy ​with this one, when you select the layers, do not move mouse cursor away from the control center before you hit DEL key because then it somehow tries to do something else than delete the layers, heh its odd. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Control-Center-Layers.png }} 
 + 
 +We deleted these layers because we dont want them to the in the project to avoid dupes and all they do is to slow down global mapper. 
 + 
 +===== Roads ===== 
 + 
 +If your cropped_roads layer is larger than 9000 (or even 5000), then you need to reduce the amount ​of roads features. 
 + 
 +Untick all other layers except cropped_roads. 
 + 
 +Select search -> search by attributes, name and description. Now you got search vector data dialog. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Search-By-Attributes.png }} 
 + 
 +Use [[road-shapefiles-real-world-data#​remove_bad_roads|road shapefiles remove bad roads]] guide to remove those extra roads. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Seach-Vector-Data-Delete-Bad-Roads.png }} 
 + 
 +When done it looks much cleaner. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-OpenStreetMap-Roads-Cleaned.png }} 
 + 
 +===== Export Shapefiles ===== 
 + 
 +Now export roads by file -> export -> vector/​lidar format -> shapefile, click OK. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Export-Vector-Lidar-Format.png }} 
 + 
 +You should only have export lines option availbale. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Shapefile-Export-Options.png }} 
 + 
 +click select file button, make sure you are in tut_austria_40km directory and give it file name "​cropped_roads.shp"​. 
 + 
 +Untick cropped_roads layer and tick cropped_areas,​ export them the same way, except this time click export areas select file button (only available). Give them "​cropped_areas.shp"​ file name. 
 + 
 +Last untick cropped_areas layer and tick cropped_points layer, export them the same way again, choose points of course. This time give "​cropped_points.shp"​ file name. 
 + 
 +File -> unload all, to close the project. Do not save or mind when global mapper asks to save changed files (the shapefile layers), ​just say no to unload the project, we dont need those cropped_* layers anymore. 
 + 
 + 
 +====== QGIS Transform ====== 
 + 
 +Open QGIS and create new project. 
 + 
 +Click add raster layer icon. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​QGIS-Add-Raster-Layer-Icon.png }} 
 + 
 +Browse to your heightmap.asc file and open it. 
 + 
 +Make sure EPSG:32631 is selected. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​QGIS-Bottom-Bar-Projection-Indicator.png }} 
 + 
 +You can RMB click on the heightmap layer and choose set layer CRS. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​QGIS-Coordinate-Reference-System-Selector.png }} 
 + 
 +You need to make sure **WGS 84 / UTM ZONE 31N** is selected. This is what I meant earlier ​that we dont need to worry about UTM zone in global mapper. Here we do worry about it heh. 
 + 
 +Click add vector layer icon. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​QGIS-Add-Vector-Layer-Icon.png }} 
 + 
 +Browse and open cropped_areas.shp,​ cropped_roads.shp and cropped_points.shp files. 
 + 
 +Now SHIFT-LMB select all the new cropped_* layers, then RMB -> set layer CRS, choose UTM 31N and click OK. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​QGIS-Layers-Panel-Select-Cropped-Layers.png }} 
 + 
 +Now in processing toolbox search and select v.transform plugin. 
 + 
 +The cropped_areas should be selected. Now paste in the heightmap.asc header coordinates you calculated earlier. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​QGIS-V-Transform-Paste-New-Coordinates.png }} 
 + 
 +Put the first xllcorner ​number ​into X shift and yllcorner number to Y shift like the image above indicates. Then click Run button and wait few moments when QGIS is processing. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​QGIS-V-Transform-Layer-Processing-Done.png }} 
 + 
 +You now should see the area shapes on top of the heightmap, perfectly aligned. 
 + 
 +Now in layers panel RMB click transformed layer and choose save as. Browse to tut_austria_40km dir and save under transformed_areas.shp file name. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​QGIS-Save-Vector-Layer-As.png }} 
 + 
 +Now on layers panel CTRL-LMB select cropped_areas,​ transformed_areas and transformed layers, then RMB -> remove, click OK to remove 3 legend entries. 
 + 
 +Again open v.transform plugin. 
 + 
 +Now cropped_points layer should be selected, if its not then select it. 
 + 
 +Type in the heightmap.asc calculated header coordinates as before and click run button, then wait. 
 + 
 +When done, you should see a lot of points on top of the heightmap. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​QGIS-V-Transform-Points-Layer-Processing-Done.png }} 
 + 
 +Now in layers panel RMB click transformed layer and choose save as. This time choose CSV file export and give it transformed_places.csv file name 
 + 
 +{{ https://​pmc.editing.wiki/​images/​QGIS-Save-Vector-Layer-As-CSV-File.png }} 
 + 
 +Now again CTRL-LMB select from layers panel the cropped_points,​ transformed_places and transformed layers. Hit RMB -> remove and OK. 
 + 
 +And one last time open v.transform plugin and transform cropped_roads,​ the only remaining layer. Save them as ESRI shapefile under transformed_roads.shp file name. 
 + 
 +Exit QGIS, were done with it. 
 + 
 + 
 +====== Global Mapper Shapefiles ====== 
 + 
 +===== Areas ===== 
 + 
 +In global mapper open transformed_areas.shp shapefile. 
 + 
 +Use file -> export -> export vector/​lidar format -> shapefile. From shapefile export options choose split export based on: FCLASS dropdown option. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Shapefile-Export-Options-FCLASS-Split.png }} 
 + 
 +Then click select file, give file name "​obj",​ just obj, click OK (twice) to export. 
 + 
 +When done, file -> unload all. 
 + 
 +Create bat file into the tut_austria_40km directory where global mapper just saved bunch of obj_*.* files. Copy paste the following into the bat file. 
 + 
 +**_run_delete_useless_area_shapefiles_and_create_TerrainProcessor_dir.bat** 
 +<code dos> 
 +@echo off 
 +echo roads.shp WILL BE DELETED! No idea why. Move it away! CTRL-C for cancel. 
 +pause 
 +if exist obj_allotments.* del obj_allotments.* 
 +if exist obj_beach.* del obj_beach.* 
 +if exist obj_cave_entrance.* del obj_cave_entrance.* 
 +if exist obj_cemetery.* del obj_cemetery.* 
 +if exist obj_cliff.* del obj_cliff.* 
 +if exist obj_glacier.* del obj_glacier.* 
 +if exist obj_heath.* del obj_heath.* 
 +if exist obj_nature_reserve.* del obj_nature_reserve.* 
 +if exist obj_park.* del obj_park.* 
 +if exist obj_peak.* del obj_peak.* 
 +if exist obj_recreation_ground.* del obj_recreation_ground.* 
 +if exist obj_retail.* del obj_retail.* 
 +if exist obj_scrub.* del obj_scrub.* 
 +if exist obj_spring.* del obj_spring.* 
 +if exist obj_vineyard.* del obj_vineyard.* 
 + 
 +md TerrainProcessor 
 +move obj_*.* TerrainProcessor 
 +</​code>​ 
 +Then just execute _run_delete_useless_area_shapefiles_and_create_TerrainProcessor_dir.bat file, it will take care of the obj_ shapefiles and creates TerrainProcessor directory for you. 
 + 
 +===== Roads ===== 
 + 
 +In global mapper open transformed_roads.shp shapefile. Use [[road-shapefiles-real-world-data#​id_and_order|road shapefiles ID and ORDER]] guide to fix the shapefile for arma3 use. 
 + 
 +When done, file -> unload all. 
 + 
 +===== Place Names ===== 
 + 
 +Next is a bit tricky part... if you do not have Qt installed. We need to use [[https://​github.com/​SnakeManPMC/​arma-3-OSM_Places_CSV2HPP|PMC OSM Places CSV to HPP converter tool]] <​del>​which we at the moment of writing this tutorial dont have binary EXE available for download, so you would have to compile that yourself in Qt. Sorry for that.</​del>​ download [[https://​pmc.editing.wiki/​downloads/​OSM_Places_CSV2HPP_v0.1.2.7z|OSM_Places_CSV2HPP_v0.1.2.7z]]. 
 + 
 +Start the tool and click QGIS button, browse to transformed_places.csv file and click open. In few seconds the arma3 config class names{} compatible text appears in the text box. 
 + 
 +(image from older version of the tool) 
 +{{ https://​pmc.editing.wiki/​images/​PMC-OSM-Places-CSV2HPP-Converter.png }} 
 + 
 +Copy paste that text, then create new file pmc_names.hpp where you paste the text. 
 + 
 + 
 +====== Create Terrain Builder Project ====== 
 + 
 +Most likely ​at this time terra incognita is still downloading (if you were fast enough to follow this tutorial hehe), so lets create terrain builder project. 
 + 
 +For this project we use TUT namespace and tag. 
 + 
 +I have copy pasteable template directories which I copy paste and then run through powergrep replace in files. That will generate the fully working WRP and config addons for any terrain, only thing I need to do is to provide the class name for it. 
 + 
 +Here is the 
 + 
 +**tut_austria_40km/​config.cpp** 
 +<code cpp> 
 +class CfgPatches 
 +
 + class tut_austria_40km_wrp 
 +
 + units[] = {}; 
 + weapons[] = {}; 
 + requiredVersion = 1; 
 + requiredAddons[] = 
 +
 + "​A3_Map_Stratis"​ 
 + }; 
 + }; 
 +}; 
 + 
 +class CfgWorlds 
 +
 + class Stratis; 
 + class tut_austria_40km:​ Stratis 
 +
 + // required by pboProject 
 + worldName = "​\tut\tut_austria_40km\tut_austria_40km.wrp";​ 
 +  
 + // required by arma3 to get forest mapview (M) working 
 + minTreesInForestSquare = 2; 
 + minRocksInRockSquare = 2; 
 + }; 
 +}; 
 +</​code>​ 
 + 
 +**tut_austria_40km_cfg/​config.cpp** 
 +<code cpp> 
 + 
 +class CfgPatches 
 +
 + class tut_austria_40km_cfg 
 +
 + units[] = 
 +
 + "​tut_austria_40km"​ 
 + }; 
 + weapons[] = {}; 
 + requiredVersion = 1; 
 + requiredAddons[] = 
 +
 + "​tut_austria_40km_wrp",​ 
 + "​tut_Terrain_Data",​ 
 + "​A3_Map_Stratis"​ 
 + }; 
 + }; 
 +}; 
 + 
 +class CfgWorldList 
 +
 + class tut_austria_40km{};​ 
 +}; 
 + 
 +class CfgWorlds 
 +
 + class Stratis; 
 + class tut_austria_40km:​ Stratis 
 +
 + cutscenes[] = {}; 
 + description = "TUT Austria 40km";​ 
 + worldName = "​\tut\tut_austria_40km\tut_austria_40km.wrp";​ 
 + author = "PMC https://​www.pmctactical.org";​ 
 + pictureMap = "​tut\tut_austria_40km_data\picturemap_ca.paa";​ 
 + pictureShot = "​tut\tut_austria_40km_data\ui_terrain_ca.paa";​ 
 + 
 + newRoadsShape = "​\tut\tut_austria_40km_data\roads\roads.shp";​ 
 + mapSize = 40960; 
 + centerPosition[] = 
 +
 + 20480, 20480 
 + }; 
 + ilsDirection[] = 
 +
 + 0, 0.08, 1 
 + }; 
 + ilsPosition[] = 
 +
 + 0, 0 
 + }; 
 + ilsTaxiIn[] = {}; 
 + ilsTaxiOff[] = {}; 
 + drawTaxiway = false; 
 + class SecondaryAirports{};​ 
 + class ReplaceObjects{};​ 
 + 
 + class Sounds 
 +
 + sounds[] = {}; 
 + }; 
 + 
 + class Animation 
 +
 + vehicles[] = {}; 
 + }; 
 + 
 + class Grid 
 +
 + offsetX = 0; 
 + offsetY = 40960; // Grid * Cell 
 + 
 + class Zoom1 
 +
 + zoomMax = 0.015; 
 + format = "​XY";​ 
 + formatX = "​0000";​ 
 + formatY = "​0000";​ 
 + stepX = 10; 
 + stepY = -10; 
 + }; 
 + 
 + class Zoom2 
 +
 + zoomMax = 0.2; 
 + format = "​XY";​ 
 + formatX = "​000";​ 
 + formatY = "​000";​ 
 + stepX = 100; 
 + stepY = -100; 
 + }; 
 + 
 + class Zoom3 
 +
 + zoomMax = 0.8; 
 + format = "​XY";​ 
 + formatX = "​00";​ 
 + formatY = "​00";​ 
 + stepX = 1000; 
 + stepY = -1000; 
 + }; 
 + 
 + class Zoom4 
 +
 + zoomMax = 1; 
 + format = "​XY";​ 
 + formatX = "​0";​ 
 + formatY = "​0";​ 
 + stepX = 10000; 
 + stepY = -10000; 
 + }; 
 + }; 
 + 
 + class Names 
 +
 + #include "​pmc_names.hpp"​ 
 + }; 
 + 
 + // new satellite vs detail texture blending setup 
 + /// maximal darkening coef for blending of satellite map and detail map on terrain (default = 0.0f, no darkening = 1.0f} 
 + terrainBlendMaxDarkenCoef = 1.0f; 
 +  
 + /// maximal brightening coef for blending of satellite map and detail map on terrain (default = 1.0f, no brightening = 0.0f) 
 + terrainBlendMaxBrightenCoef = 0.0f; 
 + 
 + // disable the stupid bis loading texts 
 + loadingTexts[] = 
 +
 + "​PMC Editing Wiki helps you develop terrains like this.",​ 
 + "​PMC Tactical https://​www.pmctactical.org",​ 
 + "​This terrain is 40km in size with 1 meter / pixel satellite resolution.",​ 
 + "​This terrain heightmap elevations are realistic made from SRTM data.",​ 
 + "​This terrain satellite texture is from ArcGIS world imagery."​ 
 + }; 
 + }; 
 +}; 
 +</​code>​ 
 + 
 +Open terrain builder and follow PMC ultimate terrain tutorial how to create terrain, no need to repeat already learned lessons here. 
 + 
 +Move TerrainProcessor directory into \tut\tut_austria_40km\source\ directory. 
 + 
 +Move heightmap.asc into \tut\tut_austria_40km\source\ directory. 
 + 
 +Move roads.* into \tut\tut_austria_40km\data\Roads\ directory. 
 + 
 +Move pmc_names.hpp into \tut\tut_austria_40km_cfg directory. 
 + 
 +In terrain builder import heightmap.asc,​ rebuild terrain and delete the heightmap.asc layer. Do not delete the heightmap.asc FILE itself. 
 + 
 +Import satellite mask. In our case its just 1024 x 1024 placeholder,​ single color yellow RGB. Adjust its properties to fit 40960 x 40960 satellite specs. 
 + 
 + 
 +====== Global Mapper Satellite Texture ====== 
 + 
 +===== Create Map Catalog ===== 
 + 
 +Global mapper map catalog is system to load many files like images, but only show few of them at one time, saving memory and performance. This is excellent for loading large numbers of oziexplorer .map files, aka satellite imagery (texture) tiles. 
 + 
 +Start global mapper and load tut_austria_40km.gmw project. 
 + 
 +File -> create map catalog. Give it name, I do "​GM_Map_Catalog_tut_austria_40km_arcgis_17_level"​ which is mouthful but leaves no questions ​when seeing it lateron. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Modify-Map-Catalog.png }} 
 + 
 +Click add directory button, then browse to the terra incognita tut_austria_40km_arcgis_17_level directory. On enter file masks to open dialog, type *.map and hit OK. Now wait until the catalog is created, this will take a few minutes again. 
 + 
 +When its done creating the catalog click OK in modify map catalog dialog. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Map-Catalog-Satellite-Imagery.png }} 
 + 
 +File -> export -> export raster/​image format -> PNG. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-Export-Raster-Image-Format.png }} 
 + 
 +Then on PNG export options dialog you choose file type 24bit RGB, sample spacing x/y 1 meters and untick the bottom three default tickboxes. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-PNG-Export-Options.png }} 
 + 
 +Select tiling tab, tick specify number of rows and columns and type 4 on both. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-PNG-Export-Options-Tiling.png }} 
 + 
 +Select export bounds tab, tick crop to selected area features. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Global-Mapper-PNG-Export-Options-Export-Bounds.png }} 
 + 
 +Click OKm give file name "​sat",​ just sat. Click save to ttart exporting satellite texture tiles in PNG image format. 
 + 
 +This will take very long time depending on your satellite size and computer speed etc. 
 + 
 + 
 +====== Darken Satellite PNG Images ====== 
 + 
 +[[real-world-data-global-mapper#​darken_and_merge_satellite_tiles|Darken and merge satellite tiles]] explains how to process the sat_*.png images. 
 + 
 +===== L3DT Background Image ===== 
 + 
 +_run_imagemagick_l3dt_8192_texture_map.bat is used to create [[tools:​l3dt|L3DT]] texture map, a basically small image of satellite texture which is used as background when editing heightmap in L3DT. 
 + 
 +**_run_imagemagick_l3dt_8192_texture_map.bat** 
 +<code dos> 
 +@echo off 
 +set MAGICK_TEMPORARY_PATH=c:​\imagemagick_temp 
 + 
 +echo Creating L3DT_texture_map.jpg 
 +echo resizing A1 ... 
 +magick convert Dsat_A1.bmp -resize 2048x2048 temp_A1.bmp 
 +echo resizing A2 ... 
 +magick convert Dsat_A2.bmp -resize 2048x2048 temp_A2.bmp 
 +echo resizing A3 ... 
 +magick convert Dsat_A3.bmp -resize 2048x2048 temp_A3.bmp 
 +echo resizing A4 ... 
 +magick convert Dsat_A4.bmp -resize 2048x2048 temp_A4.bmp 
 + 
 +echo resizing B1 ... 
 +magick convert Dsat_B1.bmp -resize 2048x2048 temp_B1.bmp 
 +echo resizing B2 ... 
 +magick convert Dsat_B2.bmp -resize 2048x2048 temp_B2.bmp 
 +echo resizing B3 ... 
 +magick convert Dsat_B3.bmp -resize 2048x2048 temp_B3.bmp 
 +echo resizing B4 ... 
 +magick convert Dsat_B4.bmp -resize 2048x2048 temp_B4.bmp 
 + 
 +echo resizing C1 ... 
 +magick convert Dsat_C1.bmp -resize 2048x2048 temp_C1.bmp 
 +echo resizing C2 ... 
 +magick convert Dsat_C2.bmp -resize 2048x2048 temp_C2.bmp 
 +echo resizing C3 ... 
 +magick convert Dsat_C3.bmp -resize 2048x2048 temp_C3.bmp 
 +echo resizing C4 ... 
 +magick convert Dsat_C4.bmp -resize 2048x2048 temp_C4.bmp 
 + 
 +echo resizing D1 ... 
 +magick convert Dsat_D1.bmp -resize 2048x2048 temp_D1.bmp 
 +echo resizing D2 ... 
 +magick convert Dsat_D2.bmp -resize 2048x2048 temp_D2.bmp 
 +echo resizing D3 ... 
 +magick convert Dsat_D3.bmp -resize 2048x2048 temp_D3.bmp 
 +echo resizing D4 ... 
 +magick convert Dsat_D4.bmp -resize 2048x2048 temp_D4.bmp 
 + 
 +echo combine temp_Ax ... 
 +magick convert +append temp_A1.bmp temp_A2.bmp temp_A3.bmp temp_A4.bmp temp_Ax.bmp 
 +echo combine temp_Bx ... 
 +magick convert +append temp_B1.bmp temp_B2.bmp temp_B3.bmp temp_B4.bmp temp_Bx.bmp 
 +echo combine temp_Cx ... 
 +magick convert +append temp_C1.bmp temp_C2.bmp temp_C3.bmp temp_C4.bmp temp_Cx.bmp 
 +echo combine temp_Dx ... 
 +magick convert +append temp_D1.bmp temp_D2.bmp temp_D3.bmp temp_D4.bmp temp_Dx.bmp 
 + 
 +echo combine L3DT_texture_map.bmp ... 
 +magick convert -append temp_Ax.bmp temp_Bx.bmp temp_Cx.bmp temp_Dx.bmp L3DT_texture_map.bmp 
 +echo creating L3DT_texture_map ... 
 +magick convert -strip -interlace Plane -quality 85% L3DT_texture_map.bmp L3DT_texture_map.png 
 +echo deleting ​temp files ... 
 +if exist temp_*.bmp del temp_*.bmp 
 +if exist L3DT_texture_map.bmp del L3DT_texture_map.bmp 
 + 
 +echo All done! 
 +pause 
 +exit 
 +</​code>​ 
 + 
 + 
 +====== Move picturemap_ca to Data ====== 
 + 
 +Next we move picturemap_ca.tga into \tut\tut_austria_40km\data\ directory. This image will be shown when selecting terrain in-game. 
 + 
 +We also rename PMC-Satellite.jpg to PMC-Austria-Satellite.jpg at the same time. This image actually has no use in arma3, its just for our purpose if we ever want to check out the satellite texture overview. 
 + 
 + 
 +====== Terrain Processor Shapefiles ====== 
 + 
 +Then we generate some objects according to OSM shapefiles. 
 + 
 +Previously we already moved the TerrainProcessor directory into \tut\tut_austria_40km\source\ directory, so now we download and copy [[https://​github.com/​SnakeManPMC/​arma-3-terrain-processor-templates|PMC terrain processor TPP files from github]]. 
 + 
 +Now you need to edit the TPP's a bit, they include mask linear task which only lists "roads.shp" ​without a path, if you run the project it will crash terrain processor as it cannot find the shapefile. So we have to manually edit the path there. 
 + 
 +I'm hardcore and edit this stuff with text editor, people with zero programming or even html skills might get lost here though. 
 + 
 +To ease up the path writing, I do one roads.shp path from terrain processor, I opened Area_Random_CUP_Chernarus_Buildings_Commercial.tpp project, browsed into \tut\tut_austria_40km\data\roads\roads.shp and saved the project. Now I can copy paste the correct path into all the other TPP's with text editor. 
 + 
 +Then run the TPP files one by one through terrain processor, leave Area_Random_CUP_Chernarus_Vegetation_Forest.tpp to last as it needs some tweaking. 
 + 
 +Run it once with the options it came with, see how many objects terrain processor created. In my case it was 107,000 objects which is nice round number 100k. 
 + 
 +With the release of arma3 64bit version terrains have object count limit so high that we have not been able to import so many objects to terrain builder that it would have crashed binarize or arma3_x64.exe itself. ​in [[https://​www.pmctactical.org/​arma3/​terrain.php|PMC Balkans Kosovo]] we tested roughly 20 million objects which worked fine, PMC Suwalki Gap which is available for download right now has 18.1 million (18,​134,​807) objects. This tutorial was written prior to 64bit binary release. 
 + 
 +Before 64bit binary release I used this formula: So lets calculate 6 / 0.107 == 56. This means we can add hectare density to 56 before running much over 6 million generated objects. 
 + 
 +50 hectare density for forests is okay-ish I guess. Lets try that first. 
 + 
 +I have the TPP open in text editor and change the hectare density line to 50 like this: 
 +<code xml> 
 +<​Parameter Name="​Hectare density"​ Type="​0"​ Value="​50"/>​ 
 +</​code>​ 
 + 
 +And then run terrain processor again, be patient as terrain processor takes long time to complete. It now generates 5,4 million objects which is very nice number. 
 + 
 + 
 +====== Terrain Builder Object Import ====== 
 + 
 +===== Template Libraries ===== 
 + 
 +First we need to download the [[https://​github.com/​SnakeManPMC/​arma-3-terrain-builder-TemplateLibs|PMC terrain builder templatelibs from github]]. 
 + 
 +Create \tut\tut_austria_40km\source\terrainbuilder\TemplateLibs\ directory, then copy the TML files there. 
 + 
 +Now load template libraries from terrain builder. 
 + 
 +===== Object Import ===== 
 + 
 +Before you import millions of objects we need to prepare terrain builder. Disable ALL views. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Terrain-Builder-Disable-All-Views-Icons.png }} 
 + 
 +When views are disabled start the import objects process, select all LBT files from TerrainProcessor directory. Then wait, very patiently as it takes forever to import millions of objects, over an hour on some cases. 
 + 
 + 
 +====== Terrain Builder Import Satellite Texture ====== 
 + 
 +In terrain builder we import satellite texture, select all of the Dsat_*.bmp images, then wait patiently as it imports. 
 + 
 +Once import is complete you see the satellite tiles in lower left corner. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Terrain-Builder-Satellite-Tiles-After-Import.png }} 
 + 
 +Now use [[import-satellite-in-smaller-tiles#​tile_example_values|import satelite in smaller tiles examples]] ​page to get the coordinates. 
 + 
 +Our tiles are 10240 x 10240 in size, we have 4 x 4 of them in 1 meter / pixel setup. 
 + 
 +Not sure what more I can say about that tile setup, you can copy paste the values from the above link and then its all aligned up nicely. 
 + 
 +{{ https://​pmc.editing.wiki/​images/​Terrain-Builder-Satellite-Tiles-After-Alignment.png }} 
 + 
 + 
 +====== Delete Global Mapper Source Files ====== 
 + 
 +Now you dont need the global mapper source files anymore, so delete *.bat, *.bmp, *.png (except L3DT_texture_map.png) and cropped_*.* and transformed_*.* from global mapper project dir. 
 + 
 +You can delete the satellite texture source images because they are not imported in terrain builder, they are not read "​live"​ from the disk as in these .bmp source images. 
 + 
 + 
 +====== Generating Satellite Texture/​Mask ====== 
 + 
 +Use terrain builder to generate satellite texture / mask layers files. 
 + 
 +Export WRP. 
 + 
 +Moveobject WRP for data_layers\ paths. 
 + 
 + 
 +====== Convert Layers PNG Fast ====== 
 + 
 +Use our [[convert-layers-png-fast|Convert Layers PNG Fast]] batch file. 
 + 
 + 
 +====== Separate Data Layers ====== 
 + 
 +Use our [[separate-data-layers|separate data layer]] guide to split the addons up. 
 + 
 + 
 +====== Move Data Dir Files ====== 
 + 
 +Go to \tut\tut_austria_40km\data\ directory and move all the TGA/PAA image files and Roads\ directory into \tut\tut_austria_40km_data\ directory. 
 + 
 + 
 +====== pboProject Batch ====== 
 + 
 +I run pboProject from command line, so it needs a batch file. 
 + 
 +Created mod.cpp file 
 + 
 +Created empty tut_austria_40km and tut_austria_40km\addons\ dirs. 
 + 
 + 
 +====== All Done! ====== 
 + 
 +Nothing left to do but run pboProject! Yeah! :)
  
  
Line 99: Line 879:
 **2007 - 2009** ArmA, fictional\\ **2007 - 2009** ArmA, fictional\\
 **2009 - 2013** ArmA 2, fictional (maybe few close to real world terrains with hard work started to appear)\\ **2009 - 2013** ArmA 2, fictional (maybe few close to real world terrains with hard work started to appear)\\
-**2013 - now?** ArmA 3, fictional + accurate real world data :)+**2013 - 2020** ArmA 3, fictional + accurate real world data :)
  
 Just a small side note, until these PMC Editing Wiki tutorials, there was no **comprehensive step by step** tutorial sets how to make real world data terrains, few guys had global mapper / qgis tutorials but none were complete tutorials from having nothing to getting terrain in-game. Just a small side note, until these PMC Editing Wiki tutorials, there was no **comprehensive step by step** tutorial sets how to make real world data terrains, few guys had global mapper / qgis tutorials but none were complete tutorials from having nothing to getting terrain in-game.
Line 109: Line 889:
  
 Credits: Credits:
 +  * [PTR] brenden0309
   * Digital Centurion   * Digital Centurion
 +  * Hydrra
   * Ice   * Ice
 +  * JakerRod
   * M1lkm8n   * M1lkm8n
   * pooface   * pooface
-  * Hydrra +  * Richie
-  * [PTR] brenden0309+
 And everyone I forgot who helped in arma discord, your help made these pages possible, you should be proud of yourselves! And everyone I forgot who helped in arma discord, your help made these pages possible, you should be proud of yourselves!
arma3/terrain/real-world-data-tutorial.1486465293.txt.gz · Last modified: 2017-02-07 11:01 by snakeman