Dev Diary PMC Taiwan Taichung ArmA 3 Terrain

This is dev diary for PMC Taiwan Taichung ArmA 3 real world data terrain by PMC.

We have been writing development or developer diaries to PMC Tactical Forums for two decades, these addon dev diaries are spread to many forum topics. One example PMC Tactical Forums: Vietnam: The Experience Neverending development diary 2007-10-13. On 2020 things changed in Farming Simulator 19 editing community when we started to write detailed dev diaries as web html pages because we had no forum, now in 2022 the same dev diary story telling continues in ofp/arma community.

These dev diaries are my current on the fly thoughts and emotions, if I feel shitty I will clearly state that. I try to avoid it but sometimes there might be hilarious profanity rants when things don't go exactly as I planned, so if you are "offended" by few curse words you probably should stop reading now Smiley ;)

Also these dev diaries are written to many addon projects and sometimes repeating the same lines of text or taking exact same screenshots feels stupid so there might be text just listing boring "I did XYZ and then ZYX" type of info without further details. And remember; this is NOT a tutorial, this is entertainment Smiley ;)

Time stamps are Universal Time Coordinated (UTC).

2021-10-10 Project Started

2021-10-10T04:49:00Z Created this page Smiley :)

2021-10-10T05:01:00Z Created Global Mapper project, user created feature grid, used the shape file from main taiwan template as guide of where to place the grid.

Goal now is to get KMZ out of this and then get Terra Incognita leeching satellite imagery which takes another 8hrs and I doubt it finished before I go to sleep tonight. We'll see.

2021-10-10T05:08:00Z Terra incognita satellite imagery download started for NW corner.

Forgot to write down when terra incognita finished 1 and 2 corners.

2021-10-10T10:13:00Z Noticed terra incognita SW corner download has finished. Started SE the last corner now.

And obviously forgot to write down when it finished before calling it a night.

2021-10-10T22:07:00Z New editing day started by global mapper adding SRTM v3 online source and exporting 4096 grid ASCII Grid heightmap.asc with 20m cell size.

2021-10-22 Global Mapper Editing

2021-10-22T08:35:00Z Back to Taichung editing, today we proceed to go through our real world data terrain tool pipeline, would be nice to get this terrain binarized, but lets proceed one step at the time.

First task for today was to do GEBCO ocean seabed heightmap data.

Global mapper exporting GEBCO featured ocean seabed heightmap data.

PMC Taiwan Taichung ArmA 3 Screenshot

Imported OpenStreetMap (OSM) shape files into global mapper and cropped them to areas, roads and points layers.

PMC Taiwan Taichung ArmA 3 Screenshot

There was over 100,000 road features so reduced those roads, got the feature count down to 5345, should work great with arma3.

Global mapper assigned colors to OSM shape areas, then started to export Sat_Mask 8 x 8 grid tiles images.

PMC Taiwan Taichung ArmA 3 Screenshot

While global mapper was tied down to exporting that satellite mask image set, I started another global mapper instance and did OSM shape file transformation to UTM 31N projection for Terrain Builder use. Then created mod.cpp with proper info and also terrain addon destination directory for arma3.

I always get frustrated when long process is running and cant do anything but wait. This is especially hard now as I'm writing these dev diaries and recording videos, if I didn't record videos I could bounce across several terrain projects and nibble few edits here and there while global mapper was busy doing its thing, but this doesn't really fit into my one project per dev diary video style.

Once global mapper finished exporting mask images I immediately started with texture, then used GraphicsMagick to fix the global mapper saved satellite mask images RGB background white color to RGB 250,250,250 which PMC_Terrain_Data layers.cfg uses.

Looks like satellite texture export takes roughly 40min. We wait again... Smiley :)

2021-10-22T10:55:00Z In the mean time I created terrain builder project, imported heightmap.asc, created mod.cpp and destination addon directory. Edit: Umm dejavu, already said to have done that previously heh. Then ran Terrain Processor for heightmap road smoothing and obj_forest object generation.

On foreground global mapper is exporting satellite texture tiles and on the background terrain builder is importing 1.16gb sized LBT object export file from terrain processor. Working hard, as usual, pushing shit in and out Smiley :D

PMC Taiwan Taichung ArmA 3 Screenshot

Then used GIMP to go through all sat_*.png satellite texture tile images, it was pretty OK overall, but yeah there were clouds and patches, I mean what large terrain doesn't, but yeah, got some clone stamp tool brushing to do again.

Also I hate doing those coastlines free hand selection in GIMP, well let me rephrase that... I love editing in GIMP, but doing just insane amount of coastline selections for hours and hours and hours just makes you want to scream and jump out of the window. All taiwan terrains have western side coastline and taipei has north and a bit east, plus that penghu or whatever island terrain obviously has full 360 coastlines on several small islands which was pure horror to do selections for (already did them, luckily). Guess this is one of the main reasons why these little wannabe bitches in the community aren't doing large terrains, they just don't have the guts to work this much for a single terrain when they can make some useless 2km piece of shit within hours (assuming they would knew how). Aaanyways.

Checked how many objects terrain processor generated, it was 12.416 million, whoah Smiley :)

It was not my intention to go well over 10 million objects, I'd like to keep that a rough limit per object layer in terrain builder, I have faint memory of some terrain import failing with more than 10 million objects per LBT terrain processor file, apparently it works fine here.

2021-10-22T11:55:00Z Lunch break to cook food and eat.

2021-10-22T13:07:00Z Feeding time over and belly is full, now back to editing.

Hmm grumble grumble guess I'll start GIMP editing all the sat_??.png images now to get them good looking for in-game use. Oh man I dread the coastline free hand selection creation, oh well you gotta do what you gotta do.

2021-10-22T15:10:00Z Finished GIMP painting satellite texture, clone stamp tool brushed clouds out and did the usual coastline free hand selection and ocean seabed pattern fill plus did mask colors from that selection as well. Uuh...

Started terrain builder up and noticed the freshly imported 12.416 million object layer file ... was 3kb in size, so indeed my memory served me well, the project saving FAILED. I need to keep objects below 10 million per object layer to make sure terrain builder saves the project properly.

2021-10-22T15:43:00Z Started importing satellite mask while satellite texture was still being Darken Merge Picturemap Satellite Texture Tiles by graphicsmagick.

While terrain builder was importing satellite mask I went and did Latitude / longitude config.cpp entry. Also moved pmc_names.hpp config.cpp class names{} into _cfg addon dir.

Satellite import finished, looking good.

PMC Taiwan Taichung ArmA 3 Screenshot

2021-10-22T16:33:00Z Started to generate layers, this will take a while again and I'm getting more and more frustrated as I cant edit in the speed I'm used to when not doing dev diary and video recording. Oh well, these dev diaries and videos rulez so I cant imagine not doing them.

Anyways lunch break is soon with food cooking so that takes my time nicely while waiting.

Looking at the real world data terrain tool pipeline 00.txt file there is not much to do before terrain is ready for binarization, I might be able to switch into the next taiwan terrain today.

2021-10-22T16:37:00Z Food cooking and lunch break, I'm starvin' ...

2021-10-22T17:35:00Z Feeding time over, lets eat some terrains.

So back from lunch... and into the waiting hehe, satellite texture generation ETA 13min and then we still have satellite mask to go. Uuh.

Convert Layers PNG to PAA Fast bat in action, putting CPU cores to full use Smiley :)

PMC Taiwan Taichung ArmA 3 Screenshot

There was no hurry to binarize (PMC Editing Wiki: search "binarize") this terrain but decided to run pboProject right away after everything was done, it can run on the background when I switch into the next terrain editing.

2021-10-22T19:31:00Z pboProject finished binarizing the terrain, we're done here (until further edits) Smiley ;)

2021-10-27 Generating Vegetation Objects

2021-10-27T22:43:00Z Back at taiwan taichung, going to add vegetation objects now as the initial terrain processor TPP created crazy 1.16gb LBT file which failed terrain builder import heh.

Used global mapper to load transformed_areas, then separated FCLASS forest from there and deleted rest. Then exported obj_forest areas and split up them into 2 x 2 grid tiles, this way we get very small, well smaller, object layers into terrain builder and should have no problem importing them.

Moved the shape files into terrain processor dir and created TPP files for A1, A2, B1 and B2. Increased forest hectare density from 17 to 50.

Some of the task generated basically nothing, probably the above ocean ones but then ones where the mountain forests are ... generated whole lotta trees Smiley ;)

A2 which is NE corner generated 7.811 million objects... that is, do-able in terrain builder but I'd prefer it to be a bit lower, lets try that anyways.

B2 generated... heh 10.617 million objects, yikes! Well, I can try to import that... but I would not be the least surprised if it failed. Lets see what happens.

Hmm but that already comes to total objects close to 20 million hehe. Yeah I'll drop the hectare density from 50 to ... 40, maybe.

With hectare density 40 it generated A2 6.989 million and B2 9.499 million objects, ouch! Smiley :)

2021-10-28T00:45:00Z Got the objects imported to terrain builder, then exported WRP and it now has 17,022,921 objects hehe.

However while saving terrain builder project that B2 object layer saving failed, the file turned up as 3kb or something, broken down useless stub. Oh well I'll have to turn down the hectare density on that or further split up the layers. In the end though there is the terrain builder limit of roughly 20 million objects, that is the most I've managed to save into WRP before terrain builder just crashes.

Started pboProject binarization right away, want to see this monster 17 million objects in-game Smiley ;)

2022-04-11 New Editing Year

2022-04-11T20:56:00Z New editing day has started, actually editing YEAR. Just released PMC Poland Krynki and now eyeballing the next terrain on my tool pipeline 00.txt list which is PMC Taiwan Taichung. Finished reading above dev diary to catch me up with latest developments, looks like this terrain has crazy big forest OSM areas as they generate that many objects, even with hectare density of 40, interesting.

I have list of fixes to do, but dunno how far this is going to go today, already feeling a bit exhausted, tired maybe, from todays prior editing and releasing.

Loaded project up in terrain builder, scanned satellite texture through and spotted airfield without keypoints so created that first. Then created poly line for airfield runways.

Exported airfield_runways.shp into pmc_taiwan_taichung_data\roads\ directory, loaded it in global mapper and set ID/ORDER values, exported back out. Then imported roads.shp to merge the two shape files and exported back out to roads.shp so that airfield runways are in the same file.

Terrain builder imported this newly saved roads.shp and exported right back out again so it will now work in arma3.

Copy-pasted the pmc_taiwan_taichung.hpp airfield keypoints class names into pmc_taiwan_taichung_cfg\pmc_names.hpp config file, now our airfield has a name in arma3 2D mapview.

2022-04-11T21:18:00Z But now I'm really exhausted and tired, just struggling to edit so decided to take a break, to muster up some energy/motivation once again...

2022-04-12T02:29:00Z Just writing a note that break is over and I have absolutely no motivation to continue this or other terrain editing right now, mind is blank, cant get anything productive started, mainly checking out arma3 multiplayer missions if I could find something interesting to play on. I definitely do not plan on returning to this terrain earlier than tomorrow, I doubt even as soon as that... Burnout is no joke, folks Smiley :(

2022-04-12 Ocean Coastline Editing

2022-04-12T17:29:00Z New editing day has started. Yesterday spend just attempting to play dedicated server MP which did not lead anywhere and then just read Ukraine war news and my local notes, watched one movie. Day was a colossal waste of time, I just waited for bed time so the day could be over with. But now I have kind of angry energy because of the yesterday and I am motivated to crush shit on taiwan taichung terrain. I just woke up, did the morning computer chores, nothing going on anywhere, havent had the days first meal yet but going to launch global mapper now and export satellite mask so I can continue editing heightmap coastline.

Hows that for opening, boom bitches!

PMC Taiwan Taichung ArmA 3 Screenshot

Now I'm off to get some grub.

2022-04-12T18:04:00Z While I ate global mapper finished exporting satellite mask tile images.

Then opened first Satellite.Mask/ sat_??.XCF image in GIMP, selected the RGB 0,0,150 ocean, opened Sat_Mask_??.png equivalent tile and copy-pasted the ocean blue into that, then saved/exported the PNG.

PMC Taiwan Taichung ArmA 3 Screenshot

Then had to repeat that for all the XCF satellite mask images to get the full ocean selection tile-set done.

2022-04-12T18:40:00Z In global mapper doing the ocean bottom edits, now its "finding areas" in satellite mask map catalog according to RGB value, this will take approximately 10min or so. My chance to read some Ukraine war news and make a quick visit to the head Smiley :)

Forgot to paint Sat_Mask_A1.png full ocean seed tile and copy it over to all other full ocean tiles.

Updated my ocean bottom coastline tool pipeline text list to fix one step I have now missed three times, no more, now its fixed, wont be making the same mistake ever again. You always need to update the tool pipeline every single time you encounter an issue of some sort.

Got it done.

PMC Taiwan Taichung ArmA 3 Screenshot

2022-04-12T19:12:00Z Got the ocean bottom coastline global mapper work done. Hopefully now the next coastline terrain is easy to do as tool pipeline works without issues.

Now I even remember my other failure which happened few times, after being excited of getting heightmap-final.asc done I immediately import it to terrain builder like a horny dog, completely forgetting that I need to level airfield runways in L3DT and terrain processor smooth roads again heh.

I have now updated tool pipeline text list for these two steps, they are fairly important Smiley :)

Then L3DT edited heightmap-final.asc to smooth the airfield runways. Next copied LBT deleting and roads.* file copying bat files into TerrainProcessor/ dir, edited the roads bat and ran it to copy roads here, then edited Smooth_Roads.tpp so it reads roads.shp in the same dir and ran it.

Now our ocean bottom coastline work is final, heightmap is ready to import into terrain builder.

Did the terrain builder heightmap import shuffle, exported WRP, saved project and exited. Ran moveObject re-pathing of WRP, then started pboProject binarization, lets get this baby to arma3.

2022-04-12T19:34:00Z As binarization takes forever, I'm going to start creating taiwan taichung web page php files already.

Hmm earlier terrain builder exported WRP with 7.522 million objects, I quickly browsed through this dev diary and I talk about seventeen million objects previously, hmm hmm, is some of the object layers broken or did I not mention when I cut ten million objects from terrain processor, hmm.

2022-04-12T20:02:00Z Got changelog page done, up until this point of editing, still few entries coming to that page. I would like to move this dev diary text to the actual web page already but we are still far from release and its such a bitch to write dev diary with html code tags, so not going to do it yet. Right now though, dunno what could I do other than wait, hmm well I could do the release news/forum post text template, but that takes whopping 37 seconds about heh.

I hate waiting...

Hmm binarize must have heard me, it finished within a minute after I wrote the above Smiley ;)

Hmm felt quite fast too, hmm hopefully terrain working ok.

*** pmc taiwan_taichung
Start time: 2022-04-12T1933
pboproject pmc_taiwan_taichung_data_layers_08
pboproject pmc_taiwan_taichung_data_layers_07
pboproject pmc_taiwan_taichung_data_layers_06
pboproject pmc_taiwan_taichung_data_layers_05
pboproject pmc_taiwan_taichung_data_layers_04
pboproject pmc_taiwan_taichung_data_layers_03
pboproject pmc_taiwan_taichung_data_layers_02
pboproject pmc_taiwan_taichung_data_layers_01
pboproject pmc_taiwan_taichung_data_layers_00
pboproject pmc_taiwan_taichung_data_layers
pboproject pmc_taiwan_taichung_data
pboproject pmc_taiwan_taichung_cfg
pboproject pmc_taiwan_taichung
End time: 2022-04-12T2005

Okay time to fire up arma3 and go see whats what Smiley :)

PMC Taiwan Taichung ArmA 3 Screenshot

I knew something was wrong, looks like... lets see, that would be Forest_B2.tpp terrain processor vegetation generation failure, meaning bottom right grid tile is completely missing all vegetation, heh how did that happen Smiley :o

But roads.shp is working, airfield runways and class names{} show up OK and most importantly... ocean coastline is cool and no water creeping to land, awesome!

Heightmap is good to go.

But man those missing objects are hilarious.

PMC Taiwan Taichung ArmA 3 Screenshot

Is that like a "crew cut" for forest? Smiley :)

2022-04-12T20:34:00Z Started investigating terrain processor project files, hmm I could find at least one object count wasting problem right away, trees are created first and way too many bushes and rocks next, for example rock hectare density is 20 whoah what was I thinking heh.

Got four TPP files done after shuffling rocks task first, trees second and bushes third. Then ran them and here are the generated objects in k thousands and m millions.

A1: 37.5k
A2: 3.452m
B1: 226k
B2: 4.694m

Pretty decent vegetation coverage. Now just have to import all that shit to terrain builder Smiley ;)

2022-04-12T20:45:00Z Terrain builder object import strated. Hopefully this finishes in less than an hour...

Updated mod.cpp for latest layout, accurate google/maps and homepage links.

Importin' ...

PMC Taiwan Taichung ArmA 3 Screenshot

Terrain builder finished importing, then exported WRP and it was total of 8,410,261 objects, hmm wonder if I squeezed the TPP settings too tight, that sounds like a low number of objects but well, it will do for now.

2022-04-12T21:17:00Z Once again, binarization begun. I'm pretty sure this is now the final time before release.

Started release procedures by opening that text list, first thing was adding pmc_taiwan_taichung to PMC "worldSize" mission pack compile bat terrains list.

2022-04-12T21:49:00Z Binarize done, in-game arma3 check, moment of truth ...

PMC Taiwan Taichung ArmA 3 Screenshot PMC Taiwan Taichung ArmA 3 Screenshot

Looking A-OK! Smiley :)

Yeah guess we're done here, next is just starting arma3 with ULTRA settings and taking those mandatory screenshots for pmctactical.org web page.

2022-04-13T02:08:00Z Had to take a what I'd like to call a short break, but looking at the times it was not that short, just finished watching some DW News channel Ukraine documentary part 2 and now its time to continue editing, break is over.

Continuing the release procedure. Created bisign keys and then 7-zip packed addons into a release packet. Next activated arma3 video settings ULTRA for a screenshot taking session.

2022-04-13T02:34:00Z Screenshots taken, sorted, renamed, edited and converted.

Then moved this dev diary text from localhost to the actual web page and started to proof read and add links to it.

2022-04-13T03:28:00Z Finished proof reading task, not my favorite activities to copy-paste those links with fingers sore heh. Then started to create torrent, release is getting closer and closer ...

2022-04-13T03:58:00Z PMC Taiwan Taichung v0.1 has been released Smiley :cool:

PMC Taiwan Taichung: Changelog, Dev Diary, Screenshots. Back to PMC Terrain's page.