Dev Diary PMC Ukraine Odesa ArmA 3 Terrain

This is dev diary for PMC Ukraine Odesa ArmA 3 real world data terrain by PMC.

We have been writing development or developer diaries to PMC Tactical Forums for two decades, these addon dev diaries are spread to many forum topics. One example PMC Tactical Forums: Vietnam: The Experience Neverending development diary 2007-10-13. On 2020 things changed in Farming Simulator 19 editing community when we started to write detailed dev diaries as web html pages because we had no forum, now in 2022 the same dev diary story telling continues in ofp/arma community.

These dev diaries are my current on the fly thoughts and emotions, if I feel shitty I will clearly state that. I try to avoid it but sometimes there might be hilarious profanity rants when things don't go exactly as I planned, so if you are "offended" by few curse words you probably should stop reading now Smiley ;)

Also these dev diaries are written to many addon projects and sometimes repeating the same lines of text or taking exact same screenshots feels stupid so there might be text just listing boring "I did XYZ and then ZYX" type of info without further details. And remember; this is NOT a tutorial, this is entertainment Smiley ;)

Time stamps are Universal Time Coordinated (UTC).

2022-03-06 Project Started

2022-03-06T12:01:00Z This page was created. While I'm writing this text, in the background PMC Ukraine Sumy is processing some hour long tasks, hopefully can manage to edit two terrains at the same time while writing and recording dev diary videos Smiley :)

Got started by creating real world data terrain tool pipeline text list for 00.txt file, started Global Mapper and got ukraine odesa location into UTM 36N zone projection. Then it was decision time of how large this terrain would be.

First I was thinking about 61.4km but it felt somehow small, there was plenty of space before PMC Ukraine Mykolaiv terrain so decision was made for 81.9km terrain size.

Then just making the user created feature grid and export KMZ file.

2022-03-06T12:16:00Z Terra Incognita satellite imagery grid section 1 started downloading.

Global mapper exported SRTM 1-arc-sec v3 ASCII Grid heightmap.asc file.

2022-03-06T12:31:00Z Finished global mapper exporting GEBCO ocean seabed heightmap data.

Then imported all OpenStreetMap (OSM) shape files and cropped them. Then set them to UTF-8 codepage and reduced road feature count to 7400 or so.

Next calculated heightmap.asc header coordinates to UTM 31N Terrain Builder compatible 200,000/0 format.

2022-03-06T12:51:00Z Finished transforming and exporting OSM shape files for areas, places and roads. All of them good to go for Terrain Processor and arma3.

Started to assign RGB colors to OSM areas, ugh I hate this part, such repetitive task which should be just handled by computer, thats why we have computer to automate repating boring stuff. At least it would be nice to import the RGB colors in. Oh well, they wont get done by bitching about it so better get started Smiley ;)

2022-03-06T13:02:00Z Finished painfully assigning RGB colors to OSM areas, uuh.

But it feels good to have another global mapper (PMC Ukraine Sumy) exporting satellite texture tiles in the background, I can hear CPU fan increasing rounds per minute when its doing the most intensive part of the tile heh, cool Smiley :cool:

2022-03-06T13:33:00Z Global mapper finished exporting satellite mask.

2022-03-06T20:35:00Z Terra incognita last grid section started to download, this is the big ocean part, interesting to see if it doesn't download anything fast or are there actually useless dark blue ocean imagery available.

2022-03-07 Global Mapper Editing

2022-03-07T10:48:00Z New editing day has started, I'm currently editing PMC Ukraine Berdyansk, but any moment there is a free time I'll plug in few edits for odesa. Right now I inspected terra incognita downloaded JPG's for those Black Tiles, none were present, good.

Didn't manage to do else than just that irFanview inspection.

2022-03-07T18:26:00Z Back into Odesa again, lets get some concrete done now.

Continued by global mapper creating map catalog for terra incognita ArcGIS 17 level satellite imagery download.

2022-03-07T18:36:00Z Started global mapper satellite texture export in 8 x 8 grid tiles setup, this will take some time...

2022-03-07T21:44:00Z Finished export some moments ago, was busy editing PMC Ukraine Ivankiv. Next is the dreaded GIMP clone stamp tool brushing off clouds, ugh.

2022-03-08 The Coast of GIMP

2022-03-08T11:57:00Z New editing day has started, next task is to GIMP clone stamp tool brush off the cloud puffs. I'm not looking forward to this task as while I like working in GIMP, I hate these clouds, they depress me Smiley :(

Satellite texture tiles with clouds: A1, B4, B1, B6, C3, C6, D3, F2

Satellite texture tiles with coastline: A3, A8, A7, B4, B3, B2, B8, B5, C4, C3, C8, C7, C6, C5, D4, D6, D5, E1, E5, F4, F2, F1, F5, G4, G3, G2, G1, H3, H2

Satellite texture tiles full ocean: D8, D7, E8, E7, E6, F8, F7, F6, G8, G7, G6, G5, H4, H8, H7, H6, H5

2022-03-08T13:18:00Z Finished fixing clouds, next is the coastlines which will take way longer time than clouds. Actually clouds were very few in this terrain, this is a coastline terrain which requires a lot of work.

2022-03-08T13:41:00Z I just feel, not physically but emotionally exhausted of editing right now. I know I'm going to laugh and shake my head for this in the future when reading this dev diary, but going to install cities skylines and play a little there to take my mind off from Ukraine war and arma3 terrain editing, I just cant keep going right now as I'm looking at the incredibly long list of coastline tiles which need well over 10 minutes of editing, each.

2022-03-08T15:44:00Z Cities skylines is stupid, back to arma3 editing Smiley ;)

2022-03-08T20:03:00Z Still editing and have almost 20 tiles to go, exhausted, just don't want to do them, need to take a break ...

2022-03-09 Endless Coastline Nightmare

2022-03-09T12:30:00Z New editing day has started. Last night I actually sneakily did one single satellite texture tile without recording video or writing dev diary about it as I was super bored, it was kind of like "off the grid" work just for my own enjoyment heh.

Currently I got 18 coast tiles to paint and then to create full ocean tiles which is of course easy with file copy bat method. This is still demoralizing amount of coastlines to paint (free hand select) in GIMP, but I'm taking it easy, one tile at the time and stopping at any moment it feels like exhausting, no rush, no pressure, just a nice and enjoyable GIMP editing.

2022-03-09T15:17:00Z Finally finished GIMP free hand selecting all coast line tiles and painting them with ocean sea bed texture.

2022-03-09T15:29:00Z Got the full ocean tiles done too. Uah now I am done with satellite texture Smiley :)

Then it was GraphicsMagick time by running Darken Merge Picturemap Satellite Texture Tiles _run_graphicsmagick_darken_and_picturemap_tiles_8x8_green.bat and _run_graphicsmagick_l3dt_8192_texture_map_tiles_8x8.bat files.

2022-03-09T16:10:00Z Oops, forgot to update dev diary again, big time. Okay so terrain builder project has been created, heightmap roads smoothed and imported, terrain processor forest objects generated and imported. Now importing satellite mask tiles.

2022-03-09T16:48:00Z Terrain builder finished importing satellite texture/mask, then created airfield_runways shape layer, the shapes for airfield 1 and keypoint.

Exported airfield_runways.shp into data\ dir.

2022-03-09T16:50:00Z Terrain builder started generate layers.

Global mapper opened airfield_runways.shp and set ID/ORDER properly, then merged with roads.shp and exported roads.shp.

2022-03-09T18:50:00Z Terrain builder finished generating layers, in the mean time I L3DT smoothed airfield runways.

Exported WRP and ran Convert Layers PNG to PAA Fast bat.

2022-03-09T19:21:00Z Started pboProject binarization, we are done here Smiley :)

2022-03-10 First ArmA 3 Check

2022-03-10T17:13:00Z Today I did first in-game test... and arma3 freezes in mission editor load right in the beginning. Did some basic investigating like checking RVMAT counts etc, but nothing I could spot right away. Hmm.

I could not see anything wrong with the files in general, so just for the sake of it ran WRP moveObject re-pathing and then started pboProject binarization (PMC Editing Wiki: search "binarize"). Lets see if it somehow would fix itself, as I cant tell whats wrong with it.

Got it binarized and tried again... no change, arma3 freezes with animated windows loading circle icon (hourglass), had to task manager kill arma3 task to recover.

Hmm wonder what the heck is going on.

2022-04-15 Hunt For The Freeze Bug

2022-04-15T08:33:00Z New editing day has started, not many hours ago I released PMC Africa Niger Tongo terrain, then checked out Hcpookie's artillery pack a bit but now I'm back to PMC terrain editing and ukraine odesa is next on the list. Just finished reading this dev diary through and uh oh, there is bad news with that strange arma3 freeze.

Hmm if too many RVMATs is not the reason, then what... hmm. Checked pmc_ukraine_odesa_data_layers addon and there was a bit over 12,000 RVMAT files, that should not cause any issues with arma3.

I know I know dev diary is clear, but just to triple check my stubborness I'm going to launch arma3 and try loading odesa, also maybe checking RPT for errors, who knows maybe there are some clueds.

Checked in arma3 and yup, it freezes when loading into mission editor, RPT is... clean, there is nothing much than basic arma3 init stuff and the it just stops, no warnings no errors.

Alright I'm going to jump straight away to hardcore debugging mode, going to load some previously well known flat heightmap, lets say todays release africa niger tongo, nice and flat, then remove all objects and then binarize, if it still freezes then the issue is definitely satellite texture/mask which means RVMAT files from the mask somehow.

Got flat style heightmap imported, rebuilt terrain and exported object-less WRP, moveObject re-pathed WRP and then started pboProject binarization, lets hope this works...

2022-04-15T09:35:00Z Binarization finished, lets see if terrain loads in arma3 ...

Negative, no go, doesn't load Smiley :(

Yeah its 99% sure the satellite mask RVMAT config files now, it cant be anything else if heightmap and objects are not the issue.

Hmm alright so I'm going to delete existing layers PAA files and RVMATs, then import YELLOW BLANK satellite mask image, generate layers and then we'll see what happens in arma3.

2022-04-15T09:46:00Z Terrain builder started to generate layers, this will take couple of hours. It might be real tight if I get this binarized tonight before my bed time...

2022-04-15T11:37:00Z Layers generated and launching convert layers PNG to PAA fast bat.

While that was running added Russian invasion of Ukraine starting day to config.cpp startDate.

As the satellite mask was single color (yellow), it generated only about 1,800 RVMAT files.

2022-04-15T12:00:00Z Started binarizing. Noon is good binarizing time.

2022-04-15T12:33:00Z Binarize finished, moment of truth, does single color satellite mask work or not...

Nope, doesn't work. Okay I'm getting tired its been a long day for me, I'm not going to even bother working on this more, going to chill rest of the evening and tomorrow we continue debugging this bitch. Most likely will go hardcore again and re-create whole terrain builder project from scratch.

Nuke it from orbit, it's the only way to be sure ...

2022-04-15 I Am The Solution To Your Problems

2022-04-15T19:54:00Z New editing day has started, was thinking about what to do with this project last night and now that I woke up, no possible bug causes come to mind, so I'm just going to nuke it from the orbit, its the only way to be sure... meaning, deleting terrain builder project and re-creating it from scratch.

Luckily I still have the full global mapper source data directory, so importing satellite texture/mask is easy.

2022-04-15T20:05:00Z Created terrain builder project, imported heightmap, satellite texture mask placeholder image and now started to import satellite texture tiles.

2022-04-15T20:36:00Z Importing done, now started to generate layers, this is the long one, couple of hours at least Smiley :(

In the mean time I was editing some tweaks to pmctactical.org/arma3/ web pages, stuff that I wanted to do several days now, like making the duplicated text descriptions as php include files, for example 40km and 81km terrain size for dummies "how big map is ...", that text.

I'm probably going to run out of things to do before terrain builder has finished generating layers. Man do I hate waiting...

2022-04-15T22:10:00Z Completely unrelated matter but it reflects on my mood so I have to mention it here, while terrain builder was generating layers I checked out Hcpookie's artillery and SAM packs, I had tested these alredy in 2018 and 2019, but now these latest updates give arma3 error dialog and huge error spam in RPT log, I was extremely disappointed to discover this as my plan was to start using these vehicles in PMC missions Smiley :(

PMC has zero tolerance policy for errors, if addon has errors we wont be using it. So I wont be using Hcpookie's arty/sam packs until those error popups have been fixed.

Now of course life ain't that black and white, if you look at CUP for example, well its RPT log is filled with errors too, but the difference is that in-game error dialog which STOPS your mission and requires to click Close/continue/OK button is completely unacceptable, luckily CUP doesn't have any of those.

Feels like such a tease, you have all these nice arty/sam vehicles available but you cant use them because of errors Smiley :(

2022-04-15T22:25:00Z Finished generating layers, started convert layers PNG to PAA fast bat.

2022-04-15T22:45:00Z Done converting PNGs to PAAs, next started to Separate Data Layers for smaller PBOs. Getting close to binarization time again.

I'm currently running arma3, but not using ukraine odesa terrain so should not cause any conflict issues when binarizing and overwriting those PBO files.

2022-04-15T22:47:00Z Started binarizing. Really hoping this would work, because if it wont... then I truly don't know what voodoo magic would Smiley ;)

2022-04-15T23:19:00Z Done, now next arma3 check, moment of truth...

Still freezes Smiley :(

Alright, what the fuck is going on with this terrain, BRAND NEW terrain builder project has been created with nothing on it except heightmap and satellite texture/mask (and mask is single color too).

Hmm going to restore some stock default config.cpp just in case and try it out.

Hmm what the hell is this, on config.cpp class names{} pmc_names.hpp I found this:

class place1
{
	name = "Одеса";
	position[] = { 125946, 5.15036e+06 };
	type = "nameLocal";

The second coordinate is all out of whack, eh how come. So commented out include pmc_names.hpp line, binarized _cfg addon and tested in arma3... it works. Shiet. Well another good piece of information learned today, make sure class names{} do not have faulty coordinates.

2022-04-15T23:53:00Z Maybe its the disappointing addon discovery or just lack of sleep dunno, but at this point was all out of editing energy even though just found a cause for the terrain freeze. Just had no motivation and will to continue editing, probably going to take a nap...

2022-04-16T03:30:00Z Woke up from nap, put Terrain Builder Template Libraries and objects layer file back in place, then started to import satellite mask tiles. In global mapper just finished transforming OSM place names properly without them being "off the known universe" with the massive second coordinate, copy pasted them into pmc_names.hpp now our arma3 mission editor freeze bug has been fixed.

Lets get this fucking terrain done.

2022-04-16T03:46:00Z Terrain builder generate layers with full satellite mask, just started...

While that process was running I was editing PMC multiplayer missions, PMC Long Pluto v1.0.4 to be exact, launched dedicated server on windows 8.1 and was game-play testing the mission while convert layers PNG to PAA fast bat was running, had some slight stutter at the same time in arma3 Smiley ;)

2022-04-16T06:13:00Z Finally binarize is chewing on the freshly created files, hopefully this works.

It works, "whohoo".. yup. Okay discovered several small cleanup items, plus the big thing which is the coastline water creeping to land issue I still need to fix. Right now got excited about multiplayer mission editing and would like to edit and play some more heh.

2022-04-16T07:09:00Z Actually to hell with it, I'll do what I want... going to edit/play some MP missions for now until I get bored with that or until the day is over Smiley ;)

2022-04-16 That Creeping Water

2022-04-16T19:19:00Z New editing day has started. Yesterday spent time editing/playing arma3 multiplayer missions, then did some PMC Gaming computer clonezilla work to clone image backup M.2 NVMe and restored Linux Mint v20.3 desktop installation with steam + proton + arma3 on it.

Today's goal is to get ukraine odesa terrain done and released. Was looking at my notes from yesterday and heightmap needs to be replaced, the ocean coastline to be done, new vegetation objects, uh bunch of stuff. But first I need to eat days first meal before getting busy with editing.

2022-04-16T19:55:00Z Alright lets get started with ocean bottom seabed and coastline heightmap editing.

Started global mapper, exported SRTM heightmap. Then did various shuffles to get its elevations clipped from below 0m to 0.99m and between 0.01m and 0.4m (or something) to 1.01m heh, shuffle indeed. Goal here is to get rid of the water creeping into land bug that SRTM heightmap have in arma3.

2022-04-16T20:06:00Z Then started the annoying process of GIMP loading Satellite.Mask/ XCF image, its equivalent Sat_Mask_??.png companion and copy-pasting RGB 0,0,150 blue ocean selection from XCF into PNG. This has to be automated somehow using ImageMagick I cant do this crap after this.

The thing is though, after I've done editing this and its no longer an issue, I wont be returning and trying to figure out imagemagick command line to get it done, because its not urgent issue... until the next time it is, heh. Yup.

Anyways for now, painstaking GIMP shuffle continues... only 28 more XCF images to go, oh my! Smiley :evil:

PMC Ukraine Odesa ArmA 3 Screenshot

2022-04-16T20:45:00Z Finished GIMP shuffle, uuh, this must be automated.

Then global mapper ocean shape generation.

PMC Ukraine Odesa ArmA 3 Screenshot

Finally exporting the final heightmap, yeah!

PMC Ukraine Odesa ArmA 3 Screenshot

Then terrain processor was hard at work. Generating new better CUP chernarus forest vegetation and smoothing roads heightmap.

PMC Ukraine Odesa ArmA 3 Screenshot

Terrain processor generated 1,712,380 vegetation objects. Quite low number for an 81.9km terrain, but its fine.

Imported heightmap from terrain processor road smoothing.

PMC Ukraine Odesa ArmA 3 Screenshot

Looking good. Then imported those 1.712 million objects, it took about hmm dunno 15min or so.

Then quickly imported airfield_runways.shp shape file and created airfield 1 keypoint, just because. Next export WRP, save and close project. Then moveObject re-pathing the WRP and now we are done with these edits.

2022-04-16T21:56:00Z Started pboProject binarizing, once again, hopefully this is now the last time for initial release.

Felt extremely bored just staring at the pboProject console waiting for binarize to finish, but didn't really want to start editing missions deeply or something like that, as soon as binarize finishes its arma3 testing time and hmm depending on the results release procedures will begin right after. Also I don't want to start release procedures before the terrain is fully finalized and ready for release, today it just feels like one step at the time.

As I've said many times, I hate waiting, terrain editing is one big waiting game...

2022-04-16T22:34:00Z Finished, uuh finally.

Everything looked great in arma3 except I forgot to uncomment config.cpp pmc_names.hpp so no names including the airfield name appeared in 2d mapview. Well that is a quick fix.

2022-04-17T03:22:00Z Took a little break here, now just finished last arma3 in-game check and everything looks great, we are good to go for release procedure Smiley :)

Added pmc_ukraine_odesa class name to PMC "worldSize" mission pack compile bat files terrain list, gotta get those missions.

2022-04-17T03:56:00Z Screenshots taken, sorted, renamed, edited and converted. Ready for web.

2022-04-17T04:15:00Z Basic web page php files done.

2022-04-17T04:30:00Z This dev diary moved from localhost to the actual web page. Starting to proof read and link add this page now...

2022-04-17T05:19:00Z Finished proof reading, got them links added as well.

Uploaded web pages online, checked that all images are working OK and there are no html/php errors. In the mean time the release .7z package was uploading to torrent seedbox which took few minutes.

2022-04-17T05:43:00Z PMC Ukraine Odesa v0.1 has been released! Smiley :cool:

PMC Ukraine Odesa: Changelog, Dev Diary, Screenshots. Back to PMC Terrain's page.