Dev Diary PMC Ukraine Donetsk ArmA 3 Terrain

This is dev diary for PMC Ukraine Donetsk ArmA 3 real world data terrain by PMC.

We have been writing development or developer diaries to PMC Tactical Forums for two decades, these addon dev diaries are spread to many forum topics. One example PMC Tactical Forums: Vietnam: The Experience Neverending development diary 2007-10-13. On 2020 things changed in Farming Simulator 19 editing community when we started to write detailed dev diaries as web html pages because we had no forum, now in 2022 the same dev diary story telling continues in ofp/arma community.

These dev diaries are my current on the fly thoughts and emotions, if I feel shitty I will clearly state that. I try to avoid it but sometimes there might be hilarious profanity rants when things don't go exactly as I planned, so if you are "offended" by few curse words you probably should stop reading now Smiley ;)

Also these dev diaries are written to many addon projects and sometimes repeating the same lines of text or taking exact same screenshots feels stupid so there might be text just listing boring "I did XYZ and then ZYX" type of info without further details. And remember; this is NOT a tutorial, this is entertainment Smiley ;)

Time stamps are Universal Time Coordinated (UTC).

2017 Project Started

PMC Ukraine Donetsk ArmA 3 terrain project was started in spring of 2017 in first few months when I learned HOWTO create and edit (ArmA 3 Real World Data Terrains). Unfortunately only dev diary entries I wrote back in 2017 was mere weekly summaries like PMC Tactical Forums: Terrain Development topic or release note type of PMC Tactical Forum posts.

2022-02-21 Orientation

2022-02-21T23:00:00Z Just unpacked pmc_ukraine_donetsk_v0.2.7z (current latest release) into my arma3 addons dir, launched arma3 v2.06 to check out how the terrain looks. Wrote down few bug reports about mod.cpp which needs some cosmetic edits. In-game I see right away that PMC_Terrain_Data roadslib.cfg is broken as wrong texture is applied to the roads and ukraine donetsk still has the extremely ugly Terrain Processor randomly generated buildings from way back in late 2017 early 2018.

After no more than 10min in-game I already had written several bug reports. Overall this terrain looks like its definitely from 2017 era because so many things are broken / unfinished or at least bad design.

2022-02-21T23:27:00Z Finished initial inspection, good bunch of bug reports written and I have a pretty good idea of the development stage right now.

2022-02-22T00:30:00Z Started operation to download brand new 2022 ArcGIS 17 level satellite imagery (texture) because the current one is from 2017 google/maps heh.

Global Mapper loaded the project up, got no errors which was surprising. Even more surprising was to see that I do have ArcGIS 17 level satellite imagery downloaded already.

Found preview-arcgis-17-level-2017.png image and it looks... just simply, broken. It is completely unusable, just horrible patches, clouds and even missing parts of the imagery.

So now I'm going to Terra Incognita generate rectangle selections for split to 4 setup for the quick download.

2022-02-22T00:40:00Z Ukraine_Donetsk_81km_1_arcgis_17_level download started.

Took backups from the whole terrain project directory setup including Terrain Builder Source/ dir.

Started terrain builder and loaded the project up, used CTRL-E to export WRP and guess what... project is broken, empty WRP got exported. So this means, to best of my recollection that I need to re-generate layers to fix the project.

2022-02-22T01:12:00Z Deleted existing layers RVMAT and PAA files, then started to generate layers. I also discovered that satellite texture/mask resolution is 40960 pixels so 2m/px scale, ugh. This needs to be fixed. But for now, I want to generate layers to fix terrain builder project.

2022-02-22T01:45:00Z Finished generating layers, now CTRL-E export WRP works, tested it, all good.

Then it was time to Convert Layers PNG to PAA Fast bat Smiley :)

ArmA 3 PMC Ukraine Donetsk Screenshot

Then split WRP and Data by doing Separate Data Layers for smaller PBOs.

ArmA 3 PMC Ukraine Donetsk Screenshot

Did the RVMAT and WRP moveObject re-pathing, then it was time to pboProject binarize.

scanning for pbos to make....
Processing \pmc\pmc_ukraine_donetsk_cfg...
<scanning files to pack>
...
</endscan>
<MakePbo...>
</MakePbo>
makepbo failed: Rapify error
Packing failed

Gotta love piece of shit mikeros stupid pboProject rules and the oh so informative error messages Smiley :D

2022-02-22T02:11:00Z Had to add minstreetsquare config.cpp entry to please pboProject rapify, finally got it error free and now its binarizing (PMC Editing Wiki: search "binarize").

2022-02-22T02:24:00Z pboProject binarization finished. Terra incognita is still downloading, 17 of 25 tiles downloaded.

2022-02-22T04:02:00Z Second terra incognita satellite grid part is now downloading 7 of 25. This is the extremely boring waiting game. I cant really edit anything else because I hate writing dev diary text and record videos when I bunny hop between terrain projects. Its fairly easy to maintain focus with dev diaries, but recording videos are technically impossible because you are editing two or maybe even three or four projects at the same time, how do you decide which projects video it is when you have maybe 3 project files open in the screen at the same time. Yup.

Terra incognita download for 81.9km terrain will take a long time, this is not even half way done yet and its been over 3 hours. So I just need to find something to do in the mean time. I will probably fix the terrain builder projects for PMC Ukraine Luhansk and PMC Ukraine Mariupol terrains.

2022-02-22T08:34:00Z Last terra incognita satellite texture grid started downloading.

2022-02-22T11:07:00Z Last grid finished downloading, finally. Now I get to start editing donetsk and downloading luhansk satellite Smiley :)

Inspected the oziexplorer .map format saved JPGs and on the very last grid (4) in last image (heh) there was three Terra Incognita Black Tiles. So loaded terra incognita back up and exported oziexplorer .map again.

Started global mapper and created new map catalog for arcgis 17 level satellite imagery I just downloaded, called it _2022.

2022-02-22T11:57:00Z Started global mapper exporting grid 4 x 4 tiles of 1m/px scale 81920 resolution satellite texture.

When the exporting started I realized that... its again waiting game before I get to GIMP editing those few cloud puffs off. Also that I could have done the OpenStreetMap (OSM) shape files to images already while terra incognita was downloading heh.

2022-02-22 GIMP Satellite Texture Fixing

2022-02-22T20:39:00Z New editing day has started, woke up and obviously global mapper satellite texture export had finished.

Exported ArcGIS 17 level preview image for new 2022 satellite imagery download, using 25m cell size.

2022-02-22T22:30:00Z GIMP edited sat_A1.png and sat_A3.png to clone stamp tool brush clouds off.

Last night when I global mapper exported satellite texture tiles I made a mistake to export 4 x 4 grid, this creates 20480 x 20480 pixel resolution tiles which are extremely heavy to edit in GIMP. I'm writing this paragraph as sat_B1.png has been exporting/saving for several minutes now and I got bored.

I'm afraid I need to delete those tiles and go back to global mapper and export 8 x 8 which generates 10240 res tiles which are much more light weight to save in GIMP. Terrain builder importing is no consideration as everything goes through PGW world file grid images.

2022-02-22T22:58:00Z Global mapper started to export 8 x 8 tiles... sigh, oh well this is another two or so hours, cant remember how long it takes. I need to write this down for the next time I'm returning from Farming Simulator 19 editing time period and cant remember shit about ArmA 3 editing.

Global mapper is very accurate about estimated time to finish counter, it now says 0150hrs, so oh wow 2h 50min uuh Smiley :(

While waiting I downloaded new OSM shape files for ukraine. Latest download was very recent as I made PMC Ukraine Yenky terrain, this was october last year, so relatively recent.

2022-02-23T02:03:00Z Global mapper satellite texture tiles export done and now GIMP loading sat_*.png images to check them out.

2022-02-23T03:11:00Z GIMP clone stamp tool brushed several sat_*.png tile images for clouds, now its lunch break time.

2022-02-23T03:47:00Z Lunch break over, back to GIMP clone stamp tool brushing last satellite texture tiles.

2022-02-23T03:49:00Z Finished GIMP clone stamp tool brushing clouds off. The last batch of images had no clouds to brush off.

Global mapper imported OSM shape files, cropped them into areas, lines and points layers. Then removed "bad" roads which have not much use in arma3. This went somewhat badly, the road feature count was over 9000 and when I deleted "track" labeled roads, it plummeted to 1200 which looks like almost no roads at all heh. Hmm might need to revisit the roads layer again.

2022-02-23T04:59:00Z Finished adding RGB color values to OSM natural area styles.

2022-02-23T05:03:00Z Started to global mapper export Sat_Mask PNG images in 8 x 8 grid tiles. This should not take too long as they are low amount of colors, these images export rather quickly.

ArmA 3 PMC Ukraine Donetsk Screenshot

2022-02-23T05:30:00Z Global mapper finished exporting Sat_Mask.

Next ran _run_graphicsmagick_convert_Sat_Mask_PNG_to_BMP_with_white_fix_8x8.bat file to GraphicsMagick convert global mapper exported Sat_Mask PNG images into BMP image format. This includes changing background white RGB 255,255,255 color to some grass or something which PMC_Terrain_Data addon layers.cfg is using.

Then ran _run_graphicsmagick_darken_and_picturemap_tiles_8x8_green.bat which is the backbone of making satellite images usable in terrain builder, which is darkening and converting PNG to BMP image format, including generating picturemap_ca.tga and L3DT texture map images.

Deleted old layers images using _run_delete_project_rvmat_png_paa_files.bat file. Was eyeballing the case sensitive terrain project directory name which is PMC_ukraine_donetsk and PMC_ukraine_donetsk_cfg, I don't like the uppercase addon tag which doesn't work in linux dedicated servers. Knowing terrain builder, the project most likely breaks if I rename the dir (tag), but then again it doesn't matter as I'm going to import satellite texture/mask anyways.

Decided to rename all these ukraine terrain addon tags to lower case like they should be.

Terrain builder deleted all raster layers, cleared unused data which had a lot of files, then saved project and exited.

2022-02-23T05:58:00Z Terrain builder started to import satellite mask.

2022-02-23T06:06:00Z Done and started to import satellite texture.

2022-02-23T06:34:00Z Finished terrain builder importing satellite texture.

Changed terrain builder mapframe properties for satellite texture/mask to 81920 x 81920 resolution. Then started to generate layers.

2022-02-23T07:47:00Z Texture has been generated, now started on mask.

Soon after I started to really get bored while waiting for terrain builder generate layers to finish so I could proceed editing THIS terrain. Sure I could have edited some others but the unfortunate side effect from these dev diary texts and video captures is that... it limits my editing speed and multitasking a great deal, I simply cannot edit two three maybe even four terrains at the same time running the data through my tool pipeline if I also have to write text like this and capture videos.

Hopefully this dev diary business wont slow me down so much that I would decide to stop writing these, because I truly believe that dev diaries are worth in gold 5, maybe 10 years from now or lets say two game engines into the future if you want to put it like that.

Its the waiting that I hate the most, I just cant stand it. Perfectly good editing time is wasted while I'm sitting on my hands waiting for piece of shit 32bit single threaded slow ass windows tools to complete their tasks Smiley :)

2022-02-23T08:54:00Z Terrain builder generate layers finished. Exported WRP. Right after started convert layers PNG to PAA fast bat, 26 core heh.

2022-02-23T09:12:00Z Convert layers PNG to PAA fast 26 core bat finished, CPU fan was screaming through the conversion heh.

There was so many RVMATs that the old free moveObject could not handle the WRP re-pathing. My WRP moveObject bat file was so old that it still used the free tool directory / exe.

2022-02-23T10:00:00Z Started pboProject binarization.

Forgot to write when it finished.

2022-02-23T10:50:00Z Tested in-game, crash to desktop (CTD) on loading. Its because satellite mask is "too detailed" with too many RVMATs. Actually the old 32bit moveObject is correct when it complained about too many RVMAT indexes. This sucks.

PMC Ukraine Donetsk now with detailed satellite mask comes down to 42,241 data_layers\ RVMAT files. This results arma3 CTD when loading the terrain.

So dunno what the best solution is, probably merging tons of terrain type layers.cfg RGB colors, or as Jakerod suggested in PMC discord to use 2048 resolution surface mask tile size.

Anyways that is tomorrows work, now I'm already way too tired and mind going blank, no more editing today. Calling it a night.

2022-02-23T11:06:00Z Well, I thought that just for the knowledge information gain I am going to generate layers with 2048 res surface mask tile size just to see how many RVMAT files it generates, this can run overnight so tomorrow morning I have fresh data waiting for me.

Good night.

2022-02-23T16:05:00Z Heh.. let the processes run while I watched a movie and once movie was over checked in-game how the 2048 pixel resolution surface mask tiles work... and they loaded ok no more CTD. Not only that but also the "mosaic" mapview bug is gone. Also maybe I was imagining it but it felt like loading larger texture tiles was faster now, but that needs more testing to verify.

Note: I'm keeping PMC addon projects in Western Digital 4tb Green hard disk drive (HDD), I just don't have SSD/NVMe capacity to store all of them. While releasing this dev diary I checked the HDD space used for my current pbo addons, it was 285gb, all released PMC torrents currently seeding are 245gb in 7-zip archives. Most of the PMC terrains have not been binarized in my HDD, mostly due file size usage. Anyways point was that HDD is slow to load arma3 stuff, just a reference to the satellite texture in-game mapview loading speed.

And now, now I'm calling it a night heh.

2022-02-24 Russia Invades Ukraine

2022-02-24T08:54:00Z New editing day has started. UTC early this morning Russian armed forces launched invasion of Ukraine.

Fuck.

Smiley :(

Okay time to start creating some new PMC Ukraine terrains.

Just terrain builder removed the terrain processor randomly generated BUILDINGS which looked real bad, exported WRP, moveObject re-pathed WRP and now pboProject binarizing it.

2022-02-25 Roads.shp Bug Hunt

2022-02-25T03:55:00Z New editing day has started, yesterday discovered cause for the roads being broken in-game, it was global mapper saving. I just confirmed it now with systematical step by step bug hunting / testing, I loaded roads.shp using terrain builder and exported them, then they work just fine, however if I global mapper load the roads.shp and simply save, it breaks, doesn't work in-game arma3 anymore.

So that pisses me off, what is wrong with my global mapper shapefile saving now Smiley :(

Okay I can get the job done by doing that stupid terrain builder import/export shuffle, but its just so lame to have to do that.

Anyways todays editing day... err not much to do, objects for buildings and meadows have been removed, so dunno what other fixes there are for next. I'm going to switch fixing roads.shp for luhansk and mariupol now.

2022-03-05 Smooth Airfield Runways

2022-03-05T07:32:00Z New editing day has started. Bug report lists airfield runway is not properly smoothed meaning the ends are either buried to ground or elevated (difficult to explain).

Created L3DT project by importing heightmap.asc, however when I was looking for L3DT_texture_map.png it was nowhere to be seen, not even in global mapper dir, so apparently... I didn't create it. Luckily the 4096 x 4096 pixel resolution PMC-Satellite-Large.png image worked fine.

The L3DT_texture_map.png is 8192 x 8192 res, so dunno if there is any details lost with smaller res image, at least it felt like importing faster.

Once L3DT exported new heightmap_airfields_smoothed.asc I imported it to terrain builder and rebuilt terrain, then exported WRP which has 4.364 million objects.

Then did latitude/longitude config using global mapper and PMC Editing Wiki guide.

Deleted all terrain builder object layers except obj_forest to get rid of the stupid bushes on farm fields. After some studying I decided to delete even the obj_forest and just re-create all the forest tree lines.

Also smoothed roads heightmap.asc.

Terrain processor generated new forest tree line objects, with that particular TPP file it came to 2.1 million objects which is only half of what we had before, mainly because the stupid bushes got removed. For 81.9km terrain that amount of forest objects is very low, but guess it depends on how much farm fields there are as those cut down tree count a great deal.

Once terrain processor was done I obviously imported new objects and heightmap.asc.

2022-03-05T08:36:00Z Number of objects in WRP file now 2,160,403.

2022-03-19 New Forests Objects

2022-03-19T23:56:00Z New editing day has started, well probably not that much editing anymore, my focus is now only on releasing terrain addons.

I just finished arma3 in-game check of PMC Ukraine Donetsk, it overall looks to be good to go, except forest tree line objects are done with the old terrain processor project file which includes the small object count wasting trees, no bushes or rocks.

Copied Forest_CUP_Chernarus_2022-03-14.tpp project file from my arma3 editing source files dir into donetsk terrain builder project dir (enuff with them projects). Then edited obj_forest density from 100 to 125 and object properties MINDIST from 10 to 7, now it should be same as in PMC Ukraine Mariupol which got released yesterday (or was it 18th hmm).

Also copied _run_copy_roads_here.bat into Source/TerrainProcessor/ dir, edited it for donetsk and ran it to copy roads.* files into this dir which TPP files use.

Then just ran terrain processor using the new Forest_CUP_Chernarus_2022-03-20.tpp file.

ArmA 3 PMC Ukraine Donetsk Screenshot

That TPP generated 4.348 million objects, nice, it should make somewhat thick tree lines and still (hopefully) no issue having that many objects in a single terrain builder objects layer.

But uh oh, when I was closing terrain processor down I noticed road cleanup task was disabled, heh I am still running my CUP Vegetation testing setup here, need to quickly fix that and re-run the project Smiley ;)

So re-ran terrain processor project and now roads got cleaned from any objects. Final object count: 4.260 million.

ArmA 3 PMC Ukraine Donetsk Screenshot

Next started terrain builder object import, it was close to half an hour ETA, it surely takes a while to import few million objects, I wish this task would be more efficient, but oh well guess I can do other tasks while it imports on the background.

2022-03-20T00:45:00Z But uh oh! once terrain builder was done importing there was template library which was missing 4 objects. Argh, forgot to update templatelibs Smiley :(

So saved terrain builder project (just in case), closed it, then in windows file explorer deleted existing Source/TerrainBuilder/TemplateLibs/ dir and copied the CUP ones from PMC Github Repo Terrain Builder Template Libraries dir over using my _run_create_TemplateLibs_generic_CUP.bat file. Then started terrain builder, went through the hoops of clicking yes I want to continue loading the project even though templatelibs are missing, then removed old non-existant templatelibs (confused yet? good HAH) and finally loaded up the new templatelibs. Ugh.

Then started to import 4.260 million objects terrain processor LBT file once again with ETA half an hour or so... who ever told you terrain developing is not a waiting game, were joking.

2022-03-20T01:20:00Z Finished importing the objects.

Then did export WRP (CTRL-E), moveObject WRP re-pathing and started pboProject binarization.

2022-03-20T02:05:00Z Finished binarizing.

2022-03-20T03:12:00Z Checked arma3 in-game and forest tree lines look good now, thick with bushes and rarely appearing rocks. Looking very nice.

Unfortunately after this I fell into the burnout boredom mind goes blank mode where I couldn't get any editing done, not even to get started writing these web pages, my motivational energy was simply gone like poof Smiley :(

2022-03-20 Web Work

2022-03-20T23:18:00Z New editing day has started, we continue where we left off yesterday. Would be nice to get some edits done today.

2022-03-20T23:47:00Z Created initial empty web page files for PMC Ukraine Donetsk. All links are working.

Then migrated this web development localhost directory page to this online dev diary page you are reading now Smiley :)

2022-03-21T01:40:00Z Finished editing this page, added links to all important parts which need or benefit from further explanation, searched, found, sorted, edited and converted all PNG screenshots seen in this page. Okay, dev diary for PMC Ukraine Donetsk seems to be good to go Smiley :)

2022-03-21T01:55:00Z Changelog page finished. Unfortunately I could not find any info from 2017 when this terrain was created, guess I started writing detailed changelogs later or something. Who knows there might be some info buried deep in PMC Tactical Forums but those will have to be added later, if found.

2022-03-21T02:37:00Z Screenshots taken, edited, converted and added to Screenshots web page.

Hmm whats next... took a screenshot of mission editor select terrain menu, it shows the latitude / longitude cross over world map which is kind of nice, though it would be good addition to the web page.

ArmA 3 PMC Ukraine Donetsk Screenshot

Then... started to run out of things to do, web pages all seem to be pretty solid. I am not perfectly happy with the hyper link colors, well the green perhaps fits to arma3 woodland camo pattern okay-ish but when link is highlighted it turns into blue background which is kind of... off. Hmm need to come up with better color scheme for that. Also pisses me off that Screenshots page now has satellite and that above mission editor select terrain image thumbnails different size than regular 3840 x 2160 4k thumbnails heh.

2022-03-21T05:09:00Z After a cooking food and lunch break added few more bits and bytes into dev diary web page, but yeah I definitely start to run out of things to edit. I am also proof reading, checking, testing, etc way too much, I should just wrap it up and ship it off, this ain't NASA rocket science just get it done and release already Smiley :p

Then generated bisign dedicated server signature key files for all PBOs. Next was to create the 7-zip release packet including editing pmc_ukraine_donetsk_readme.txt of course to all the latest info. That .7z packet came to pmc_ukraine_donetsk_v0.3.7z 3.56gb, heh nice size for a single terrain in tightly squeezed 7-zip archive Smiley ;)

2022-03-21T05:54:00Z Created pmc_ukraine_donetsk_v0.3.7z.torrent using qBittorrent v4.4.

Uploaded the release to our "seedbox" so to speak. Got torrent magnet link from qBittorrent and added it to PMC Tactical ArmA 3 Downloads page with file size info.

2022-03-21T06:55:00Z PMC Ukraine Donetsk v0.3 has been released! Smiley :D

PMC Ukraine Donetsk: Changelog, Dev Diary, Screenshots. Back to PMC Terrain's page.