SRTM data conversion to Falcon 4 Terrain format

T_Rex · Post by **T_Rex** » 2007-03-01 14:06:22

Closter - I think you're on to something!

Just to make sure I understand, though, the idea is that the F4 lats remain constant from the bare DEM data through to the L2 data.

I guess the first question I would have is whether there is some way to make sure the SRTM data starts at exactly the same lats. If it is off even a little, won't that offset the data, too?

Also, the big question is whether Bluey and Zaggy already took this into account.

Closter · Post by **Closter** » 2007-03-02 00:56:01

I have sent Blueprint a PM with some of my equations (too boring for posting here). These are some related images.

Dem2terrain:

NW: Lat 75.6031 Long -177.514
NE: Lat 75.6031 Long -140.486
SW: Lat 66.3969 Long -170.496
SE: Lat 66.3969 Long -147.504

My calculation

NW Corner = 75.6031º (N) / -177.513º (W)
NE Corner = 75.6031º (N) / -140,487º (W)

SW Corner = 66,3969º (N) / -170.496º (W)

SE Corner = 66,3969º (N) / -147.503º (W)

If someone is interested I can send him the link to the pdf or simply post it here.

(OK: here it is Little pdf with the trigonometric equations for obtaining the geographic coordinates of the corners in the previous example.)

Sherlock · Post by **Sherlock** » 2007-03-02 01:01:36

@Closter: I am definitely interested in reading it. Could you please post a link to the pdf? Thanks!

Blueprint · Post by **Blueprint** » 2007-03-02 14:42:43

The equations are very good and might just lead us to what we are missing. I'm looking into it now. The results look very promising so far.
Kudos to Closter!

Zaggy · Post by **Zaggy** » 2007-03-02 14:53:06

Closter, cool...

Snakey, do you know what Projection is being used by DEM2Terrain? That would be VERY Useful to know...

As for my Interpolation Tool, I must have a look for the HACK code version... Its VERY ugly tho! As, like bluey said, a specialist installer then needs to be made to install the L1/L0, and do the normals and etc... Bluey, do you still have the code you wrote to take the Elevation maps, put them back into Lx/Ox formats and etc?

Ive been Sick again the last week, and missing LOTS of school... I will see what i can get together over the next few months tho, but a warning, I am in my last six months of my Adv Dip, and working on my major project, which will have to come first.

Blueprint · Post by **Blueprint** » 2007-03-02 15:01:44

Zaggy wrote:Snakey, do you know what Projection is being used by DEM2Terrain? That would be VERY Useful to know...

I think Closter may have found it.

Zaggy wrote:Bluey, do you still have the code you wrote to take the Elevation maps, put them back into Lx/Ox formats and etc?

Yeah, should be here somewhere (I hope).

T_Rex · Post by **T_Rex** » 2007-03-02 15:08:03

If you guys can make command line utilities to do all this stuff, we can do an installer to run them.

You guys are doing great! My head starts hurting just looking at the trig stuff, though.

Zaggy · Post by **Zaggy** » 2007-03-02 15:59:51

Command line stuff to do what? Intelligently Interpolate existing terrains?

That probably wouldnt be a good idea... My Alg's used a degree of randomness in them at various points (depending on things like gradient and etc), so if you were to want to run that command line on existing installs, you would get slightly differing results on different installs... a BAD thing for MP...

The Intelligent Interpolation is more designed to be used by the theater maker, to add data to the terrain, that he/she can then package and distribute, so all ppl are running the SAME terraform...

BTW TRex, remember when Wooters was BAGGING Bluey and I for this stuff?!?!? HAHAHAAHA...

T_Rex · Post by **T_Rex** » 2007-03-02 16:13:23

Zaggy wrote:Command line stuff to do what? Intelligently Interpolate existing terrains?

I dunno - I don't understand half this stuff.

But, if you guys need an installer, or some way to automate this stuff even for devs, I think we can get that covered.

BTW TRex, remember when Wooters was BAGGING Bluey and I for this stuff?!?!? HAHAHAAHA...

Well, the way I remember it, the idea was to make sure you guys did it right.

The radar LOS calcs were... uh... inconsistent in early SRTM stuff.

Zaggy · Post by **Zaggy** » 2007-03-02 16:29:07

HAHAHAHA... that was an issue we discovered AFTER the initial WOoters attack, regarding L2E... and how we were going to screw things up and have airbases on the sides of hills and etc! hahahaha...

But yeah, the LOS stuff was a curly one for a short while, til Bluey got BaldEagle to make a 4Byte version of that app, that currently eludes me... ummm, yeah!!! hahahaah

Closter · Post by **Closter** » 2007-03-02 18:59:50

Here it is my trial with Taiwan's data:

Dem2terrain:

Theater name: my_asia1
Theater size: Regular
Size (L2 segments): 64 x 64
Size (L2 tiles): 1024 x 1024
Size (Nautical Miles): 552.377 x 552.377
Resolution: 0.134858 Nautical Miles
Centered on: Lat 25 Long 120

NW: Lat 29.6031 Long 114.706
NE: Lat 29.6031 Long 125.294
SW: Lat 20.3969 Long 115.089
SE: Lat 20.3969 Long 124.911

My calculations:
NW = Lat 29.6301º Lon: 114.7058º
NW = Lat 29.6301º Lon: 125.2941º

SW = Lat 20.3969º Lon: 115.0888º
SE = Lat 20.3969º Lon: 124.9111º

Pretty much the same precision....

So it could be possible to use the SRTM data in the theaters as they are. But for sure, the Bluey's new tool will be much better than this, specially for high latitude theaters: Behring, Scandinavia, Iceland...

@TRex, you asked if there is a "offset" between SRTM data and the Dem2terrain. Not in latitude AFAIK, but then it could be in longitude. We need to interpolate the SRTM grid of heights in longitude to find the correct ones for the actual grid used in Falcon.

An example: the SRTM files are SRTM3, or 3 arcsec gap between heights. This means in the previous Taiwan example that in the N row there are 10.1579º / 3 arcsec = 12189 samples. In the S row there are 9.8222 / 3 arcsec = 11786 samples. As you need a row of only 4096 numbers to fill your L0 level, you must interpolate differently in each row. But in the end, all is ok.

Blueprint · Post by **Blueprint** » 2007-03-03 11:24:45

Here's my first test for a 64 size theater.
This is the SRTM data in its geographic form:

Now I'm interpolating between the coordinates that came out of the equation and load the coast line trace of an L0 created from the dem2terrain L2. And this is what I get:

My next test will be with a 128 theater.

ccc · Post by **ccc** » 2007-03-03 13:14:23

wow..sweet!!

Zaggy · Post by **Zaggy** » 2007-03-03 16:23:28

Regarding SRTMv1 and SRTMv2 (v2 is the corrected Data, v1 is the RAW Data... v2 Was released maybe 12 months back or so, JUST after Bluey had finished his project, basically doing the same thing to the v1 Data... Bluey's Data, IMHO, is of better quality tho, fixing one or two of the things the v2 Data missed) and Berings Theater...

SRTM Data sits between 59S and 60N. Any Theater that will encompass Lat's outside that Range, will need to find a substitute for SRTM outside those ranges... BUT, its not overly difficult...

I'd start by resampling your SRTM source from 3ArcSec to 250m. Then acquiring 30ArcSec GLOBE or GTOPO30, which you'd interpolate to 250m. You would then create an overlap of say 30-60minutes of Lat, and interpolate the overlaps, so at the SRTM edge you'd be 100% biased to SRTM, and by the Interpolated GLOBE edge, the bias would have been gradually reversed to the GLOBE data...

As you can tell, i thought about this for the Berings Theater, but never put it into practice... Its something you'd probably like to see put into any SRTM app that Bluey creates; the ability to SPECIFY Source Elevation Array resolution, so you could use SRTM3, SRTM1 (just incase you got access to full res data somehow or wanted to do continental US) as well as User defined res, where we'd specify 250m, for the example above...

Just some thoughts... Would be relevant to Berings of course, as well as the Nordic guys...

T_Rex · Post by **T_Rex** » 2007-03-03 16:39:27

Also, isn't the Falklands South of that range?

Blueprint · Post by **Blueprint** » 2007-03-03 19:16:44

Blueprint wrote:My next test will be with a 128 theater.

Out of memory.

87th_striker · Post by **87th_striker** » 2007-03-03 19:32:19

hehe, another reason to swap to Win Xp 64... can I run it for you ?

Zaggy · Post by **Zaggy** » 2007-03-04 10:03:01

Bluey, segmentise it... Divide the thing into 4 x 64seg theaters, and process them one after the other... Then just stitch the results together...

Zaggy · Post by **Zaggy** » 2007-03-04 10:10:26

T_Rex wrote:Also, isn't the Falklands South of that range?

Nope... Cos all of South America IIRC, is in the data-take... Falklands is probably soemwhere between 51S-53S, without looking at a map... maybe 60W...

Blueprint · Post by **Blueprint** » 2007-03-04 12:29:32

In general we're getting a good fit. There are differences though in coast lines due to the differences between high-res SRTM data and the inferior GLOBE data.

Closter · Post by **Closter** » 2007-03-04 16:24:07

Blueprint, you are the man!

Thank you for taking our theater as example, it`s fascinating to see how our terrain could be made with this kind of detail.

I can see the slightly annoying offset that increases as we go to the east, we must study the reason.
...

I have a guess. It seems near the corner the offset is the least.
We could calculate over each row or calculate only for the corners and interpolate linearly. The problem could come from here.

Blueprint · Post by **Blueprint** » 2007-03-04 19:15:02

The offset appears to be only of horizontal nature. Smallest at the center and increasing with more longitude.
I'm going to play with the eastern and western borders. Reducing the longitudinal range should get us the close to perfect match. Earth radius might me a variable to consider as well.

Sherlock · Post by **Sherlock** » 2007-03-04 19:22:45

You guys are doing incredible work here. My hat's off to you!

I wish I could contribute but as Closter already knows you are already playing in the "major leagues" while I am still playing "T-ball league". Keep up the great work!

Zaggy · Post by **Zaggy** » 2007-03-05 01:40:35

What can I say that I ahvent said before? Bluey, you da man!

Closter · Post by **Closter** » 2007-03-07 00:50:12

@ Blueprint : Offset!

I am writting my theory about the offset. The guess about the projection system was based in the data we knew from dem2terrain: the center and the corners of the terrain, and a clever assumption: data in the rows are the same latitude and the rows have same size in km (mi). Perhaps my assumption was not so clever. Let's improve it!

Blueprint · Post by **Blueprint** » 2007-03-10 16:34:17

New attempt based on Globe data:

Closter · Post by **Closter** » 2007-03-10 18:45:16

It fits much, much better now, as my calculations told. So the dem2terrain projection is disclosed now, cheers!

Sherlock · Post by **Sherlock** » 2007-03-10 20:49:27

Closter wrote:It fits much, much better now, as my calculations told. So the dem2terrain projection is disclosed now, cheers!

YEH! You guys are my heros!

Zaggy · Post by **Zaggy** » 2007-03-11 11:09:55

So, Bluey, whats your thoughts? Are we accurate enough now to be able to generate an SRTM based series of Lx/Ox files for OLD theaters?

Blueprint · Post by **Blueprint** » 2007-03-11 12:00:40

Yes

Sherlock · Post by **Sherlock** » 2007-03-11 17:08:37

@ Blueprint, Zaggy & Closter:

Are you guys ready to share the "How to" with the unwashed masses?

Blueprint · Post by **Blueprint** » 2007-03-12 16:22:09

I'm planning a tool that will take projected SRTM data and apply it to a dem2terrain theater. The SRTM theater installer will have to be run during theater installation.

The projected SRTM data will be created by myself on request because the dataset is very large (~40 GB) and the available data from NASA still has errors in it (mine doesn't).

I think theater creators should take the dem2terrain way and then add SRTM later if needed.

T_Rex · Post by **T_Rex** » 2007-03-12 17:08:38

I guess my ignorance will be further revealed....

Why would a theater-maker NOT prefer the SRTM data?

I shudder to think how much work you'd be taking on by doing this on a by-request basis.

(I personally have about 3 requests for you, though, if you really want to do it that way.)

What is the workflow you envision? :thumbs:

Blueprint · Post by **Blueprint** » 2007-03-12 19:08:33

T_Rex wrote:Why would a theater-maker NOT prefer the SRTM data?

Dunno 8)

T_Rex wrote:I shudder to think how much work you'd be taking on by doing this on a by-request basis. (I personally have about 3 requests for you, though, if you really want to do it that way.)

The data creation app is nowhere near a releasable state. I'd rather not finish it (too much work) and create data on requests instead.

T_Rex wrote:What is the workflow you envision?

I'll make an announcement when I know all the parameters.

Question to theater makers:
I can't remember, is there a 4-byte version of CATE?

I'm going to need a finished theater (64 size, center coordinates must be known) for testing the SRTM installer. Any suggestions?

T_Rex · Post by **T_Rex** » 2007-03-12 19:30:36

Blueprint wrote:I'm going to need a finished theater (64 size, center coordinates must be known) for testing the SRTM installer. Any suggestions?

How about stock Korea?

Only thing I'm not sure of is the center coordinates.

Otherwise, I'd suggest Taiwan or Israel.

Sherlock · Post by **Sherlock** » 2007-03-12 19:44:25

Blueprint wrote: Question to theater makers:
I can't remember, is there a 4-byte version of CATE?

Unfortunately, no.

ranger822 · Post by **ranger822** » 2007-03-12 20:23:33

I will second the motion for SRTM for Israeli Theater - that could be a great addition. Lead me to the promised land

Falklands and Taiwan would also be great

Malc · Post by **Malc** » 2007-03-13 08:04:30

Blueprint wrote: I'm going to need a finished theater (64 size, center coordinates must be known) for testing the SRTM installer. Any suggestions?

Hmmm... Panama would be worth a blast surely? Nice and small, etc etc etc 8)

i_hawk · Post by **i_hawk** » 2007-03-13 14:58:17

Sounds like amazing stuff, thanx to all involved

ranger822 · Post by **ranger822** » 2007-03-14 15:24:30

The opporutunity Israeli Theater SRTM seems more difficult than I had hoped. I opened the Theater.map file in Pathmaker thinking all I needed to do was add 5 degrees to the the SW coordinates for latitude and longitude recorded in the map file. However, upon inspection of the coordinates and doing a visual verification with GoogleEarth indicates the map coordinates for Israeli Theater are way way off.

Does anyone happen to have the original Theater.map file for Israeli Theater or can confirm that those coordinates do not match up with the actual terrain for Israeli Theater.

It seems a real shame not to have collected up the center coordinates for various theaters as a normal course of registering theaters in PMC. I.e. where each theater is described the SW and NE coordinates might be given to establish the terrain "BOX". Is there any merit in doing that now, if for no other reason than to see what theaters might be doable for Bluey's SRTM data workup. Maybe Snakeman will post up the center points for his theaters. I sure wouldn't mind seeing Falklands 128 in SRTM too. That is a theater that is in early stages where perhaps anything is possible.

Is is safe to assume that most if not all 64x64 theaters use a 10 degree box and 128x128 theaters use a 20 degree box? Is so then perhaps it should be possible to simply add 5 or 10 degrees of latitude and longitude to the values given in the Theater.map file for each theater?

I just hate the idea of letting Bluey off the hook for making some SRTM data

PMC Tactical forums

SRTM data conversion to Falcon 4 Terrain format

SRTM for Israeli Theater

Theater Coordinates