TerraGear-cs Build

external: index
internal: preamble
build TG: git source simgear terragear folders downloads
build: scenery elevation airports land use construction viewing
end

2011-06-06: Construction ceased!
Under heavy construction
but remains under construction?

Preamble:

2011-06-06: The full completion of this page has been abandoned! Building TerraGear with 'multiple' dependencies will always be a nightmare ;=((. So, commenced a NEW build system, hopefully with 'less' dependencies, using perl scripts - see here, fgfs--054.htm - But, as usual, a lot of this page, and the information contained, remains very relevant...

2009-02-26: Wow, it has been nearly a year time since I tried a TerraGear (TG) build in WIN32 (circa March, 2008)! This build is using the git repository and MSVC8 (2005) in Windows XP. The main dependencies are PLIB and Simgear (SG), and even SG has to be the git repository, which has some changes which have not yet made it into SG CVS.

Additionally, this time I have Ubuntu linux on a dual boot, thus are able to compare the results of what is built, and created in there with this WIN32 project...

Since the TG suite of scenery building tools also depends on many of the dependants of FlightGear (FG), it makes sense to ONLY attempt a TG build AFTER a complete compile of FG! See fgfs-047.htm. In fact, I used the same base root WORK folder, c:\FG\27 in my case, for the build. However, as usual, I did the source download, called a 'clone' in git terminology, into my usual SOURCE C:\FGCVS folder, and x-copied that source to my WORK folder.

top


Git Tool

'Git' is the newest repository, or Source Control Management (SCM), tool on the block. The main page is - http://git-scm.com/ - where, in 'Download', a WIN32 native binary can be found at - http://code.google.com/p/msysgit/ - I downloaded the Git-1.6.1-preview20081227.exe, and during installation, allowed it to add itself to my PATH, so I could run it from the command line. I have read there is also a GUI version, but have never tried this.

Take care when using 'git' in a batch file, since the actual executable 'git.exe' is run from within a 'batch' file, git.cmd, so you have to use 'call git ...' in batch files if you want to do something after git is completed. Also you will often see 'use git-clone' - that is a hyphenated form of a command - in unix/linux environments. In WIN32 this has to be space separated, like 'git clone', etc...

top


Source Downloads

The details for the source downloads, called a 'git clone', can be found at -
 http://mapserver.flightgear.org/git/
but essentially it is -

[root folder]> git clone http://mapserver.flightgear.org/git/simgear-cs/
[root folder]> git clone http://mapserver.flightgear.org/git/terragear-cs/

This will create [root folder]\simgear-cs and [root folder]\terragear-cs. As stated, in my case I then copied these sources to my WORK folder. As stated, this was _AFTER_ a full compile of FG - see my fgfs-047.htm for this.

top


Building SimGear

For SG-cs build, I used the SimGear.dsw, although there are VC7.1 and VC8 'solution' files available. Of course, after conversion, I had to adjust all the include and library directories relative to my WORK folder. After adjustment, I used my sln2dsw02.pl perl script tool to write a new SimGear.dsp file, included in the zip below, if you are using my folder structure, and like me use an installed OpenAL SDK.

Date Zip Size MD5
03/03/2009 sg-msvc-01.zip 4,056 8fb23401cc68596f3e45f7e751408863

Of course, like CVS SimGear, simgear-cs has all the same dependencies. That is why I compiled it in the same root folder where I have previously compiled SG and FG. As stated see my fgfs-047.htm for my full FG building, and all the prerequisites - see my prereq.htm for more details on where to get all these sources...

top


Building TerraGear Suite of Tools

TerraGear is _NOT_ a single application!. It is a suite of command line tools for :-

So there are some 40 plus applications - some are just 'test' applications - to process the above. But this section is not about 'using' the TG tools, that is given below, but about building the TG tools in native win32 using msvc.

The 'cloned' source contains one(1) dsw and dsp file, and in the folder projects/VC7.1 one(1) 'sln', and then forty(40) 'vcproj' files, but I used none of these! I started a new folder terragear-cs/projects/msvc, and added one TGSuite.dsw, with some sixty eigth(68) 'dsp' files, plus other helpful things...

This 'msvc' zip below also contains the all important hand-crafted config.h. The two(2) additional static library sources - generate polygon clip (gpc), and new materials (newmat) - necessary for the building the the TG Suite - are available on the web, but I have includes a 'libs' zip below...

Further, until the online source git repository is updated, (if ever ;=)) then there is also a patch file, to add the modifications I made, to get the suite of tools built in native WIN32, using MSVC, and some minor, and other changes, usually under a _MSC_VER flag. Of course there were a few related to the path separator, '/' in unix, versus '\' in windows, and other things.

I used MSVC8 Express (2005), but it should work with any version of MSVC later than MSVC6 (1998) - I have found certain constructs in SG can NOT be compile by MSVC6, and assume earlier versions.

So it should be a simple matter, _AFTER_ you have -
(a) built FG, including all the prerequisites, especially PLIB and zlib - see my fgfs-047.htm
(b) built simgear-cs - as detailed above,
(c) downloaded (cloned) the latest terragear-cs source,
then to :-

The result should be a set of EXE files in the terragear-cs/projects/msvc/bin folder, ready to run.

top


Folder Structure

My WORK folder structure - as stated above, it would be normal to build FG before undertaking Terragear

work folders for terragear-cs

Note the new sub-directories, created in projects. VC7.1 exists already, but I have put the additional static libraries, gpc and newmat into lib-srcs, a new utility source in win32, and the msvc build files in msvc...

top


Downloads:

Following downloads are available -

Date Zip Size MD5
01/03/2009 tg-exes-01.zip 1,720,346 15c6af2822646b1cba87ba0b86538910
01/03/2009 tg-msvc-01.zip 136,405 1dacba5ae28cf091d09798908590b343
17/02/2009 tg-libs-01.zip 307,436 83db94d9266462b92f35afd94874ad42

As always, take care downloading and running executables from the web! ;=((

To repeat, you must to 'patch' the current 'git clone' terragear-cs source with the patch file in tg-msvc-01.zip, or available separately here as tg-diff-01.txt. or tg-diff-01.patch - same thing in unix format.

For more DETAIL on each file changes, and why, visit fgfs-050a.htm, where each patch is separated file by file, with a brief explanation.

The WIN32 ported GNU 'patch' utility is available from - http://unxutils.sourceforge.net/

And although it is recommended to 'git clone' your own source from the respective repositories, so you will be getting the latest, which MAY already include some of the above patches, or other changes or updates, below are the complete sources I used, but the above patch still has to be applied :-

Date Zip Size MD5
19/02/2009 sg-cs-src-01.zip 5,315,338 c7c0c9eb54b3ef1fa0da83c4ff443bf2
24/02/2009 tg-cs-src-01.zip 8,512,796 892265da4cbe44f844cdb5aa65ab85b6

But to repeat, it is easier and better to install 'git', clone the latest from the repository, and check if all of my patch is still required...

top


Building Scenery

For a test, if you have all the tg-cs exes, and other tools available, there is a terragear-cs/projects/msvc/test-scenery.bat file, to download and build a small section of Australian scenery - part of chunk e150s40 - a 'chunk' is the label of the lower left corner of a 10x10 degree block of the world. This batch file uses several tools from a GNU WIN32 port of various unix tools, from - http://unxutils.sourceforge.net/ - like gzip, and tar...

Started: Friday, February 20, 2009 - Ended: Sunday, March 1, 2009

TerraGear-cs - Processing CHUNK e150s40 ... Sydney, NSW, Australia. A 'CHUNK' is the lower left corner of a 10x10 degree block...

So I would NOT have to particularly install the TerraGear tools into a PATH, I chose to do all this near where the exes were created - namely in <some root>/terrragear-cs/projects/msvc/test folder

In this folder I created a 'data' and 'work' sub-directories, to be populated with files...
1. Elevation Data
2. Airport Data
3. Land Use Data
4. Final Scenery Construction
5. Viewing Results with FG

1. Elevation Data - Choosing a source of this data... A., B., and eventually C.

A. National Aeronautics and Space Administration (NASA) - SRTM

The Shuttle Radar Topography Mission (SRTM) data sets result from a collaborative effort by NASA and the National Geospatial-Intelligence Agency (NGA - previously known as the National Imagery and Mapping Agency, or NIMA), and others.

(a) From : ftp://e0srp01u.ecs.nasa.gov/srtm/version2/SRTM3/Australia/ downloaded the following set of files - of course each is of the form .hgt.zip
OZSET="S31E150 S31E151 S31E152 S31E153 S32E150 S32E151 S32E152 S32E153 S32E159 S33E150 S33E151 S33E152 S34E150 S34E151 S35E150 S35E151 S36E150 S37E150 S38E150"

These were downloaded in <some root>/terrragear-cs/bin/data/SRTM-3. Then ran my 'unzip' tool in this folder to extract the HGT file from each.

(b) Chop up elevation data
Through the use of a batch file, run in <some root>/terrragear-cs/projects/msvc, I ran 'hgtchop' on each of the .hgt files, and output the results into <some root>/terrragear-cs/projects/msvc/test/work/SRTM-30
using the general command line to hgtchop

test> ..\bin\hgtchop 3 <in_hgt_file_name> work\SRTM=30

I was rewarded with some 403 files, of the form -
...\e150s40\e150s31\5410496.arr.gz
... through to ...
...\e150s40\e159s32\5557920.arr.gz

BUT, this missed e150s40, that is there is no S40E150.tar.gz on the site, and perhaps others, for the Australian region I wanted!

Abandoned this source!

top


B. National Geophysical Data Center (NGDC) - DEM

(a) This is the Global Land One-km Base Elevation Project (GLOBE)
from : http://www.ngdc.noaa.gov/mgg/topo/gltiles.html
For Australia, it is TILE L (0-50S, 90-180E)
Download :
http://www.ngdc.noaa.gov/mgg/topo/DATATILES/elev/l10g.gz
to, and unpack -
<some root>\terragear-cs\projects\msvc\test\work\DEM2
to get -
18/09/1998 17:44 129,600,000 l10g

(b) Since this file has no DEM header, you either need to create one, or modify raw2ascii.exe to handle.
from : http://www.ngdc.noaa.gov/mgg/topo/report/s11/s11D.html
The data are stored in row major order (all the data for row 1, followed by all the data for row 2, etc.). All files have 10800 columns, and either 4800 or 6000 rows ...
from : http://www.ngdc.noaa.gov/mgg/topo/report/s11/s11C.html#table3 ...
L10G -50 0 90 180 1 5179 10800 6000
...
So this allows for the construction of a HEADER file, something like -

BYTEORDER      M
LAYOUT       BIL
NROWS         6000
NCOLS         10800
NBANDS        1
NBITS         16
BANDROWBYTES         9600
TOTALROWBYTES        9600
BANDGAPBYTES         0
NODATA        -9999
ULXMAP        90
ULYMAP        0
XDIM          0.00833333333333
YDIM          0.00833333333333

But will this work? No, the current raw2ascii.exe has FIXED (hard coded) rows and column values - 6000 x 7200 ... so would need to either cut this raw-DEM into the appropriate sizes, or use a different raw2ascii ;=()

For now this also abandoned!!!

top


C. U.S. Geological Survey (USGS)

(a) From : http://edcwww.cr.usgs.gov/products/elevation/gtopo30/gtopo30.html
or Mirror : http://www1.gsi.go.jp/geowww/globalmap-gsi/gtopo30/gtopo30.html
download : http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/e140s10.tar.gz

(b) Using tar/gzip, or alternatives, unpack to -
<some root>\terragear-cs\bin\work\DEM2 -
and get list -

22/01/1997  00:23        57,600,000 E140S10.DEM
22/01/1997  00:23               222 E140S10.DMW
22/01/1997  19:00            35,870 E140S10.GIF
23/01/1997  16:53               328 E140S10.HDR
22/01/1997  00:23               648 E140S10.PRJ
23/01/1997  16:53               327 E140S10.SCH
22/01/1997  00:20        28,800,000 E140S10.SRC
22/01/1997  00:23                28 E140S10.STX

These files represent -

Ext Contents
DEM digital elevation model data
HDR header file for DEM
DMW world file
STX statistics file
PRJ projection information file
GIF shaded relief image
SRC source map
SCH header file for source map

(c) Run raw2asci.exe to convert this DEM file into a 'staged' ASCII DEM set.

> bin\raw2ascii test\data\DEM2\E140S10 test\staged\DEM2

This produced 2000 files (213MB) of the form -

21/02/2009  14:29           106,543 e140s11.dem
...
21/02/2009  14:25           106,543 e140s60.dem
21/02/2009  14:29           106,543 e141s11.dem
... etc - until -
21/02/2009  14:26           106,543 e179s60.dem

Since I am ONLY interested in the chunk e150s40 (10x10 - from lower left), then I only want a sub-set of these...
e150s31 - e159s31
e150s40 - e159s40

And actually it is less than that, since a lot of that is WATER!
The full block I download, per its own GIF file (600x750) is -

e140s11 s179s11
original gif image from e140s10.tar.gz data
e140s60 e179s60

So 600 pixels wide = 40 degrees, and 750 pixels high = 49 degrees, so
I want 10 degrees in = 150, and 20 degrees from top (or bottom) = 306, and the final image to be 10x10 = 150x150
> convert E140S10.GIF -crop 150x150+150+306 crop.gif

clipped image

And just the approximate LAND region from Google Earth -

e150s30 e154s30
image from google earth
e150s37 e154s37

As an enhancement, it would be nice is raw2ascii.exe ONLY output the desired region, rather than simply processing the whole DEM file!!! Due the fact that I only want a small sub-set, decided to implement this RFC, rather crudely, but effectively. The patch does not take account that perhaps SOME of the strip being written may be within the min/max lat/lon bounds given, AND all the 'work' is still done before checking if the file should be written...

So this command (as one line) -

test> ..\bin\raw2ascii --min-lat=-37 --max-lat=-30 --min-lon=150 --max-lon=154 \
data\DEM2\E140S10 staged\DEM3

only wrote 40 DEM (ascii) files, 4.2MB, instead of the 2000, 213MB, when the whole raw DEM is processed. An 80% reduction in file count - a 98% reduction in disk usage -  a GOOD job ;=))

Processing this DEM data
For this, demchop.exe is used -

test> ..\bin\demchop staged\DEM3\<each dem file> work\DEM-USGS3
..\bin\demchop.exe data\DEMASCII\e150s30.dem work\SRTM-3
..\bin\demchop.exe data\DEMASCII\e150s31.dem work\SRTM-3
... etc ... through to ...
..\bin\demchop.exe data\DEMASCII\e154s36.dem work\SRTM-3
..\bin\demchop.exe data\DEMASCII\e154s37.dem work\SRTM-3

Generate the fit.gz files.
Hmmm, this presents no choice in Win32, unless you have python installed. With python there is a terrafit.py to build the fit.gz files from the arr.gz files, one of one. But in without there is only terrafit.exe, which I hope does the same job.

test>..\bin\terrafit.exe work\SRTM-3

top


2. Airport Data

*TBD*

test>..\bin\genapts.exe --input=data\apt.dat --work=work --min-lon=150 \
   --max-lon=154 --min-lat=-37 --max-lat=-30

top


3. Land Use Data

*TBD*

test>cd work
..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=LandMass \
 --material=Default c:\FG\data\vmap0\v0sas\vmaplv0 sasaus bnd polbnda
 ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Cities \
 --material=Urban c:\FG\data\vmap0\v0sas\vmaplv0 sasaus pop builtupa
..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Towns \
 --material=Town c:\FG\data\vmap0\v0sas\vmaplv0 sasaus pop mispopp
..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Crops \
 --material=MixedCropPastureCover c:\FG\data\vmap0\v0sas\vmaplv0 sasaus veg cropa
..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Forest \
 --material=MixedForestCover c:\FG\data\vmap0\v0sas\vmaplv0 sasaus veg treesa
..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Freeways 
 --material=Freeway --width=50m --att=med:1 c:\FG\data\vmap0\v0sas\vmaplv0 sasaus trans roadl
..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Roads \
 --material=Road --width=20m --att=!med:1 c:\FG\data\vmap0\v0sas\vmaplv0 sasaus trans roadl
..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Railroads \
 --material=Railroad --width=10m c:\FG\data\vmap0\v0sas\vmaplv0 sasaus trans railrdl
..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Lakes \
 --material=Lake c:\FG\data\vmap0\v0sas\vmaplv0 sasaus hydro inwatera
..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=PolarIce \
--material=Glacier --att=f_code:BJ080 c:\FG\data\vmap0\v0sas\vmaplv0 sasaus phys seaicea

Done LIST...
LandMass Cities Towns Crops Forest Freeways Roads Railroads Lakes PolarIce Rivers

top


4. Final Scenery Construction

There are the tools fgfs-server.exe and fgfs-client.exe which will process whole blocks, but in actual fact the 'client' uses fgfs-construct.exe for the final stage, so I just directly used this 'construction' tool.

First, FG expects the scenery to be in a directory called 'Terrain', so... also the lon and lat should be the center of the area to be done, since fgfs-construct will work +/- xdist ydist ...

test>..\bin\fgfs-construct.exe --work-dir=work --output-dir=Scenery\Terrain \
   --lon=154 --lat=-30 --xdist=4 --ydist=7 \
   AirportArea SRTM-3 AirportObj LandMass Cities Towns Crops Forest \
   Freeways Roads Railroads Lakes PolarIce Rivers

*TBD*

top


5. Setting it up for viewing

To 'test' my new scenery, I ran - of course the '\' indicates it has to be all one line -

@set FG_SCENERY=C:\FG\27\terragear-cs\projects\msvc\test\Scenery
fgfs --fg-root=..\data --airport=YSSY --fog-disable --disable-clouds --timeofday=noon \
   --fdm=ufo --aircraft=ufo --altitude=40000 --prop:/sim/rendering/fps-display=true \ 
   --prop:/environment/clouds/layer/coverage=clear --heading=0

and to my pleasant surprise -

My First View
image04

This is from 40,000 feet, in the UFO. Very FLAT - maybe my 'elevation' stage somehow failed - but I essentially got the airport, YSSY, the harbor, the 'city', more or less, and some minor roads, and rivers - I thinks - NOT BAD FOR A FIRST TRY ;=))

The FLATNESS can be seen better in these two clipped images - the first is from Scenery-1.0.1, and then mine -

from scenery-1.0.1 in ufo at --altitude=500 --heading=270 --lon=151.2927 --lat=-33.8693
from my scenery in ufo at --altitude=500 --heading=270 --lon=151.2927 --lat=-33.8693

Could this be the use of terrafit.exe instead of terrfit.py??? The difference is the python wrote some 'pgm' file first, then used terra.exe, to write an .obj, then converted the 'obj' to fit.gz. I will have to take a careful look at my 'elevation' stage (when I get a chance) again. Either the information is not there, not in the right place, the right folder, format, or something... ;=(( And this was part of the reason for choosing this location. I had noted some brown shear cliffs, which I thought were quite realistic, but alas I have none of these, YET!

And maybe 'soon' I can compare it with my ubuntu parallel build... ;=))

top


But that's it for now...

2011-06-06: Construction ceased!
Under heavy construction
but remains under construction?

Geoff R. McLane
February 28, 2009
EOP - fgfs-050.htm

checked by tidy  Valid HTML 4.01 Transitional