external: index
internal: preamble
build TG: git source simgear terragear folders downloads
build: scenery elevation airports land use construction viewing
end
2011-06-06: Construction ceased!
but remains under construction?
2011-06-06: The full completion of this page has been abandoned! Building TerraGear with 'multiple' dependencies will always be a nightmare ;=((. So, commenced a NEW build system, hopefully with 'less' dependencies, using perl scripts - see here, fgfs--054.htm - But, as usual, a lot of this page, and the information contained, remains very relevant...
2009-02-26: Wow, it has been nearly a year time since I tried a TerraGear (TG) build in WIN32 (circa March, 2008)! This build is using the git repository and MSVC8 (2005) in Windows XP. The main dependencies are PLIB and Simgear (SG), and even SG has to be the git repository, which has some changes which have not yet made it into SG CVS.
Additionally, this time I have Ubuntu linux on a dual boot, thus are able to compare the results of what is built, and created in there with this WIN32 project...
Since the TG suite of scenery building tools also depends on many of the dependants of FlightGear (FG), it makes sense to ONLY attempt a TG build AFTER a complete compile of FG! See fgfs-047.htm. In fact, I used the same base root WORK folder, c:\FG\27 in my case, for the build. However, as usual, I did the source download, called a 'clone' in git terminology, into my usual SOURCE C:\FGCVS folder, and x-copied that source to my WORK folder.
'Git' is the newest repository, or Source Control Management (SCM), tool on the block. The main page is - http://git-scm.com/ - where, in 'Download', a WIN32 native binary can be found at - http://code.google.com/p/msysgit/ - I downloaded the Git-1.6.1-preview20081227.exe, and during installation, allowed it to add itself to my PATH, so I could run it from the command line. I have read there is also a GUI version, but have never tried this.
Take care when using 'git' in a batch file, since the actual executable 'git.exe' is run from within a 'batch' file, git.cmd, so you have to use 'call git ...' in batch files if you want to do something after git is completed. Also you will often see 'use git-clone' - that is a hyphenated form of a command - in unix/linux environments. In WIN32 this has to be space separated, like 'git clone', etc...
The details for the source downloads, called a 'git clone', can be found at -
http://mapserver.flightgear.org/git/
but essentially it is -
[root folder]> git clone http://mapserver.flightgear.org/git/simgear-cs/
[root folder]> git clone http://mapserver.flightgear.org/git/terragear-cs/
This will create [root folder]\simgear-cs and [root folder]\terragear-cs. As stated, in my case I then copied these sources to my WORK folder. As stated, this was _AFTER_ a full compile of FG - see my fgfs-047.htm for this.
For SG-cs build, I used the SimGear.dsw, although there are VC7.1 and VC8 'solution' files available. Of course, after conversion, I had to adjust all the include and library directories relative to my WORK folder. After adjustment, I used my sln2dsw02.pl perl script tool to write a new SimGear.dsp file, included in the zip below, if you are using my folder structure, and like me use an installed OpenAL SDK.
Date | Zip | Size | MD5 |
---|---|---|---|
03/03/2009 | sg-msvc-01.zip | 4,056 | 8fb23401cc68596f3e45f7e751408863 |
Of course, like CVS SimGear, simgear-cs has all the same dependencies. That is why I compiled it in the same root folder where I have previously compiled SG and FG. As stated see my fgfs-047.htm for my full FG building, and all the prerequisites - see my prereq.htm for more details on where to get all these sources...
TerraGear is _NOT_ a single application!. It is a suite of command line tools for :-
So there are some 40 plus applications - some are just 'test' applications - to process the above. But this section is not about 'using' the TG tools, that is given below, but about building the TG tools in native win32 using msvc.
The 'cloned' source contains one(1) dsw and dsp file, and in the folder projects/VC7.1 one(1) 'sln', and then forty(40) 'vcproj' files, but I used none of these! I started a new folder terragear-cs/projects/msvc, and added one TGSuite.dsw, with some sixty eigth(68) 'dsp' files, plus other helpful things...
This 'msvc' zip below also contains the all important hand-crafted config.h. The two(2) additional static library sources - generate polygon clip (gpc), and new materials (newmat) - necessary for the building the the TG Suite - are available on the web, but I have includes a 'libs' zip below...
Further, until the online source git repository is updated, (if ever ;=)) then there is also a patch file, to add the modifications I made, to get the suite of tools built in native WIN32, using MSVC, and some minor, and other changes, usually under a _MSC_VER flag. Of course there were a few related to the path separator, '/' in unix, versus '\' in windows, and other things.
I used MSVC8 Express (2005), but it should work with any version of MSVC later than MSVC6 (1998) - I have found certain constructs in SG can NOT be compile by MSVC6, and assume earlier versions.
So it should be a simple matter, _AFTER_ you have -
(a) built FG, including all the prerequisites, especially PLIB and zlib - see my
fgfs-047.htm
(b) built simgear-cs - as detailed above,
(c) downloaded (cloned) the latest terragear-cs source,
then to :-
The result should be a set of EXE files in the terragear-cs/projects/msvc/bin folder, ready to run.
My WORK folder structure - as stated above, it would be normal to build FG before undertaking Terragear
Note the new sub-directories, created in projects. VC7.1 exists already, but I have put the additional static libraries, gpc and newmat into lib-srcs, a new utility source in win32, and the msvc build files in msvc...
Following downloads are available -
Date | Zip | Size | MD5 |
---|---|---|---|
01/03/2009 | tg-exes-01.zip | 1,720,346 | 15c6af2822646b1cba87ba0b86538910 |
01/03/2009 | tg-msvc-01.zip | 136,405 | 1dacba5ae28cf091d09798908590b343 |
17/02/2009 | tg-libs-01.zip | 307,436 | 83db94d9266462b92f35afd94874ad42 |
As always, take care downloading and running executables from the web! ;=((
To repeat, you must to 'patch' the current 'git clone' terragear-cs source with the patch file in tg-msvc-01.zip, or available separately here as tg-diff-01.txt. or tg-diff-01.patch - same thing in unix format.
For more DETAIL on each file changes, and why, visit fgfs-050a.htm, where each patch is separated file by file, with a brief explanation.
The WIN32 ported GNU 'patch' utility is available from - http://unxutils.sourceforge.net/
And although it is recommended to 'git clone' your own source from the respective repositories, so you will be getting the latest, which MAY already include some of the above patches, or other changes or updates, below are the complete sources I used, but the above patch still has to be applied :-
Date | Zip | Size | MD5 |
---|---|---|---|
19/02/2009 | sg-cs-src-01.zip | 5,315,338 | c7c0c9eb54b3ef1fa0da83c4ff443bf2 |
24/02/2009 | tg-cs-src-01.zip | 8,512,796 | 892265da4cbe44f844cdb5aa65ab85b6 |
But to repeat, it is easier and better to install 'git', clone the latest from the repository, and check if all of my patch is still required...
For a test, if you have all the tg-cs exes, and other tools available, there is a terragear-cs/projects/msvc/test-scenery.bat file, to download and build a small section of Australian scenery - part of chunk e150s40 - a 'chunk' is the label of the lower left corner of a 10x10 degree block of the world. This batch file uses several tools from a GNU WIN32 port of various unix tools, from - http://unxutils.sourceforge.net/ - like gzip, and tar...
Started: Friday, February 20, 2009 - Ended: Sunday, March 1, 2009
TerraGear-cs - Processing CHUNK e150s40 ... Sydney, NSW, Australia. A 'CHUNK' is the lower left corner of a 10x10 degree block...
So I would NOT have to particularly install the TerraGear tools into a PATH, I chose to do all this near where the exes were created - namely in <some root>/terrragear-cs/projects/msvc/test folder
In this folder I created a 'data' and 'work' sub-directories, to be populated with files...
1. Elevation Data
2. Airport Data
3. Land Use Data
4. Final Scenery Construction
5. Viewing Results with FG
The Shuttle Radar Topography Mission (SRTM) data sets result from a collaborative effort by NASA and the National Geospatial-Intelligence Agency (NGA - previously known as the National Imagery and Mapping Agency, or NIMA), and others.
(a) From :
ftp://e0srp01u.ecs.nasa.gov/srtm/version2/SRTM3/Australia/
downloaded the following set of files - of course each is of the form .hgt.zip
OZSET="S31E150 S31E151 S31E152 S31E153 S32E150 S32E151 S32E152 S32E153 S32E159 S33E150 S33E151
S33E152 S34E150 S34E151 S35E150 S35E151 S36E150 S37E150 S38E150"
These were downloaded in <some root>/terrragear-cs/bin/data/SRTM-3. Then ran my 'unzip' tool in this folder to extract the HGT file from each.
(b) Chop up elevation data
Through the use of a batch file, run in <some root>/terrragear-cs/projects/msvc, I ran
'hgtchop' on each of the .hgt files, and output the results into <some
root>/terrragear-cs/projects/msvc/test/work/SRTM-30
using the general command line to hgtchop
test> ..\bin\hgtchop 3 <in_hgt_file_name> work\SRTM=30
I was rewarded with some 403 files, of the form -
...\e150s40\e150s31\5410496.arr.gz
... through to ...
...\e150s40\e159s32\5557920.arr.gz
BUT, this missed e150s40, that is there is no S40E150.tar.gz on the site, and perhaps others, for the Australian region I wanted!
Abandoned this source!
(a) This is the Global Land One-km Base Elevation Project (GLOBE)
from :
http://www.ngdc.noaa.gov/mgg/topo/gltiles.html
For Australia, it is TILE L (0-50S, 90-180E)
Download :
http://www.ngdc.noaa.gov/mgg/topo/DATATILES/elev/l10g.gz
to, and unpack -
<some root>\terragear-cs\projects\msvc\test\work\DEM2
to get -
18/09/1998 17:44 129,600,000 l10g
(b) Since this file has no DEM header, you either need to create one, or modify raw2ascii.exe
to handle.
from :
http://www.ngdc.noaa.gov/mgg/topo/report/s11/s11D.html
The data are stored in row major order (all the data for row 1, followed by all the data for
row 2, etc.). All files have 10800 columns, and either 4800 or 6000 rows ...
from :
http://www.ngdc.noaa.gov/mgg/topo/report/s11/s11C.html#table3
...
L10G -50 0 90 180 1 5179 10800 6000
...
So this allows for the construction of a HEADER file, something like -
BYTEORDER M LAYOUT BIL NROWS 6000 NCOLS 10800 NBANDS 1 NBITS 16 BANDROWBYTES 9600 TOTALROWBYTES 9600 BANDGAPBYTES 0 NODATA -9999 ULXMAP 90 ULYMAP 0 XDIM 0.00833333333333 YDIM 0.00833333333333
But will this work? No, the current raw2ascii.exe has FIXED (hard coded) rows and column values - 6000 x 7200 ... so would need to either cut this raw-DEM into the appropriate sizes, or use a different raw2ascii ;=()
For now this also abandoned!!!
(a) From :
http://edcwww.cr.usgs.gov/products/elevation/gtopo30/gtopo30.html
or Mirror :
http://www1.gsi.go.jp/geowww/globalmap-gsi/gtopo30/gtopo30.html
download :
http://edcftp.cr.usgs.gov/pub/data/gtopo30/global/e140s10.tar.gz
(b) Using tar/gzip, or alternatives, unpack to -
<some root>\terragear-cs\bin\work\DEM2 -
and get list -
22/01/1997 00:23 57,600,000 E140S10.DEM 22/01/1997 00:23 222 E140S10.DMW 22/01/1997 19:00 35,870 E140S10.GIF 23/01/1997 16:53 328 E140S10.HDR 22/01/1997 00:23 648 E140S10.PRJ 23/01/1997 16:53 327 E140S10.SCH 22/01/1997 00:20 28,800,000 E140S10.SRC 22/01/1997 00:23 28 E140S10.STX
These files represent -
Ext Contents DEM digital elevation model data HDR header file for DEM DMW world file STX statistics file PRJ projection information file GIF shaded relief image SRC source map SCH header file for source map
(c) Run raw2asci.exe to convert this DEM file into a 'staged' ASCII DEM set.
> bin\raw2ascii test\data\DEM2\E140S10 test\staged\DEM2
This produced 2000 files (213MB) of the form -
21/02/2009 14:29 106,543 e140s11.dem ... 21/02/2009 14:25 106,543 e140s60.dem 21/02/2009 14:29 106,543 e141s11.dem ... etc - until - 21/02/2009 14:26 106,543 e179s60.dem
Since I am ONLY interested in the chunk e150s40 (10x10 - from lower left), then I only want a
sub-set of these...
e150s31 - e159s31
e150s40 - e159s40
And actually it is less than that, since a lot of that is WATER!
The full block I download, per its own GIF file (600x750) is -
e140s11 | s179s11 |
e140s60 | e179s60 |
So 600 pixels wide = 40 degrees, and 750 pixels high = 49 degrees, so
I want 10 degrees in = 150, and 20 degrees from top (or bottom) = 306, and the final image to
be 10x10 = 150x150
> convert E140S10.GIF -crop 150x150+150+306 crop.gif
And just the approximate LAND region from Google Earth -
e150s30 | e154s30 |
e150s37 | e154s37 |
As an enhancement, it would be nice is raw2ascii.exe ONLY output the desired region, rather than simply processing the whole DEM file!!! Due the fact that I only want a small sub-set, decided to implement this RFC, rather crudely, but effectively. The patch does not take account that perhaps SOME of the strip being written may be within the min/max lat/lon bounds given, AND all the 'work' is still done before checking if the file should be written...
So this command (as one line) -
test> ..\bin\raw2ascii --min-lat=-37 --max-lat=-30 --min-lon=150 --max-lon=154 \ data\DEM2\E140S10 staged\DEM3
only wrote 40 DEM (ascii) files, 4.2MB, instead of the 2000, 213MB, when the whole raw DEM is processed. An 80% reduction in file count - a 98% reduction in disk usage - a GOOD job ;=))
Processing this DEM data
For this, demchop.exe is used -
test> ..\bin\demchop staged\DEM3\<each dem file> work\DEM-USGS3 ..\bin\demchop.exe data\DEMASCII\e150s30.dem work\SRTM-3 ..\bin\demchop.exe data\DEMASCII\e150s31.dem work\SRTM-3 ... etc ... through to ... ..\bin\demchop.exe data\DEMASCII\e154s36.dem work\SRTM-3 ..\bin\demchop.exe data\DEMASCII\e154s37.dem work\SRTM-3
Generate the fit.gz files.
Hmmm, this presents no choice in Win32, unless you have python installed. With python there is
a terrafit.py to build the fit.gz files from the arr.gz files, one of one. But in without there
is only terrafit.exe, which I hope does the same job.
test>..\bin\terrafit.exe work\SRTM-3
*TBD*
test>..\bin\genapts.exe --input=data\apt.dat --work=work --min-lon=150 \ --max-lon=154 --min-lat=-37 --max-lat=-30
*TBD*
test>cd work ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=LandMass \ --material=Default c:\FG\data\vmap0\v0sas\vmaplv0 sasaus bnd polbnda ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Cities \ --material=Urban c:\FG\data\vmap0\v0sas\vmaplv0 sasaus pop builtupa ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Towns \ --material=Town c:\FG\data\vmap0\v0sas\vmaplv0 sasaus pop mispopp ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Crops \ --material=MixedCropPastureCover c:\FG\data\vmap0\v0sas\vmaplv0 sasaus veg cropa ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Forest \ --material=MixedForestCover c:\FG\data\vmap0\v0sas\vmaplv0 sasaus veg treesa ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Freeways --material=Freeway --width=50m --att=med:1 c:\FG\data\vmap0\v0sas\vmaplv0 sasaus trans roadl ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Roads \ --material=Road --width=20m --att=!med:1 c:\FG\data\vmap0\v0sas\vmaplv0 sasaus trans roadl ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Railroads \ --material=Railroad --width=10m c:\FG\data\vmap0\v0sas\vmaplv0 sasaus trans railrdl ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=Lakes \ --material=Lake c:\FG\data\vmap0\v0sas\vmaplv0 sasaus hydro inwatera ..\bin\tgvpf.exe --min-lon=150 --max-lon=154 --min-lat=-37 --max-lat=-30 --work-dir=PolarIce \ --material=Glacier --att=f_code:BJ080 c:\FG\data\vmap0\v0sas\vmaplv0 sasaus phys seaicea
Done LIST...
LandMass Cities Towns Crops Forest Freeways Roads Railroads Lakes PolarIce Rivers
There are the tools fgfs-server.exe and fgfs-client.exe which will process whole blocks, but in actual fact the 'client' uses fgfs-construct.exe for the final stage, so I just directly used this 'construction' tool.
First, FG expects the scenery to be in a directory called 'Terrain', so... also the lon and lat should be the center of the area to be done, since fgfs-construct will work +/- xdist ydist ...
test>..\bin\fgfs-construct.exe --work-dir=work --output-dir=Scenery\Terrain \ --lon=154 --lat=-30 --xdist=4 --ydist=7 \ AirportArea SRTM-3 AirportObj LandMass Cities Towns Crops Forest \ Freeways Roads Railroads Lakes PolarIce Rivers
*TBD*
To 'test' my new scenery, I ran - of course the '\' indicates it has to be all one line -
@set FG_SCENERY=C:\FG\27\terragear-cs\projects\msvc\test\Scenery fgfs --fg-root=..\data --airport=YSSY --fog-disable --disable-clouds --timeofday=noon \ --fdm=ufo --aircraft=ufo --altitude=40000 --prop:/sim/rendering/fps-display=true \ --prop:/environment/clouds/layer/coverage=clear --heading=0
and to my pleasant surprise -
My First View
This is from 40,000 feet, in the UFO. Very FLAT - maybe my 'elevation' stage somehow failed - but I essentially got the airport, YSSY, the harbor, the 'city', more or less, and some minor roads, and rivers - I thinks - NOT BAD FOR A FIRST TRY ;=))
The FLATNESS can be seen better in these two clipped images - the first is from Scenery-1.0.1, and then mine -
Could this be the use of terrafit.exe instead of terrfit.py??? The difference is the python wrote some 'pgm' file first, then used terra.exe, to write an .obj, then converted the 'obj' to fit.gz. I will have to take a careful look at my 'elevation' stage (when I get a chance) again. Either the information is not there, not in the right place, the right folder, format, or something... ;=(( And this was part of the reason for choosing this location. I had noted some brown shear cliffs, which I thought were quite realistic, but alas I have none of these, YET!
And maybe 'soon' I can compare it with my ubuntu parallel build... ;=))
But that's it for now...
2011-06-06: Construction ceased!
but remains under construction?
Geoff R. McLane
February 28, 2009
EOP - fgfs-050.htm