[osg-users] [VPB] Getting started with parallel builds

[Glenn Waldron]

Robert, this is exactly what I needed. This will give me something to chew
on for a while ;)

Much thanks - Glenn

On Wed, Mar 26, 2008 at 1:32 PM, Robert Osfield <robert.osfield at gmail.com>
wrote:

> Hi Glenn,
>
> The most simple example would be (based on yours):
>
>   osgdem --terrain -t imagery.tif -o out.ive
>
> Becomes:
>
>   vpbmaster --terrain -t imagery.tif -o output/out.ive
>
> If you have a multi-core machine It'll automatically divide the job up
> into tasks and then run them on all the cores you have available.
>
> The above example takes advantage of the fact that vpbmaster uses the
> same command line parser as osgdem, it also adds its own options too,
> but those are for another round of testing.
>
> Note the change of -o out.ive to -o output/out.ive is just done so
> that the generated database is self contained in its own directory.
> VPB will create the required directories if they aren't already there.
>
> The vpbmaster run will generate a build_master.tasks and
> build_master.source files at the start - this is effect the recipe for
> the build, and also generate logs and tasks subdirectories contains
> task files and log files.  I'd recommend spending some time having a
> look at all the various files it generates as it'll give you a picture
> of all the things that is it doing during a build.
>
> As an appetizer, at the end of last week VPB completed its first
> terrabyte build, I'm not sure of the latest build size but it was well
> of 3TB whole earth model.  It took a week to build, running over a
> network of 8 quad core machines.  Previously osgdem on its own would
> struggle to get over a 10GB database, with some users leaving osgdem
> for months in vain hope that it couldn't crash and burn.   With
> vpbmaster now anyone with the data can now give Google Earth a good
> run for it's money.
>
> Robert.
>
> On Wed, Mar 26, 2008 at 4:35 PM, Glenn Waldron <gwaldron at gmail.com> wrote:
> > Robert et al,
> >
> > I am finally getting around to experimenting with the new VPB, and so
> far so
> > good. Now I would like to try the new parallel build features and I'm
> > looking for the simplest possible example. I've searched the archives
> and
> > perused the source but I could still use a quick jump start.
> >
> > Given a simple build command like this:
> >
> >   osgdem --terrain -t imagery.tif -o out.ive
> >
> > How would I split that into a parallel build, say with 2 processes each
> > building half the database. Not looking to do multiple machines or
> anything
> > fancy just yet.
> >
> > Thanks for the tips!
> > Glenn
> >
> > --
> > Glenn Waldron : Pelican Mapping : http://pelicanmapping.com :
> 703-652-4791
> > _______________________________________________
> >  osg-users mailing list
> >  osg-users at lists.openscenegraph.org
> >
> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
> >
> >
> _______________________________________________
> osg-users mailing list
> osg-users at lists.openscenegraph.org
> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
>



-- 
Glenn Waldron : Pelican Mapping : http://pelicanmapping.com : 703-652-4791
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openscenegraph.org/pipermail/osg-users-openscenegraph.org/attachments/20080326/d0645666/attachment.htm