[osg-users] Refactoring DatabasePagerNeedToRemovestringflagging technique

Robert Osfield robert.osfield at gmail.com
Tue Nov 24 02:06:44 PST 2009


HI Wojtek,

On Mon, Nov 23, 2009 at 5:22 PM, Wojciech Lewandowski
<lewandowski at ai.com.pl> wrote:
> Our case is following: We mostly have to update camera position and some
> objects around camera. And ocassionaly we reposition a camera to new
> location in the world. We are doing our intersections in update traversal. I
> assume its allowed to use IntersectVisitor in update and load new tiles
> then. We load many objects in update traversal.

The crucial part you don't talk about is the performance constraints.
Can you pause a frame for hundreds of milliseseconds while the
required high res tiles are pulled in off disk?  Or are you trying to
stick with a solid 60Hz?  Loading any tiles in the update phase is not
possible if you want to keep to a solid 60Hz.


> And believe it or not but our scheme was working. Cache kept nodes bit
> longer than DatabasePager wanted, but not used tiles were eventually freed.
> I know it did work, because we made quick dirty fix by renaming
> "NeedToRemove" nodes to empty string name when fetching PagedLODs back to
> activePageLOD list from cache (described in Pawel Ksiezopolski post I
> mentioned earlier). Unfortuantely this fix was not appropriate as elegant
> submission. Sharing  loaded PageLOD tiles through cache was actually working
> as some preload for DatabasePager. DatabasePager still could load whatever
> it wanted.

I think something that might provide a "work around" such as replacing
the"NeedToRemove" is just a work around it's very unlikely to be a
solution of trying to use the DatabasePager beyond it's current design
assumptions.

>> Using a cache to prevent a subgraph from being deleted defeats the
>> load balancing that the DatabasePager will be attempting to do, so
>> it's a dangerous thing to do - it's a recipe for relentless growth in
>> memory usage.
>
> Well yes, but in our case PagedLOD usage timeframe should be extended to
> period when the tiles were used for intersections and cache maybe in bit
> hacky way provided this prolonged life time. And use of cache is not direct
> cause of leaks becaus osgDB cache is freed when node ref_count reaches zero
> so when DatabasePager & IntersectionVisitor stopped using the tile it was
> effectively removed from the cache as well.
> Memory leaks were the result of extra NeedToRemove nodes that appeared in
> activePageLOD lists when some nodes were resurected from Cache because
> camera moved over them again. I know its sophistry, but in some way, it was
> more effective than native DatabasePager management because it loaded nodes
> in no time while DatabasePager would have to load them from disk in this
> case ;-).

I can't understand why you want to do what you are trying to do, but
it still doesn't change the fact that DatabasePager hasn't been
designed with this in mind.  We can certainly look to take this into
account in another rev of DatabasePager.

As a general note a good OS will actually cache file accesses,
especially if you don't overload virtual memory/swap space too much so
the cost of reloading a tile might not be quite as expensive as you
think...  I know that when I benchmark under Linux I have the file
cache makes a huge difference, so I can to be careful about running
test apps in sequence.

> Well I thought that if PagedLOD class is public and offers public interface
> and methods we could use it as long as we do not hit compiler errors.
> Perhaps different set of classes should be created for use in more VPB
> specific closed way.

I don't think this is a VPB database specific issue.  The problem you
are trying to solve is really to do with trying to juggle user manage
caching with how the DatabasePager manages expiry.

When I get on to reviewing the multiple viewpoint issue with
DatabasePager I'll have a think about the consequences of users
caching subgraphs

Robert.


Robert.


More information about the osg-users mailing list