[osg-users] Refactoring DatabasePagerNeedToRemovestringflagging technique
lewandowski at ai.com.pl
Mon Nov 23 09:22:11 PST 2009
>> How should we tackle it, if current
>> approach is wrong ?
> If you do want to mix the two then you need to ask questions about how
> you want to do intersections and how your thread them. Do you do
> intersections in the main loop? In a separate thread? Do you run the
> intersections multi-threaded? I.e. multiple intersections traversals
> at one time? Should the intersection traversal wait to load external
> tiles to get the result, or should then just return the intersections
> for what is already loaded into memory.
> What approach you will want to use will depend upon all of this.
Our case is following: We mostly have to update camera position and some
objects around camera. And ocassionaly we reposition a camera to new
location in the world. We are doing our intersections in update traversal. I
assume its allowed to use IntersectVisitor in update and load new tiles
then. We load many objects in update traversal.
And believe it or not but our scheme was working. Cache kept nodes bit
longer than DatabasePager wanted, but not used tiles were eventually freed.
I know it did work, because we made quick dirty fix by renaming
"NeedToRemove" nodes to empty string name when fetching PagedLODs back to
activePageLOD list from cache (described in Pawel Ksiezopolski post I
mentioned earlier). Unfortuantely this fix was not appropriate as elegant
submission. Sharing loaded PageLOD tiles through cache was actually working
as some preload for DatabasePager. DatabasePager still could load whatever
>> Second important issue for me is usage of _name in scene graph. I always
>> expected that _name is reserved for users and its a normal rule in all
>> Graph implementations that libraries do not change it. Names are ususlly
>> used to identify certain portions of models and hook up the code
>> Thats something that provide standard linking mechanisms between artists
>> programmers works.
> I agree, but... in this instance the DatabasePager's algorithm was
> about deleting a subgraph that would no longer have any role to play
> in the applications life so the changing of name should never have got
> outside that algorithm as the subgraph would be just deleted. So it
> is in theory just a black box, how it does it's job shouldn't effect
> anything else. Alas in this case it looks like the algorithm in
> DatabasePager is flawed.
>> Please also note that we use cache becuse we otherwise were loading
>> files twice. Is it reasonable ?.
> Using a cache to prevent a subgraph from being deleted defeats the
> load balancing that the DatabasePager will be attempting to do, so
> it's a dangerous thing to do - it's a recipe for relentless growth in
> memory usage.
Well yes, but in our case PagedLOD usage timeframe should be extended to
period when the tiles were used for intersections and cache maybe in bit
hacky way provided this prolonged life time. And use of cache is not direct
cause of leaks becaus osgDB cache is freed when node ref_count reaches zero
so when DatabasePager & IntersectionVisitor stopped using the tile it was
effectively removed from the cache as well.
Memory leaks were the result of extra NeedToRemove nodes that appeared in
activePageLOD lists when some nodes were resurected from Cache because
camera moved over them again. I know its sophistry, but in some way, it was
more effective than native DatabasePager management because it loaded nodes
in no time while DatabasePager would have to load them from disk in this
>> Even if we skip intersections, other
>> situations are also possible. For example we may have few highly detailed
>> special PageLOD tiles with ariports which we want to preload and keep in
>> memory for whole application runtime. So we modify readFileCallback to
>> for such cache and return these preloaded models each time thery are
> Use of the cache in conjunction with a paged database should be used
> very sparingly and for only very specific types of assets. It does
> also open the question of how DatabasePager should deal with such
> datasets, without lots of reflection on the issue I can't say. I can
> say in the design and development of DatabasePager I have made the
> assumption that it'd be the master of the PagedLOD's and manage all
> reading and expiring, and not have code on the outside managing things
> in a parallel.
Well I thought that if PagedLOD class is public and offers public interface
and methods we could use it as long as we do not hit compiler errors.
Perhaps different set of classes should be created for use in more VPB
specific closed way.
> It's worth noting that PagedLOD has settings that allow you to control
> what happens with expiry - so you can individually switch off the
I have not thought about it. I will have to learn what we can do using this
mechanism, perhaps we could do something smarter.
More information about the osg-users