[osg-users] Bug in OverlayNode.cpp

Chris Long clong at setcorp.com
Tue Apr 5 11:16:47 PDT 2011


Hi,

I have found a bug at OverlayNode.cpp:335:


Code:
for(unsigned int i=0; i < baseVertices.size()-1; ++i)




The problem is when baseVertices is empty, baseVertices.size() is 0 and (here's the surprising part) the compiler treats baseVertices.size()-1 as an unsigned int, which turns it into some huge positive number (18446744073709551615 on Linux on amd64). So the loop actually executes and seg faults.

I have this case occurring in my application, I think because of an OverlayNode that has a valid overlay subgraph node with no children of its own.

I'm sure there are other ways to fix this, but I would suggest fixing it by changing the above line to:


Code:
  for(unsigned int i=0; i+1 < baseVertices.size(); ++i)




Thank you!

Cheers,
Chris

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=38233#38233








More information about the osg-users mailing list