[osg-users] Bug in OverlayNode.cpp

Chris Long clong at setcorp.com
Tue Apr 5 11:16:47 PDT 2011


I have found a bug at OverlayNode.cpp:335:

for(unsigned int i=0; i < baseVertices.size()-1; ++i)

The problem is when baseVertices is empty, baseVertices.size() is 0 and (here's the surprising part) the compiler treats baseVertices.size()-1 as an unsigned int, which turns it into some huge positive number (18446744073709551615 on Linux on amd64). So the loop actually executes and seg faults.

I have this case occurring in my application, I think because of an OverlayNode that has a valid overlay subgraph node with no children of its own.

I'm sure there are other ways to fix this, but I would suggest fixing it by changing the above line to:

  for(unsigned int i=0; i+1 < baseVertices.size(); ++i)

Thank you!


Read this topic online here:

More information about the osg-users mailing list