[osg-users] Bug in OverlayNode.cpp
clong at setcorp.com
Tue Apr 5 11:16:47 PDT 2011
I have found a bug at OverlayNode.cpp:335:
for(unsigned int i=0; i < baseVertices.size()-1; ++i)
The problem is when baseVertices is empty, baseVertices.size() is 0 and (here's the surprising part) the compiler treats baseVertices.size()-1 as an unsigned int, which turns it into some huge positive number (18446744073709551615 on Linux on amd64). So the loop actually executes and seg faults.
I have this case occurring in my application, I think because of an OverlayNode that has a valid overlay subgraph node with no children of its own.
I'm sure there are other ways to fix this, but I would suggest fixing it by changing the above line to:
for(unsigned int i=0; i+1 < baseVertices.size(); ++i)
Read this topic online here:
More information about the osg-users