[osg-submissions] Fix for CustomPolytope in OverlayNode.cpp + minor tweaks for osg::Plane

[Wojciech Lewandowski]

Hi, 

----1----

Attached is a fixed version of OverlayNode.cpp. I fixed CustomPolytope::cut( osg::Plane ) method.  Bug was apparent in such scenario: 

Let P1 be some random frustum polytope
Let P2 be the polytope that was created from P1 bounding box (P2 contains P1 entirely)

Then ignoring precision errors: P1.cut( P2 ) == P2.cut( P1 ) == P1.  But this condition was not always met. Cut failed when some of the polytope reference points happened to lie exactly on some intersecting planes in both P1 & P2  (plane distance was = 0). 

I only use CustomPolytope for my shadowing stuff so I did not test how this affects rest of OverlayNode.cpp. 

----2----

Also attached is a minor precision improvement for osg::Plane intersect method (double version). 

----3----

I have also one observation regarding osg::Plane - There are two intersect vertices methods (float and double flavour): 

inline int intersect(const std::vector<Vec3>& vertices) const
inline int intersect(const std::vector<Vec3d>& vertices) const

I guess osg::Plane won't compile when someone changes default vec3 typedef to vec3d. Shouldn't the first method be changed to use vec3f explicitly ? Ie: 

inline int intersect(const std::vector<Vec3f>& vertices) const

----

Modifications are based on latest SVN  code.

Cheers,
Wojtek Lewandowski
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openscenegraph.org/pipermail/osg-submissions-openscenegraph.org/attachments/20080207/0d3fd0ed/attachment-0001.html 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: OverlayNode.cpp
Url: http://lists.openscenegraph.org/pipermail/osg-submissions-openscenegraph.org/attachments/20080207/0d3fd0ed/attachment-0001.pot 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Plane
Type: application/octet-stream
Size: 11146 bytes
Desc: not available
Url : http://lists.openscenegraph.org/pipermail/osg-submissions-openscenegraph.org/attachments/20080207/0d3fd0ed/attachment-0001.obj