Graphics New Features

Yuzhu Lu, Wolfram Research, Inc.

Join the Conversation #WolframTechConf

ABSTRACT

This presentation will present some new features and optimizations of Mathematica 2D and 3D Graphics language including more texture mapping support for 3D primitives, named VertexTextureCoordinates generation, interactive view center picking, non photorealistic rendering directives(NPO), SurefaceAppearance for customized rendering. It will also introduce our new drawing layers of Metal and Direct3D11, performance benefits and new drawing layer based features.

New Features and Optimizations: 
(Syntax might have some small changes in V12.1, please refer to the documentation)

HiDPI support for all 2D/3D Graphics

Texture Mapping for More 3D Primitives

Named VertexTextureCoordinates

Interactive ViewCenter Picking

Non Photorealistic Rendering Directives(NPO)

Surface Appearance

New Drawing Layers:

Metal

Direct3D11

Pixel  Based Lighting

List and Choosing Graphic Hardware

Build-In StereoScopic

HiDPI support for all 2D/3D Graphics

Supported in all drawing layers
Supported by all primitives including Raster/Image
Everything can be Rasterize/Copy/Paste in HiDPI

Plot[Sin[x],{x,0,6Pi}]

Graphics3D[Sphere[]]

Texture Mapping for More 3D Primitives

More primitives support texture mapping:
Sphere, Cylinder, Cuboid, Cone, BSplineSurface, Tube

The default value for VertextureCoordinates:
old: None
new: Automatic

a={{-1,0},{0,1},{1,0}};b={{0,-(2/3)},{-1,0}};
Graphics[{Texture[ExampleData[{"TestImage","Lena"}]],FilledCurve[{BezierCurve[a],Line[b]},VertexTextureCoordinates->#]}]&/@{None,Automatic}

Graphics[{Texture[ExampleData[{"TestImage","Lena"}]],Polygon[a,VertexTextureCoordinates->#]}]&/@{None,Automatic}

Graphics3D[{Texture[ExampleData[{"TestImage","Lena"}]],Polygon[{{0,0,0},{1,0,0},{0.5,1,1}},VertexTextureCoordinates->#]}]&/@{None,Automatic}

Graphics3D[{Texture[ExampleData[{"TestImage","Lena"}]],Hexahedron[{{0,0,0},{1,0,0},{2,1,0},{1,1,0},{0,0,1},{1,0,1},{2,1,1},{1,1,1}},VertexTextureCoordinates->#]}]&/@{None,Automatic}

Graphics3D[{Texture[ExampleData[{"TestImage","Lena"}]],Prism[{{1,0,1},{0,0,0},{2,0,0},{1,2,1},{0,2,0},{2,2,0}},VertexTextureCoordinates->#]}]&/@{None,Automatic}

Graphics3D[{Texture[ExampleData[{"TestImage","Lena"}]],Tetrahedron[{{1,0,0},{1,0,1},{1,1,1},{0,0,1}},VertexTextureCoordinates->#]}]&/@{None,Automatic}

Graphics3D[{Texture[ExampleData[{"TestImage","Lena"}]],Pyramid[{{0,0,0},{2,0,0},{2,2,0},{0,2,0},{1,1,2}},VertexTextureCoordinates->#]}]&/@{None,Automatic}

Named VertexTextureCoordinates

“Planar”

Flat texture mapping along X/Y/Z directions

v={{0,0,0},{2,0,0},{2,2,0},{0,2,0},{1,1,2}};
i={{1,2,5},{2,3,5},{3,4,5},{4,1,5}};
Graphics3D[{Texture[ExampleData[{"TestImage","Lena"}]],GraphicsComplex[v,Polygon[i,VertexTextureCoordinates->{"Planar",#}]]}]&/@{-3,-2,-1,3,2,1}

“Cubic”

cubic texture mapping along 3 directions

v=PolyhedronData["Icosahedron","GraphicsComplex"][[1]];
i=PolyhedronData["Icosahedron","GraphicsComplex"][[2,1]];
Graphics3D[{Texture[ExampleData[{"TestImage","Lena"}]],GraphicsComplex[v,Polygon[i,VertexTextureCoordinates->{"Cubic",#}]]}]&/@{True,False}

“Box”

cubic texture mapping along 6 directions

Graphics3D[{Texture[ExampleData[{"TestImage","Lena"}]],GraphicsComplex[v,Polygon[i,VertexTextureCoordinates->"Box"]]}]

“Spherical”

spherical texture mapping

Graphics3D[{Texture[ExampleData[{"TestImage","Lena"}]],GraphicsComplex[v,Polygon[i,VertexTextureCoordinates->{"Spherical",#}]]}]&/@{-3,-2,-1,3,2,1}

Transform texture mapping only

v=PolyhedronData["Icosahedron","GraphicsComplex"][[1]];
i=PolyhedronData["Icosahedron","GraphicsComplex"][[2,1]];
Manipulate[With[{transform=RotationMatrix[o,Prepend[d,0]]},Graphics3D[{Texture[ExampleData[{"TestImage","Lena"}]],GeometricTransformation[GraphicsComplex[Dot[transform,#]&/@v,Polygon[i,VertexTextureCoordinates->{"Planar",-3}]],Dynamic@Inverse@transform]}]],{o,0,10},{{d,{0,1}},{-1,-1},{1,1}}]

Interactive ViewCenter Picking

Hold Shift+Option key to pick

Graphics3D[{Yellow,Sphere[],Blue,Sphere[{10,0,0},0.3],Red,Sphere[{8.8,0,0},0.1],Green,Sphere[{0,6,2.5},0.5],Gray,Opacity[0.2],Ellipsoid[{0,0,0},{10,5,0.1}]},Axes->True,AxesLabel->{X,Y,Z},Background->Black,ImageSize->Large]

Non Photorealistic Rendering Directives(NPO)

New NPO for 3D graphics
ToonShading, GoochShading, StippleShading, HalftoneShading, HatchShading

ToonShading

Graphics3D[{ToonShading[],Sphere[]},Lighting->#,ImageSize->300]&/@{"Accent",{{"Directional",White,{{1,-1,2},{0,0,0}}}}}

Graphics3D[{ToonShading[ColorData[35,"ColorList"]],KnotData["SolomonSeal","ImageData"]},Lighting->#,ImageSize->300]&/@{"Accent",{{"Directional",White,{{1,-1,2},{0,0,0}}}}}

GoochShading

Graphics3D[{GoochShading[],Sphere[]},Lighting->#,ImageSize->300]&/@{"Accent",{{"Directional",White,{{1,-1,2},{0,0,0}}}}}

Graphics3D[{GoochShading[ColorData[35,"ColorList"]],KnotData["SolomonSeal","ImageData"]},Lighting->#,ImageSize->300]&/@{"Accent",{{"Directional",White,{{1,-1,2},{0,0,0}}}}}

StippleShading

Graphics3D[{StippleShading[],Sphere[]},Lighting->#,ImageSize->300]&/@{"Accent",{{"Directional",White,{{1,-1,2},{0,0,0}}}}}

Manipulate[Graphics3D[{StippleShading[d],ExampleData[{"Geometry3D","Cow"},"GraphicsComplex"]},Lighting->#,ImageSize->300]&/@{"Accent",{{"Directional",White,{{1,-1,2},{0,0,0}}}}},{{d,0.5},0,1}]

HatchShading

Graphics3D[{HatchShading[],Sphere[]},Lighting->#,ImageSize->300]&/@{"Accent",{{"Directional",White,{{1,-1,2},{0,0,0}}}}}

tones=ImageData[#]&/@{,,,,,,,};
cow=ExampleData[{"Geometry3D","Cow"},"GraphicsComplex"]/.Polygon[pts_]:>Polygon[pts,VertexTextureCoordinates->"Box"];

Graphics3D[{HatchShading[],Texture[tones],cow},Lighting->#,ImageSize->300,Boxed->False]&/@{"Accent",{{"Directional",White,{{1,-1,2},{0,0,0}}}}}

tones=ImageData[#]&/@{,,,,,,,};
Graphics3D[{HatchShading[],Texture[tones],cow},Lighting->#,ImageSize->300,Boxed->False]&/@{"Accent",{{"Directional",White,{{1,-1,2},{0,0,0}}}}}

HalftoneShading

Graphics3D[{HalftoneShading[],Sphere[]},Lighting->#,ImageSize->300]&/@{"Accent",{{"Directional",White,{{1,-1,2},{0,0,0}}}}}

Graphics3D[{HalftoneShading[0.5,Black,#],ExampleData[{"Geometry3D","Galleon"},"GraphicsComplex"]},Boxed->False,Method->{"ShrinkWrap" -> True},ImageSize->200,Lighting->{{"Directional",White,{{1,-1,2},{0,0,0}}}}]&/@{"Disk","Hexagon", "Line","Square","Triangle"}

Line pattern and tiling directives for 2D graphics
Hatching, Tiling

Hatching

Manipulate[Graphics[{Hatching[a],Polygon[{{0,0},{1,0},{1/2,1}}]}],{{a,0.3},0,4}]

Plot[{Sin[x]+x/2,Sin[x]+x}, {x, 0, 10},Filling->{1->{2}},FillingStyle->Hatching[.3]]

Tiling

Manipulate[Graphics[{Tiling[,a],Polygon[{{0,0},{1,0},{1/2,1}}]}],{{a,1},.1,5}]

Plot[{Sin[x]+x/2,Sin[x]+x}, {x, 0, 10},Filling->{1->{2}},FillingStyle->Tiling[]]

Surface Appearance

1. Customize Your Own Rendering With Shaders

2. Work with both Graphics and Graphics3D

3. Supported everywhere on all platforms with all drivers including Mesa
4. GLSL 1.2 standard

NPR Directives are built on top of SurfaceAppearance

Graphics3D[{ToonShading[],Sphere[]},Lighting->"Accent"]

Association syntax

shader=\[LeftAssociation]
"Name"->"DepthOfField",
"Attributes"->\[LeftAssociation]"a_vertex"->"ATTRIB_VERTEX"\[RightAssociation],"Parameters"->\[LeftAssociation]"Matrix"->{"u_transform","TRANSFORMMATRIX","MATRIX4"},"ColorRamp"->{"u_colorRamp",Automatic,"TEXTURE"},"Size"->{"size",360,"NUMBER"},"FocusDepth"->{"focus",0.5,"NUMBER"},"Maxfilter"->{"maxfilter",20,"NUMBER"}\[RightAssociation],"ShaderPrograms"->\[LeftAssociation]"GLFragmentProgram"->"uniform sampler2D u_colorRamp; uniform float size; uniform float focus; uniform float maxfilter; varying vec4 v_pos; void main(){ float offset=1.0/size; vec2 texCoord = v_pos.xy/2.0+vec2(.5,0.5); vec3 sum = texture2D(u_colorRamp, texCoord).rgb; float block = max(0, texture2D(u_colorRamp, texCoord).a - focus) * maxfilter; float count = 1.0; for(float i = -block; i < block; i = i+1.0) { for(float j = -block; j < block; j = j+1.0) { sum = sum + texture2D(u_colorRamp, clamp(texCoord+vec2(offset * i, offset * j), 0.0, 1.0)).rgb; count = count + 1.0;}} gl_FragColor.rgb = sum/count; gl_FragColor.a = 1.0;}","GLVertexProgram"->"attribute vec4 a_vertex; uniform mat4 u_transform; varying vec4 v_pos; void main(){v_pos = u_transform * a_vertex; gl_Position = u_transform * a_vertex;}"\[RightAssociation]
\[RightAssociation];

Dataset[shader]

Example 1: Depth of Field

ColorDepth[g_,size_]:=ColorCombine[Append[ColorSeparate[Rasterize@Graphics3D[g,Method->{"OneLayer"->{"Color",1}},Boxed->False,ImageSize->size]],First@ColorSeparate@Rasterize@Graphics3D[g,Method->{"OneLayer"->{"Depth",1}},Boxed->False,ImageSize->size]],"RGB"];
FE`Evaluate@Evaluate[FEPrivate`AddSurfaceAppearanceDefinition@shader];
Graphics3DWithDOF[g_,size_,focus_,maxFilter_]:=
Graphics[{Texture[ColorDepth[g,size]],Directive[SurfaceAppearance["DepthOfField","Size"->size,"FocusDepth"->focus,"Maxfilter"->maxFilter]],Polygon[{{0,0},{1,0},{1,1},{0,1}}]},PlotRange->{{0,1},{0,1}},ImageSize->size];

Manipulate[Graphics3DWithDOF[{Blue,Cylinder[],Red,Sphere[{0,0,2}],Black,Thick,Dashed,Line[{{-2,0,2},{2,0,2},{0,0,4},{-2,0,2}}],Yellow,Polygon[{{-3,-3,-2},{-3,3,-2},{3,3,-2},{3,-3,-2}}],Green,Cuboid[{-2,-2,-2},{2,2,-1}]},300,f,m],{{f,0.5},0,1},{{m,20},5,50}]

Manipulate[Graphics3DWithDOF[Normal[ExampleData[{"Geometry3D","Galleon"},"GraphicsComplex"]],300,f,m],{{f,0.5},0,1},{{m,20},5,50}]

Manipulate[Graphics3DWithDOF[Normal[ExampleData[{"Geometry3D","Beethoven"},"GraphicsComplex"]],300,f,m],{{f,0.5},0,1},{{m,20},5,50}]

Manipulate[Graphics3DWithDOF[Normal[ExampleData[{"Geometry3D","Cow"},"GraphicsComplex"]],300,f,m],{{f,0.5},0,1},{{m,20},5,50}]

Metal

Fully Implemented New 3D Drawing Layer on Mac

Performance comparing to OpenGL

Grid[{{,"OpenGL","Metal"},{"MengerMesh[1,3]","133 fps","151 fps"},{"MengerMesh[2,3]","125 fps","132 fps"},{"MengerMesh[3,3]","80 fps","66 fps"},{"MengerMesh[4,3]","2.4 fps","7.1 fps"}},Frame->All,ItemStyle->{Automatic,Automatic,{{{1,1},{1,4}}->{Bold,16,RGBColor[0,0,0.8]}}}]

{MengerMesh[4,3]}

Direct3D11

Fully Implemented New 3D Drawing Layer on Windows

Performance comparing to Direct3D 9

Grid[{{,"Direct3D 9","Direct3D 11"},{"MengerMesh[1,3]","140 fps","250 fps"},{"MengerMesh[2,3]","127 fps","242 fps"},{"MengerMesh[3,3]","124 fps","136 fps"},{"MengerMesh[4,3]","113 fps","121 fps"}},Frame->All,ItemStyle->{Automatic,Automatic,{{{1,1},{1,4}}->{Bold,16,RGBColor[0,0,0.8]}}}]

Pixel Based Lighting

Better lighting with less polygons

Plot3D[Sin[x] Cos[y^1.1] Log[x],{x,0,2 Pi},{y,0,2 Pi},Mesh->None, MaxRecursion->0,PlotPoints->10,Method->{"PerFragmentLighting"->#},ImageSize->300,BoxRatios->{1, 1, 1}]&/@{True,False}

List and Choosing Graphic Hardware

List informations for all graphics hardware

User can choose graphic hardware for rendering: 
Automatic, Integrated, Discrete, External, Software

Build-In Stereoscopic

Build-In Stereoscopic support for Direct3D11

Hold left Ctrl to interact with any 3D with bring up the stereo window

Graphics3D[{Specularity[White,30],Raster3D[RawArray["Byte",Import["ExampleData/CThead.tiff","Data"]],{{-1,-1,-1},{1,1,1}},{0,255},ColorFunction->(Blend[{{0, RGBColor[0.035,0.058,0,0]},{0.1, RGBColor[0.015,0,0.035,0]},{0.2, RGBColor[0,0,0,0.032]},{0.3, RGBColor[0.103,0.032,0,0]},{0.4, RGBColor[0,0,0,0]},{0.5, RGBColor[0.202,0.203,0.218,0.337]},{0.6, RGBColor[0.9,0.9,0.9,0.457]},{0.7, RGBColor[0.98,0.97,1,0.682]},{0.8, RGBColor[1,1,1,0.794]},{0.9, RGBColor[0.95,0.97,0.95,0.842]},{1, RGBColor[1,1,1,0.962]}}, #] &),Method->{ "VolumeLighting"->"EnhancedEdge","InterpolateValues"->True,"SampleLayers"->1000}]},ViewPoint->{-3.2,0.89,-0.34},Lighting->{{"Directional",White,ImageScaled[{0,0,2}]}}]