What the equivalent of OpenGL glStencilFunc in Metal? - objective-c

Question

How to translate to Metal the following OpenGL instruction:
glStencilFunc(Func, Ref, Mask);

Answer 1

Let's try to extend Hamid's correct answer with some actual Swift code...
Assuming you already have the basic Metal setup done:
let metalView: MTKView = /* ... */
let commandBuffer: MTLCommandBuffer = /* ... */
let device: MTLDevice = metalView.device!
Let's create a MTLTexture to store the stencil buffer:
let stencilTexture: MTLTexture
// Set up a texture for rendering the *8 bit unsigned* stencil buffer.
let stencilTextureDescriptor = MTLTextureDescriptor()
stencilTextureDescriptor.textureType = .type2D
stencilTextureDescriptor.width = Int(metalView.frame.width)
stencilTextureDescriptor.height = Int(metalView.frame.height)
stencilTextureDescriptor.pixelFormat = .stencil8
stencilTextureDescriptor.storageMode = .private
// Set this option if you use the given texture
// as a stencil render target in any render pass.
stencilTextureDescriptor.usage = [.renderTarget]
stencilTexture = device.makeTexture(descriptor: stencilTextureDescriptor)
Create the MTLRenderPipelineState as usual but be sure to set the stencil attachment too:
let rendePipelineState: MTLRenderPipelineState
let renderPipelineDescriptor = MTLRenderPipelineDescriptor()
/* ...standard render pipeline configuration (e.g., render states, etc)... */
renderPipelineDescriptor.stencilAttachmentPixelFormat = .stencil8
rendePipelineState = try! device.makeRenderPipelineState(descriptor: renderPipelineDescriptor)
Next configure the MTLRenderPassDescriptor and create a MTLRenderCommandEncoder:
let renderEncoder: MTLRenderCommandEncoder
let renderPassDescriptor = MTLRenderPassDescriptor()
renderPassDescriptor.colorAttachments[0].loadAction = .clear
renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColorMake(0, 0, 0, 1)
renderPassDescriptor.colorAttachments[0].storeAction = .store
// Configure these depending on your rendering pass stencil semantics.
renderPassDescriptor.stencilAttachment.texture = stencilTexture
renderPassDescriptor.stencilAttachment.clearStencil = 0
renderPassDescriptor.stencilAttachment.loadAction = .clear
renderPassDescriptor.stencilAttachment.storeAction = .store
renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor)!
renderEncoder.setRenderPipelineState(rendePipelineState)
Now create a MTLDepthStencilState with the desired stencil buffer configuration:
let depthAndStencilState: MTLDepthStencilState
let stencilDescriptor = MTLStencilDescriptor()
stencilDescriptor.depthStencilPassOperation = .incrementClamp
// ## glStencilFunc(FUNC, _, _) ##
stencilDescriptor.stencilCompareFunction = .equal
// ## glStencilFunc(_, _, MASK) ##
stencilDescriptor.readMask = 0xFF_FF_FF_FF
let depthAndStencilDescriptor = MTLDepthStencilDescriptor()
depthAndStencilDescriptor.depthCompareFunction = .always
depthAndStencilDescriptor.isDepthWriteEnabled = false
depthAndStencilDescriptor.frontFaceStencil = stencilDescriptor
depthAndStencilDescriptor.backFaceStencil = stencilDescriptor
depthAndStencilState = device.makeDepthStencilState(descriptor: depthAndStencilDescriptor)!
renderEncoder.setDepthStencilState(depthAndStencilState)
// ## glStencilFunc(_, REF, _) ##
renderEncoder.setStencilReferenceValue(1)
Finally, do the rendering as usual:
/* ...encode rendering primitives... */
renderEncoder.endEncoding()

Answer 2

MTLStencilDescriptor an object that defines the front-facing or back-facing stencil operations of a depth and stencil state object.
A stencil test is a comparison between a masked reference value and a
masked value stored in a stencil attachment. (A value is masked by
performing a logical AND operation on it with the readMask value.) The
MTLStencilDescriptor object defines how to update the contents of the
stencil attachment, based on the results of the stencil test and the
depth test.
The stencilCompareFunction property defines the stencil test. The
stencilFailureOperation, depthFailureOperation, and
depthStencilPassOperation properties specify what to do to a stencil
value stored in the stencil attachment for three different test
outcomes: if the stencil test fails, if the stencil test passes and
the depth test fails, or if both stencil and depth tests succeed,
respectively. writeMask determines which stencil bits can be modified
as the result of a stencil operation.
I see where to put the value of Func and Mask with
MTLStencilDescriptor, but where go the value of REF ?
setStencilReferenceValue sets a stencil reference value for both front and back stencil comparison tests.