使用MTKView显示JPEG图像
有没有办法通过MTKView和MTLBuffer（和iPhone 6 +内）显示JPEG图像。 我已经尝试了以下方式（这只是testing）：
- (id<MTLBuffer>)testBuffer { if (!_testBuffer) { // NSString *path = [[NSBundle mainBundle] pathForResource:@"testImage" ofType:@"jpg"]; NSData *imageData = [NSData dataWithContentsOfFile:path]; _testBuffer = [self.device newBufferWithBytes:imageData.bytes length:imageData.length options:MTLResourceCPUCacheModeWriteCombined]; } return _testBuffer; } - (void)drawInMTKView:(MTKView *)view { MTLRenderPassDescriptor* renderPassDescriptor = view.currentRenderPassDescriptor; id <MTLCommandBuffer> commandBuffer = [_commandQueue commandBuffer]; id <MTLRenderCommandEncoder> renderEncoder = [commandBuffer renderCommandEncoderWithDescriptor:renderPassDescriptor]; [renderEncoder drawPrimitives:MTLPrimitiveTypePoint indirectBuffer:self.testBuffer indirectBufferOffset:0]; [renderEncoder endEncoding]; [commandBuffer presentDrawable:view.currentDrawable]; [commandBuffer commit]; } 我在这个方法调用中遇到一个错误：
[renderEncoder drawPrimitives:MTLPrimitiveTypePoint indirectBuffer:self.testBuffer indirectBufferOffset:0];
/BuildRoot/Library/Caches/com.apple.xbs/Sources/Metal/Metal-54.31/ToolsLayers/Debug/MTLDebugRenderCommandEncoder.mm:2123：failed assertion` – [MTLDebugRenderCommandEncoder drawPrimitives：indirectBuffer：indirectBufferOffset：]仅在MTLFeatureSet_iOS_GPUFamily3_v1上受支持，后来'
这是因为这个API仅支持iPhone 6s（+） 。
我想我正在做一些完全错误的事情。 也许我需要从另一个angular度思考。 有人可以帮助我或指出正确的方向吗？ 谢谢。
有没有简单的方法来使用MTLBuffer通过MTKView呈现图像数据。 在这种情况下，您必须准备正确的像素表示，以便在可绘制图层中使用MTKView显示的正确输出纹理。 最简单的方法是使用MTKTextureLoader，或从UIImage / CGImage实例加载数据到MTLTexture对象。
我做了一个类似的实验，在iOS设备上使用MTL Layer和MTLBuffer（而不是MTLBuffer）展示图像，并且我决定了最好的显示图像的方式（如果这只需要显示图像），它使用直通内核函数没有渲染。
在Swift2中，这看起来如下所示：
... let textureLoader = MTKTextureLoader(device: self.device!) if let image = UIImage(named: file){ imageTexture = try! textureLoader.newTextureWithCGImage(image.CGImage!, options: nil) threadGroups = MTLSizeMake( (imageTexture.width+threadGroupCount.width)/threadGroupCount.width, (imageTexture.height+threadGroupCount.height)/threadGroupCount.height, 1) } } ... let function:MTLFunction! = library.newFunctionWithName("kernel_passthrough") ... pipeline = try! self.device.newComputePipelineStateWithFunction(function) ... let commandBuffer = commandQueue.commandBuffer() let encoder = commandBuffer.computeCommandEncoder() encoder.setComputePipelineState(pipeline) encoder.setTexture(actualImageTexture, atIndex: 0) encoder.setTexture(metalView.currentDrawable!.texture, atIndex: 1) encoder.setBuffer(self.saturationUniform, offset: 0, atIndex: 0) encoder.dispatchThreadgroups(threadGroups!, threadsPerThreadgroup: threadGroupCount) encoder.endEncoding() commandBuffer.presentDrawable(metalView.currentDrawable!) commandBuffer.commit() ...
在金属底纹文件中：
kernel void kernel_passthrough(texture2d<float, access::read> inTexture [[texture(0)]], texture2d<float, access::write> outTexture [[texture(1)]], uint2 gid [[thread_position_in_grid]]) { float4 inColor = inTexture.read(gid); // // flip texture vertically if it needs to display with right orientation // outTexture.write(inColor, gid); }
完整的示例源： ImageMetalling-00
由于忙，我忘了写解决scheme。 @丹恩的答案看起来是正确的，所以我会接受。 我的解决scheme如下（在Objective-C中）。 我简化了苹果的例子（以及他们的着色器文件），并做了testing项目 。 我创build了一个名为MetalView的MTKView的子类。 它具有“configuration”方法，为绘图准备MetalView：设置MTKView的属性，并创buildpipelineState对象以及顶点缓冲区。
- (void)configure { self.device = MTLCreateSystemDefaultDevice(); self.colorPixelFormat = MTLPixelFormatBGRA8Unorm; self.framebufferOnly = YES; self.sampleCount = 1; // create command queue for usage during drawing self.commandQueue = [self.device newCommandQueue]; // load shaders functions from texturedQuad.metal file. These functions needed for configuration of MTLRenderPipelineDescriptor id <MTLLibrary> shaderLibrary = [self.device newDefaultLibrary]; id <MTLFunction> fragmentProgram = [shaderLibrary newFunctionWithName:@"texturedQuadFragment"]; id <MTLFunction> vertexProgram = [shaderLibrary newFunctionWithName:@"texturedQuadVertex"]; // create a pipeline state MTLRenderPipelineDescriptor *pQuadPipelineStateDescriptor = [MTLRenderPipelineDescriptor new]; pQuadPipelineStateDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormatBGRA8Unorm; pQuadPipelineStateDescriptor.sampleCount = self.sampleCount; pQuadPipelineStateDescriptor.vertexFunction = vertexProgram; pQuadPipelineStateDescriptor.fragmentFunction = fragmentProgram; NSError *pipErr = nil; self.pipelineState = [self.device newRenderPipelineStateWithDescriptor:pQuadPipelineStateDescriptor error:&pipErr]; if (pipErr) NSLog(@"newRenderPipelineStateWithDescriptor err: %@", pipErr); // buffers for MTLRenderCommandEncoder in drawRect: method self.vertexBuffer = [self.device newBufferWithBytes:kQuadVertices length:kSzQuadVertices options:MTLResourceOptionCPUCacheModeDefault]; self.texCoordBuffer = [self.device newBufferWithBytes:kQuadTexCoords length:kSzQuadTexCoords options:MTLResourceOptionCPUCacheModeDefault]; self.paused = YES; self.enableSetNeedsDisplay = NO; }
这些东西将由renderEncoder在drawRect：方法中用于实际绘图：
- (void)drawRect:(CGRect)rect { id <MTLCommandBuffer> commandBuffer = [self.commandQueue commandBuffer]; MTLRenderPassDescriptor *renderPassDescriptor = self.currentRenderPassDescriptor; id <MTLRenderCommandEncoder> renderEncoder = [commandBuffer renderCommandEncoderWithDescriptor:renderPassDescriptor]; // Encode into a renderer [renderEncoder setRenderPipelineState:self.pipelineState]; [renderEncoder setVertexBuffer:self.vertexBuffer offset:0 atIndex:0]; [renderEncoder setVertexBuffer:self.texCoordBuffer offset:0 atIndex:1]; [renderEncoder setFragmentTexture:self.texture atIndex:0]; // tell the render context we want to draw our primitives. We will draw triangles that's // why we need kQuadVertices and kQuadTexCoords (arrays of points) [renderEncoder drawPrimitives:MTLPrimitiveTypeTriangle vertexStart:0 vertexCount:6 instanceCount:1]; [renderEncoder endEncoding]; [commandBuffer presentDrawable:self.currentDrawable]; [commandBuffer commit]; }
着色器文件代码（texturedQuad.metal）：
struct VertexInOut { float4 m_Position [[position]]; float2 m_TexCoord [[user(texturecoord)]]; }; vertex VertexInOut texturedQuadVertex(constant float4 *pPosition [[ buffer(0) ]], constant packed_float2 *pTexCoords [[ buffer(1) ]], constant float4x4 *pMVP [[ buffer(2) ]], uint vid [[ vertex_id ]]) { VertexInOut outVertices; outVertices.m_Position = pPosition[vid]; outVertices.m_TexCoord = pTexCoords[vid]; return outVertices; } fragment half4 texturedQuadFragment(VertexInOut inFrag [[ stage_in ]], texture2d<half> tex2D [[ texture(0) ]]) { constexpr sampler quad_sampler; half4 color = tex2D.sample(quad_sampler, inFrag.m_TexCoord); return color; }
在ViewController的storyboard中，MetalView类被用作view outlet的类。 ViewController的viewDidLoad方法检索指定图像的对象，并将此对象设置为MetalView。
- (void)viewDidLoad { [super viewDidLoad]; self.metalView = (MetalView *)self.view; NSString *path = [[NSBundle mainBundle] pathForResource:@"testImage" ofType:@"jpg"]; MTKTextureLoader *loader = [[MTKTextureLoader alloc] initWithDevice:self.metalView.device]; NSError *err = nil; self.metalView.texture = [loader newTextureWithContentsOfURL:[NSURL fileURLWithPath:path] options:nil error:&err]; if (err) NSLog(@"newTextureWithContentsOfURL err: %@", [err localizedDescription]); }
最后，我们需要为MetalView调用draw方法
- (void)viewDidLayoutSubviews { [self.metalView draw]; }
示例代码在这里 。