Curently based on Sascha Willems examples I've cerated samplerCube texture for fragment shader.
It has same JPG image copied to all 6 layers (faces).
I use stbi image library for image loading, it works okay if I use it for regular 2D texture, but if it's mapped on cube mesh it creates distorted image:
int width = 0, height = 0, channel = 0;
float* pixels = stbi_loadf("textures/test.hdr", &width, &height, &channel, STBI_rgb_alpha);
if(!pixels) throw std::runtime_error("failed to load texture image!");
this->texture_image.create_image(width, height, VK_FORMAT_R32G32B32A32_SFLOAT, VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_SAMPLED_BIT);
this->texture_image.fill_memory(width, height, 4*sizeof(float), pixels);
this->texture_image.create_image_view(VK_FORMAT_R32G32B32A32_SFLOAT, VK_IMAGE_ASPECT_COLOR_BIT);
stbi_image_free(pixels);
Found how to do it here: https://learnopengl.com/PBR/IBL/Diffuse-irradiance
Even if it's in OpenGL the concenpt is the same.
Related
I have an app written with RXSwift which processes 500+ days of HealthKit data to draw a chart for the user.
The chart image is drawn incrementally using the code below. Starting with a black screen, previous image is drawn in the graphics context, then a new segment is drawn over this image with certain offset. The combined image is saved and the process repeats around 70+ times. Each time the image is saved, so the user sees the update. The result is a single chart image which the user can export from the app.
Even with autorelease pool, I see spikes of memory usage up to 1Gb, which prevents me from doing other resource intensive processing.
How can I optimize incremental drawing of very large (1440 × 5000 pixels) image?
When image is displayed or saved at 3x scale, it is actually 4320 × 15360.
Is there a better way than trying to draw over an image?
autoreleasepool {
//activeEnergyCanvas is custom data processing class
let newActiveEnergySegment = activeEnergyCanvas.draw(in: CGRect(x: 0, y: 0, width: 1440, height: days * 10), with: energyPalette)
let size = CGSize(width: 1440, height: height)
UIGraphicsBeginImageContextWithOptions(size, false, 0.0)
//draw existing image
self.activeEnergyImage.draw(in: CGRect(origin: CGPoint(x: 0, y: 0),
size: size))
//calculate where to draw smaller image over larger one
let offsetRect = CGRect(origin: CGPoint(x: 0, y: offset * 10),
size: newActiveEnergySegment.size)
newActiveEnergySegment.draw(in: offsetRect)
//get the combined image
let newImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
//assign combined image to be displayed
if let unwrappedImage = newImage {
self.activeEnergyImage = unwrappedImage
}
}
Turns out my mistake was in passing invalid drawing scale (0.0) when creating graphics context, which defaulted to drawing at the device's native screen scale.
In case of iPhone 8 it was 3.0 The result is needing extreme amounts of memory to draw, zoom and export these images. Even if all debug logging prints that image is 1440 pixels wide, the actual canvas ends up being 1440 * 3.0 = 4320.
Passing 1.0 as the drawing scale makes the image more fuzzy, but reduces memory usage to less than 200mb.
// UIGraphicsBeginImageContext() <- also uses #3x scale, even when all display size printouts show
let drawingScale: CGFloat = 1.0
UIGraphicsBeginImageContextWithOptions(size, true, drawingScale)
I have a grayscale image that serves as a soft mask and I want to use it on a group of PDF objects (images or paths).
The mask and the objects do not necessarily use the same transformation matrix, and there might be more than one object to mask, so that excludes the possibility of using the SMask attribute of the ImageXObject dictionary.
So after reading some of the PDF specification, it looks like I should do the following: create a transparency group with the objects to mask, then draw it with the soft mask set on the graphics state.
Will that work? How can I achieve this, preferably with PDFBox?
Here's an example. I have these two images: the mask and another image.
The mask image is 200x200. It is drawn with the matrix [[4 0 100] [0 4 100]].
The image is 400x300. It is drawn with the matrix [[2 0 100] [0 2 150]].
Additionally, a 400x400 black square is drawn below the image with no transform matrix.
So a transparency group is created with the image and the square, then it's drawn with the mask image. Here's the expected result:
Rather ugly as far as the effect goes, but that's just an example.
As far as I can see establishing an extended graphics state soft mask is a very manual task in PDFBox. You can do so as follows:
try ( PDDocument document = new PDDocument() ) {
final PDImageXObject image = RETRIEVE PHOTO IMAGE;
final PDImageXObject mask = RETRIEVE MASK IMAGE;
PDTransparencyGroupAttributes transparencyGroupAttributes = new PDTransparencyGroupAttributes();
transparencyGroupAttributes.getCOSObject().setItem(COSName.CS, COSName.DEVICEGRAY);
PDTransparencyGroup transparencyGroup = new PDTransparencyGroup(document);
transparencyGroup.setBBox(PDRectangle.A4);
transparencyGroup.setResources(new PDResources());
transparencyGroup.getCOSObject().setItem(COSName.GROUP, transparencyGroupAttributes);
try ( PDFormContentStream canvas = new PDFormContentStream(transparencyGroup) ) {
canvas.drawImage(mask, new Matrix(400, 0, 0, 400, 100, 100));
}
COSDictionary softMaskDictionary = new COSDictionary();
softMaskDictionary.setItem(COSName.S, COSName.LUMINOSITY);
softMaskDictionary.setItem(COSName.G, transparencyGroup);
PDExtendedGraphicsState extendedGraphicsState = new PDExtendedGraphicsState();
extendedGraphicsState.getCOSObject().setItem(COSName.SMASK, softMaskDictionary);
PDPage page = new PDPage(PDRectangle.A4);
document.addPage(page);
try ( PDPageContentStream canvas = new PDPageContentStream(document, page) ) {
canvas.saveGraphicsState();
canvas.setGraphicsStateParameters(extendedGraphicsState);
canvas.setNonStrokingColor(Color.BLACK);
canvas.addRect(100, 100, 400, 400);
canvas.fill();
canvas.drawImage(image, new Matrix(400, 0, 0, 300, 100, 150));
canvas.restoreGraphicsState();
}
document.save(new File(RESULT_FOLDER, "SoftMaskedImageAndRectangle.pdf"));
}
The result:
If I were you, though, I would not use a bitmap image for the soft mask but instead a PDF gradient. The result most likely will be much less pixelated.
I am trying to create jigsaw puzzle shapes using P5.js. After creating puzzle shapes, I want to cut areas from main image into pieces. For that I have options of using GET() or COPY():
But both of them take fix height and width as parameter. How can I copy a custom area like given in following shapes:
https://editor.p5js.org/techty/sketches/h7qwatZRb
let cutout = createGraphics(w, h);
cutout.background(255, 255);
cutout.blendMode(REMOVE);
//draw shape on cutout
let newshapeimagegraphic = createGraphics(w, h);
newshapeimagegraphic.image(myImg, 0, 0);
newshapeimagegraphic.blendMode(REMOVE);
newshapeimagegraphic.image(cutout, 0, 0);
image(newshapeimagegraphic, 0, 0);
I was using CGImageCreate with CGColorSpaceCreateDeviceGray to convert a buffer (CVPixelBufferRef) to grayscale image. It was very fast and did work well until iOS 12... now the returned image is empty.
The code look like this:
bitmapInfo = kCGBitmapByteOrder32Little | kCGImageAlphaNoneSkipFirst;
CGDataProviderRef provider = CGDataProviderCreateWithData((void *)i_PixelBuffer,
sourceBaseAddr,
sourceRowBytes * height,
ReleaseCVPixelBuffer);
retImage = CGImageCreate(width,
height,
8,
32,
sourceRowBytes,
CGColorSpaceCreateDeviceGray(),
bitmapInfo,
provider,
NULL,
true,
kCGRenderingIntentDefault);
CGDataProviderRelease(provider);
This is a known bug in iOS 12? If device gray is no supported anymore in this function, can you suggest me another way to do it?
Note that conversion should take less than 0.1 seconds for a 4K image.
Thanks in advance!
According to the list of Supported Pixel Formats in the Quartz 2D Programming Guide, iOS doesn't support 32 bits per pixel with gray color spaces. And even on macOS, 32 bpp gray requires the use of kCGBitmapFloatComponents (and float data).
Is your data really 32 bpp? If so, is it float? What are you using for bitmapInfo?
I would not expect CGImageCreate() to "convert" a buffer, including to grayscale. The parameters you're supplying are telling it how to interpret the data. If you're not using floating-point components, I suspect it was just taking one of the color channels and interpreting that as the gray level and ignoring the other components. So, it wasn't a proper grayscale conversion.
Apple's advice is to create an image that properly represents the image; create a bitmap context with the colorspace, pixel layout, and bitmap info you desire; draw the former into the latter; and create the final image from the context.
I finally found a workaround for my purpose. Note that the CVPixelBuffer is coming from the video camera.
Changed camera output pixel format to
kCVPixelFormatType_420YpCbCr8BiPlanarFullRange
(AVCaptureVideoDataOutput)
Extract the Y plane from YpCbCr
Build a CGImage with the Y plane
Code:
// some code
colorSpace = CGColorSpaceCreateDeviceGray();
sourceRowBytes = CVPixelBufferGetBytesPerRowOfPlane(i_PixelBuffer, 0);
sourceBaseAddr = (unsigned char*)CVPixelBufferGetBaseAddressOfPlane(i_PixelBuffer,0);
bitmapInfo = kCGImageByteOrderDefault;
// some code
CGContextRef context = CGBitmapContextCreate(sourceBaseAddr,
width,
height,
8,
sourceRowBytes,
colorSpace,
bitmapInfo);
retImage = CGBitmapContextCreateImage(context);
// some code
You can also look at this related post:
420YpCbCr8BiPlanarVideoRange To YUV420 ?/How to copy Y and Cbcr plane to Single plane?
In the fragment function of a Metal Shader file, is there a way to redefine the "bounds" of the texture with respect to what the sample will consider it's normalized coordinates to be?
By default, a value of 0,0 for the sample is the top-left "pixel" and 1,1 is the bottom right "pixel" of the texture. However, I'm re-using textures for drawing and at any given render pass there's only a portion of the texture that contains the relevant data.
For example, in a texture of width: 500 and height: 500, I might have only copied data into the region of 0,0,250,250. In my fragment function, I'd like the sampler to interpret a normalized coordinate of 1.0 to be 250 and not 500. Is that possible?
I realize I can just change the sampler to use pixel addressing, but that comes with a few restrictions as noted in the Metal Shader Specification.
No, but if you know the region you want to sample from, it's quite easy to do a little math in the shader to fix up your sampling coordinates. This is used often with texture atlases.
Suppose you have an image that's 500x500 and you want to sample the bottom-right 125x125 region (just to make things more interesting). You could pass this sampling region in as a float4, storing the bounds as (left, top, width, height) in the xyzw components. In this case, the bounds would be (375, 375, 125, 125). Your incoming texture coordinates are "normalized" with respect to this square. The shader simply scales and biases these coordinates into texel coordinates, then normalizes them to the dimensions of the whole texture:
fragment float4 fragment_main(FragmentParams in [[stage_in]],
texture2d<float, access::sample> tex2d [[texture(0)]],
sampler sampler2d [[sampler(0)]],
// ...
constant float4 &spriteBounds [[buffer(0)]])
{
// original coordinates, normalized with respect to subimage
float2 texCoords = in.texCoords;
// texture dimensions
float2 texSize = float2(tex2d.get_width(), tex2d.get_height());
// adjusted texture coordinates, normalized with respect to full texture
texCoords = (texCoords * spriteBounds.zw + spriteBounds.xy) / texSize;
// sample color at modified coordinates
float4 color = tex2d.sample(sampler2d, texCoords);
// ...
}