I am referencing this link -> Converting a SoftwareBitmap or WriteableBitmap to cv::Mat in c++/cx
But every time I try to put this into the code it shows me this error...enter image description here
You can see the background highlighted part.. where I have used this conversion part in the code....
and this is the reference link for code
https://github.com/Microsoft/Windows-universal-samples/blob/master/Samples/CameraGetPreviewFrame/cpp/MainPage.xaml.cpp
Related
I'm trying to port my OpenGL code to Metal one. As part of it I need to draw PNG textures, and I'm using MTKTextureLoader to load them. Here is my pipeline code:
texturePipelineDescriptor.vertexFunction = textureVertexFunc;
texturePipelineDescriptor.fragmentFunction = textureFragmentFunc;
texturePipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormatRGBA8Unorm;
texturePipelineDescriptor.colorAttachments[0].blendingEnabled = YES;
texturePipelineDescriptor.colorAttachments[0].rgbBlendOperation = MTLBlendOperationAdd;
texturePipelineDescriptor.colorAttachments[0].sourceRGBBlendFactor = MTLBlendFactorSourceAlpha;
texturePipelineDescriptor.colorAttachments[0].destinationRGBBlendFactor = MTLBlendFactorOneMinusSourceAlpha;
And here is my loader code:
NSData* imageData = [NSData dataWithBytes:imageBuffer length:imageBufferSize];
id<MTLTexture> newMTLTexture = [m_metal_renderer.metalTextureLoader newTextureWithData:imageData options:nil error:&error];
And here is the result. The entire image should be orange, like the small parts.
If I change this line:
texturePipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormatRGBA8Unorm;
To this line:
texturePipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormatBGRA8Unorm;
I will get the following results:
As you can see,images changed R and B channels, When I loading the textures, in ALL OF THEM
newMTLTexture.pixelFormat = MTLPixelFormatBGRA8Unorm;
regardless whether they are drawn correctly or not.
I would suspect there is something wrong with PNG files, but they seems absolutely normal in any PNG viewer.
Any clues what I can do to resolve it? Is it a bug In MTLTextureLoader? Is there another way of loading PNG picture?
P.S. The software is written for MacOS and not iOS.
I found the problem. It seems that some of the PNG files were stored in RGB (24 bit) format, while another were stored in RGBA (32 bit) format. All color became right after converting images to 32bit format.
I would expect MTKTextureLoader to be able to handle that, but apparently it is not. If someone knows the option I missed or another way to correctly load those PNG's to Metal (Other than adding 4th channel manually) it will be great!
I try to add a B&W filter to the camera images of an ARSCNView, then render colored AR objects over it.
I'am almost there with the following code added to the beginning of - (void)renderer:(id<SCNSceneRenderer>)aRenderer updateAtTime:(NSTimeInterval)time
CVPixelBufferRef bg=self.sceneView.session.currentFrame.capturedImage;
if(bg){
char* k1 = CVPixelBufferGetBaseAddressOfPlane(bg, 1);
if(k1){
size_t x1 = CVPixelBufferGetWidthOfPlane(bg, 1);
size_t y1 = CVPixelBufferGetHeightOfPlane(bg, 1);
memset(k1, 128, x1*y1*2);
}
}
This works really fast on mobile, but here's the thing: sometimes a colored frame is displayed.
I've checked and my filtering code is executed but I assume it's too late, SceneKit's pipeline already processed camera input.
Calling the code earlier would help, but updateAtTime is the earliest point one can add custom frame by frame code.
Getting notifications on frame captures might help, but looks like the whole AVCapturesession is unaccessible.
The Metal ARKit example shows how to convert the camera image to RGB and that is the place where I would do filtering, but that shader is hidden when using SceneKit.
I've tried this possible answer but it's way too slow.
So how can I overcome the frame misses and convert the camera feed reliably to BW?
Here's the key for this problem:
session:didUpdateFrame:
Provides a newly captured camera image and accompanying AR information to the delegate.
So just moved CVPixelBufferRef manipulation, the image filtering code from
- (void)renderer:(id<SCNSceneRenderer>)aRenderer updateAtTime:(NSTimeInterval)time
to
- (void)session:(ARSession *)session didUpdateFrame:(ARFrame *)frame
Made sure to set self.sceneView.session.delegate = self to have this delegate called.
I'm working with the PImage class. Normally I make 2 PImage objects, load an image into one of them (my input picture) and create a blank image using createImage(), which will become the output. I then use the loadPixels() method to access the data on the input, do some manipulation then set the respective output pixel to the result. I have not had any trouble with this so far.
The dimensions of the input and the output PImage objects need to be the same to make the pixel-by-pixel manipulations as straight forward as possible.
So here is the pickle:
PImage myinput;
PImage myoutput;
void setup() {
size(350, 350);
myinput = loadImage("myfile.jpg");
// the pic is 300 x 300
//myoutput = createImage(myinput.width, myinput.height, RGB);
//I've hardcoded the width and height below
myoutput = createImage(300, 300, RGB);
}
void draw() {
image(myoutput, 0, 0);
}
The result of the above is a black square 300 x 300 which overlaps a grey canvas of 350 x 350. Given the code I've written, this is the result I would expect.
Now, in the above example, I've hardcoded the width and height of 'myoutput' with the line:
myoutput = createImage(300, 300, RGB);
My question relates to the bit that follows:
Instead of hardcoding the values, I would rather do something like this:
myoutput = createImage(myinput.width, myinput.height, RGB);
But it isn't working. I just get a big 350 x 350 grey box. And I'm not sure why. Though I do have my suspicions. When I work with pictures in javascript, I've got wait for the page to load (using an event listener like window.onload() {} etc.) before I can access the width/height properties of the image.
UPDATE:
I saw another post which had the following:
/* #pjs preload="myfile.jpg"; */
So I just included this before I declared my PImage objects and now the following line works.
myoutput = createImage(myinput.width, myinput.height, RGB);
I'm quite confused by the new piece of code.
When you run your sketch in Java mode, you're running as Java. Java loads images synchronously, which means that the code won't continue running until the image is fully loaded. That's why it works in Java mode.
But when you're running using Processing.js, you're running as JavaScript. JavaScript loads images asynchronously, which means that the image is loaded in the background while your code continues. That means you aren't guaranteed that the image is done loading when the next line executes, which is why the image's width and height are unset.
The preload command tells Processing.js to load the images before the sketch starts executing, so that you're guaranteed that the image loads before you try to access its width and height.
From the Processing.js reference:
This directive regulates image preloading, which is required when using loadImage() or requestImage() in a sketch. Using this directive will preload all images indicated between quotes, and comma separated if multiple images are used, so that they will be ready for use when the sketch begins running. As resources are loaded via the AJAX approach, not using this directive will result in the sketch loading an image, and then immediately trying to use this image in some way, even though the browser has not finished downloading and caching it.
I have an application that displays graphs and because the results are often interesting (either due to bugs or intentionally) I wan't an ability to quickly save screenshots. So I made a screenshot button.
I used the code from wxWidgets forum FAQ but unfortunatelly, this method only saves images on screenshots (applies on fullscreen screenshot as well). Everything else is left transparent.
For some reason, this only happens with PNG image export. Exporting as BMP or JPG is just fine.
There must be something wrong with:
screenshot.SaveFile("image.png", wxBITMAP_TYPE_PNG);
I have the PNG procesor loaded in wxWidgets:
wxImage::AddHandler(new wxPNGHandler);
Code:
//Create a DC for the main window
wxClientDC dcScreen(GetParent());
//Get the size of the screen/DC
wxCoord screenWidth, screenHeight;
dcScreen.GetSize(&screenWidth, &screenHeight);
//Create a Bitmap that will later on hold the screenshot image
//Note that the Bitmap must have a size big enough to hold the screenshot
//-1 means using the current default colour depth
screenshot.Create(screenWidth, screenHeight,-1);
//Create a memory DC that will be used for actually taking the screenshot
wxMemoryDC memDC;
//Tell the memory DC to use our Bitmap
//all drawing action on the memory DC will go to the Bitmap now
memDC.SelectObject(screenshot);
//Blit (in this case copy) the actual screen on the memory DC
//and thus the Bitmap
memDC.Blit( 0, //Copy to this X coordinate
0, //Copy to this Y coordinate
screenWidth, //Copy this width
screenHeight, //Copy this height
&dcScreen, //From where do we copy?
0, //What's the X offset in the original DC?
0 //What's the Y offset in the original DC?
);
//Select the Bitmap out of the memory DC by selecting a new
//uninitialized Bitmap
memDC.SelectObject(wxNullBitmap);
Images:
Instead of (made with Alt+PrintSreen in windows):
If the image comes out correctly in BMP but not PNG, the problem is probably due to the transparency, i.e. somehow all the rest of the image must have its alpha channel set to wxIMAGE_ALPHA_TRANSPARENT. If this is really the case, then using
wxImage image = bmp.ConvertToImage();
image.ClearAlpha();
image.SaveFile("foo.png", wxBITMAP_TYPE_PNG);
should help, but I still have no idea why would it be transparent in the first place.
If this still happens with wxWidgets 3.0 (currently RC2 is available, final will be next week) and if you can find a simple of reproducing the problem, it would be worth reporting it as a bug.
You should initialize the bitmap to any object like staticbitmap before this line:
memDC.SelectObject(wxNullBitmap);
I am trying to extract all the images in a PDF and then convert them into DIB format. First part is easy. I extract all the contents in the PDF, then iterate through them and whenever I find a PDEImage, I put them in an array.
But I am clueless about how to go about the second part. Looks like all the AVConversion methods allow you to convert a whole page of a PDF, not just images, into other formats.
Is there any way I can accomplish this task? Thanks in advance!
EDIT: Further elaborating the problem.
I am writing an Adobe Acrobat Plug-in using Visual C++ with .NET Framework 4.
The purpose of the plug-in is to (among other things) extract image data from a PDF file, then convert those data to DIBs. The need to convert to DISs is because I then pass those DIBs to another library which do some image correction work on them.
Now my problem is with converting the said image data in PDFs to DIBs. The image data on PDFs are found in a format called PDEImage (Ref Link) where apparently it contains all the color data of the image. Now I'm using the following code to extract the said image data bits from the image to be used with CreateCompatibleBitmap() and SetBitmapBits() to obtain a HBITMAP handle. Then, I pass that along with other necessary parameters to the GetDIBits() to obtain a DIB in the form of a byte array as stated in the MSDN.
void GetDIBImage(PDEElement element)
{
//Obtaining a PDEImage
PDEImage image;
memset(&image, 0, sizeof(PDEImage));
image = (PDEImage)element;
//Obtaining attributes (such as width, height)
//of the image for later use
PDEImageAttrs attrs;
memset(&attrs, 0, sizeof(attrs));
PDEImageGetAttrs(image, &attrs, sizeof(attrs));
//Obtainig image data from PDEImage to a byte array
ASInt32 len = PDEImageGetDataLen(image);
byte *data = (byte *)malloc(len);
PDEImageGetData(image, 0, data);
//Creating a DDB using said data
HDC hdc = CreateCompatibleDC(NULL);
HBITMAP hBmp = CreateCompatibleBitmap(hdc, attrs.width, attrs.height);
LONG bitsSet = SetBitmapBits(hBmp, len, data); //Here bitsSet gets a value of 59000 which is close to the image's actual size
//Buffer which GetDIBits() will fill with DIB data
unsigned char* buff = new unsigned char[len];
//BITMAPINFO stucture to be passed to GetDIBits()
BITMAPINFO bmpInfo;
memset(&bmpInfo, 0, sizeof(bmpInfo));
bmpInfo.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
bmpInfo.bmiHeader.biWidth = (LONG)attrs.width;
bmpInfo.bmiHeader.biHeight = (LONG)attrs.height;
bmpInfo.bmiHeader.biPlanes = 1;
bmpInfo.bmiHeader.biBitCount = 8;
bmpInfo.bmiHeader.biCompression = BI_RGB;
bmpInfo.bmiHeader.biSizeImage = ((((bmpInfo.bmiHeader.biWidth * bmpInfo.bmiHeader.biBitCount) + 31) & ~31) >> 3) * bmpInfo.bmiHeader.biHeight;
bmpInfo.bmiHeader.biXPelsPerMeter = 0;
bmpInfo.bmiHeader.biYPelsPerMeter = 0;
bmpInfo.bmiHeader.biClrUsed = 0;
bmpInfo.bmiHeader.biClrImportant = 0;
//Callling GetDIBits()
//Here scanLines get a value of 0, while buff receives no data.
int scanLines = GetDIBits(hdc, hBmp, 0, attrs.height, &buff, &bmpInfo, DIB_RGB_COLORS);
if(scanLines > 0)
{
MessageBox(NULL, L"SUCCESS", L"Message", MB_OK);
}
else
{
MessageBox(NULL, L"FAIL", L"Message", MB_OK);
}
}
Here are my questions / concerns.
Is it correct the way I'm using CreateCompatibleDC(), CreateCompatibleBitmap() and SetBitmapBits() functions? My thinking is that I use CreateCompatibleDC() to obtain current DC, then create a DDB using CreateCompatibleBitmap() and then set the actual data to the DDB using SetBitmapBits(). Is that correct?
Is there a problem with the way I've created the BITMAPINFO structure. I am under the assumption that it need to contain all the details regarding the format of the DIB I will eventually obtain.
Why am I not getting the bitmap data as a DIB to the buff when I call GetDIBits()?
I do not know the library you use to access the inner structures of a PDF file but the problem at hand will have tree distinct subproblems:
Find all images in the PDF file
Decode the images to their components
Convert the decoded image to a DIB
Find all Images
Images can occur inside content streams or in streams attached to dictionaries. To find all images in content streams, you need to find all content streams in either Pages, XObjects or Patterns. Each of those can have a Resources -> XObject dictionary that references all XObjects (and an XObject can be an Image).
If you avoid the inline images you might simply scan the PDF file and each dectionary that is of type XObject subtype Image can be decoded.
Decode
All streams (inline in content streams) of in separate objects in the PDF file are encoded and mught need post processing using the Decode arrays. There are several filters that you need to be able to perform for decoding. Flate decode (ZLIB), JPEG and CCITT (fax G3/G4) are probable the most used for images. Hopefully the PDF library you use will know how to decode the streams..
Next there are Decode arrays (a bit rare) where each color component can be scaled from an input value to an output value. This is a linear transformation.
To DIB
Next in line is the conversion of the decoded image to a DIB. This means you need to convert the color components to something Windows can 'get' (eg, Palette, grayscale (special palette) of RGB. PDF supports a very very large variety of color spaces and converting them to RGB is no sinecure. You best hope here is that the PDFs you need to process only use a select subset (like RGB and palette). Now a DIB can be simply created by creating the bitmap header (BITMAPINFO), fill in all data and call the DIB creation function CreateDIBSection and them process the DIB the way you application needs.
Epilogue
All in all: to be able to process all PDF files and find all images is quite a daunting task, if you control the source if teh PDFs and you know they are always in DeviceRGB format and always JPEG etc and never inlined into the content stream it is do-able.