I would like to use a VNImageRequestHandler with raw pixels data. The method initWithData:options: of VNImageRequestHandler is not documented, and I can't find any example using this. What is the accepted format, and what is this "options" parameter ?
Apple documentation (empty) page : https://developer.apple.com/documentation/vision/vnimagerequesthandler/2866551-initwithdata?language=objc
Note that a CVPixelBuffer is basically a wrapper around raw pixel data. You can easily create one that points at your own pixel data using CVPixelBufferCreateWithBytes().
Related
I'm using the bincode crate to write a structure into a file. The structure contains a slice with a fixed size. How can I force bincode to write only the slice's content without the slice's length?
#![allow(unstable)]
#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]
extern crate serde;
extern crate bincode;
use std::fs::File;
use bincode::serde::serialize_into;
use bincode::SizeLimit;
#[derive(Serialize)]
struct Foo([u8; 16]);
fn main() {
let data = Foo([0; 16]);
let mut writer = File::create("test.bin").unwrap();
serialize_into::<File, Foo>(&mut writer, &data, SizeLimit::Infinite).unwrap();
}
File 'test.bin' has 24 bytes size instead of 16.
I saw related remark in documentation of bincode, but I did not understand how to use it.
a slice with a fixed size
[u8; 16] is not a slice. It is an array which may be coerced to a slice.
Anyway... I do not believe that you can. The important function appears to be Serializer::serialize_fixed_size_array which is not implemented by the current serializer. That means it defaults to behaving the same as a slice.
Since slices do not have a length known at compile time, they must have their size written when serialized.
If no one else can provide a better suggestion, it's possible that the maintainer could find a way to make this happen. You may want to politely ask the maintainer for this feature or offer to help with the work.
Beyond that, it sounds like you are trying to make the bincode output fit a pre-existing format. That doesn't really make sense; bincode is its own format and had already made various choices and tradeoffs.
If you need to, you could implement your own encoder / decoder (either using serde or not). If you are concerned about file size, you can combine bincode with a compression step as well.
I have read (after running into the limitation myself) that for copying data from the host to a VK_IMAGE_TILING_OPTIMAL VkImage, you're better off using a VkBuffer rather than a VkImage for the staging image to avoid restrictions on mipmap and layer counts. (Here and Here)
So, when it came to implementing a glReadPixels-esque piece of functionality to read the results of a render-to-texture back to the host, I thought that reading to a staging VkBuffer with vkCmdCopyImageToBuffer instead of using a staging VkImage would be a good idea.
However, I haven't been able to get it to work yet, I'm seeing most of the intended image, but with rectangular blocks of the image in incorrect locations and even some bits duplicated.
There is a good chance that I've messed up my synchronization or layout transitions somewhere and I'll continue to investigate that possibility.
However, I couldn't figure out from the spec whether using vkCmdCopyImageToBuffer with an image source using VK_IMAGE_TILING_OPTIMAL is actually supposed to 'un-tile' the image, or whether I should actually expect to receive a garbled implementation-defined image layout if I attempt such a thing.
So my question is: Does vkCmdCopyImageToBuffer with a VK_IMAGE_TILING_OPTIMAL source image fill the buffer with linearly tiled data or optimally (implementation defined) tiled data?
Section 18.4 describes the layout of the data in the source/destination buffers, relative to the image being copied from/to. This is outlined in the description of the VkBufferImageCopy struct. There is no language in this section which would permit different behavior from tiled images.
The specification even has pseudo code for how copies work (this is for non-block compressed images):
rowLength = region->bufferRowLength;
if (rowLength == 0)
rowLength = region->imageExtent.width;
imageHeight = region->bufferImageHeight;
if (imageHeight == 0)
imageHeight = region->imageExtent.height;
texelSize = <texel size taken from the src/dstImage>;
address of (x,y,z) = region->bufferOffset + (((z * imageHeight) + y) * rowLength + x) * texelSize;
where x,y,z range from (0,0,0) to region->imageExtent.width,height,depth}.
The x,y,z part is the location of the pixel in question from the image. Since this location is not dependent on the tiling of the image (as evidenced by the lack of anything stating that it would be), buffer/image copies will work equally on both kinds of tiling.
Also, do note that this specification is shared between vkCmdCopyImageToBuffer and vkCmdCopyBufferToImage. As such, if a copy works one way, it by necessity must work the other.
I am writing a EbGL based HTML application that uses ASTC (Adaptive Scalable Texture Compression) compressed textures to be loaded on my triangle. I would like to know that does there exists a way to know whether the internal format of the compressed ASTC image(that in my case might be located on a remote web server) is "linear" or "srgb encoded", by parsing the ASTC header. I can then use that internalFormat information obtained to pass my ASTC texture to glCompressedTexImage2D(). In other words, for eg. I want to know whether my internal format is COMPRESSED_RGBA_ASTC_4x4_KHR or COMPRESSED_SRGB8_ALPHA8_ASTC_4x4_KHR from the header of any ASTC compressed image. Any clues?
It seems that ASTC file header indeed doesn't fully describe its contents. It has only dimensions and some strange 'magic number' which seems to be just a constant.
Information about file header: http://malideveloper.arm.com/downloads/Stacy_ASTC_white%20paper.pdf (pages 4-5, it also refers to code samples from Mali Developer Center for more clues).
'Magic number' explained here as a mere constant value 0x5CA1AB13:
http://community.arm.com/thread/3981
You should ask a question at Mali Developer Center forums - these guys are very helpful and usually respond quite fast.
EDIT: Header format in case external links go down:
struct astc_header
{
uint8_t magic [ 4 ];
uint8_t blockdim_x;
uint8_t blockdim_y;
uint8_t blockdim_z ;
uint8_t xsize [ 3 ];
uint8_t ysize [ 3 ];
uint8_t zsize [ 3 ];
};
The ASTC header has no such information. You could try a file name extension, perhaps, such as .srgb.astc. Using KTX, the Khronos alternative storage container for ASTC data, you can add key-value pair for whatever you like, though the glInternalFormat should be good enough in this case.
That said, if you store your data in the ASTC file as sRGB (non-linear gamma) then you can choose whether you want the data to be read as non-linear gamma or linear gamma by setting COMPRESSED_RGBA_ASTC_4x4_KHR or COMPRESSED_SRGB8_ALPHA8_ASTC_4x4_KHR when you read it. That is, the sRGB-ness probably shouldn't be considered a property of the file format but a property of the texel read operation and/or your graphics pipeline. Use the appropriate kind to produce the style of output you want.
Just now i start a small work related to video capture I want all supported pixel formats by a video capture device using avfoundation. I am using CMFormatDescriptionGetMediaSubtype function to get pixel format in mac(CMPixelFormatType). But i am always getting a value which is not matching from any of the available pixel format (predefined pixel fromat like: kCMPixelFormat_422YpCbCr8_yuvs). Is there any other function present for that purpose.. I searched but till now not found. Returned value is always 1684890161 as pixel format type.
Actual thing I want to ask is :
How can I have all supported pixel formats by a video capture device in Mac. Remember I am not asking for codec type. Its pixel format I need. Please help.. if possible.
Thanks in advance..
To get all supported pixel formats, you need to call CMIOObjectGetPropertyData with the second parameter CMIOObjectPropertyAddress as follows:
CMIOObjectPropertyAddress theAddress;
theAddress.mSelector = kCMIOStreamPropertyFormatDescriptions;
theAddress.mScope = kCMIOObjectPropertyScopeGlobal;
theAddress.mElement = 0;
This will give you a CFArray of CMFormatDescriptionRef.
Then you iterate over this array and call:
CMFormatDescriptionGetMediaSubType(yourCMFormatDescriptionRef);
Your CMFormatDescriptionGetMediaSubType translates into a FourCharacterCode (OSType) of 'dmb1' which seems to be a Matrox specific hardware compression type.
The reason, you won't find your format in the CMPixelFormatType enum is because there are only the most common formats there, which yours is not. The most common on the Mac are kCMPixelFormat_422YpCbCr8 and kCMPixelFormat_422YpCbCr10, both are raw 8 and 10 bit YUV video.
For example, there are QR scanners which scan video stream in real time and get QR codes info.
I would like to check the light source from the video, if it is on or off, it is quite powerful so it is no problem.
I will probably take a video stream as input, maybe make images of it and analyze images or stream in real time for presence of light source (maybe number of pixels of certain color on the image?)
How do I approach this problem? Maybe there is some source of library?
It sounds like you are asking for information about several discreet steps. There are a multitude of ways to do each of them and if you get stuck on any individual step it would be a good idea to post a question about it individually.
1: Get video Frame
Like chaitanya.varanasi said, AVFoundation Framework is the best way of getting access to an video frame on IOS. If you want something less flexible and quicker try looking at open CV's video capture. The goal of this step is to get access to a pixel buffer from the camera. If you have trouble with this, ask about it specifically.
2: Put pixel buffer into OpenCV
This part is really easy. If you get it from openCV's video capture you are already done. If you get it from an AVFoundation you will need to put it into openCV like this
//Buffer is of type CVImageBufferRef, which is what AVFoundation should be giving you
//I assume it is BGRA or RGBA formatted, if it isn't, change CV_8UC4 to the appropriate format
CVPixelBufferLockBaseAddress( Buffer, 0 );
int bufferWidth = CVPixelBufferGetWidth(Buffer);
int bufferHeight = CVPixelBufferGetHeight(Buffer);
unsigned char *pixel = (unsigned char *)CVPixelBufferGetBaseAddress(Buffer);
cv::Mat image = cv::Mat(bufferHeight,bufferWidth,CV_8UC4,pixel); //put buffer in open cv, no memory copied
//Process image Here
//End processing
CVPixelBufferUnlockBaseAddress( pixelBuffer, 0 );
note I am assuming you plan to do this in OpenCV since you used its tag. Also I assume you can get the OpenCV framework to link to your project. If that is an issue, ask a specific question about it.
3: Process Image
This part is by far the most open ended. All you have said about your problem is that you are trying to detect a strong light source. One very quick and easy way of doing that would be to detect the mean pixel value in a greyscale image. If you get the image in colour you can convert with cvtColor. Then just call Avg on it to get the mean value. Hopefully you can tell if the light is on by how that value fluctuates.
chaitanya.varanasi suggested another option, you should check it out too.
openCV is a very large library that can do a wide wide variety of things. Without knowing more about your problem I don't know what else to tell you.
Look at the AVFoundation Framework from Apple.
Hope it helps!
You can try this method: start by getting all images to an AVCaptureVideoDataOutput. From the method:captureOutput:didOutputSampleBuffer:fromConnection,you can sample/calculate every pixel. Source: answer
Also, you can take a look at this SO question where they check if a pixel is black. If its such a powerful light source, you can take the inverse of the pixel and then determine using a set threshold for black.
The above sample code only provides access to the pixel values stored in the buffer; you cannot run any other commands but those that change those values on a pixel-by-pixel basis:
for ( uint32_t y = 0; y < height; y++ )
{
for ( uint32_t x = 0; x < width; x++ )
{
bgraImage.at<cv::Vec<uint8_t,4> >(y,x)[1] = 0;
}
}
This—to use your example—will not work with the code you provided:
cv::Mat bgraImage = cv::Mat( (int)height, (int)extendedWidth, CV_8UC4, base );
cv::Mat grey = bgraImage.clone();
cv::cvtColor(grey, grey, 44);