I am writing a EbGL based HTML application that uses ASTC (Adaptive Scalable Texture Compression) compressed textures to be loaded on my triangle. I would like to know that does there exists a way to know whether the internal format of the compressed ASTC image(that in my case might be located on a remote web server) is "linear" or "srgb encoded", by parsing the ASTC header. I can then use that internalFormat information obtained to pass my ASTC texture to glCompressedTexImage2D(). In other words, for eg. I want to know whether my internal format is COMPRESSED_RGBA_ASTC_4x4_KHR or COMPRESSED_SRGB8_ALPHA8_ASTC_4x4_KHR from the header of any ASTC compressed image. Any clues?
It seems that ASTC file header indeed doesn't fully describe its contents. It has only dimensions and some strange 'magic number' which seems to be just a constant.
Information about file header: http://malideveloper.arm.com/downloads/Stacy_ASTC_white%20paper.pdf (pages 4-5, it also refers to code samples from Mali Developer Center for more clues).
'Magic number' explained here as a mere constant value 0x5CA1AB13:
http://community.arm.com/thread/3981
You should ask a question at Mali Developer Center forums - these guys are very helpful and usually respond quite fast.
EDIT: Header format in case external links go down:
struct astc_header
{
uint8_t magic [ 4 ];
uint8_t blockdim_x;
uint8_t blockdim_y;
uint8_t blockdim_z ;
uint8_t xsize [ 3 ];
uint8_t ysize [ 3 ];
uint8_t zsize [ 3 ];
};
The ASTC header has no such information. You could try a file name extension, perhaps, such as .srgb.astc. Using KTX, the Khronos alternative storage container for ASTC data, you can add key-value pair for whatever you like, though the glInternalFormat should be good enough in this case.
That said, if you store your data in the ASTC file as sRGB (non-linear gamma) then you can choose whether you want the data to be read as non-linear gamma or linear gamma by setting COMPRESSED_RGBA_ASTC_4x4_KHR or COMPRESSED_SRGB8_ALPHA8_ASTC_4x4_KHR when you read it. That is, the sRGB-ness probably shouldn't be considered a property of the file format but a property of the texel read operation and/or your graphics pipeline. Use the appropriate kind to produce the style of output you want.
Related
im trying to use nanopb, according to the example:
https://github.com/nanopb/nanopb/blob/master/examples/simple/simple.c
the buffer size is initialized to 128:
uint8_t buffer[128];
my question is how do i know (in advance) this 128-length buffer is enough to transmit my message? how to decide a proper(enough but not waste too much due to over-large) size of buffer before initial (or coding) it?
looks like a noob question :) , but thx for your quick suggestion.
When possible, nanopb adds a define in the generated .pb.h file that has the maximum encoded size of a message. In the file examples/simple/simple.pb.h you'll find:
/* Maximum encoded size of messages (where known) */
#define SimpleMessage_size 11
And could specify uint8_t buffer[SimpleMessage_size];.
This define will be available only if all repeated and string fields have been specified (nanopb).max_count and (nanopb).max_size options.
For many practical purposes, you can pick a buffer size that you estimate will be large enough, and handle error conditions. It is also possible to use pb_get_encoded_size() to calculate the encoded size and dynamically allocate storage, but in general that is not a great solution in embedded applications. When total system memory size is limited, it is often better to have a constant sized buffer that you can test with, instead of having the available amount of dynamic memory vary at the runtime.
I have a compute shader that reads a signed normalized integer image using imageLoad.
The image itself (which contains both positive and negative values) is created as a R16G16_SNORM and is written by a fragment shader in a previous gpass.
The imageview bound to the descriptorsetlayout binding in the compute shader is also created with the same R16G16_SNORM format.
Everything works as expected.
Yesterday I realized that in the compute shader I used the wrong image format qualifier rg16.
A bit puzzled (I could not understand how it could work properly reading an unsigned normalized value) I corrected to rg16_snorm, and.. nothing changed.
I performed several tests (I even specified a rg16f) and always had the same (correct, [-1,1] signed) result.
It seems like Vulkan (at least my implementation) silently ignores any image format qualifier, and falls back (I guess) to the imageview format bound to the descriptorset.
This seems to be in line with the spec regarding format in imageview creation
format is a VkFormat describing the format and type used to interpret texel blocks in the image
but then in Appendix A (Vulkan Environment for SPIR-V - "Compatibility Between SPIR-V Image Formats And Vulkan Formats") there is a clear distinction between Rg16 and Rg16Snorm.. so:
is it a bug or a feature?
I am working with an Nvidia 2070 Super under ubuntu 20.04
UPDATE
The initial image writing operation happens as the result of a fragment shader color attachment output, and as such, there is no descriptorsetlayout binding declaration. The fragment shader outputs a vec2 to the R16G16_SNORM color attachment as specified by the active framebuffer and renderpass.
The resulting image (after the relevant barriers) is then read (correctly, despite the wrong layout qualifier) by a compute shader as an image/imageLoad operation.
Note that validation layers are enabled and silent.
Note also that the resulting values are far from random, and exactly match the expected values (both positive and negative), using either rg16, rg16f or rg16_snorm.
What you're getting is undefined behavior.
There is a validation check on Image Write Operations that prevents the OpTypeImage's format (equivalent to the layout format specifier in GLSL) from being incompatible with the backing VkImageView's format:
If the image format of the OpTypeImage is not compatible with the VkImageView’s format, the write causes the contents of the image’s memory to become undefined.
Note that when it says "compatible", it doesn't mean image view compatibility; it means "exactly match". Your OpTypeImage format did not exactly match that of the shader, so your writes were undefined. And "undefined" can mean "works as if you had specified the correct format".
WebCodecs is released in Chrome 86. But there's no real code example on how to use it yet. Given a video url, how to extract video frames as ImageData using webcodecs?
What you describe is the entire complex process of acquiring raw bitmap-like data (e.g. something you can dump on a canvas), from a formatted file or a stream of data chunks.
In case of files (including the case where your URL points to a complete file, such as an .mp4 file), this is generally made of 2 steps:
Parsing the container file into individual chunks of encoded video and/or audio
Decoding these chunks of encoded video/audio
WebCodecs only facilitates step 2 of this process, i.e. what is called decoding. The reasoning behind this decision was that parsing the container is computationally trivial, so you can efficiently do this with the File APIs already, but you still need to implement parsing/processing the container yourself.
Luckily, plenty of libraries exist already, many of which ironically existed long before the emergence of the WebCodecs API.
MP4Box is one example, helping you acquire encoded video and audio chunks, which you can then feed into a VideoDecoder or AudioDecoder.
With MP4Box, the key piece of your code will be centered around the onSamples callback you provide, and it'll look something like this:
mp4BoxFile.onSamples = (trackId, user, chunks) =>
{
for (let i = 0; i < chunks.length; i++)
{
let chunk = chunks[i];
let encodedChunk = new EncodedVideoChunk({
// you'll need to deep-inspect chunk to figure these out
type: "key", // or "delta"
timestamp: ...
duration: ...
data: chunk.data
});
// pass encodedChunk to a VideoDecoder instance's decode method
}
};
This is just a rough sketch of how your code will probably look, it probably won't work without more inspection, and it'll take a lot of trial and error, because this is very low level stuff.
WebCodecs is not the silver bullet you probably expected, but it can help you build one.
I'm using the bincode crate to write a structure into a file. The structure contains a slice with a fixed size. How can I force bincode to write only the slice's content without the slice's length?
#![allow(unstable)]
#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]
extern crate serde;
extern crate bincode;
use std::fs::File;
use bincode::serde::serialize_into;
use bincode::SizeLimit;
#[derive(Serialize)]
struct Foo([u8; 16]);
fn main() {
let data = Foo([0; 16]);
let mut writer = File::create("test.bin").unwrap();
serialize_into::<File, Foo>(&mut writer, &data, SizeLimit::Infinite).unwrap();
}
File 'test.bin' has 24 bytes size instead of 16.
I saw related remark in documentation of bincode, but I did not understand how to use it.
a slice with a fixed size
[u8; 16] is not a slice. It is an array which may be coerced to a slice.
Anyway... I do not believe that you can. The important function appears to be Serializer::serialize_fixed_size_array which is not implemented by the current serializer. That means it defaults to behaving the same as a slice.
Since slices do not have a length known at compile time, they must have their size written when serialized.
If no one else can provide a better suggestion, it's possible that the maintainer could find a way to make this happen. You may want to politely ask the maintainer for this feature or offer to help with the work.
Beyond that, it sounds like you are trying to make the bincode output fit a pre-existing format. That doesn't really make sense; bincode is its own format and had already made various choices and tradeoffs.
If you need to, you could implement your own encoder / decoder (either using serde or not). If you are concerned about file size, you can combine bincode with a compression step as well.
I have read (after running into the limitation myself) that for copying data from the host to a VK_IMAGE_TILING_OPTIMAL VkImage, you're better off using a VkBuffer rather than a VkImage for the staging image to avoid restrictions on mipmap and layer counts. (Here and Here)
So, when it came to implementing a glReadPixels-esque piece of functionality to read the results of a render-to-texture back to the host, I thought that reading to a staging VkBuffer with vkCmdCopyImageToBuffer instead of using a staging VkImage would be a good idea.
However, I haven't been able to get it to work yet, I'm seeing most of the intended image, but with rectangular blocks of the image in incorrect locations and even some bits duplicated.
There is a good chance that I've messed up my synchronization or layout transitions somewhere and I'll continue to investigate that possibility.
However, I couldn't figure out from the spec whether using vkCmdCopyImageToBuffer with an image source using VK_IMAGE_TILING_OPTIMAL is actually supposed to 'un-tile' the image, or whether I should actually expect to receive a garbled implementation-defined image layout if I attempt such a thing.
So my question is: Does vkCmdCopyImageToBuffer with a VK_IMAGE_TILING_OPTIMAL source image fill the buffer with linearly tiled data or optimally (implementation defined) tiled data?
Section 18.4 describes the layout of the data in the source/destination buffers, relative to the image being copied from/to. This is outlined in the description of the VkBufferImageCopy struct. There is no language in this section which would permit different behavior from tiled images.
The specification even has pseudo code for how copies work (this is for non-block compressed images):
rowLength = region->bufferRowLength;
if (rowLength == 0)
rowLength = region->imageExtent.width;
imageHeight = region->bufferImageHeight;
if (imageHeight == 0)
imageHeight = region->imageExtent.height;
texelSize = <texel size taken from the src/dstImage>;
address of (x,y,z) = region->bufferOffset + (((z * imageHeight) + y) * rowLength + x) * texelSize;
where x,y,z range from (0,0,0) to region->imageExtent.width,height,depth}.
The x,y,z part is the location of the pixel in question from the image. Since this location is not dependent on the tiling of the image (as evidenced by the lack of anything stating that it would be), buffer/image copies will work equally on both kinds of tiling.
Also, do note that this specification is shared between vkCmdCopyImageToBuffer and vkCmdCopyBufferToImage. As such, if a copy works one way, it by necessity must work the other.