Rendering a cube in Vulkan vs OpenGL - vulkan

I wrote a simple OpenGL program which merely renders a cube from an angle. It's as simple as you can get: vertex buffer only (no index buffer), a vertex shader which only multiplies the vertices by an MVP matrix from a uniform buffer, and a static fragment shader which just returns red. More recently, I have tried writing this same program in Vulkan, but I have run into some issues.
I started by following the Intel API without secrets tutorial to setup a simple 2d texture rendering program, but when I took the leap into 3d, I started having issues. In order to debug this, I simplified the program to match my older OpenGL program (removed texturing and some other extra stuff I did in Vulkan), and even went as far as to use the exact same vertex and MVP data. However, I just can't get the cube to render correctly in Vulkan.
I am aware that OpenGL coordinates do not map directly to Vulkan coordinates, as the Y coordinate is flipped, but if anything that should just flip the image upside down, and I already tried switching the Y values in the MVP. I feel like there is some other detail I am missing here with coordinates, but I just can't figure it out searching around and looking at guides about converting OpenGL code bases to Vulkan.
I'm including the data I am uploading to the shaders, and some of the core code from the Vulkan code base. The Vulkan code is in D, so it's similar to C++, but a little different. With the library I'm using for wrapping Vulkan (erupted), the device level functions are loaded into a device dispatch (access as device.dispatch in the code), and when they are called on the dispatch without the vk prefix, the device and command buffer (which is assigned to the dispatch in code) arguments of the function are auto populated.
Vertex Data:
[ [1, 1, 1, 1],
[1, 1, -1, 1],
[-1, 1, -1, 1],
[1, 1, 1, 1],
[-1, 1, -1, 1],
[-1, 1, 1, 1],
[1, 1, 1, 1],
[1, -1, 1, 1],
[1, -1, -1, 1],
[1, 1, 1, 1],
[1, -1, -1, 1],
[1, 1, -1, 1],
[1, 1, -1, 1],
[1, -1, -1, 1],
[-1, -1, -1, 1],
[1, 1, -1, 1],
[-1, -1, -1, 1],
[-1, 1, -1, 1],
[-1, 1, -1, 1],
[-1, -1, -1, 1],
[-1, -1, 1, 1],
[-1, 1, -1, 1],
[-1, -1, 1, 1],
[-1, 1, 1, 1],
[-1, 1, 1, 1],
[-1, -1, 1, 1],
[1, -1, 1, 1],
[-1, 1, 1, 1],
[1, -1, 1, 1],
[1, 1, 1, 1],
[1, -1, 1, 1],
[1, -1, -1, 1],
[-1, -1, -1, 1],
[1, -1, 1, 1],
[-1, -1, -1, 1],
[-1, -1, 1, 1] ]
MVP:
[ [-1.0864, -0.993682, -0.687368, -0.685994],
[0, 2.07017, 0.515526, -0.514496],
[-1.44853, 0.745262, 0.515526, 0.514496],
[-8.04095e-16, 0, 5.64243, 5.83095] ]
Graphics Pipeline Setup:
VkPipelineShaderStageCreateInfo[] shader_stage_infos = [
{
stage: VK_SHADER_STAGE_VERTEX_BIT,
_module: vertex_shader,
pName: "main"
},
{
stage: VK_SHADER_STAGE_FRAGMENT_BIT,
_module: fragment_shader,
pName: "main"
}
];
VkVertexInputBindingDescription[] vertex_binding_descriptions = [
{
binding: 0,
stride: VertexData.sizeof,
inputRate: VK_VERTEX_INPUT_RATE_VERTEX
}
];
VkVertexInputAttributeDescription[] vertex_attribute_descriptions = [
{
location: 0,
binding: vertex_binding_descriptions[0].binding,
format: VK_FORMAT_R32G32B32A32_SFLOAT,
offset: VertexData.x.offsetof
},
{
location: 1,
binding: vertex_binding_descriptions[0].binding,
format: VK_FORMAT_R32G32_SFLOAT,
offset: VertexData.u.offsetof
}
];
VkPipelineVertexInputStateCreateInfo vertex_input_state_info = {
vertexBindingDescriptionCount: vertex_binding_descriptions.length.to!uint,
pVertexBindingDescriptions: vertex_binding_descriptions.ptr,
vertexAttributeDescriptionCount: vertex_attribute_descriptions.length.to!uint,
pVertexAttributeDescriptions: vertex_attribute_descriptions.ptr
};
VkPipelineInputAssemblyStateCreateInfo input_assembly_state_info = {
topology: VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST,
primitiveRestartEnable: VK_FALSE
};
VkPipelineViewportStateCreateInfo viewport_state_info = {
viewportCount: 1,
pViewports: null,
scissorCount: 1,
pScissors: null
};
VkPipelineRasterizationStateCreateInfo rasterization_state_info = {
depthBiasClamp: 0.0,
polygonMode: VK_POLYGON_MODE_FILL,
cullMode: VK_CULL_MODE_FRONT_AND_BACK,
frontFace: VK_FRONT_FACE_COUNTER_CLOCKWISE,
lineWidth: 1
};
VkPipelineMultisampleStateCreateInfo multisample_state_info = {
rasterizationSamples: VK_SAMPLE_COUNT_1_BIT,
minSampleShading: 1
};
VkPipelineColorBlendAttachmentState[] color_blend_attachment_states = [
{
blendEnable: VK_FALSE,
srcColorBlendFactor: VK_BLEND_FACTOR_ONE,
dstColorBlendFactor: VK_BLEND_FACTOR_ZERO,
colorBlendOp: VK_BLEND_OP_ADD,
srcAlphaBlendFactor: VK_BLEND_FACTOR_ONE,
dstAlphaBlendFactor: VK_BLEND_FACTOR_ZERO,
alphaBlendOp: VK_BLEND_OP_ADD,
colorWriteMask:
VK_COLOR_COMPONENT_R_BIT |
VK_COLOR_COMPONENT_G_BIT |
VK_COLOR_COMPONENT_B_BIT |
VK_COLOR_COMPONENT_A_BIT
}
];
VkPipelineColorBlendStateCreateInfo color_blend_state_info = {
logicOpEnable: VK_FALSE,
logicOp: VK_LOGIC_OP_COPY,
attachmentCount: color_blend_attachment_states.length.to!uint,
pAttachments: color_blend_attachment_states.ptr,
blendConstants: [ 0, 0, 0, 0 ]
};
VkDynamicState[] dynamic_states = [
VK_DYNAMIC_STATE_VIEWPORT,
VK_DYNAMIC_STATE_SCISSOR
];
VkPipelineDynamicStateCreateInfo dynamic_state_info = {
dynamicStateCount: dynamic_states.length.to!uint,
pDynamicStates: dynamic_states.ptr
};
VkGraphicsPipelineCreateInfo pipeline_info = {
stageCount: shader_stage_infos.length.to!uint,
pStages: shader_stage_infos.ptr,
pVertexInputState: &vertex_input_state_info,
pInputAssemblyState: &input_assembly_state_info,
pTessellationState: null,
pViewportState: &viewport_state_info,
pRasterizationState: &rasterization_state_info,
pMultisampleState: &multisample_state_info,
pDepthStencilState: null,
pColorBlendState: &color_blend_state_info,
pDynamicState: &dynamic_state_info,
layout: pipeline_layout,
renderPass: render_pass,
subpass: 0,
basePipelineHandle: VK_NULL_HANDLE,
basePipelineIndex: -1
};
VkPipeline[1] pipelines;
checkVk(device.dispatch.CreateGraphicsPipelines(VK_NULL_HANDLE, 1, [pipeline_info].ptr, pipelines.ptr));
pipeline = pipelines[0];
Drawing:
if(device.dispatch.WaitForFences(1, [fence].ptr, VK_FALSE, 1000000000) != VK_SUCCESS)
throw new StringException("timed out waiting for fence");
device.dispatch.ResetFences(1, [fence].ptr);
uint image_index;
switch(device.dispatch.AcquireNextImageKHR(swapchain.swapchain, uint64_t.max, image_available_semaphore, VK_NULL_HANDLE, &image_index)) {
case VK_SUCCESS:
case VK_SUBOPTIMAL_KHR:
break;
case VK_ERROR_OUT_OF_DATE_KHR:
on_window_size_changed();
break;
default:
throw new StringException("unhandled vk result on swapchain image acquisition");
}
if(framebuffer != VK_NULL_HANDLE) device.dispatch.DestroyFramebuffer(framebuffer);
VkFramebufferCreateInfo framebuffer_info = {
renderPass: swapchain.render_pass,
attachmentCount: 1,
pAttachments: [swapchain.image_resources[image_index].image_view].ptr,
width: swapchain.extent.width,
height: swapchain.extent.height,
layers: 1
};
checkVk(device.dispatch.CreateFramebuffer(&framebuffer_info, &framebuffer));
VkCommandBufferBeginInfo cmd_begin_info = { flags: VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT };
VkImageSubresourceRange image_subresource_range = {
aspectMask: VK_IMAGE_ASPECT_COLOR_BIT,
baseMipLevel: 0,
levelCount: 1,
baseArrayLayer: 0,
layerCount: 1,
};
VkImageMemoryBarrier barrier_from_present_to_draw = {
srcAccessMask: VK_ACCESS_MEMORY_READ_BIT,
dstAccessMask: VK_ACCESS_MEMORY_READ_BIT,
oldLayout: VK_IMAGE_LAYOUT_PRESENT_SRC_KHR,
newLayout: VK_IMAGE_LAYOUT_PRESENT_SRC_KHR,
srcQueueFamilyIndex: device.present_queue.family_index,
dstQueueFamilyIndex: device.graphics_queue.family_index,
image: swapchain.image_resources[image_index].image,
subresourceRange: image_subresource_range
};
VkImageMemoryBarrier barrier_from_draw_to_present = {
srcAccessMask: VK_ACCESS_MEMORY_READ_BIT,
dstAccessMask: VK_ACCESS_MEMORY_READ_BIT,
oldLayout: VK_IMAGE_LAYOUT_PRESENT_SRC_KHR,
newLayout: VK_IMAGE_LAYOUT_PRESENT_SRC_KHR,
srcQueueFamilyIndex: device.graphics_queue.family_index,
dstQueueFamilyIndex: device.present_queue.family_index,
image: swapchain.image_resources[image_index].image,
subresourceRange: image_subresource_range
};
VkViewport viewport = {
x: 0,
y: 0,
width: swapchain.extent.width,
height: swapchain.extent.height,
minDepth: 0,
maxDepth: 1
};
VkRect2D scissor = {
offset: {
x: 0,
y: 0
},
extent: swapchain.extent
};
VkClearValue[] clear_values = [
{ color: { [ 1.0, 0.8, 0.4, 0.0 ] } }
];
VkRenderPassBeginInfo render_pass_begin_info = {
renderPass: swapchain.render_pass,
framebuffer: framebuffer,
renderArea: {
offset: {
x: 0,
y: 0
},
extent: swapchain.extent
},
clearValueCount: clear_values.length.to!uint,
pClearValues: clear_values.ptr
};
device.dispatch.commandBuffer = command_buffer;
device.dispatch.BeginCommandBuffer(&cmd_begin_info);
if(device.graphics_queue.handle != device.present_queue.handle)
device.dispatch.CmdPipelineBarrier(
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
0, 0, null, 0, null, 1,
&barrier_from_present_to_draw
);
device.dispatch.CmdBeginRenderPass(&render_pass_begin_info, VK_SUBPASS_CONTENTS_INLINE);
device.dispatch.CmdBindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, swapchain.pipeline);
device.dispatch.CmdSetViewport(0, 1, &viewport);
device.dispatch.CmdSetScissor(0, 1, &scissor);
const(ulong) vertex_buffer_offset = 0;
device.dispatch.CmdBindVertexBuffers(0, 1, &vertex_buffer, &vertex_buffer_offset);
device.dispatch.CmdBindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline_layout, 0, 1, &descriptor_set, 0, null);
device.dispatch.CmdDraw(draw_count, 1, 0, 0);
device.dispatch.CmdEndRenderPass();
if(device.graphics_queue.handle != device.present_queue.handle)
device.dispatch.CmdPipelineBarrier(
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT,
0, 0, null, 0, null, 1,
&barrier_from_draw_to_present
);
checkVk(device.dispatch.EndCommandBuffer());
device.dispatch.commandBuffer = VK_NULL_HANDLE;
VkSubmitInfo submit_info = {
waitSemaphoreCount: 1,
pWaitSemaphores: [image_available_semaphore].ptr,
pWaitDstStageMask: castFrom!(VkPipelineStageFlagBits*).to!(const(uint)*)([VK_PIPELINE_STAGE_TRANSFER_BIT].ptr),
commandBufferCount: 1,
pCommandBuffers: [command_buffer].ptr,
signalSemaphoreCount: 1,
pSignalSemaphores: [rendering_finished_semaphore].ptr
};
checkVk(device.dispatch.vkQueueSubmit(device.graphics_queue.handle, 1, [submit_info].ptr, fence));
VkPresentInfoKHR present_info = {
waitSemaphoreCount: 1,
pWaitSemaphores: [rendering_finished_semaphore].ptr,
swapchainCount: 1,
pSwapchains: [swapchain.swapchain].ptr,
pImageIndices: [image_index].ptr
};
switch(device.dispatch.vkQueuePresentKHR(device.present_queue.handle, &present_info)) {
case VK_SUCCESS:
break;
case VK_ERROR_OUT_OF_DATE_KHR:
case VK_SUBOPTIMAL_KHR:
on_window_size_changed();
break;
default:
throw new StringException("unhandled vk result on presentation");
}
(I can't embed the images because my rep is too low, sorry)
Program Outputs:
OpenGL draws the cube as expected
OpenGL Output
Vulkan does not render anything except for the clear color.
UPDATE:
After fixing the cull mode by changing it to VK_CULL_MODE_NONE, this is the result I get:
Output after cull mode fix

VK_CULL_MODE_FRONT_AND_BACK
I think this is your problem :)

After cull mode fix, seems that your problem in your vertex data layout. Vulkan expects (accordingly to your layout binding) something like
struct Vertex {
vec4 x;
vec2 u;
};
Vertex VertexData[] = {...};
because you set VK_VERTEX_INPUT_RATE_VERTEX in your vertex_binding_descriptions.inputRate field.
And it seems that in your case you should set VK_VERTEX_INPUT_RATE_INSTANCE instead to work with buffers after each other.
Fix: Have seen your new comment, it looks like i misunderstood your vertices layout, so it won't help.

Related

Identify vectors being a multiple of another in rectangular matrix

Given a nxm matrix (n > m) of integers, I'd like to identify rows that are a multiple of a single other row, so not a linear combination of multiple other rows.
I could scale all rows to their length and find unique rows, but that is prone to numerical issues on floating points and would also not detect vectors being opposite (pointing in the other directon) of each other.
Any ideas?
Example
A = array([[-1, -1, 0, 0],
[-1, -1, 0, 1],
[-1, 0, -1, 0],
[-1, 0, 0, 0],
[-1, 0, 0, 1],
[-1, 0, 1, 1],
[-1, 1, -1, 0],
[-1, 1, 0, 0],
[-1, 1, 1, 0],
[ 0, -1, 0, 0],
[ 0, -1, 0, 1],
[ 0, -1, 1, 0],
[ 0, -1, 1, 1],
[ 0, 0, -1, 0],
[ 0, 0, 0, 1],
[ 0, 0, 1, 0],
[ 0, 1, -1, 0],
[ 0, 1, 0, 0],
[ 0, 1, 0, 1],
[ 0, 1, 1, 0],
[ 0, 1, 1, 1],
[ 1, -1, 0, 0],
[ 1, -1, 1, 0],
[ 1, 0, 0, 0],
[ 1, 0, 0, 1],
[ 1, 0, 1, 0],
[ 1, 0, 1, 1],
[ 1, 1, 0, 0],
[ 1, 1, 0, 1],
[ 1, 1, 1, 0]])
For example Rows 0 and -3 just point in the opposite direction (multiply one by -1 to make them equal).
You can normalize each row dividing it by its GCD:
import numpy as np
def normalize(a):
return a // np.gcd.reduce(a, axis=1, keepdims=True)
And you can define a distance that considers opposite vectors as equal:
def distance(a, b):
equal = np.all(a == b) or np.all(a == -b)
return 0 if equal else 1
Then you can use standard clustering methods:
from scipy.spatial.distance import pdist
from scipy.cluster.hierarchy import linkage, fcluster
def cluster(a):
norm_a = normalize(a)
distances = pdist(norm_a, metric=distance)
return fcluster(linkage(distances), t=0.5)
For example:
>>> A = np.array([( 1, 2, 3, 4),
... ( 0, 2, 4, 8),
... (-1, -2, -3, -4),
... ( 0, 1, 2, 4),
... (-1, 2, -3, 4),
... ( 2, -4, 6, -8)])
>>> cluster(A)
array([2, 3, 2, 3, 1, 1], dtype=int32)
Interpretation: cluster 1 is formed by rows 4 and 5, cluster 2 by rows 0 and 2, and cluster 3 by rows 1 and 3.
You can take advantage of the fact that inner product of two normalized linearly dependent vectors gives 1 or -1, so the code could look like this:
>>> A_normalized = (A.T/np.linalg.norm(A, axis=-1)).T
>>> M = np.absolute(np.einsum('ix,jx->ij', A_normalized, A_normalized))
>>> i, j = np.where(np.isclose(M, 1))
>>> i, j = i[i < j], j[i < j] # Remove repetitions
>>> print(i, j)
output: [ 0 2 3 6 7 9 11 13] [27 25 23 22 21 17 16 15]

CoreML: Failed in 2nd reshape after missing custom layer info

I our very big Tensorflow (2.x) model there is a part where we use InceptionResNetV2 and create multi-pooled model from it:
model_base = InceptionResNetV2(weights = 'imagenet',
include_top = False,
input_shape = input_shape)
ImgResizer = Lambda(lambda x: tf.image.resize(x, pool_size, method='area'),
name='feature_resizer')
feature_layers = [l for l in model_base.layers if 'mixed' in l.name]
feature_layers = [feature_layers[i] for i in indexes]
pools = [ImgResizer(l.output) for l in feature_layers]
conc_pools = Concatenate(name='conc_pools', axis=3)(pools)
model = Model(inputs = model_base.input, outputs = conc_pools)
Here we use Lambda function to resize 4D tensor ([batch, height, width, channels]) to a new 4D tensor with pool_size (5,5), input_shape=(None, None, 3) and indexes=list(range(43)). Unfortunately, this operation (ResizeArea) is not supported in the current version of coremltools (5.0) and I have to do a custom operation wrapper and then implement it on a swift.
#register_op(doc_str='Custom ResizeArea Layer', is_custom_op=True)
class custom_resize_area(Operation):
input_spec = InputSpec(
x = TensorInputType(),
s = ScalarOrTensorInputType()
)
bindings = { 'class_name' : 'CustomResizeArea',
'input_order' : ['x', 's'],
'parameters' : [],
'description' : "Resize area custom layer"
}
def __init__(self, **kwargs):
super(custom_resize_area, self).__init__(**kwargs)
def type_inference(self):
x_type = self.x.dtype
x_shape = self.x.shape
s = list(self.s.val)
ret_shape = list(x_shape)
ret_shape[1] = s[0]
ret_shape[2] = s[1]
#print(x_shape, ret_shape)
return types.tensor(x_type, ret_shape)
# Override ResizeArea op with override=True flag
#register_tf_op(tf_alias=['ResizeArea'], override=True)
def CustomResizeArea(context, node):
#input: "model_2/mixed_5b/concat"
#input: "model_2/lambda/resize/size"
x = context[node.inputs[0]]
s = context[node.inputs[1]]
x = mb.custom_resize_area(x=x, s=s, name=node.name)
context.add(node.name, x)
The conversion was successful and now I started to implement this custom layer on swift. Here is a part from my code:
#objc(CustomResizeArea) class CustomResizeArea: NSObject, MLCustomLayer {
required init(parameters: [String : Any]) throws {
super.init()
}
func setWeightData(_ weights: [Data]) throws {}
func outputShapes(forInputShapes inputShapes: [[NSNumber]]) throws
-> [[NSNumber]] {
print(#function, inputShapes)
let outputShape = inputShapes[0]
// [ sequence, batch, channel, height, width ] - why?
if outputShape.count == 5 {
print(outputShape)
return [outputShape]
}
print([outputShape[0], 5, 5, outputShape[3]])
return [[outputShape[0], 5, 5, outputShape[3]]]
}
...
}
I am publishing only outputShapes function since the CoreML gives me an error before evaluate function will actually be executed. Here the last logs (with some output shapes parameters inside):
outputShapes(forInputShapes:) [[0, 0, 0, 0, 0], [0, 0, 0, 0, 0]]
[0, 0, 0, 0, 0]
outputShapes(forInputShapes:) [[0, 0, 0, 0, 0], [0, 0, 0, 0, 0]]
[0, 0, 0, 0, 0]
.....
outputShapes(forInputShapes:) [[1, 0, 0, 448], [2]]
[1, 5, 5, 448]
outputShapes(forInputShapes:) [[1, 0, 0, 448], [2]]
[1, 5, 5, 448]
outputShapes(forInputShapes:) [[1, 0, 0, 448], [2]]
[1, 5, 5, 448]
2021-11-06 10:15:55.085694+0100 snafu[11263:4717917] [espresso] [Espresso::handle_ex_plan] exception=Failed in 2nd reshape after missing custom layer info.
2021-11-06 10:15:55.086108+0100 snafu[11263:4717917] [coreml] Error in adding network -1.
2021-11-06 10:15:55.086477+0100 snafu[11263:4717917] [coreml] MLModelAsset: load failed with error Error Domain=com.apple.CoreML Code=0 "Error in declaring network." UserInfo={NSLocalizedDescription=Error in declaring network.}
I have no idea why I get [[0, 0, 0, 0, 0], [0, 0, 0, 0, 0]] as an inout, and why are there zeros in, for example, [[1, 0, 0, 448], 2]. Actually my model should support any image size on input, that's why? I think I have tried everything...
P.S.:
I also tried to replace area reize method with bilinear in the python code (Lambda) because there is a BilinearResize converter in coremltools. But I can not convert it, because I get the following error:
File "/home/alex/anaconda3/envs/tfgpu/lib/python3.8/site-packages/coremltools/converters/mil/mil/builder.py", line 75, in _add_const
raise ValueError("Cannot add const {}".format(val))
ValueError: Cannot add const 5.0001/is57
I suppose that 5.0001 here is my pool_size, but I don't understand why is it incorrect.

Is there a way to slice out multiple 2D numpy arrays from one 2D numpy array in one batch operation?

I have a numpy array heatmap of shape (img_height, img_width) and another array bboxes of shape (K, 4), where K is a number of bounding boxes.
Each bounding box is defined
like so: [x_top_left, y_top_left, width, height].
Here's an example of such array:
bboxes = np.array([
[0, 0, 4, 7],
[3, 4, 3, 4],
[7, 2, 3, 7]
])
heatmap is initally filled with zeros.
What I need to do is to put value 1 for each bounding box in it's corresponding place.
The resulting heatmap should be:
heatmap = np.array([
[1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0],
[1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0],
[1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0],
[1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0],
[1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0],
[0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0],
[0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0],
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
])
Important things to note:
axis 0 corresponds to image height
axis 1 corresponds to image width
I've already solved this using python for loop, like so:
for bbox in bboxes:
# y_top_left:y_top_left + img_height, x_top_left:x_top_left + img_width
heatmap[bbox[1] : bbox[1] + bbox[3], bbox[0] : bbox[0] + bbox[2]] = 1
I would like to avoid using python for loops (if it's possible) and be able to do something like this:
heatmap[bboxes[:,1] : bboxes[:,1] + bboxes[:,3], bboxes[:,0]:bboxes[:,0] + bboxes[:,2]] = 1
Is there a way of doing such multiple slicing in numpy?
I am aware of numpy integer array indexing, but to generate such indices I am also unable to avoid python for loops.

Custom tetris block placement algorithm

Well it's not entirely a generic problem, but that's why we're here after all.
A bit of background
My task is to create a tetris like simulation with an AI playing it. The lines do not disappear when they are completed. The end result should be a matrix filled with the neatly placed blocks, with very few or no gaps.
What I chose to do, was a genetic approach, with constant weights and methods for evaluation. The AI would try to place the blocks in all possible places, rotations, evaluate the temporary matrices, and go with the best one.
The problem
In tetris, you can move to the left or right even when the block is on the ground. This allows to solve many positions that would otherwise be impossible. The real problem however, that these holes can even occur mid-air, something like this:
falling J shape, with optimal choice occurring mid-air
The only solution I see here, would be trying all positions, rotations, and all possible combinations of mid-air moves, which I assume is "not an optimal solution" to say it formally.
My question
Is if someone has an idea or another approach, to find these possibilities for placement with realistic amounts of computing power
A single piece can be positioned in a maximum of 10x20x4 = 800 positions on a single board. These will be the nodes of your graph. Where you can move from one node to another in a single move, there is an edge. You can then prune nodes that are illegal (e.g. overlap with existing obstacles on the board). You can also mark nodes as "final" - there, the tile has an obstacle directly under at least one part of it (but they can still have nodes connected to them).
You can then check which final nodes are reachable from the initial node, which is a standard problem.
You could optimize this further by ignoring nodes where the piece is above the current height of the board.
Example code:
import copy
import time
from os import system, name
from random import randint
from wrapt import synchronized
from asciimatics.screen import Screen
import asciimatics
from asciimatics.effects import Cycle, Stars
from asciimatics.renderers import FigletText
from asciimatics.scene import Scene
from asciimatics.screen import Screen
class Tile:
shapes = \
[
[
[0, 0, 2],
[2, 2, 2]
],
[
[3, 3, 0],
[0, 3, 3]
],
[
[0, 4, 4],
[4, 4, 0]
],
[
[5, 0, 0],
[5, 5, 5]
],
[
[6, 6],
[6, 6]
],
[
[0, 0, 0, 0],
[7, 7, 7, 7],
[0, 0, 0, 0]
],
[
[0, 8, 0],
[8, 8, 8]
]
]
def __init__(self, id=-1):
if id >= 0:
self.id = id
else:
self.id = randint(0, len(self.shapes)-1)
self.shape = self.shapes[id]
x = 8
y = 0
id = 0
def rotate(self):
self.shape = list(zip(*self.shape[::-1]))
class Model:
_height = 25
_width = 20
_score = 0
def __init__(self):
self._view = None
self._field = [[0] * self._width for i in range(self._height)]
for i in range(5):
for j in range(self._height):
self._field[j][i] = 1
self._field[j][-i-1] = 1
for i in range(5):
for j in range(self._width):
self._field[-i-1][j] = 1
self._tile = Tile()
self._nexttile = Tile()
def set_view(self, view):
self._view = view
def get_height(self):
i = 0
for r in self._field[:-5]:
full_line = True
if sum(r[5:-5]) > 0:
return i
i += 1
return i
def _merge(self, field, tile):
field_copy = copy.deepcopy(field)
for i in range(len(tile.shape)):
for j in range(len(tile.shape[0])):
field_copy[tile.y + i][tile.x + j] += tile.shape[i][j]
return field_copy
#synchronized
def _is_valid(self, field, tile):
for i in range(len(tile.shape)):
for j in range(len(tile.shape[0])):
if tile.shape[i][j] > 0:
if (field[tile.y + i][tile.x + j] > 0):
return False
return True
def get_board(self):
return self._merge(self._field, self._tile)
def rotate(self):
self._tile.rotate()
if not self._is_valid(self._field, self._tile):
self._tile.rotate()
self._tile.rotate()
self._tile.rotate()
if self._view is not None:
self._view.show()
def _which_lines_completed(self):
lines = []
i = 0
for r in self._field[:-5]:
full_line = True
for c in r:
if c == 0:
full_line = False
if full_line:
lines.append(i)
i += 1
return lines
def _remove_lines(self, lines):
for l in lines:
for i in list(range(1, l+1))[::-1]:
self._field[i] = self._field[i-1].copy()
if len(lines) == 4:
self._score += 5000
elif len(lines) == 3:
self._score += 1000
elif len(lines) == 2:
self._score += 500
elif len(lines) == 1:
self._score += 100
#synchronized
def move_down(self):
self._tile.y += 1
if not self._is_valid(self._field, self._tile):
self._tile.y -= 1
self._field = self._merge(self._field, self._tile)
self._tile = self._nexttile
self._nexttile = Tile()
# Check if any lines need to be removed
lines = self._which_lines_completed()
# If lines need to be removed, notify the view
if len(lines) > 0:
self._view.remove_lines(lines)
# Remove the lines
self._remove_lines(lines)
if self._view is not None:
self._view.show()
#synchronized
def move_left(self):
self._tile.x -= 1
if not self._is_valid(self._field, self._tile):
self._tile.x += 1
else:
if self._view is not None:
self._view.show()
#synchronized
def move_right(self):
self._tile.x += 1
if not self._is_valid(self._field, self._tile):
self._tile.x -= 1
if self._view is not None:
self._view.show()
class AsciimaticView:
def __init__(self, model):
self.screen = Screen.open()
self.model = model
def _show_board(self, board):
#self.screen.clear()
b = board
x = 0
y = 0
for r in b[:-4]:
x = 0
for c in r[4:-4]:
if c == 1:
self.screen.print_at(u'██', x, y, Screen.COLOUR_BLUE, Screen.A_BOLD)
elif c == 2:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_RED, Screen.A_BOLD)
elif c == 3:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_GREEN, Screen.A_BOLD)
elif c == 4:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_GREEN, Screen.A_BOLD)
elif c == 5:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_RED, Screen.A_BOLD)
elif c == 6:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_CYAN, Screen.A_BOLD)
elif c == 7:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_YELLOW, Screen.A_BOLD)
elif c == 8:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_MAGENTA, Screen.A_BOLD)
else:
self.screen.print_at(u' ', x, y, Screen.COLOUR_BLUE, Screen.A_BOLD)
x += 2
y += 1
self.screen.print_at(u' ', 0, y, Screen.COLOUR_RED, Screen.A_BOLD)
self.screen.print_at(u' ', 0, y+1, Screen.COLOUR_RED, Screen.A_BOLD)
self.screen.print_at(u' ', 0, y+2, Screen.COLOUR_RED, Screen.A_BOLD)
self.screen.print_at(u' ', 0, y+3, Screen.COLOUR_RED, Screen.A_BOLD)
for i in range(len(self.model._nexttile.shape)):
x = 0
if (i == 1):
self.screen.print_at(u'Next: ', x, y, Screen.COLOUR_WHITE, Screen.A_BOLD)
x = x + 6
for j in range(len(self.model._nexttile.shape[0])):
c = self.model._nexttile.shape[i][j]
if c == 1:
self.screen.print_at(u'██', x, y, Screen.COLOUR_BLUE, Screen.A_BOLD)
elif c == 2:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_RED, Screen.A_BOLD)
elif c == 3:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_GREEN, Screen.A_BOLD)
elif c == 4:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_GREEN, Screen.A_BOLD)
elif c == 5:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_RED, Screen.A_BOLD)
elif c == 6:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_CYAN, Screen.A_BOLD)
elif c == 7:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_YELLOW, Screen.A_BOLD)
elif c == 8:
self.screen.print_at(u'[]', x, y, Screen.COLOUR_MAGENTA, Screen.A_BOLD)
else:
self.screen.print_at(u' ', x, y, Screen.COLOUR_BLUE, Screen.A_BOLD)
x = x + 2
y = y + 1
x = 0
y = 24
self.screen.print_at(u'Score: ' + str(self.model._score), x, y, Screen.COLOUR_WHITE, Screen.A_BOLD)
self.screen.refresh()
def show(self):
self._show_board(self.model.get_board())
def remove_lines(self, lines):
b = self.model.get_board()
for i in range(5):
for l in lines:
b[l][5:-5] = [1-el for el in b[l][5:-5]]
self._show_board(b)
time.sleep(0.1)
class Node:
x = 0
y = 0
rot = 0
final = False
edges = []
def __eq__(self, other):
"""Overrides the default implementation"""
if isinstance(other, Node):
return (self.x == other.x) and (self.y == other.y) and (self.rot == other.rot)
return False
def is_neighbour(self, other):
if (abs(self.x - other.x) + abs(self.y - other.y) + abs(self.rot - other.rot) == 1) and (other.y >= self.y):
return True
return False
def __hash__(self):
return hash((self.x, self.y, self.rot))
def get_possible_moves(model, tile):
start_node = Node()
start_node.x = model._tile.x
start_node.y = model.get_height() - len(tile.shape)-1
frontier = [start_node]
visited = {start_node: True}
final_nodes = []
while len(frontier) > 0:
n = frontier.pop()
for dx, dy, rot in [(-1, 0, 0), (1, 0, 0), (0, 1, 0), (0, 0, 1)][::-1]:
nn = Node()
nn.x = n.x + dx
nn.y = n.y + dy
nn.rot = (n.rot + rot) % 4
if nn not in visited:
visited[nn] = True
t = Tile(tile.id)
t.x = nn.x
t.y = nn.y
for r in range(nn.rot):
t.rotate()
if model._is_valid(model._field, t):
frontier.append(nn)
# check if node is final
for i in range(len(t.shape)):
for j in range(len(t.shape[0])):
if (t.shape[i][j] > 0) and (model._field[nn.y + i + 1][nn.x + j] > 0):
nn.final = True
final_nodes.append(nn)
break
if (nn.final):
break
print(len(visited))
print(len(final_nodes))
return final_nodes
m = Model()
m._field = [
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 3, 3, 3, 3, 3, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 3, 3, 0, 3, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 3, 3, 3, 3, 3, 0, 0, 0, 0, 3, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 3, 3, 3, 3, 3, 0, 0, 0, 3, 3, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
]
m._tile = Tile(3)
import threading
import keyboard
# define a thread which takes input
class InputThread(threading.Thread):
def run(self):
#self.daemon = True
self.last_user_input = None
while True:
try:
if keyboard.is_pressed('left'):#if key 'q' is pressed
self.last_user_input = 'a'
m.move_left()
elif keyboard.is_pressed('right'):#if key 'q' is pressed
self.last_user_input = 'a'
m.move_right()
elif keyboard.is_pressed('z'):
print('z')
self.last_user_input = 'z'
m.rotate()
elif keyboard.is_pressed('down'):
m.move_down()
elif keyboard.is_pressed('q'):
break
else:
pass
# do something based on the user input here
# alternatively, let main do something with
# self.last_user_input
except:
pass
time.sleep(0.1)
# main
#it = InputThread()
#it.start()
fn = get_possible_moves(m, m._tile)
time.sleep(2)
v = AsciimaticView(m)
m.set_view(v)
v.show()
time.sleep(2)
for n in fn:
m._tile = Tile(m._tile.id)
m._tile.x = n.x
m._tile.y = n.y
for r in range(n.rot):
m._tile.rotate()
v.show()
time.sleep(1)
time.sleep(500)

Lazy High Charts: change colors of a pie chart [rails 3]

I'm working in a rails 3 project, I want to change the pie chart colors that I generate with the LazyHighChart gem and I don;t know how to do that
This is my method controller
def set_pie_chart(data)
fixed_data = []
data.each_pair do |key, value|
fixed_data << [key.name, value]
end
#color = data.keys.map {|e| "#" + e.colour } # e.colour is like '333333'
#chart = LazyHighCharts::HighChart.new('pie') do |c|
c.chart({:defaultSeriesType=>"pie" , :margin=> [0, 0, 0, 0]})
series = {
type: 'pie',
name: 'total expenses',
data: fixed_data,
colors: ['green','red'] # intent
}
c.series(series)
c.colors = ['red','blue','black'] # intent
c.options[:colors] = ['green','blue','yellow'] # intent
c.options['colors'] = ['red','blue','yellow'] # intent
c.options[:title][:text] = nil
c.plot_options(:pie=>{
cursor: "pointer",
center: ['50%','37%'],
color: 'red', #intent
dataLabels: { enabled: false }
})
end
end
this method doesn't leave any error, what is the correct way or its not possible with this gem?
or what other good alternative gems could I use for my project?
I had the same problem and I solved it by putting the color options in my view. This is what I did:
#my_view_helper.rb
def chart_colors
html =
"[{linearGradient: [0, 0, 0, 200], stops: [[0, '#e28b02'],[1, '#f1bc70']]},
{linearGradient: [0, 0, 0, 200], stops: [[0, '#a5ba57'],[1, '#a8bb51']]},
{linearGradient: [0, 0, 0, 200], stops: [[0, '#1e93d6'],[1, '#35aff6']]},
{linearGradient: [0, 0, 0, 200], stops: [[0, '#c8cf99'],[1, '#cdcfa9']]},
{linearGradient: [0, 0, 0, 200], stops: [[0, '#709ab1'],[1, '#a7c5d0']]},
{linearGradient: [0, 0, 0, 200], stops: [[0, '#c76f4e'],[1, '#fba98e']]},
{linearGradient: [0, 0, 0, 200], stops: [[0, '#95d6e3'],[1, '#bbe8ed']]}]"
html
end
#my_view.html.haml
#chart
= high_chart("pie_1", #chart) do |c|
= "options.colors = #{chart_colors}".html_safe
#chart = LazyHighCharts::HighChart.new('pie') do |c|
c.colors(["red","green","blue"]);
end