VTK cutter output - line

I am seeking a solution of connecting all the lines that have the same slope and share a common point. For example, after I load a STL file and cut it using a plane, the cutter output includes the points defining the contour. Connecting them one by one forms a (or multiple) polyline. However, some lines can be merged when their slopes are the same and they share a common point. E.g., [[0,0,0],[0,0,1]] and [[0,0,1],[0,0,2]] can be represented by one single line [[0,0,0],[0,0,2]].
I wrote a function that can analyse all the lines and connect them if they can be merged. But when the number of lines are huge, this process is slow. I am thinking in the VTK pipeline, is there a way to do the line merging?
Cheers!
plane = vtk.vtkPlane()
plane.SetOrigin([0,0,5])
plane.SetNormal([0,0,1])
cutter = vtk.vtkCutter()
cutter.SetCutFunction(plane)
cutter.SetInput(triangleFilter.GetOutput())
cutter.Update()
cutStrips = vtk.vtkStripper()
cutStrips.SetInputConnection(cutter.GetOutputPort())
cutStrips.Update()
cleanDataFilter = vtk.vtkCleanPolyData()
cleanDataFilter.AddInput(cutStrips.GetOutput())
cleanDataFilter.Update()
cleanData = cleanDataFilter.GetOutput()
print cleanData.GetPoint(0)
print cleanData.GetPoint(1)
print cleanData.GetPoint(2)
print cleanData.GetPoint(3)
print cleanData.GetPoint(4)
The output is:
(0.0, 0.0, 5.0)
(5.0, 0.0, 5.0)
(10.0, 0.0, 5.0)
(10.0, 5.0, 5.0)
(10.0, 10.0, 5.0)
Connect the above points one by one will form a polyline representing the cut result. As we can see, the line [point0, point1] and [point1, point2] can be merged.
Below is the code for merging the lines:
Assume that the LINES are represented by list: [[(p0),(p1)],[(p1),(p2)],[(p2),(p3)],...]
appended = 0
CurrentLine = LINES[0]
CurrentConnectedLine = CurrentLine
tempLineCollection = LINES[1:len(LINES)]
while True:
for HL in tempLineCollection:
QCoreApplication.processEvents()
if checkParallelAndConnect(CurrentConnectedLine, HL):
appended = 1
LINES.remove(HL)
CurrentConnectedLine = ConnectLines(CurrentConnectedLine, HL)
processedPool.append(CurrentConnectedLine)
if len(tempLineCollection) == 1:
processedPool.append(tempLineCollection[0])
LINES.remove(CurrentLine)
if len(LINES) >= 2:
CurrentLine = LINES[0]
CurrentConnectedLine = CurrentLine
tempLineCollection = LINES[1:len(LINES)]
appended = 0
else:
break
Solution:
I figured out a way of further accelerating this process using some vtk data structure. I found out that a polyline line will be stored in a cell, which can be checked by using GetCellType(). Since the point order for a polyline is sorted already, We do not need to search globally which lines are colinear with the current one. For each point on the polyline, I just need to check the point[i-1], point[i], point[i+1]. And if they are colinear, the end of the line will be updated to the next point. This process continues until the end of the polyline is reached. The speed increases by a huge amount compared with the global search approach.

Not sure if it is the main source of slowness (depends on how many positive hits on the colinearity you have), but removing items from a vector is costly (O(n)), since it requires reorganizing the rest of the vector, you should avoid it. But even without hits on colinearity, the LINES.remove(CurrentLine) call is surely slowing things down and there isn't really any need for it - just leave the vector untouched, write the final results to a new vector (processedPool) and get rid of the LINES vector in the end. You can modify your algorithm by making a bool array (vector), initialized at "false" for each item, then when you remove a line, you don't actually remove it, but only mark it as "true" and you skip all lines for which you have "true", i.e. something like this (I don't speak python so the syntax is not accurate):
wasRemoved = bool vector of the size of LINES initialized at false for each entry
for CurrentLineIndex = 0; CurrentLineIndex < sizeof(LINES); CurrentLineIndex++
if (wasRemoved[CurrentLineIndex])
continue // skip a segment that was already removed
CurrentConnectedLine = LINES[CurrentLineIndex]
for HLIndex = CurrentLineIndex + 1; HLIndex < sizeof(LINES); HLIndex++:
if (wasRemoved[HLIndex])
continue;
HL = LINES[HLIndex]
QCoreApplication.processEvents()
if checkParallelAndConnect(CurrentConnectedLine, HL):
wasRemoved[HLIndex] = true
CurrentConnectedLine = ConnectLines(CurrentConnectedLine, HL)
processedPool.append(CurrentConnectedLine)
wasRemoved[CurrentLineIndex] = true // this is technically not needed since you won't go back in the vector anyway
LINES = processedPool
BTW, the really correct data structure for LINES to use for that kind of algorithm would be a linked list, since then you would have O(1) complexity for removal and you wouldn't need the boolean array. But a quick googling showed that that's not how lists are implemented in Python, also don't know if it would not interfere with other parts of your program. Alternatively, using a set might make it faster (though I would expect times similar to my "bool array" solution), see python 2.7 set and list remove time complexity
If this does not do the trick, I suggest you measure times of individual parts of the program to find the bottleneck.

Related

Best way to parallelize multi-table function in Python (using Pandas)

I have this function below that iterates through every row of a data frame (using pandas apply) and determines what values are valid from a prediction-probability matrix (L2) by referencing another data frame (GST) to obtain the valid values for a given row. The function just returns the row back with the maximum valid probability assigned to the previously blank value for that row (Predicted Level 2) in the data frame passed to the function (test_x2)
Not a terribly complex function and it works fine on smaller datasets but when I scale to like 3-5 million records, it starts to take way too long. I tried using the multiprocessing module as well as dask/numba but nothing was able to improve the runtime (not sure if this is just due to the fact the function is not vectorizable).
My question is two fold:
1) Is there a better way to write this? (I'm guessing there is)
2) If not, what parallel computing strategies could work with this type of function? I've already tried a number of different python options but I'm just leaning more towards running the larger datasets on totally separate machines at this point. Feel free to provide any suggested code to parallelize something like this. Thanks in advance for any guidance provided.
l2 = MNB.predict_proba(test_x)
l2_classes = MNB.classes_
L2 = pd.DataFrame(l2, columns = MNB.classes_)
test_x2["Predicted Level 2"] = ""
def predict_2(row):
s = row["Predicted Level 1"]
s = GST.loc[s,:]
s.reset_index(inplace = True)
Valid_Level2s = s["GST Level 2"].tolist()
p2 = L2.ix[row.name, Valid_Level2s]
max2 = p2.idxmax(axis = 1)
output = row["Predicted Level 2"] = max2
return row
test_x2 = test_x2.apply(predict_2, axis = 1)

Maya: Having trouble writing a script to cut a mesh into equal pieces

I want to split a mesh into sections based on a number of vertices. Essentially, I want a mesh cut into sections of 300 verts each with a remainder section of whatever is left over.
I've done this for the most part (i can get verts/faces, etc) but I'm having trouble figuring a graceful way of iterating through the extracted meshes.
I'm using polyChipOff which has no return value for the faces it chipped, so it's entirely new objects that are created that i have no handle to so i can't just continue chipping away from the previous piece as it no longer exists.
Any advice on how to go about this better?
I've thought of either iterating through all meshes in the scene for new ones (cache them at the start) or using a scriptJob to detect new objects being made. Both of those seem very hacky so was curious if anyone had advice.
You can try this method:
import maya.cmds as cmds
shape = cmds.listRelatives(p=True)
object = cmds.listRelatives(a, p=True)
selectedFace = cmds.ls(sl=True)
cmds.select(object[0] + '.f[:]', tgl=True)
unselecetedFace = cmds.ls(sl=True)
duplicated = cmds.duplicate(object, un=True)[0]
cmds.delete(duplicated, ch=True)
cmds.delete(selectedFace)
for i in range(len(unselecetedFace)):
unselecetedFace[i] = unselecetedFace[i].replace(object[0],duplicated)
cmds.delete(unselecetedFace)
cmds.select(duplicated)

Implementing real time plot with Qt5 charts

I am new to Qt and trying to implement a real time plot using QSplineSeries with Qt 5.7. I need to scroll the x axis as new data comes in every 100ms. It seems the CPU usage reaches 100% if I do not purge the old data which was appended to the series, using graphSeriesX->remove(0). I found two ways of scrolling the x axis.
const uint8_t X_RANGE_COUNT = 50;
const uint8_t X_RANGE_MAX = X_RANGE_COUNT - 1;
qreal y = (axisX->max() - axisX->min()) / axisX->tickCount();
m_x += y;
if (m_x > axisX->max()) {
axisX->setMax(m_x);
axisX->setMin(m_x - 100);
}
if (graphSeries1->count() > X_RANGE_COUNT) {
graphSeries1->remove(0);
graphSeries2->remove(0);
graphSeries3->remove(0);
}
The problem with the above is that m_x is of type qreal and at some time if I keep the demo running continuously, it will reach it's MAX value and the axisX->setMax call will fail making the plot not work anymore. What would be the correct way to fix this use case?
qreal x = plotArea().width() / X_RANGE_MAX;
chart->scroll(x, 0)
if (graphSeries1->count() > X_RANGE_COUNT) {
graphSeries1->remove(0);
graphSeries2->remove(0);
graphSeries3->remove(0);
}
However it's not clear to me how can I use the graphSeriesX->remove(0) call in this scenario. The graph will keep getting wiped out since once the series get appended with X_RANGE_COUNT values, the if block will always be true removing 0th value but the scroll somehow does not work the way manually setting maximum for x axis works and after a while I have no graph. scroll works if I do not call remove but then my CPU usage reaches 100%.
Can someone point me in the right direction on how to use scroll while using remove to keep the CPU usage low?
It seems like the best way to update data for a QChart is through void QXYSeries::replace(QVector<QPointF> points). From the documentation, it's much faster than clearing all the data (and don't forget to use a vector instead of a list). The audio example from the documentation does exactly that. Updating the axes with setMin, setMax and setRange all seem to use a lot of CPU. I'll try to see if there's a way around that.
What do you mean by "does not work the way manually setting maximum for x axis works"? The Second method you have shown works if you define x-axis range to be between 0 and X_RANGE_MAX. Is this not what you are after?
Something like: chart->axisX()->setRange(0, X_RANGE_MAX);

Faster calculation for large amounts of data / inner loop

So, I am programming a simple Mandelbrot renderer.
My inner loop (which is executed up to ~100,000,000 times each time I draw on screen) looks like this:
Complex position = {re,im};
Complex z = {0.0, 0.0};
uint32_t it = 0;
for (; it < maxIterations; it++)
{
//Square z
double old_re = z.re;
z.re = z.re*z.re - z.im*z.im;
z.im = 2*old_re*z.im;
//Add c
z.re = z.re+position.re;
z.im = z.im+position.im;
//Exit condition (mod(z) > 5)
if (sqrt(z.re*z.re + z.im*z.im) > 5.0f)
break;
}
//Color in the pixel according to value of 'it'
Just some very simple calculations. This takes between 0.5 and a couple of seconds, depending on the zoom and so on, but i need it to be much faster, to enable (almost) smooth scrolling.
My question is: What is my best bet to achieve the maximum possible calculation speed?
OpenCl to use the GPU? Coding it in assembly? Dividing the image into small pieces and dispatch the calculation of each piece on another thread? A combination of those?
Any help is appreciated!
I have written a Mandelbrot set renderer several times... and here are the things that you should keep in mind...
The things that take the longest are the ones that never escape and take all the iterations.
a. so you can make a region in the middle out of a few rectangles and check that first.
any starting point with a real and imaginary part between -1 and 1 will never escape.
you can cache points (20, or 30) in a rolling buffer and if you ever see a point in the buffer that you just calculated means that you have a cycle and it will never escape.
You can use a more general logic that doesn't require a square root... in that if any part is less than -2 or more than 2 it will race out of control and can be considered escaped.
But you can also break this up because each point is its own thing, so you can make a separate thread or gcd dispatch or whatever for each row or quadrant... it is a very easy problem to divide up and run in parallel.
In addition to the comments by #Grady Player you could start just by optimising your code
//Add c
z.re += position.re;
z.im += position.im;
//Exit condition (mod(z) > 5)
if (z.re*z.re + z.im*z.im > 25.0f)
break;
The compiler may optimise the first, but the second will certainly help.
Why are you coding your own complex rather than using complex.h

Pygame with strange variable issues - a couple variables are increasing for unknown reason

I am having a really strange problem with Pygame, and it's got me stumped for the last few (OK, more like 5) hours. There are free programs out there for making photo mosaics, but ever since my early days tinkering with VB5, I've wanted to write my own version. You know how that is. I have all kinds of cool parts written for loading source images, finding color averages and everything. But here I'm stuck and confused. So very stuck and confused.
This part of the program converts a 'target image' (the one that will be made up of small source images) to smaller blocks of color that other source images will try to match and then replace eventually. But for some reason, the size of the blocks keeps increasing with every iteration. I've tried so many different things that I've had to go through the script and delete out a bunch of things and add a couple more comments before posting.
The target img is a 1280x800 random google image, but any other picture should work just the same. Watch as the Y size of the blit increases with every block going down, and the X size increases as new rows are made. I hard coded in a set size for the solid color rectangle (2 pixels across, much smaller than I'll use), but for whatever reason this keeps increasing. The first row of blits is so small right now that it's hard to see. That quickly changes.
**Here's the link to what I am the image I'm using (http://www.travelimg.org/wallpapers/2012/01/iceland-golden-falls-druffix-europe-golden-falls-golden-falls-iceland-natur-waterfall-waterfalls-800x1280.jpg), but any other pic/size renamed to target.jpg should do the same.
If anyone can point me in the right direction it would be much appreciated. I want to cover this whole source pic in nice 12x12 blocks of solid color to start with. I can't figure out what is changing these block sizes as it goes.
import pygame
import os
from time import sleep
okformats = ['png','jpg','bmp','pcx','tif','lbm','pbm','pgm','ppm','xpm']
targetimg = 'C:\\Python27\\mosaic\\target.jpg'
if targetimg[-3:] not in okformats:
print 'That format is unsupported, get ready for some errors...'
else:
print 'Loading...'
pygame.init()
screen = pygame.display.set_mode((100,100)) #picked a size just to start it out
clock = pygame.time.Clock() #possibly not needed in this script
targetpic = pygame.image.load(targetimg).convert()
targetrect = targetpic.get_rect() #returns something like [0,0,1280,800]
targetsize = targetrect[2:]
targetw = targetrect[2]
targeth = targetrect[3]
numpicsx = 100 #number of pictures that make up the width
sourceratio = 1 #testing with square pics for now
picxsize = targetw/numpicsx
numpicsy = targeth/(picxsize*sourceratio)
picysize = targeth/numpicsy
print 'Blitting target image'
screen = pygame.display.set_mode(targetsize)
screen.fill((255,255,255)) #set to white in case of transparency
screen.blit(targetpic,(0,0))
#update screen
pygame.display.update()
pygame.display.flip()
clock.tick(30)
SLOWDOWN = .1 #temp slow down to watch it
print numpicsx #here are some print statements just to show all the starting values are correct
print numpicsy
print '---'
print picxsize
print picysize
sleep(1)
for x in xrange(numpicsx):
for y in xrange(numpicsy):
currentrect = [x*picxsize,y*picysize,x*picxsize+picxsize,y*picysize+picysize]
avgc = pygame.transform.average_color((targetpic), currentrect) #average color
avgc = avgc[:3] #drops out the alpha if there is one
pygame.draw.rect(screen, avgc, currentrect)
#pygame.draw.rect(screen, avgc, (currentrect[0],currentrect[1],currentrect[0]+2,currentrect[1]+2)) #hard coded 2s (rather than 12s in this case) to help pin point the problem
pygame.display.update()
pygame.display.flip()
clock.tick(30) #probably not needed
sleep(SLOWDOWN)
print 'Done./nSleeping then quitting...'
sleep(3)
pygame.quit()
A friend of mine took a look at my code and showed me the problem. I was thinking that the rect format for drawing was (x1,y1,x2,y2), but it's actually (x,y,width,height). This is the new line:
currentrect = [x*picxsize,y*picysize,picxsize,picysize]
I also dropped the clock.tick(30) lines to speed it all up.