Hello everyone
I have a pool of binary data with n elements.
k of those elements (the private keys) have been xored with each other, generating another element (a public key), which is also in the pool. All the other elements are irrelevant.
So when xoring the private keys and the public key, the result should be 0.
I don't know the position of the k private keys and the position of the public key. I have to figure those out.
What would be an approach I could try, a phrase I could search for or an existing algorithm?
What I've tried (and failed because of runtime)
Of course I tried the brute force approach.
My first attempt was the worst. I made an iterator, and checked every option until I got there.
I did the same thing like in the first attempt but with an imaginary tree.
Those worked, but the runtime was to bad, with n = 115; k = 11.
I've tried something like this but I couldn't get something in this direction work for my needs.
Please help I put way to much time in this,
Hellow2
Related
The requirements:
100k lines
One of the columns is not text - its custom painted with wxDC*.
The items addition is coming from another thread using wxThreadEvent.
Up until now I used wxDataViewListCtrl, but it takes too long to AppendItem 100 thousand time.
wxListCtrl (in virtual mode) does not have the ability to use wxDC* - please correct me if I am wrong.
The only thing I can think of is using wxDataViewCtrl + wxDataViewModel. But I can't understand how to add items.
I looked at the samples (https://github.com/wxWidgets/wxWidgets/tree/WX_3_0_BRANCH/samples/dataview), too complex for me.
I cant understand them.
I looked at the wiki (https://wiki.wxwidgets.org/WxDataViewCtrl), also too complex for me.
Can somebody please provide a very simple example of a wxDataViewCtrl + wxDataViewModel with one string column and one wxDC* column.
Thanks in advance.
P.S.
Per #HajoKirchhoff's request in the comments, I am posting some code:
// This is called from Rust 100k times.
extern "C" void Add_line_to_data_view_list_control(unsigned int index,
const char* date,
const char* sha1) {
wxThreadEvent evt(wxEVT_THREAD, 44);
evt.SetPayload(ViewListLine{index, std::string(date), std::string(sha1)});
wxQueueEvent(g_this, evt.Clone());
}
void TreeWidget::Add_line_to_data_view_list_control(wxThreadEvent& event) {
ViewListLine view_list_line = event.GetPayload<ViewListLine>();
wxVector<wxVariant> item;
item.push_back(wxVariant(static_cast<int>(view_list_line.index)));
item.push_back(wxVariant(view_list_line.date));
item.push_back(wxVariant(view_list_line.sha1));
AppendItem(item);
}
Appending 100k items to a control will always be slow. That's because it requires moving 100k items from your storage to the controls storage. A much better way for this amount of data is to have a "virtual" list control or wxGrid. In both cases the data is not actually transferred to the control. Instead when painting occurs, a callback function will transfer only the data required to paint. So for a 100k list you will only have "activity" for the 20-30 lines that are visible.
With wxListCtrl see https://docs.wxwidgets.org/3.0/classwx_list_ctrl.html, specify the wxLC_VIRTUAL flag, call SetItemCount and then provide/override
OnGetItemText
OnGetItemImage
OnGetItemColumnImage
Downside: You can only draw items contained in a wxImageList, since the OnGetItemImage return indizes into the list. So you cannot draw arbitrary items using a wxDC. Since the human eye will be overwhelmed with 100k different images anyway, this is usually acceptable. You may have to provide 20/30 different images beforehand, but you'll have a fast, flexible list.
That said, it is possible to override the OnPaint function and use that wxDC to draw anything in the list. But that'll get difficult pretty soon.
So an alternative would be to use wxGrid, create a wxGridTableBase derived class that acts as a bridge between the grid and your actual 100k data and create wxGridCellRenderer derived classes to render the actual data onscreen. The wxGridCellRenderer class will get a wxDC. This will give you more flexibility but is also much more complex than using a virtual wxListCtrl.
The full example of doing what you want will inevitably be relatively complex. But if you decompose in simple parts, it's really not that difficult: you do need to define a custom model, but if your list is flat, this basically just means returning the value of the item at the N-th position, as you can trivially implement all model methods related to the tree structure. An example of such a model, although with multiple columns can be found in the sample, so you just need to simplify it to a one (or two) column version.
Next, you are going to need a custom renderer too, but this is not difficult neither and, again, there is an example of this in the sample too.
If you have any concrete questions, you should ask them, but it's going to be difficult to do much better than what the sample shows and it does already show exactly what you want to do.
Thank you every one who replied!
#Vz.'s words "If you have any concrete questions, you should ask them" got me thinking and I took another look at the samples of wxWidgets. The full code can be found here. Look at the following classes:
TreeDataViewModel
TreeWidget
TreeCustomRenderer
What I Have
Using OptaPlanner 7.45.0.Final on Java 8.
I am working on a proof-of-concept for scheduling of shifts, where I have created CrewAssignment object as a #PlanningEntity.
I have created my own EasyScoreCalculator implementation (from the new non-deprecated EasyScoreCalculator).
The score function returns a HardMediumSoftScore, with Hard representing physical possibility (can't be in 2 places at once, can't take on an activity that starts in a different place than the previous activity ended), Medium representing legal issues (maximum work day, etc.), and Soft roughly representing financial concerns.
The behavior I am getting
The schedule seemed to work well for small datasets where there are many good possible Solutions. However, when I moved to larger datasets, it had a harder time returning good final schedules. There would be scores like -60hard/-390060medium/-1280457soft, which seems really bad. Moreover, if I increased the time available, the score would sometimes get worse!
Things I have tried
I put a print in my score function for the score being calculated. It would often give scores such as 0hard/0medium/-2250411soft, which in comparison to the final scores is great! However, the final result would still be a bad score.
I added a SolverEventListener to the Solver, and it is only called once at the very end of the solving.
I thought maybe there was a problem with cloning of the Solution, so I created my own SolutionCloner. It is called once at the beginning, and twice at the end. Since it is called so infrequently, I suspect the solver uses the first Solution clone as the best score, tries to copy the values from the current Solution iteration to the best Solution, but given that the SolverEventListener is only called once, that may indicate that it is not recognizing the best solution.
I tried simplifying to a HardSoftScore by combining Hard and Medium values, but the behavior is the same.
I tried calling solverConfig.setEnvironmentMode(EnvironmentMode.NON_INTRUSIVE_FULL_ASSERT); (yes, I'm programatically creating the SolverConfig, but the behavior does not change.
Relevant Code Snippets
Subset of the CrewAssignment #PlanningEntity:
#PlanningEntity
public final class CrewAssignment
{
private Long crewMemberId;
#PlanningVariable(valueRangeProviderRefs = {"crewMemberId"})
public Long getCrewMemberId() { return crewMemberId; }
public void setCrewMemberId(Long value) { crewMemberId = value; }
#Override
protected CrewAssignment clone()
{
CrewAssignment newAssignment = new CrewAssignment(getActivities());
newAssignment.setCrewMemberId(crewMemberId);
return newAssignment;
}
}
Subset of the Solution:
#PlanningSolution(solutionCloner = CrewSchedSolutionCloner.class)
public final class CrewSchedSolution
{
private final List<Long> crewMemberIds;
#ValueRangeProvider(id="crewMemberId")
#ProblemFactCollectionProperty
public List<Long> getCrewMemberIds() { return crewMemberIds; }
// assignments
private List<CrewAssignment> crewAssignments = new ArrayList<>();
#PlanningEntityCollectionProperty
public List<CrewAssignment> getCrewAssignments() { return crewAssignments; }
HardSoftScore score;
#PlanningScore
public HardSoftScore getScore() { return score; }
public void setScore(HardSoftScore value) { score = value; }
public CrewSchedSolution cloneSolution()
{
List<CrewAssignment> newCrewAssignments = new ArrayList<>(crewAssignments);
for (int i = 0; i < newCrewAssignments.size(); i++)
{
CrewAssignment existingAssignment = newCrewAssignments.get(i);
CrewAssignment newAssignment = existingAssignment.clone();
newCrewAssignments.set(i, newAssignment);
}
return new CrewSchedSolution(/* Various additional data */,
crewMemberIds,
newCrewAssignments, score);
}
}
The SolutionCloner:
public final class CrewSchedSolutionCloner
implements SolutionCloner<CrewSchedSolution>
{
#Override
public CrewSchedSolution cloneSolution(CrewSchedSolution originalSolution)
{
return originalSolution.cloneSolution();
}
}
Summary
I have run out of obvious (to me) ways to debug this further. It kind of feels like the good values are getting written over, but I can't prove that yet. I might be able to come up with a way of saving the values set in the CrewAssignment #PlanningEntity just to get it to work, but it seems like that is something OptaPlanner was designed to be able to handle.
The problem is too large to post a SSCCE. I have posted what I think might be relevant. Let me know if there is any other part of the code you would need to see. Thanks in advance!
P.S. I realize this may be a duplicate of OptaPlanner return the best score but not its associated solution, but the original poster never followed up with the posted answer.
if I increased the time available, the score would sometimes get worse! That is impossible (if it gets more steps in the second run, which it should, and it is a reproducible run, which it is).
Does the DEBUG log show it's running more "LS step" lines? If it does, you might have score corruption, but NON_INTRUSIVE_FULL_ASSERT or FULL_ASSERT would detect that if they run long enough (much longer than without because they are slower).
Normally, optaplanner runs are 100% reproducible, given the same amount of steps (~ same amount of time give or take a few steps). Check your DEBUG log if that's the case. Simulated Annealing isn't reproducible, but that's off by default.
OptaPlanner Benchmark is your best friend. Especially the BEST_SCORE graph and the score calculation speed number. Run it 4 times longer, so you can see how the BEST_SCORE graph behaves if given more time. Also, subSingleCount might be an interesting thing to turn on here, to see how sensitive the optimization is to a good random seed (it shouldn't be).
Ow, wait. You wrote your own solution cloner? That's probably it, because solution cloners are extremely difficult to write correctly. Disable that and see what happens.
OptaPlanner was (of course) working correctly. I had a conceptual misunderstanding.
I was focusing on the Scores I was calculating, and not focusing on the scores that OptaPlanner was actually using. OptaPlanner can augment the Scores that are returned from the scoring function by adding an "init" value representing the number of uninitialized #PlanningVariables. Scores that had 0 Hard and Medium scores may have looked reasonable, but may still be uninitialized, which takes precedence over Hard/Medium/Soft values.
Once the duration of the simulation was set sufficiently high, all #PlanningVariables were being set. No scores with init of 0 were created after that, and completed optimized solutions (whether infeasible or feasible) were computed.
See the steps Geoffrey outlined for debugging. If you are using JDK logging, you may have to add slf4j-api and slf4j-jdk14 jars to your path.
See the documentation, section 2.3.6 "Gather the domain objects in a planning solution", where it describes the difference between an "uninitialized", "infeasible", and "feasible" solution.
Note: It appears that the solver does not notify SolverEventListeners when the best solution so far is uninitialized.
I am working on a vb.net auto-focus routine and have the image processing part worked out, basically I do some edge detection, convert to gray-scale and then measure the standard deviation to work out the most 'in focus' point of the image.
I have done this with a number of images, and it almost comes out as a normal distribution, now I want to start to integrate this with my microscope and a stepper motor.
The concept is that I would move through a lower and upper limit on the stepper motor, and measure the above through live-view, recording the values in a list. In my case the two things I want to record are the position, and the double standard deviation value.
I am wondering what the best way to record these are, should it be
recorded as a typed list, or a dictionary or another method?
Once I record all of these values, I would want to go through the values to conduct some simple analysis of them, so if that was the case
how would I then be able to determine the average, min, max etc?
My first attempt of storing the information was in a typed list, where I had essentially done the below;
Public ZPositions As New List(Of Zfocus)
Public Class Zfocus
Public Position As Integer
Public GreyStDev As Double
End Class
The second way was to use a dictionary;
Public ZPosition As New Dictionary(Of Integer, Double)
However in both cases, I am not sure how I can either pull out a single maximum position value (e.g. Position integer,) or from the dictionary the position value (integer) which (sort of) corrosponds to the best auto-focus position.
The Third added bonus, is to be able to pull out any postions above a
specific value, which may corrospond to having some focus information
within them for focus stacking?
Many thanks
Big thanks to jmcilhinney, this solved my issue and works a treat!
Went with a strongly typed list (the ZFocus list) and then I could do the below;
MaxPosition = ZPositions.First(Function(zp1) zp1.GreyStDev = ZPositions.Max(Function(zp2) zp2.GreyStDev))
This allowed be to set up an auto-focus routine which loops through a number of images (as a test), stores the position (e.g. image number in this case) and the intensity edge information, and at the end then pull out the strongest intensity information which forms the best auto-focus point in my case
I've been reading this article on elliptic-curve crypto and how it works:
http://arstechnica.com/security/2013/10/a-relatively-easy-to-understand-primer-on-elliptic-curve-cryptography/
In the article, they state:
It turns out that if you have two points [on an elliptic curve], an initial point "dotted" with itself n times to arrive at a final point [on the curve], finding out n when you only know the final point and the first point is hard.
It goes on to state that the only way to find out n (if you only have the first and final points, and you know the curve eqn), is to repeatedly dot the initial point until you finally have the matching final point.
I think I understand all this - but what confuses me is - if n is the private key, and the final point corresponds to the public key (which I think is the case), then doesn't it take the exact same amount of work to compute the public key from the private, as it does the private from the public (both just have to recursively dot a point on the curve)? am I misunderstanding something about what the article is saying?
The one-way attribute of ECC and RSA is due to the Chinese Reminder Theoreom (CRT). A series of arithmetic divisions where only the remainder is kept (aka Modulo operation %), which results in information loss in the output. As a result, the person with the keys takes one direct path to generate the output - and any would-be attacker has to exhaust a massive number of possible paths in order to determine what key was used to create the output. If the simple division was used instead of a modulo - then key data would be present in the output and it couldn't be used for cryptography.
If you lived in a world where you had a powerful enough computer to exhaust all possibilities - then the CRT wouldn't be useful as a cryptographic primitive. The computers we have now a fairly powerful - so we balance the power of our modern machines with a keysize that introduces enough range of possibilities so that they cannot be exhausted in a timeframe that matters.
The CRT is a subset of the P vs NP problem set - so perhaps proving P=NP may lead to a way of undermining the oneway aspect of asymmetric cryptography. We know that there is a way to factor CRT using a quantum computer running Shor's Algorithm. Shor's Algorithm has proven that we can defeat the so-called "trapdoor", or one-way attributes of CRT, it is still however an expensive attack to conduct.
The following lecture is my favorite description of the CRT. It shows that there are many possible solutions for one direction forcing an attacker to exhaust them all and only one solution for the other:
https://www.youtube.com/watch?v=ru7mWZJlRQg
EDIT: I previously stated that n is not the private key. In your example, n is either server or client private key.
How it works is that there is a starting point known to anybody.
You select random integer k and do the "dotting operation" k-times. Then you send this new point to the server. (k is your private key)
Server does the same with the starting point, but q-times and sends it to you. (q is server's private key)
You take the point you got from server and "dot" it k-times. The final point would be the starting point "dotted" k*q-times.
Server does the same with point it got from you. And again its final point would be the starting point "dotted" q*k-times.
That means the final point (= the starting point "dotted" k*q-times) is the shared secret since all what any attacker would know is the starting point, the starting point dotted k-times and the starting point dotted q-times. And given only those data, it's practically impossible to find the final point as a product of k*q unless any of those known.
EDIT: No, it doesn't take the same time to compute k from G = kP given known values of G (sent point) and P (starting point). More in comment section and:
For rising to power, see Exponentiation by squaring.
For ECC point multiplication, see point multiplication.
I'm running an update() method n times per second to "update" the keyboard input from the user so I can read it later in the logic part of the program. So I find two ways of implementing this in the SDL Docs and I'm not sure which one should I use.
1; Loop for all events using SDL_PollEvent searching for key down/up events and saving the key states in a map so I can check for each key state in the logic of the program.
Note: Alternatively, I can also use SDL_PeepEvents instead of SDL_PollEvent to take only the event types that matter; so, it would not "thrown away" the events on the queue.
std::map<int, bool> keyboard; // Saves the state(true=pressed; false=released) of each SDL_Key.
void update()
{
SDL_Event event;
while(SDL_PollEvent(&event))
{
switch(event.type)
{
case SDL_KEYDOWN:
keyboard[event.key.keysym.sym] = false;
break;
case SDL_KEYUP:
keyboard[event.key.keysym.sym] = true;
break;
}
}
}
2; Taking a snapshot from the keyboard each frame so I can read it easily.
Uint8* keyboard;
void update()
{
SDL_PumpEvents();
keyboard = SDL_GetKeyState(NULL);
}
With any of above implementations I can read keyboard just like this:
if (key_map[SDLK_Return]) printf("Return has been pressed.");
Also, is there another way to do so?
I prefer to do a variation of 1, where I fill three arrays, indicating not only the current state, but also which keys just went down and which keys just went up. This allows me to easily check for those events in code (without comparing to the previous snapshot), but, most importantly, it won't miss events that last less than a frame. For example, if your game is running at 10 fps due to a slow machine, the user might press and release an important key between two calls of your update routine, and then your system will never register it. This is extremely frustrating.
SDL also sends key events when the key is held down, which allow you to have multiple key down events for each key up. I find this particularly useful when implementing keyboard scrolling through a list of items, e.g. a keyboard-controlled menu.
You should use solution 2.
Why? As SDL_GetKeyState() docs point out, before using it you are expected to call SDL_PumpEvents() to update the state array.
When you are calling SDL_PollEvent(), it implicitly calls SDL_PumpEvents(). So, it basically updates the array for SDL_GetKeyState() anyway. By parsing these events manually, you just create a second array (well, actually a much slower map) holding the same information which SDL already collected for you.
So, I would dare say that first solution means doing the same thing twice. And if you ever decide to support things such as repeated keystrokes (SDL_EnableKeyRepeat()), you'll be reimplementing even a larger part of SDL.
I realize this question is quite old, but my answer could benefit someone. Personally, I use two arrays with SDL_GetKeyState. I store one array holding the current frame's keyboard state, and one array holding that last frame's keyboard state. (With some memcpy commands, it's really easy to update them.) Along with those two arrays, I have a map that converts strings like "A" to the SDL scancode values, but that is optional.
Then, when you need to check if something is released or pressed, you can combine the two arrays to check. (I made this a function.) For example, if you know that the key is pressed this frame, but wasn't pressed last frame, it was clearly just pressed this frame. if (currentFrame["A"] == true && lastFrame["A"] == false) {/*just pressed*/}
You would then do the opposite for the released. I find that method super easy to implement and use.