EXCEPTION: Update is unnecessary as statistics is up-to-date - azure-data-lake

Does it exists any way to change the U-SQL's behavior? I would like if it can ignore silently any statistics that is up to date instead of throw an exception. The reason why i need it. I just wanna schedule triggering of update the stats from ADF. (I would like to have a temporary solution like set #opt='cryptic__undocumented_option')

Answering explicitly to have an answer appear.
This will be fixed in the upcoming refresh. It was a regression that was introduced by another bug fix. We have now put regression tests into place.
Thanks for reporting!

Related

Is there a way to see where a schema validation fails?

I'm trying to validate a json against a somehow complex 'schema' (defined as in karate docs). The error I'm getting is not so explicit:
reason: not equal
How can I check where it actually fails validating?
Really sorry about that, I'm to blame. I know this won't help you but I'm re-writing the core of Karate right now and this is how a match failure looks like in the improved version:
Would it be possible for you to create a representative sample of what you have right now (create an issue on GitHub). I would like to see if that is handled better in the future or tweak things right away.

Problem with PDDL code using conditional effects

I'm trying to solve a Pacman-styled PDDL problem and there's a particular scenario I've been stuck on for days now. I'm getting the classic
ff: goal can be simplified to FALSE. No plan will solve it
which means the issue is trivial and logic related. However, I'm new to PDDL and can't seem to figure out what's causing it.
The problem is simple, Pacman (P) has to eat the Food (F), but two ghost agents (G) are blocking it. To get past them, Pacman needs to consume the Capsule (C), making him invisible.
(Edit: I've deleted the question as it was part of an assignment. I managed to solve the issue and will post the solution as soon as the assignment is graded.)
In this thread: About PDDL in AI planning #haz mentioned a good methodology to debug your PDDL Model when the goail is unreachable from the initial state.
The best way to test this out is to follow the following strategy: (1) write down a plan you know will solve it; (2) starting with the first action, set the goal to the precondition; (3) repeat to the end. If that fails, start changing the initial state to what you expect the complete state to be during the execution of the plan. – haz May 1 at 2:04
I uploaded a new version which is able to find a solution (if Ghosts are not in the path of foods):
http://editor.planning.domains/#read_session=c7Vez9nrti
Two main issues:
you never delete GhostPos, but it appears in the goal formula
If GhostPos is on the way to a FoodPos, then you can never reach the food as the move action requires not to be a Ghost.

Trigger action realtime based on keyword in Logs

I have a requirement for which I want to trigger an action (like calling a REST-ful service) in the event a keyword is found in the logs. The trigger would have to be fairly real time. I was evaluating open source solutions like GrayLog2, ELK stack (which I believe can't analyse real time), fluentd etc. but wanted to know your opinion on that. It would be great if the tool also allows setting up rules against key words to eliminate false positives and easy to set up.
I hope this makes sense and apologies if this has been discussed elsewhere!
You can try Massalyzer. It's a real-time analyzer too, very fast (up to 10 millinon line per sec), and you can analyze unlimited size with free demo version.
So, I tried Logstash+Graylog2 combination for the scenario I described in the question and it works quite well. I had to tweak a few things to make Logstash work with Graylog2, especially around capturing the right log levels. I will try this out on a highly loaded clustered environment and update my findings here.

Pentaho kettle: How to prevent db transformation step from always executing?

I have a transformation that has a switch case which can either run a database retrieval transformation or do nothing based on the switch case value.
The problem is that the database transformation seems to get executed always no matter what the result of the switch case is. The database name is parametrized and the switch case tries to make sure that non existing database names are ignored and not queries (as this causes an error). But now the database transformation runs every time and causes an error.
So the question is: Is there a way to prevent the a database transformation from automatically executing? I've tried adding a blocking step before it but with no results.
I've tried to do this before and hit the exact same problem. It's fundamentally to how PDI works, if your step wont initialize then nothing will run - there is no solution to this. In fact, I even have a jira about this, but it doesn't seem to be going anywhere.
However perhaps you're doing this about face. Why do you have conditional connections? IF you could explain the use case then perhaps we can propose a better solution.

Should triggers only be used for something that's not possible with SQL?

I'm currently writing an application that uses SQLite. SQLite doesn't have the ON UPDATE function for timestamps so I wrote my first trigger for it. Then I wrote a trigger to add the current timestamp on the modified and created fields on an insert.
The problem came when I went and deleted the setting of the modified/created fields for the insert. I felt like I was hiding something from developers that might see my code in the future. It could be a source of confusion.
How will they know that the sql is coming from a trigger. Should I comment it? Is it bad practice?
As a rule of thumb, triggers are meant to implement SQL functional rules, such as inclusions, exclusions, partitions etc.
This kind of thing belongs to the model and should be implemented as triggers whenever it is possible. It has to be delivered with the database otherwise the model would be broken.
Regarding to your case, it is more a hack than anything. If you can't do differently, do it and then add a comment like you said. But it should remain an exception.
Keep in mind that almost everything a trigger is doing could be done at the application layer (whichever you want)
Good observation. There are some things only triggers can do. However I suggest that if there is any alternative to using a trigger then use the alternative. I'm not familiar with SQLite, but in any other database I would use a DEFAULT rather than a trigger to timestamp a new record. To capture updated date I would enclose this in a stored procedure, or whatever database side logic you have (kind of what RandomUs1r suggested). I might consider a trigger but only for very basic operations.
You are correct that triggers can be confusing and difficult to debug.
"I felt like I was hiding something from developers..." - this is a very good point. I've came across many developers who use ##Identity and were genuinely shocked that if somebody put a trigger on the table which inserted another row, they'd end up with the wrong identity. (As opposed to SCOPE_IDENTITY() - I know these are sql server specific, but that's pretty much all I know...)
It is hidden - other than documentation I'm not sure you can make it more visible either.
Which is why many avoid them where possible - I guess if there's no easy way around using them in some cases then as long as its well documented, etc. I think like cursors, although scorned by many they can be very powerful and useful...but if they can be avoided probably for the best.
On the code that modifies the record, to get the current timestamp... in SQLLite, try:
DATETIME('NOW')