This is my Transformation
Run in local gives me a good log and allows me to Preview results of a step. (eg what went wrong and where, what data do I have in each step)
When I run it in Repository, it just gives me a link to check status, where logs are present BUT text only and I cannot Preview Data in each step.
Is there any way to get this? (I also have a Carte server running, my code call it to run, but for debugging I need to run from Kettle UI)
Related
I have a strange dbt behaviour.
When I run models or tests in dbt cloud, some of them passes, some fails. When I click on 'System Logs' dropdown or logs of each models, it shows the result of run and information about the run. The command, which I used (for instance, 'dbt test') also marked as completed. But on the status bar that maps execution process of each task with its results all the steps of the run are placed only in queue tab.
What's the problem?
I'm trying to create a scheduled job based out of saved query in big query. I'm following steps as per documentation here
After entering all the filelds, I'm getting response "Error creating scheduled query: Aq" with no additional details.
In on BigQuery documentation also, I didn't find anything.
I could run the query which I'm trying to create a schedule for, is running fine and giving appropriate results.
It turned out that, when schedule query is created for first time, the console is taking permission by take user to new tab in browser. Browser settings was suppressing it.
After acknowledging the required permissions, I could create scheduled query successfully.
I realise normally a debug run is not visible in the data factory v2 UI after closing the browser window, however unfortunately I needed to restart my machine unexpectedly and it's a long running pipeline.
I thought maybe the runs might be available via powershell, but I haven't had any luck.
The pipeline is likely still running.
We do have external logging, however ideally I'd like to see how long each activity is taking as I'm load testing.
And more importantly I do not want to do another run until I'm sure it's finished.... notably I'll run it from a trigger next time (just in case!).
EDIT:
It looks like a sandbox id is used which is stored in the browser local storage and there appears to be undocumented API endpoints for gathering info using the sandbox id. But there doesn't appear to be a way of getting old sandbox id's so I'm probably out of luck.
There is a button for view all debug runs.
Taken from Microsoft documentation:
To view a historical view of debug runs or see a list of all active debug runs, you can go into the Monitor experience.
I have many builds with failed tests.
I learn one test and want to find last build, where this test was succesfully.
How I can find this build or how I can get test results history?
If as a post-build step you are publishing your test results, you should be able to go to the job page and see the test results graph. If you are looking for the last build where all tests passed, you can click on "enlarge" under the graph. You will then see a high level view of all the builds from which you should be able to see the last build where all tests passed.
If you are looking for the last time a particular test passed, you can click on the latest build in the test results graph, drill down to the particular test you are interested in, and then click the previous build button until you find a build where that test passed.
If you aren't publishing your test results, you may be able to write some groovy code to scan through your build results, but beyond that I don't know of a way to find what your looking for without publishing test results in your post-build.
I am doing a simple screenshot compare Jenkins build.
The idea is very simple:
1, run build with "generate" parameter to generate screenshot and save in workspace.
2, run build with "compare" parameter to take screenshot and compare with the saved screenshot in above step.
To make the captured screenshot carry over to next build: I use the option of "Force polling using workspace". That works fine when build success. But, once the build failed, the workspace is clean up. And the generated screenshot in step one are lost.
Is there a better why to handle the screenshot? There is a solution to put the screenshot to separate server. But, is there a simpler way?