I am facing errors when I deploy the hello samza tutorial on yarn following the documentation. Particularly, I was getting errors when I run the run-app.sh script as mentioned.
I am currently using Samza 1.1.0 on AWS EMR (emr - 5.13.0, amazon 2.8.3, zookeeper 3.4.10) and I am trying to deploy hello samza using the documentation provided on samza (https://samza.apache.org/learn/documentation/latest/deployment/yarn.html). Firstly, I could not find bin/build-package.sh but I used bin/deploy.sh to build the maven package. Then, I tried running the following script run-app.sh
./deploy/samza/bin/run-app.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path ./deploy/samza/config/filter-example.properties
The documentation says that I should be able to deploy the job now but I get the following error which says
Error: Could not find or load main class org.apache.samza.runtime.ApplicationRunnerMain
Related
I am not sure why this file cannot be found.
I am using Dagster to run the DBT Docs Generate command, upload those files to S3 so that they can be deployed via CI/CD to an ngnix server.
The documentation will sometimes load after a fresh deploy, but quickly will throw the above error. I can confirm, though, that the debug and docs generate CLI commands produce no error(s), and all the files (including the catalog.json) are present on the ngninx server.
I am not sure how to debug this weird issue. Thanks!
I tried running the docs generate command with the --no-compile flag, but it produces the same result.
I am seeing an issue running testcafe tests on the gitlab runner. The tests runs fine locally on the gitlab runner, but when run off the CI/CD pipeline it fails. Below is my .yaml code -
test-e2e-windows:
stage: test
cache:
<<: *global_cache
policy: pull
#needs: ['yarn-install']
tags:
- win32
script:
- yarn install
- yarn run test:e2e
Below is the error I see -
$ yarn run test:e2e
Running tests in:
Chrome 99.0.4844.82 / Windows 10
Audio Configuration Combination
× Inbuilt_Mic_Recording_1Ch_48 khz_16 bit_64 kbps_opus_webm
A JavaScript error occurred on
"https://dolby-web-sdk-dev.net/sandbox/index.html".
Repeat test actions in the browser and check the console for errors.
To ignore client-side JavaScript errors, enable the
"--skip-js-errors" CLI option, or set the "skipJsErrors"
configuration file property to "true".
If the website only throws this error when you test it with
TestCafe, please create a new issue at:
"https://github.com/DevExpress/testcafe/issues/new?template=bug-report.md".
JavaScript error details:
NotFoundError: Requested device not found
No stack trace available
Am I missing something here?
According to the "Requested device not found" error message, this issue is related to your application implementation:
DOMException: Requested device not found GetUserMedia.
Requested device not found - Unable to access Camera and Microphone
I can assume that it tries to access some device on a virtual machine but fails due to the fact that this device does not exist. In my opinion, the best way to resolve this issue is to manually handle this error in your application.
If my assumptions are wrong, please create a new GitHub issue using this template and share a minimal working example illustrating the issue.
I'm using the latest helm chart to install Airflow 2.1.1 on k8s. I have a problem with s3 logging - I'm keep getting the error message:
*** Falling back to local log
*** Log file does not exist: /opt/airflow/logs/test_connection/send_slack_message/2021-07-16T08:48:27.337421+00:00/2.log
*** Fetching from: http://airflow2-worker-1.airflow2-worker.airflow2.svc.cluster.local:8793/log/test_connection/send_slack_message/2021-07-16T08:48:27.337421+00:00/2.log
in the task logs.
this is the relevant part from the chart values:
AIRFLOW__LOGGING__REMOTE_LOGGING: "True"
AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID: "s3_logs"
AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER: "s3://.../temp/airflow_logs/stg"
The s3_logs connection is defined like this:
What am I missing?
Technical details:
chart - airflow-8.4.0
app version - 2.1.1
eks version - 1.17
So it seems that the S3 target folder should exist before writing the first log and that solves the issue. I hope that it will help someone in the future!
I'm trying to get Scalding working on Zeppelin while using YARN. I followed the steps in the docs here to build the interpreter and set up the classpath override. When I run in local mode, code executes properly. However when I run on my cluster via YARN my jobs fail with:
Error: java.lang.ClassNotFoundException: cascading.CascadingException
or
Error: java.lang.ClassNotFoundException: cascading.tuple.TupleException
What is even stranger to me is that I can go into Zeppelin and execute:
import cascading.tuple.TupleException
import cascading.CascadingException
And both appear to have no problem finding those classes. It is only when I try to actually use scalding (on YARN), like loading data into a typed pipe and dumping that I get the ClassNotFoundException. Any ideas on how to debug or what to fix?
It looks like the cascading jars are not distributed to the YARN cluster. Please add "zeppelin/interpreter/scalding/*" to the args.string property of the scalding interpreter.
Here's the args.string we use:
-libjars /home/zeppelin-user/zeppelin/interpreter/scalding/,/home/zeppelin-user/deploy-bundle-201608111417/libs/ -Dscalding.reducer.estimator.classes=com.twitter.scalding.reducer_estimation.InputSizeReducerEstimator -Delephantbird.use.combine.input.format=true -Delephantbird.combine.split.size=134217728 --hdfs --repl
tmpjars contains jars that are distributed to the YARN cluster. You can see its contents with the command below:
%scalding
mode.asInstanceOf[Hdfs].conf.get("tmpjars").split(",").foreach(println)
"Error executing command: URL [mvn:org.apache.activemq/activemq-web-console/5.4.0/war] could not be resolved."
is thrown up when i do features:install activemq-web-console
i am using fuse 4.3
removed the old Url for activemq from features:removeUrl and added the new one
mvn:org.apache.activemq/activemq-karaf/5.4.0/xml/features
and re-installed activemq features . if u are unable to install mvn url's then,u must add the fuse repo's to your maven settings xml.
apart from this there is a well known bug
Add the following lines to etc/system.properties
webconsole.type=properties
webconsole.jms.url=tcp://localhost:61616
webconsole.jmx.url=service:jmx:rmi:///jndi/rmi://localhost:1099/karaf-root
webconsole.jmx.user=smx
webconsole.jmx.password=smx
this would make ur console work
For newbie follow these steps:
1. list all features using following command:
karaf#root> features:listurl
2. you can remove url using following command:
features:removeurl
you can find the url to remove in the above removeurl command from the step 1 where you list urls.
add url that you want to using following command:
features:addurl <>
and then follow steps that are mentioned by sanre6 above.