I have simple scala application with spark dependencies. I am just trying to create spark context using the follwing code.
def main(args: Array[String]) {
var sparkConfig : SparkConf = new SparkConf() ;
sparkConfig.setAppName("ProxySQL").setMaster("local");
var sc = new SparkContext(sparkConfig)
}
When i try to run this code inside main - it throws security execption at new SparkContext(sparkConfig) with the following message .
Exception in thread "main" java.lang.SecurityException: class "javax.servlet.ServletRegistration"'s signer information does not match signer information of other classes in the same package .
At problem tab of Eclipse, it shows one warning
Description Path Resource Location Type
More than one scala library found in the build path (D:/workspaces/scala/scalaEclipse/eclipse/plugins/org.scala-ide.scala210.jars_4.0.0.201503031935/target/jars/scala-library.jar, C:/Users/prems.bist/.m2/repository/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar).This is not an optimal configuration, try to limit to one Scala library in the build path. SQLWrapper Unknown Scala Classpath Problem
I have scala installation of 2.10.4 at windows machine.
Scala compiler version set at eclipse is 2.10.5 . What is causing this security exception? Is this the incompatiblity version issues or what exaclty else? How would i solve it?
The problem was more or less related with conflicting dependencies.
The following task resolve my issue.
Go to Project
Build Path -> Order and Export tab -> Change the order of
javax.servlet jar
either to bottom or top.
This Resolved the problem.
Well,as I follow the suggestion:Go to Project Build Path -> Order and Export tab -> Change the order of javax.servlet jar either to bottom or top.
I find my buidpath libiaries was changed and it seems mussy(too many small libs),maybe this was caused by maven.
So I try to remove all of them and reimport the libs and chose Project -> Maven ->Update Project !
Now ,it goes well.
The name of your object with the main method shoul be the same as the setAppName("ProxySQL"), also you can exttend it with app and do not use main method, but this is only if you want I find it easy.
package spark.sample
import org.apache.spark.{ SparkContext, SparkConf }
/**
* Created by anquegi on 18/05/15.
*/
object ProxySQL {
def main(args: Array[String]) {
var sparkConfig: SparkConf = new SparkConf();
sparkConfig.setAppName("ProxySQL").setMaster("local");
var sc = new SparkContext(sparkConfig)
}
}
I normally use and object like for using Spark
package spark.sample
import org.apache.spark.{ SparkContext, SparkConf }
/**
* Created by anquegi on 18/05/15.
*/
object ProxySQL extends App {
val sparkConfig: SparkConf = new SparkConf();
sparkConfig.setAppName("ProxySQL").setMaster("local[4]");
val sc = new SparkContext(sparkConfig)
}
I prefer to use val instead of var
You can also setMaster with .setMaster("local[4]"), and not work only with one
It means you did not exclude the Servlet APIs from some dependency in your app, and one of them is bringing it in every time. Look at the dependency tree and exclude whatever brings in javax.servlet.
It should be already available in Spark, and the particular javax.servlet JAR from Oracle has signing info that you have to strip out, or simply exclude the whole thing.
some of the libraries were mention here
Related
We are migrating the Javalin from 3 to 4 in our current kotlin project. the dynamicGzip has been deprecated and replaced with compression strategy.
The pom.xml part will look like below.
<properties>
<javalin.version>4.1.1</javalin.version>
<jackson.version>2.13.0</jackson.version>
</properties>
The code part of kotlin is as follows
import io.javalin.Javalin
import io.javalin.apibuilder.ApiBuilder.*
import io.javalin.http.BadRequestResponse
import io.javalin.http.NotFoundResponse
import io.javalin.http.staticfiles.Location
import io.javalin.plugin.json.JavalinJackson
import io.javalin.core.compression.*
val app = Javalin.create { config ->
config.defaultContentType = "application/json"
config.enableWebjars()
config.addStaticFiles("", Location.CLASSPATH)
config.enableCorsForAllgOrigins()
//it.dynamicGzip = true // deprecated method which was used in 3.12.0
config.compressionStrategy(Gzip(6))
}
We are using the migrating document from this link
https://javalin.io/migration-guide-javalin-3-to-4
When we try to build the project in intelij Idea with this change, ended with the below error.
D:\app\src\main\kotlin\app\app.kt:78:40
Kotlin: Unresolved reference: Gzip
What is that we are missing here?
Also it will be helpfull if config.addStaticFiles syntax is also added wrt javalin 4
Compression
The compressionStrategy method of the JavalinConfig class takes two parameters:
void compressionStrategy(Brotli brotli, Gzip gzip)
See the JavaDoc here.
The related classes are found in Javalin here:
import io.javalin.core.compression.Brotli;
import io.javalin.core.compression.Gzip;
So, you can do something like this in your set-up (my example is Java not Kotlin):
// my Java example:
config.compressionStrategy(new Brotli(6), new Gzip(6));
Static Files
You can use something like this (again, a Java example not Kotlin):
// my Java example:
config.addStaticFiles("/public", Location.CLASSPATH);
In this case, because I want my files to be on the runtime classpath, I have also created a public directory in my application's resources directory. Your specific implementation may differ.
You can also use Location.EXTERNAL if you prefer, to place the files somewhere else in your filesystem (outside the application).
Note also there is a small typo in config.enableCorsForAllgOrigins(). It should be:
config.enableCorsForAllOrigins()
Given this code, where should I place the file.json to be able to be found in the runtime?
// path: src/main/kotlin/Server.kt
fun main() {
val serviceAccount = require("file.json")
}
I tried place it under src/main/resources/ without luck. I also use Gradle to compile kotlin to js with kotlin2js plugin.
Assuming the Kotlin compiler puts the created JS file (say server.js) into the default location at build/classes/kotlin/main and the resource file (file.json) into build/resources/main.
And you are running server.js by executing node build/classes/kotlin/main/server.js
According to the NodeJS documentation:
Local modules and JSON files can be imported using a relative path (e.g. ./, ./foo, ./bar/baz, ../foo) that will be resolved against the directory named by __dirname (if defined) or the current working directory.
(https://nodejs.org/api/modules.html#modules_require_id)
In our case __dirname is build/classes/kotlin/main
So the correct require statement is:
val serviceAccount = js("require('../../../resources/main/file.json')")
or if require is defined as a Kotlin function like in the question
val serviceAccount = require("../../../resources/main/file.json")
You may just use js("require('./file.json')") if you do not have import for the require function in Kotlin. The result will be dynamic, so you may cast it to Map.
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.js/js.html
I am very new to perf testing, i went through the sample project created using karate scripts in gatling, but unable to understand how to do it,
Can anyone please provide explanation on how to use karate for performance testing using gatling by using some public api like below
Scenario: Get State specific information - one state
Given url 'http://services.groupkt.com/state/get/IND/AP'
When method get
Then status 200
* def resp = response.RestResponse.result.name
* print resp
so that we can use it in our project. unable to understand the current demo project available in github karate
You just have to Git-clone and run (using Maven) this simple, stand-alone project: https://github.com/ptrthomas/karate-gatling-demo
Take the help of someone who knows their way around a Maven project if needed.
Once you get this running, you will be able to understand and modify this in no time.
package mock
import com.intuit.karate.gatling.PreDef._
import io.gatling.core.Predef._
import scala.concurrent.duration._
class CatsSimulation extends Simulation {
val protocol = karateProtocol(
"/cats/{id}" -> Nil,
"/cats" -> Nil
)
val create = scenario("create").exec(karateFeature("classpath:mock/cats-create.feature"))
val delete = scenario("delete").exec(karateFeature("classpath:mock/cats-delete.feature"))
setUp(
create.inject(rampUsers(10) over (5 seconds)).protocols(protocol),
delete.inject(rampUsers(5) over (5 seconds)).protocols(protocol)
)
}
When using Kotlin's Script Engine, trying to import packages or use any class throws an "unresolved reference"
javax.script.ScriptException: error: unresolved reference: mrpowergamerbr
fun loritta(context: com.mrpowergamerbr.loritta.commands.CommandContext) {
^
This doesn't happen when running the class within IntelliJ IDEA, however it does happen when running the class on production.
While this YouTrack issue is related to fat JARs, this also can happen if you aren't using fat JARs (loading all the libraries via the startup classpath option or the Class-Path manifest option)
To fix this, or you can all your dependencies on your startup script like this:
-Dkotlin.script.classpath=jar1:jar2:jar3:jar4
Example:
java -Dkotlin.script.classpath=libs/dependency1.jar:libs/dependency2.jar:yourjar.jar -jar yourjar.jar
Or, if you prefer, set the property via code, using your Class-Path manifest option.
val path = this::class.java.protectionDomain.codeSource.location.path
val jar = JarFile(path)
val mf = jar.manifest
val mattr = mf.mainAttributes
// Yes, you SHOULD USE Attributes.Name.CLASS_PATH! Don't try using "Class-Path", it won't work!
val manifestClassPath = mattr[Attributes.Name.CLASS_PATH] as String
// The format within the Class-Path attribute is different than the one expected by the property, so let's fix it!
// By the way, don't forget to append your original JAR at the end of the string!
val propClassPath = manifestClassPath.replace(" ", ":") + ":Loritta-0.0.1-SNAPSHOT.jar"
// Now we set it to our own classpath
System.setProperty("kotlin.script.classpath", propClassPath)
While I didn't test this yet, in another unrelated answer it seems you can also supply your own classpath if you initialize the KotlinJsr223JvmLocalScriptEngine object yourself (as seen here)
I've been trying to use Apache Sling's Scala 2.8 Script Engine recently updated last month. I came from using Scala 2.7 along with Sling's Scala 2.7 Script Engine and that worked great. I run into a problem when I try to use the new implementation. When calling ScalaScriptEngine's eval function I always receive an "Error executing script" due to a NullPointerException. Has anyone else worked with the new build of the script engine and run into this as well?
Thanks!
Steven
There is a bug which prevent the Scala scripting engine from being used standalone. See https://issues.apache.org/jira/browse/SLING-1877 for details and a patch.
Also note, that with the patch applied you still need to set the class path when using the scripting engine. This is a change from 2.7.7 where the default java class path (i.e. java.class.path) was used automatically. In 2.8 you have to set this explicitly thorough the '-usejavacp' argument.
Here is some sample code demonstrating the standalone usage of the Scala scripting engine:
def testScalaScriptEngine() {
val scriptEngineFactory = new ScalaScriptEngineFactory
val settings = new ScalaSettings()
settings.parse("-usejavacp")
scriptEngineFactory.getSettingsProvider.setScalaSettings(settings)
val scriptEngine = scriptEngineFactory.getScriptEngine
val script = """
package script {
class Demo(args: DemoArgs) {
println("Hello")
}
}
"""
scriptEngine.getContext.setAttribute("scala.script.class", "script.Demo", ScriptContext.ENGINE_SCOPE)
scriptEngine.eval(script)
}