I am trying to create an Unmanaged ApplicationMaster and having issues with creating AMRMtokens right. I peeked into the TestAMRMtokens.java test cases and here is what I have come up with. (This is scala code)
def getNextAttemptid() : ApplicationAttemptId = {
val t = java.lang.System.currentTimeMillis()
ConverterUtils.toApplicationAttemptId(ConverterUtils.APPLICATION_ATTEMPT_PREFIX + "_" + t.toString() + "_0001" + "_0001")
}
appAttemptId = getNextAttemptid()
UserGroupInformation.setConfiguration(yarnConf)
val ugi = UserGroupInformation.getCurrentUser()
val tokenIdentifier = new AMRMTokenIdentifier(appAttemptId)
val secretManager = new AMRMTokenSecretManager(yarnConf)
val token: Token[_ <: TokenIdentifier] = new Token[AMRMTokenIdentifier](tokenIdentifier, secretManager)
ugi.addToken(token)
amClient = AMRMClient.createAMRMClient()
amClient.init(yarnConf)
amClient.start()
val appMasterResponse = amClient.registerApplicationMaster("localhost", 0, "")
Yarn does not like this request and says:
2014-01-27 10:47:10,938 WARN SecurityLogger.org.apache.hadoop.ipc.Server: Auth failed for 127.0.0.1:63085:null (DIGEST-MD5: IO error acquiring password)
2014-01-27 10:47:10,938 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8030: readAndProcess from client 127.0.0.1 threw exception [org.apache.hadoop.security.token.SecretManager$InvalidToken: Password not found for
ApplicationAttempt appattempt_1390848430314_0001_000001]
I am sure I am doing something wrong. Is there documentation or example code that shows how to create the tokens right? Do I have to setup these users somewhere in Yarn/Hadoop configuration.
Configuration:
hadoop 2.2.0
All services are listening on localhost interface Just
using out of the box configuration. No special changes.
Related
I have a Kotlin class, its task is to push a jar file to codeartifact, i do get a token successfully but i keep on getting a 401 back from codeartifact service, i am reading the following doc[1]
The documentation does not show examples other than curl and maven plugin, which is not what i use, so i am trying to mimic this behavior with a javax.ws.rs.client.HttpClient:
val authTokenReq = GetAuthorizationTokenRequest.builder()
.domain(domainName)
.domainOwner(domainOwner)
.durationSeconds(1000)
.build()
val token = CodeartifactClient.create().getAuthorizationToken(authTokenReq).authorizationToken()
val authString = "aws:$token"
val encodedAuth = Base64.getEncoder().encode(authString.toByteArray(Charsets.UTF_8))
val upload = client.target(to)
.request()
.header("X-Checksum-Sha1", sha1)
.header("Authorization", "Basic " + encodedAuth.toString(Charsets.UTF_8))
.put(Entity.entity(packageVersionAssetAsBytes.asInputStream(), "application/java-archive"))
Unfortunatelly i get:
"exception":"java.lang.RuntimeException: Upload failed, status=401
Does anyone has a tip about how i should properly use codeartifact token?
[1]: https://docs.aws.amazon.com/codeartifact/latest/ug/maven-curl.html
I Use ElasticSearch 8.1.2
and Nest 17.7.1
var settings = new ConnectionSettings(new Uri("http://localhost:9200/"))
.CertificateFingerprint("A5:8B:07:2D:A9:E8:53:CE:GB:C0:15:CE:6E:DF:9C:65:89:A3:AC:D2:94:2C:46:BD:85:23:20:6B:F2:69:B3:88")
.BasicAuthentication("elastic", "-L-uXRg5=iOXGFgebP68")
.DeadTimeout(TimeSpan.FromSeconds(300))
.DefaultIndex("people");
var client = new ElasticClient(settings);
var person = new Person
{
Id = 1,
FirstName = "Martijn",
LastName = "Laarman"
};
var asyncIndexResponse = await client.IndexDocumentAsync(person);
return Task.CompletedTask;
But I have Error
enter image description here
error message:
Message = "The client is unable to verify that the server is Elasticsearch due to an unsuccessful product check call. Some functionality may not be compatible if the server is running an unsupported product. Call: Status code unknown from: GET /"
Enable the compatibility header in the connection settings:
settings.EnableApiVersioningHeader(); // enable ES 7.x compatibility on ES 8.x servers
Docu here under Enabling Compatibility Mode
I have this test that launches a service at port 7000 and inside the only endpoint I do a failing assertion:
#Test
fun `javalin assertion should fail`() {
Javalin.create()
.get("/") { assertTrue(false) }
.start()
newHttpClient().send(
HttpRequest.newBuilder()
.uri(URI.create("http://localhost:7000/"))
.GET().build(),
discarding()
)
}
The problem is that the test always passes (but it should fail):
(same behavior happens by running ./gradlew test)
... even though there's a console output claiming that a test failed:
[Test worker] INFO io.javalin.Javalin - Listening on http://localhost:7000/
[Test worker] INFO io.javalin.Javalin - Javalin started in 356ms \o/
[qtp2100106358-22] ERROR io.javalin.Javalin - Exception occurred while servicing http-request
org.opentest4j.AssertionFailedError: expected: <true> but was: <false>
at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
..
Probably, it's being run in another thread, but I wonder if there's a way to attach it to the same context.
(Weirdly, in another scenario in my app - that I couldn't isolate - it properly fails.)
TLDR
To make the test fail as you'd expect, add an assertion on the response you get from your Javalin instance.
#Test
fun `javalin assertion should fail`() {
Javalin.create()
.get("/") { assertTrue(false) } // or any expression that throws an Exception, like Kotlin's TODO()
.start()
val javalinResponse: HttpResponse<Void> = newHttpClient().send(
HttpRequest.newBuilder()
.uri(URI.create("http://localhost:7000/"))
.GET().build(),
discarding()
)
assertThat(javalinResponse.statusCode()).isEqualTo(200) // will fail with Expected: 200, Actual: 500
}
Details
There're two distinct steps in the test: new Javalin instance configuration + build, and calling that instance with a HttpClient.
In Javalin configuration + build step it gets instructed to do assertTrue(false) when called on the / endpoint. assertTrue(false) will throw an AssertionFailedError, but the behavior will be the same if you throw something else there. Now, as many (all?) other webservers, Javalin / Jetty will try to catch any uncaught exceptions that happen within it and return a HTTP response with code 500 (Internal Server Error).
Indeed, this all happens in another thread, as internally a Jetty webserver instance is being launched, which does the port listening, HTTP request / response handling and other important stuff.
So when later in the test an HTTP call is performed to the new Javalin instance, it gets the 500 (Internal Server Error) response successfully, and as originally there are no assertions on the response and there were no uncaught exceptions, the test is deemed successful.
Never do assertions inside the Javalin handler, because if the test fails, the JUnit exception is swallowed by Javalin and the test fails silently (better explained in the other answer). The solution is to make an assertion outside, in the end, as in the Arrange, Act, Assert pattern.
How? You store what you want to assert inside the handler and assert it later. For example, if it's a POST.
var postedBody: String? = null
fakeProfileApi = Javalin.create().post("profile") {
postedBody = it.body()
}.start(1234)
val profileGateway = ProfileGateway(apiUrl = "http://localhost:1234")
profileGateway.saveProfile( // contains the HTTP POST
Profile(id = "abc", email = "john.doe#gmail.com".toEmail())
)
JSONAssert.assertEquals(
""" { "id": "abc", "email": "johndoe#gmail.com" } """,
postedBody, true
)
If it's a GET, it's easier:
fakeProfileApi = Javalin.create().get("profile/abc") {
it.result(""" {"id": "abc", "email": "johndoe#gmail.com"} """)
}.start(1234)
val profileGateway = ProfileGateway(apiUrl = "http://localhost:1234")
val result = profileGateway.fetchProfile("abc") // contains the HTTP GET
assertEquals(
Profile(id = "abc", email = "john.doe#gmail.com".toEmail()),
result
)
More info: Unit testing a gateway with Javalin
I forked this simple server-client akka project:
https://github.com/roclas/akka-irc
which is an IRC-like chat and I'm trying to encode messages.
In my master branch, if I start a server (sbt run and then select option 2) and then a client (sbt run and then select option 1),
if I write something in the client, the message is correctly sent to the server.
If I start wireshark and listen to the messages that meet these conditions:
tcp.port==1099 and tcp.len>200
I can read the messages in plain text.
How could I encode them using SSL?
You can see what I am trying to do by modifying the src/main/resources/application.conf file in the develop branch
What would I have to modify?
How should my src/main/resources/application.conf file look like?
Thank you
You should enable SSL at yout custom .conf file with:
akka {
actor {
provider = "akka.remote.RemoteActorRefProvider"
}
remote {
enabled-transports = ["akka.remote.netty.ssl"]
netty.ssl{
enable-ssl = true
security {
key-store = "path-to-your-keystore"
key-store-password = "your-keystore's-password"
key-password = "your-key's-password"
trust-store = "path-to-your-truststore"
trust-store-password = "your-trust-store's-password"
protocol = "TLSv1"
random-number-generator = "AES128CounterSecureRNG"
enabled-algorithms = ["TLS_RSA_WITH_AES_128_CBC_SHA"]
}
}
}
}
And don't forget to change your actor path's prefix to:
akka.ssl.tcp://YourActorSystemName#ip:port:/...
In addition to what J.Santos said, I had forgotten to create these two files:
trust-store = "path-to-your-truststore"
trust-store-password = "your-trust-store's-password"
that I changed by:
key-store = "src/main/resources/keystore"
trust-store = "src/main/resources/truststore"
in my ./src/main/resources/common.conf
as J.Santos reminded me after looking at my project.
Thank you very much!!
I'm working on a mobile prof-of-concept using IBM's Worklight (6.1) to retrieve info via HTTP server (Apache) running on a mainframe (z/OS). I'm using the HTTP adapter procedure to log-on and retrieve data but I so far no success logging on via Worklight HTTP adapter. If I open a browser and provide the 'user:password' headers, the log-in is successful but if I try it via Worklight procedure, the '401 authorization required' error is returned. The HTTP server error log shows:
.. (139)EDC5139I Operation not permitted. (errno2=0x0BE800DB): SAF
authentication failure for "/cgi-bin/itil_v11_main.sh": SAFRunAs
failure on switching SAF UID from Authorization header using
%%CLIENT%% .. user (\xe1\xcb: authentication failure for
"/cgi-bin/itil_v11_main.sh": Password Mismatch
That 'password mismatch' may suggest the 'headers' are not correct? Here's the procedure:
var user_id = 'userid';
var user_psw = 'userpassword';
var loginstring ;
var base64= new com.worklight.customcode.Base64Encoding();
function getITIL() {
loginstring = base64.encode(user_id+':'+user_psw);
var path = '/cgi-bin/itil_v11_main.sh';
var input = {
method : 'get',
headers : {
'Authorization' : 'Basic ' + loginstring
},
returnedContentType : 'html',
path : path
};
return WL.Server.invokeHttp(input);
}
It seems like you've implemented it correctly, however the complaint is on the password, which in your case originates from var base64= new com.worklight.customcode.Base64Encoding();.
Because you do not supply the code that you are using in said class, it's difficult to say what the error is, but that is where you should look at for the cause of your error.
You'll need to provide the class's implementation in order to further debug the question.