I want to post http requests to a secured server with a given ca cert.
I'm using Spray 1.3.1, the code looks something like this:
val is = this.getClass().getResourceAsStream("/cacert.crt")
val cf: CertificateFactory = CertificateFactory.getInstance("X.509")
val caCert: X509Certificate = cf.generateCertificate(is).asInstanceOf[X509Certificate];
val tmf: TrustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
val ks: KeyStore = KeyStore.getInstance(KeyStore.getDefaultType());
ks.load(null);
ks.setCertificateEntry("caCert", caCert);
tmf.init(ks);
implicit val sslContext: SSLContext = SSLContext.getInstance("TLS");
sslContext.init(null, tmf.getTrustManagers(), null);
implicit val timeout: Timeout = Timeout(15.seconds)
import spray.httpx.RequestBuilding._
val respFuture = (IO(Http) ? Post( uri=Uri(url), content="my content")).mapTo[HttpResponse]
The problem is that the defined implicit SSLContext is not taken and I'm getting: "unable to find valid certification path to requested target" on runtime.
How can I define a SSLContext to be used with spray client?
I use the following to define an SSLContext in spray. In my case, I'm using a very permissive context that does not validate the remote server's certificate. Based on the first solution in this post - works for me.
import java.security.SecureRandom
import java.security.cert.X509Certificate
import javax.net.ssl.{SSLContext, X509TrustManager, TrustManager}
import akka.actor.ActorRef
import akka.io.IO
import akka.util.Timeout
import spray.can.Http
import scala.concurrent.Future
trait HttpClient {
/** For the HostConnectorSetup ask operation. */
implicit val ImplicitPoolSetupTimeout: Timeout = 30 seconds
val hostName: String
val hostPort: Int
implicit val sslContext = {
/** Create a trust manager that does not validate certificate chains. */
val permissiveTrustManager: TrustManager = new X509TrustManager() {
override def checkClientTrusted(chain: Array[X509Certificate], authType: String): Unit = {
}
override def checkServerTrusted(chain: Array[X509Certificate], authType: String): Unit = {
}
override def getAcceptedIssuers(): Array[X509Certificate] = {
null
}
}
val initTrustManagers = Array(permissiveTrustManager)
val ctx = SSLContext.getInstance("TLS")
ctx.init(null, initTrustManagers, new SecureRandom())
ctx
}
def initClientPool(): Future[ActorRef] = {
val hostPoolFuture = for {
Http.HostConnectorInfo(connector, _) <- IO(Http) ? Http.HostConnectorSetup(hostName, port = hostPort,
sslEncryption = true)
} yield connector
}
}
I came up with this replacement for sendReceive which allows passing a custom SSLContext (as an implicit)
def mySendReceive( request: HttpRequest )( implicit uri: spray.http.Uri, ec: ExecutionContext, futureTimeout: Timeout = 60.seconds, sslContext: SSLContext = SSLContext.getDefault): Future[ HttpResponse ] = {
implicit val clientSSLEngineProvider = ClientSSLEngineProvider { _ =>
val engine = sslContext.createSSLEngine( )
engine.setUseClientMode( true )
engine
}
for {
Http.HostConnectorInfo( connector, _ ) <- IO( Http ) ? Http.HostConnectorSetup( uri.authority.host.address, port = uri.authority.port, sslEncryption = true )
response <- connector ? request
} yield response match {
case x: HttpResponse ⇒ x
case x: HttpResponsePart ⇒ sys.error( "sendReceive doesn't support chunked responses, try sendTo instead" )
case x: Http.ConnectionClosed ⇒ sys.error( "Connection closed before reception of response: " + x )
case x ⇒ sys.error( "Unexpected response from HTTP transport: " + x )
}
}
Then use it as "usual" (almost see below):
val pipeline: HttpRequest => Future[ HttpResponse ] = mySendReceive
pipeline( Get( uri ) ) map processResponse
There are a couple of things I really do not like though:
it is a hack. I would expect spray-client to allow support of a custom SSLContext natively. These are very useful during dev and test, to force custom TrustManagers typically
there is an implicit uri: spray.http.Uri parameter to avoid hard coding of the host and port on the connector. So uri must be declared implicit.
Any improvement to this code or even better, a patch to spray-client, is most welcome (externalization of the creation of the SSLEngine being an obvious one)
The shortest I've gotten to work is this:
IO(Http) ! HostConnectorSetup(host = Conf.base.getHost, port = 443, sslEncryption = true)
i.e. what's in #reed-sandberg's answer, but no ask pattern seems to be needed. I don't pass a connection parameter to sendReceive but instead:
// `host` is the host part of the service
//
def addHost = { req: HttpRequest => req.withEffectiveUri(true, Host(host, 443)) }
val pipeline: HttpRequest => Future[Seq[PartitionInfo]] = (
addHost
~> sendReceive
~> unmarshal[...]
)
This seems to work, but I would naturally be interested to hear if there are downsides to this approach.
I agree with all spray-client SSL support criticism. It's awkward that something like this is so hard. I probably spent 2 days on it, merging data from various sources (SO, spray documentation, mailing list).
Related
I am trying to use http4k's built in oauth module to implement Google sign-in in my backend app.
Here is what I have so far:
val googleClientId = "<GoogleClientID>"
val googleClientSecret = "<GoogleClientSecret>"
// this is a test implementation of the OAuthPersistence interface, which should be
// implemented by application developers
val oAuthPersistence = InsecureCookieBasedOAuthPersistence("Google")
// pre-defined configuration exist for common OAuth providers
val oauthProvider = OAuthProvider.google(
JavaHttpClient(),
Credentials(googleClientId, googleClientSecret),
Uri.of("http://localhost:9000/oauth/callback"),
oAuthPersistence
)
val app: HttpHandler = routes(
"/oauth" bind routes(
"/" bind GET to oauthProvider.authFilter.then {
val user = contextFn(it)
Response(OK).body("authenticated!")
},
"/callback" bind GET to oauthProvider.callback
)
app.asServer(SunHttp(9000)).start()
This lets me go to http://localhost:9000/oauth and I can sign-in to my google account. Cool!
However, after the redirect, I go to the following function contextFn, which looks like this atm:
val transport = NetHttpTransport()
val jsonFactory = GsonFactory.getDefaultInstance()
val verifier = GoogleIdTokenVerifier.Builder(transport, jsonFactory)
.setAudience(listOf(googleClientId))
.build()
fun contextFn(request: Request): Principal {
// TODO: get the id token somehow, but the request header only contains the following in cookie:
// - GoogleCsrf
// - GoogleAccessToken
// - GoogleOriginalUri
val idTokenString = ""
val idToken: GoogleIdToken = verifier.verify(idTokenString)
val payload: GoogleIdToken.Payload = idToken.payload
// Print user identifier
val userId: String = payload.subject
println("User ID: $userId")
// Get profile information from payload
val email: String = payload.email
val emailVerified: Boolean = payload.emailVerified
val name = payload["name"]
return GoogleUser(email)
}
How can i get the id token? Currently I am getting the access token from google.
Can you try to add this scope(openid)?
I am not sure listOf or addScope support in http4k
But it missed openid scope.
val oauthProvider = OAuthProvider.google(
JavaHttpClient(),
Credentials(googleClientId, googleClientSecret),
Uri.of("http://localhost:9000/oauth/callback"),
listOf("openidScope"),
oAuthPersistence
)
oauthProvider.addScope('openid');
I have a simple function in kotlin like that :
suspend fun createTicket(#Valid request: CreateTicketRequest, authentication: Authentication): HttpResponse<Any> {
request.customerId = "customerId"
logger().info("Receive by the client $request")
return HttpResponse.created(service.create(request))
}
I've already Mock the request and the authentication.
So, I call it on Spock:
def 'It should create a ticket with success'() {
given:
def request = createRequest(
TICKET_ID,
TICKET_NAME,
TICKET_PHONE,
TICKET_CPF,
TICKET_EMAIL,
TICKET_COMMENT,
TICKET_SUBJECT,
TICKET_TAG
)
when:
response = controller.createTicket(
request,
authentication
)
then:
response != null
}
I'm getting the following error :
Suspend function 'create' should be called only from a coroutine or another suspend function.
Can anyone help me with this question ?
Best regards
Solved I created a Kotlin class code
class CallCreateTicket {
private lateinit var response: HttpResponse<Any>
private fun createTicket(
request: CreateTicketRequest,
authenticator: Authenticator,
controller: TicketController,
): HttpResponse<Any> {
runBlocking {
response = controller.createTicket(request, authenticator)
}
return response
}
}
and I called it on groovy ...
#Mockable(TicketCreateServiceImpl)
class TicketControllerTest extends Specification {
def mockUtil = new MockUtil()
def service = Mock(TicketCreateServiceImpl)
def authenticator = Mock(Authenticator)
def response = Mock(HttpResponse)
def controller = new TicketController(service)
def callCreateTicket = new CallCreateTicket()
def 'check if all instances are mocked'() {
mockUtil.isMock(authentication)
mockUtil.isMock(service)
}
def 'It should call the index function and return a valid String'() {
when:
response = controller.index()
then:
response == INDEX_RETURN
}
def 'It should call the index function and return a invalid String'() {
when:
response = controller.index()
then:
response != INVALID_INDEX_RETURN
}
def 'It should create a ticket with success'() {
given:
def request = createRequest(
TICKET_ID,
TICKET_NAME,
TICKET_PHONE,
TICKET_CPF,
TICKET_EMAIL,
TICKET_COMMENT,
TICKET_SUBJECT,
TICKET_TAG
)
when:
response = callCreateTicket.createTicket(
request,
authenticator,
controller
)
then:
response.status(HttpStatus.CREATED)
}
}
I have the following test http, and I would like to get all the information from the request in the default route.
class HttpTestServer(port: Int = 6666)(implicit as: ActorSystem, ec: ExecutionContext, mat: ActorMaterializer)
extends JsonSupport
with LazyLogging{
var numRequest: Int = 0
def start(): Future[ServerBinding] = {
def routes = {
// print request information here
logRequest("test")
numRequest += 1
println(s"HttpServer request received: $numRequest")
complete(Map("key"-> 1.0))
}
Http().bindAndHandle(routes, "0.0.0.0", port)
}
}
And I have the following line at logback.xml, to enable the debug at akka-http:
<logger name="akka.http" level="DEBUG"/>
As a references, I am using akka-http DebuggingDirective
UPDATED: My guess is that there is a problem with the LoggingAdapter or incompatibility between LazyLogging and the akka LoggingAdapter. If I print to stdout works fine:
val logRequestPrintln = DebuggingDirectives.logRequest(LoggingMagnet(_ => println _))
....
logRequestPrintln(complete(Map("key"-> 1.0)))
I am using Vertx 3 with Kotlin, and at times I need to return a specific URI from the perspective of the public URL which is not the same as what the Vertx-web request thinks my URL is. This is likely due to my load balancer or proxy receiving one URL, and then forwarding to my application on an internal URL.
So if I do this:
val publicUrl = context.request().absoluteURI()
I end up with a URL like http://10.10.103.22:8080/some/page instead of https://app.mydomain.com/some/page. Everything is wrong about that URL!
I found a header that supposedly tell me more about the original request such as X-Forwarded-Host but it only includes app.mydomain.com or sometimes it has the port app.mydomain:80 but that isn't enough to figure out all parts of the URL, I end up with something like http://app.mydomain.com:8080/some/page which is still not the correct public URL.
I also need to handle not just my current URL, but peer URL's, like while on page "something/page1" go to "something/page2" on same server. The same problems mentioned about when I try to resolve to another URL because important parts of the public URL are unobtainable.
Is there a method in Vertx-web I'm missing to determine this public URL, or some idiomatic way to solve this?
I'm coding in Kotlin, so any examples for that language are great!
Note: this question is intentionally written and answered by the author (Self-Answered Questions), so that solutions for interesting problems are shared in SO.
This is a more complicated issue, and the logic is the same for most App servers if they do not already provide an URL externalization function.
To do this correctly, you need to handle all of these headers:
X-Forwarded-Proto (or X-Forwarded-Scheme: https, and maybe oddballs like X-Forwarded-Ssl: on, Front-End-Https: on)
X-Forwarded-Host (as "myhost.com" or "myhost.com:port")
X-Forwarded-Port
And if you want to resolve and return a URL that is not the current one you need to also consider:
partial without host, for example "/something/here" or "under/me" resolving to the servers public protocol, host, port as well as that abosolute or relative path
partial with host/port, for example "//somehost.com:8983/thing" would add the same scheme (http/https) as this server and keep the rest
full, URL's that are fully qualified are returned untouched, so they are safe to pass to this function ("http://...", "https://...") and won't be modified
Here is a pair of extension functions to RoutingContext that will handle all these cases and fall back when the load balancer / proxy headers are not present so will work in both cases of direct connections to the server and those going through the intermediary. You pass in the absolute or relative URL (to the current page) and it will return a public version of the same.
// return current URL as public URL
fun RoutingContext.externalizeUrl(): String {
return externalizeUrl(URI(request().absoluteURI()).pathPlusParmsOfUrl())
}
// resolve a related URL as a public URL
fun RoutingContext.externalizeUrl(resolveUrl: String): String {
val cleanHeaders = request().headers().filterNot { it.value.isNullOrBlank() }
.map { it.key to it.value }.toMap()
return externalizeURI(URI(request().absoluteURI()), resolveUrl, cleanHeaders).toString()
}
Which call an internal function that does the real work (and is more testable since there is no need to mock the RoutingContext):
internal fun externalizeURI(requestUri: URI, resolveUrl: String, headers: Map<String, String>): URI {
// special case of not touching fully qualified resolve URL's
if (resolveUrl.startsWith("http://") || resolveUrl.startsWith("https://")) return URI(resolveUrl)
val forwardedScheme = headers.get("X-Forwarded-Proto")
?: headers.get("X-Forwarded-Scheme")
?: requestUri.getScheme()
// special case of //host/something URL's
if (resolveUrl.startsWith("//")) return URI("$forwardedScheme:$resolveUrl")
val (forwardedHost, forwardedHostOptionalPort) =
dividePort(headers.get("X-Forwarded-Host") ?: requestUri.getHost())
val fallbackPort = requestUri.getPort().let { explicitPort ->
if (explicitPort <= 0) {
if ("https" == forwardedScheme) 443 else 80
} else {
explicitPort
}
}
val requestPort: Int = headers.get("X-Forwarded-Port")?.toInt()
?: forwardedHostOptionalPort
?: fallbackPort
val finalPort = when {
forwardedScheme == "https" && requestPort == 443 -> ""
forwardedScheme == "http" && requestPort == 80 -> ""
else -> ":$requestPort"
}
val restOfUrl = requestUri.pathPlusParmsOfUrl()
return URI("$forwardedScheme://$forwardedHost$finalPort$restOfUrl").resolve(resolveUrl)
}
And a few related helper functions:
internal fun URI.pathPlusParmsOfUrl(): String {
val path = this.getRawPath().let { if (it.isNullOrBlank()) "" else it.mustStartWith('/') }
val query = this.getRawQuery().let { if (it.isNullOrBlank()) "" else it.mustStartWith('?') }
val fragment = this.getRawFragment().let { if (it.isNullOrBlank()) "" else it.mustStartWith('#') }
return "$path$query$fragment"
}
internal fun dividePort(hostWithOptionalPort: String): Pair<String, Int?> {
val parts = if (hostWithOptionalPort.startsWith('[')) { // ipv6
Pair(hostWithOptionalPort.substringBefore(']') + ']', hostWithOptionalPort.substringAfter("]:", ""))
} else { // ipv4
Pair(hostWithOptionalPort.substringBefore(':'), hostWithOptionalPort.substringAfter(':', ""))
}
return Pair(parts.first, if (parts.second.isNullOrBlank()) null else parts.second.toInt())
}
fun String.mustStartWith(prefix: Char): String {
return if (this.startsWith(prefix)) { this } else { prefix + this }
}
I managed to implement an https restlet with bot client and server certificated. I can prove it works since if I call the server with an untrusted certification communication fails. Unfortunately I can't find the certificate of the client on the server. I'm using this code:
List<Certificate> certs = request.getClientInfo().getCertificates();
but list is empty. What I'm doing wrong?
EDIT:
version is Restlet-Framework/2.3m2
The problem is related to the use of the default server implementation via com.sun.httpserver.
The class org.restlet.engine.connector.HttpExchangeCall should return the certificates in the getCertificates() method,
but it always returns null. This class is used in org.restlet.engine.connector.HttpsServerHelper
which in turn is the helper for the Restlet framework when using the server implementation com.sun.httpserver.
To fix this, a couple of things are needed.
First, a new class HttpsExchangeCall:
package org.restlet.engine.connector;
import java.security.cert.Certificate;
import java.util.ArrayList;
import java.util.List;
import org.restlet.Server;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpsExchange;
/**
* The default {#link HttpExchangeCall} fails to extract certificates from the SSL connection.
* This class implements {#link #getCertificates()} to extract certificates.
*/
#SuppressWarnings("restriction")
public class HttpsExchangeCall extends HttpExchangeCall {
private static final Logger log = LoggerFactory.getLogger(HttpsExchangeCall.class);
private final HttpsExchange sexchange;
public HttpsExchangeCall(Server server, HttpExchange exchange) {
this(server, exchange, true);
}
public HttpsExchangeCall(Server server, HttpExchange exchange, boolean confidential) {
super(server, exchange, confidential);
if (exchange instanceof HttpsExchange) {
sexchange = (HttpsExchange) exchange;
} else {
sexchange = null;
}
}
#Override
public List<Certificate> getCertificates() {
if (sexchange == null) {
log.debug("Cannot extract peer certificates from unsecure connection.");
return null;
}
Certificate[] certs = null;
try {
certs = sexchange.getSSLSession().getPeerCertificates();
if (log.isDebugEnabled()) {
log.debug("Found " + (certs == null ? "no" : Integer.toString(certs.length)) + " peer certificate(s).");
}
} catch (Exception e) {
log.debug("Unable to find peer certificates - " + e);
}
List<Certificate> lcerts = null;
if (certs != null) {
lcerts = new ArrayList<Certificate>();
for (int i = 0; i < certs.length; i++) {
lcerts.add(certs[i]);
}
}
return lcerts;
}
}
Then a copy of HttpsServerHelper
renamed to HttpsServerHelper2 with one line modified. Replace the line
HttpsServerHelper.this.handle(new HttpExchangeCall(getHelped(),
with the line:
HttpsServerHelper2.this.handle(new HttpsExchangeCall(getHelped(),
This helper needs to be registered:
Engine.getInstance().getRegisteredServers().add(new HttpsServerHelper2(null));
and creating a Server now becomes very explicit:
Component component = new Component();
Server server = new Server(
(Context) null, Arrays.asList(Protocol.HTTPS),
(String) null, Constants.PORT_TEST, component.getServers().getNext(),
HttpsServerHelper2.class.getName()
);
component.getServers().add(server);
I'm hoping Restlet's own HttpExchangeCall will be updated to extract the certificates:
it is a minor fix and saves a lot of unneeded code required to work around the issue.
In the mean time, you can find all the source code (using Restlet 2.3.4) and a working example in the
restlet-clientcert Github project.
This method should provide what you are looking for: request.getClientInfo().getCertificates()
http://restlet.com/learn/javadocs/2.3/jse/api/org/restlet/data/ClientInfo.html
You can also retrieve the cipher suites