So I am creating a distributed Key-Value datastore and have a broker to manage/query.
Here is my client:
val client = HttpClient(CIO) {
install(WebSockets)
}
runBlocking {
client.ws(
method = HttpMethod.Get,
host = ip,
port = port,
path = "/thepath"
) {
...
}
}
client.close()
So far I can connect only one server to the client (Obviously by the code above).
What I tried is to create an array of all the available servers and randomly pick one and work with the broker (client). But this works only for the connected server and the others should wait until the connection is closed.
val clients: Array<HttpClient?> = arrayOfNulls(replicationFactor)
for (i in 0 until replicationFactor) {
clients[i] = HttpClient(CIO) {
install(WebSockets)
}
}
runBlocking {
clients[0]?.ws(
method = HttpMethod.Get,
host = "some ip",
port = the_port,
path = "/thepath"
) {
....
}
...
...
}
Any ideas of how to tackle this problem? Maybe I can keep the connection with each server on a separate thread.
You can create any number of HTTP clients and connect them to a server concurrently. Here is an example:
suspend fun main() {
val clients = (0 until 3).map {
HttpClient(CIO) {
install(WebSockets)
}
}
val connections = coroutineScope {
clients.mapIndexed { index, client ->
async {
client.ws("wss://echo.websocket.org") {
outgoing.send(Frame.Text("Hello server"))
incoming.consumeEach { frame ->
if (frame is Frame.Text) {
println("[$index] Server replied ${frame.readText()}")
}
}
}
}
}.toTypedArray()
}
awaitAll(*connections)
}
Related
I'm building my first desktop application using JetBrains Compose & Ktor. I want to connect to the Spotify API using the Spotify Auth PCKE extension. The flow should be
Send a Get request for auth to accounts.spotify.com/authorize
If the user not auth'd -> launch a webpage for the user to sign-in
Spotify sends a callback to a url provided in the initial request.
Recieve a Token from Spotify
I'm facing an issue when launching the Webpage for user sign-in and getting the response on the URL I've provided. I'm launching the webpage using the desktop's default browse and opening a websocket that should be listening to "http://localhost:8888/callback)." I noticed when I interact with the launched webpage, the url shows the response that Spotify sent back ("http://localhost:8888/callback?error=access_denied&state=initial" but my websocket code is never called. Is this an issue with how I'm launching the browser or the websocket... or am I going about this wrong in general?
class SpotifyClient {
private val client: HttpClient = HttpClient(CIO) {
followRedirects = true
install(WebSockets) {
contentConverter = KotlinxWebsocketSerializationConverter(Json)
}
handleSpotifyAccountAuthResponse()
}
private val clientId = "<Removed for Post>"
suspend fun authorizeSpotifyClient(): HttpResponse {
val withContext = withContext(Dispatchers.IO) {
val response: HttpResponse =
client.get(NetworkConstants.BASE_URL_SPOTIFY_ACCOUNTS + NetworkConstants.PATH_SPOTIFY_AUTH) {
header("Location", NetworkConstants.BASE_URL_SPOTIFY_AUTH_REDIRECT_URI)
parameter("client_id", clientId)
parameter("response_type", "code")
parameter("redirect_uri", NetworkConstants.BASE_URL_SPOTIFY_AUTH_REDIRECT_URI)
parameter("state", ClientStates.INITIAL.value)
parameter("show_dialog", false)
parameter("code_challenge_method", PCKE_CODE_CHALLENGE_METHOD)
parameter("code_challenge", generatePCKECodeChallenge())
parameter(
"scope",
NetworkUtils.getSpotifyScopes(
ImmutableList.of(
SpotifyScopes.USER_READ_PLAYBACK_STATE,
SpotifyScopes.USER_READ_CURRENTLY_PLAYING
)
)
)
}
println("Auth Spotify API Response $response")
println("Auth Spotify API Response Code ${response.status}")
client.close()
return#withContext response
}
accountAuthRedirectWebsocket()
return withContext
}
private fun HttpClientConfig<CIOEngineConfig>.handleSpotifyAccountAuthResponse() {
HttpResponseValidator {
validateResponse { response ->
handleValidAccountResponse(response)
}
handleResponseExceptionWithRequest { exception, request ->
}
}
}
private fun handleValidAccountResponse(response: HttpResponse) {
if (response.status.value == 200) { // success
val responseUrl = response.call.request.url
if (responseUrl.toString().contains("continue")) {
println("Needs a redirect, ${responseUrl.toString()}")
openWebpage(responseUrl.toURI())
}
}
}
private fun openWebpage(uri: URI?): Boolean {
val desktop = if (Desktop.isDesktopSupported()) Desktop.getDesktop() else null
if (desktop != null && desktop.isSupported(Desktop.Action.BROWSE)) {
try {
desktop.browse(uri)
return true
} catch (e: Exception) {
e.printStackTrace()
}
}
return false
}
private suspend fun accountAuthRedirectWebsocket(){
client.webSocket(method = HttpMethod.Get, host = "localhost", port = 8888, path = "/customer/1") {
println("Got a response in the socket")
}
}
To receive the callback I needed to implement a local server like below. Using the WebSocket was the wrong approach in this case and didn't provide the functionality I was expecting.
object SpotifyServer {
suspend fun initServer() {
embeddedServer(CIO,
host = "127.0.0.1",
port = 8080,
configure = {}) {
routing {
get("/callback") {
call.respondText("Got a callback response")
}
}
}.start(wait = true)
}
}
So im trying to request a side with proxies that is protected with cloudflare. The problem is i get 403 forbidden cloduflare error but only when im using proxies without it works. But the proxies are not the problem i tried them with python(requests module) and in my browser there i dont get blocked. My code
suspend fun scrape() {
val client = HttpClient {
followRedirects = true
install(ContentNegotiation) {
json(Json {
ignoreUnknownKeys = true
})
}
engine {
proxy =
ProxyBuilder.http("http://ProxyIP:proxyPort")
}
defaultRequest {
val credentials = Base64.getEncoder().encodeToString("ProxyUser:ProxyPassword".toByteArray())
header(HttpHeaders.ProxyAuthorization, "Basic $credentials")
}
}
val response = client.get("http://example.com")
val body = response.bodyAsText()
println(body)
println(response.status.hashCode())
Fixxed it
suspend fun scrape() {
val client = HttpClient(Apache) {
install(ContentNegotiation) {
json(Json {
ignoreUnknownKeys = true
})
}
engine {
followRedirects = false
customizeClient {
setProxy(HttpHost("hostname", port))
val credentialsProvider = BasicCredentialsProvider()
credentialsProvider .setCredentials(
AuthScope("hostname", port),
UsernamePasswordCredentials("username", "password")
)
setDefaultCredentialsProvider(credentialsProvider )
}
}
}
val response =
client.get("http://example.com") {
}
val body = response.bodyAsText()
println(body)
println(response.status.hashCode())
}
There is a problem with making a request through a proxy server using the CIO engine. I've created an issue to address this problem. As a workaround, please use the OkHttp engine instead of CIO.
Here is how you can use a proxy with the Basic authentication:
import io.ktor.client.*
import io.ktor.client.engine.okhttp.*
import io.ktor.client.request.*
import kotlinx.coroutines.runBlocking
import okhttp3.Authenticator
import okhttp3.Credentials
import okhttp3.OkHttpClient
import java.net.Proxy
fun main(): Unit = runBlocking {
val proxyAuthenticator = Authenticator { _, response ->
response.request.newBuilder()
.header("Proxy-Authorization", Credentials.basic("<username>", "<password>"))
.build()
}
val client = HttpClient(OkHttp) {
engine {
preconfigured = OkHttpClient.Builder()
.proxy(Proxy(Proxy.Type.HTTP, java.net.InetSocketAddress("127.0.0.1", 3128)))
.proxyAuthenticator(proxyAuthenticator)
.build()
}
}
val response = client.get("http://eu.kith.com/products.json")
println(response.status)
}
I'm receiving data through bluetooth with a custom protocol so I need to "decode" the packet and then show the result. I'm using a function of an object (singleton) to decode the bytes received but the communication is async so it's possible that I'm decoding the bytes/packet while I'm receiving other bytes. I need a solution to process the incoming data sequentially and not call the decode function while I'm already decoding a previous packet.
Here is my actual code. I tried to use mutex without success. Using a defined coroutineScope I should work on the same coroutine. Obviously the decode process have to run in background.
// Activity
bluetoothService.incomingData.observe(this, {
viewModel.getData(it) })
// ViewModel
private var coroutineJob = Job()
private val uiScope = CoroutineScope(Dispatchers.Default + coroutineJob)
fun getData(bytes: ByteArray) {
getDataSafeCall(bytes)
}
private fun getDataSafeCall(bytes: ByteArray) {
uiScopeJob = uiScope.launch {
Packet.processMessage(bytes.toUByteArray()).collect {
updateData(it)
}
}
}
// Packet Object
object Packet {
private val mutex = Mutex()
suspend fun processMessage(buffer: UByteArray) = flow {
mutex.withLock {
decode(buffer).collect { emit(it) }
}
}
private fun decode(buffer: UByteArray) = channelFlow {
// Here I'm processing my packet and then
offer()
}
}
EDIT: This is part of the Bluetooth Service if it helps to understand more, even if there's no problem receiving data.
// BluetoothService
class BluetoothService : Service() {
private val btChannel: BtChannel by inject()
private var socket: BluetoothSocket? = null
private val receivedLive = MutableLiveData<ByteArray>()
fun incomingData(): LiveData<ByteArray> {
if (socket != null) {
socket.let {
btChannel.read(socket).onEach
{
withContext(Dispatchers.Main) {
receivedLive.value = it
}
}.launchIn(CoroutineScope(Dispatchers.IO))
}
}
return receivedLive
}
}
I have the following code, that it going to start a http server:
class MainVerticle : CoroutineVerticle() {
override suspend fun start() {
val server = vertx.createHttpServer()
val router = Router.router(vertx)
router.route("/api/genders*")
.subRouter(GenderApi(vertx).create())
server.requestHandler(router)
.listen(8080)
.await()
}
}
Now, I would like to output, if the server has been successfully started or failed(in case the port has been already occupied).
Without the Coroutine, the codes would be:
class MainVerticle : AbstractVerticle() {
override fun start(startPromise: Promise<Void>) {
val server = vertx.createHttpServer()
val router = Router.router(vertx)
server.requestHandler(router).listen(8888) { http ->
if (http.succeeded()) {
startPromise.complete()
println("HTTP server started on port 8888")
} else {
println(http.cause())
startPromise.fail(http.cause());
}
}
}
}
Here I do output, if the server has been started success or not.
If you use coroutines, add a try/catch block:
class MainVerticle : CoroutineVerticle() {
override suspend fun start() {
val server = vertx.createHttpServer()
val router = Router.router(vertx)
router.route("/api/genders*")
.subRouter(GenderApi(vertx).create())
try {
server.requestHandler(router)
.listen(8080)
.await()
println("HTTP server started on port 8888")
} catch (e: Exception) {
println(http.cause())
throw e
}
}
}
I am sending a message(custom protocol, no HTTP) to my server and want to wait for a response. It is working with the following code:
class Connection {
val messages: Observable<Message>
fun sendMessageWithAnswer(message: Message, timeout:Int = 10): Observable<Answer> {
if (!isConnected) {
return Observable.just(Answer.NoConnection)
}
val result = BehaviorSubject.create<Answer>()
val neverDisposed = messages.filter {
it.header.messageId == message.header.messageId
}
.map { Answer.Success(it) as Answer}
.mergeWith(Observable.timer(timeout.toLong(), TimeUnit.SECONDS)
.map { Answer.Timeout })
.take(1).singleOrError()
.subscribe(
{result.onNext(it)},
{
// Should never happen
throw IllegalStateException("Waiting for answer failed: $it")
}
)
sendMessage(message)
return result
}
}
The problem with this solution that "neverDisposed" gets never disposed, is this a memory leak?
My other solutions are not working for this test case:
#Test
fun ImmediateAnswer() {
prepare()
val message = ...
val answerObservable = connection.sendMessageWithAnswer(message, timeout = 1)
connection.receiveMessage(message)
val answer = answerObservable.test()
answer.awaitCount(1)
Thread.sleep(1000)
Assert.assertEquals(1, answer.valueCount())
Assert.assertEquals(Answer.Success(message), answer.values()[0])
}
Do you have a cleaner solution for this problem?