I'm trying to initialize Ktor http client and setup json serialization. I need to allow non-strict deserialization which JSON.nonstrict object allows. Just can't get how to apply this setting to serializer.
val client = HttpClient {
install(JsonFeature) {
serializer = KotlinxSerializer()
}
}
You can specify json configurations using the Json builder, which you pass into the KotlinxSerializer.
val client = HttpClient {
install(JsonFeature) {
serializer = KotlinxSerializer(Json {
isLenient = true
ignoreUnknownKeys = true
})
}
}
The exact fields for the Json builder is experimental and subject to change, so check out the source code here.
Figured out - we can pass in constructor:
serializer = KotlinxSerializer(Json.nonstrict)
After Kotlin 1.4.0 released:
use this for converting string to Object:
val response = Json {
ignoreUnknownKeys = true
}.decodeFromString(ResponseObject.serializer(), jsonString)
And for your httpClient use:
HttpClient {
install(JsonFeature) {
serializer = KotlinxSerializer()
}
install(Logging) {
logger = Logger.DEFAULT
level = LogLevel.ALL
}
}
This change very often, but with Kotlin 1.4.10 and Ktor 1.4.1 you need to pass a kotlinx Json (be careful because there is also a io.ktor.client.features.json.Json, I used an import alias to distinguish them because I needed both import kotlinx.serialization.json.Json as KotlinJson)
val client = HttpClient {
install(JsonFeature) {
serializer = KotlinxSerializer(KotlinJson { ignoreUnknownKeys = true })
}
...
For those who use retrofit, you might want to consider using JsonConfiguration(strictMode = false) on the retrofit builder.
E.g:
// your retrofit builder
Retrofit.Builder()
.baseUrl(url)
.client(okHttpClient)
.client(httpClient)
.addConverterFactory(
Json(JsonConfiguration(strictMode = false))
.asConverterFactory(MediaType.get("application/json")
)
)
Source: issue on the kotlinx github
Working from Rodion Altshuler's answer above, this is what worked for me in my KMP project:
install(JsonFeature) {
serializer = KotlinxSerializer(kotlinx.serialization.json.Json(JsonConfiguration.Stable.copy(strictMode = false))).apply {
useDefaultTransformers = true
}
}
With the "1.0.0RC" version, the use with retrofit is as follows.
Retrofit.Builder()
.client(okHttpClient)
.baseUrl(Env.BASE_URL)
.addConverterFactory(Json{
isLenient = true
ignoreUnknownKeys = true
}.asConverterFactory(MediaType.get("application/json")))
.addCallAdapterFactory(CoroutineCallAdapterFactory())
.build()
This is how you can configure the JsonConfig for the Spring reactive Webclient:
val json = Json { ignoreUnknownKeys = true isLenient = true }
val strategies = ExchangeStrategies
.builder()
.codecs { clientDefaultCodecsConfigurer ->
run {
clientDefaultCodecsConfigurer.defaultCodecs()
.kotlinSerializationJsonDecoder(KotlinSerializationJsonDecoder(json))
clientDefaultCodecsConfigurer.defaultCodecs()
.kotlinSerializationJsonEncoder(KotlinSerializationJsonEncoder(json))
}
}.build()
return WebClient
.builder()
.exchangeStrategies(strategies)
.baseUrl(baseUrl!!)
.build()
It seems that for 1.4.32 I have it as follows:
install(JsonFeature) {
serializer = KotlinxSerializer(json = kotlinx.serialization.json.Json {
isLenient = true
ignoreUnknownKeys = true
})
}
For ktor 2.0+
return HttpClient(CIO) {
engine {
maxConnectionsCount = 10
}
install(ContentNegotiation) {
json(kotlinx.serialization.json.Json {
ignoreUnknownKeys = true
})
}
install(HttpTimeout) {
requestTimeoutMillis = 1000L
connectTimeoutMillis = 1000L
socketTimeoutMillis = 1000L
}
}
Related
So im trying to request a side with proxies that is protected with cloudflare. The problem is i get 403 forbidden cloduflare error but only when im using proxies without it works. But the proxies are not the problem i tried them with python(requests module) and in my browser there i dont get blocked. My code
suspend fun scrape() {
val client = HttpClient {
followRedirects = true
install(ContentNegotiation) {
json(Json {
ignoreUnknownKeys = true
})
}
engine {
proxy =
ProxyBuilder.http("http://ProxyIP:proxyPort")
}
defaultRequest {
val credentials = Base64.getEncoder().encodeToString("ProxyUser:ProxyPassword".toByteArray())
header(HttpHeaders.ProxyAuthorization, "Basic $credentials")
}
}
val response = client.get("http://example.com")
val body = response.bodyAsText()
println(body)
println(response.status.hashCode())
Fixxed it
suspend fun scrape() {
val client = HttpClient(Apache) {
install(ContentNegotiation) {
json(Json {
ignoreUnknownKeys = true
})
}
engine {
followRedirects = false
customizeClient {
setProxy(HttpHost("hostname", port))
val credentialsProvider = BasicCredentialsProvider()
credentialsProvider .setCredentials(
AuthScope("hostname", port),
UsernamePasswordCredentials("username", "password")
)
setDefaultCredentialsProvider(credentialsProvider )
}
}
}
val response =
client.get("http://example.com") {
}
val body = response.bodyAsText()
println(body)
println(response.status.hashCode())
}
There is a problem with making a request through a proxy server using the CIO engine. I've created an issue to address this problem. As a workaround, please use the OkHttp engine instead of CIO.
Here is how you can use a proxy with the Basic authentication:
import io.ktor.client.*
import io.ktor.client.engine.okhttp.*
import io.ktor.client.request.*
import kotlinx.coroutines.runBlocking
import okhttp3.Authenticator
import okhttp3.Credentials
import okhttp3.OkHttpClient
import java.net.Proxy
fun main(): Unit = runBlocking {
val proxyAuthenticator = Authenticator { _, response ->
response.request.newBuilder()
.header("Proxy-Authorization", Credentials.basic("<username>", "<password>"))
.build()
}
val client = HttpClient(OkHttp) {
engine {
preconfigured = OkHttpClient.Builder()
.proxy(Proxy(Proxy.Type.HTTP, java.net.InetSocketAddress("127.0.0.1", 3128)))
.proxyAuthenticator(proxyAuthenticator)
.build()
}
}
val response = client.get("http://eu.kith.com/products.json")
println(response.status)
}
I am testing an API written in Kotlin using the KTOR framework. For the testing, I am using JUnit5 and Mockito. There is a route class where a route is defined which I need to test. Here is the route class :-
fun Application.configureRouting() {
routing {
post("/someRoute") {
val service = MyService()
val request: JsonNode = call.receive()
launch {
service.dummyFunction(request)
}
val mapper = ObjectMapper()
val responseStr = "{\"status\":\"success\",\"message\":\"Request has been received successfully\"}"
val response: JsonNode = mapper.readTree(responseStr)
call.fireHttpResponse(HttpStatusCode.OK, response)
}
}
}
This is the test case I am writing for it :-
class RouteTest {
#Mock
var service = MyService()
// read the configuration properties
private val testEnv = createTestEnvironment {
config = HoconApplicationConfig(ConfigFactory.load("application.conf"))
}
#Before
fun setUp() = withApplication(testEnv) {
MockitoAnnotations.openMocks(MyService::class)
}
#Test
fun test() = withApplication(testEnv) {
withTestApplication(Application::configureRouting) {
runBlocking {
Mockito.`when`(service.dummyFunction(Mockito.any()).thenReturn(true)
with(handleRequest(HttpMethod.Post, "/someRoute") {
setBody("some body")
}) {
assertEquals(HttpStatusCode.OK, response.status())
}
}
}
}
}
When I run the test, it calls the actual "dummyFunction()" method instead of the mocked one and hence, it is failing. Am I doing something wrong?
Because your service in test is different from the service you mocked. To solve this, you need to inject the service into your class, or pass the service as an argument.
Read more: IoC, DI.
The simplest way to solve your problem is to define the service parameter for the configureRouting method and pass a corresponding argument in the test and production code when calling it.
fun Application.configureRouting(service: MyService) {
routing {
post("/someRoute") {
val request: JsonNode = call.receive()
launch {
service.dummyFunction(request)
}
val mapper = ObjectMapper()
val responseStr = "{\"status\":\"success\",\"message\":\"Request has been received successfully\"}"
val response: JsonNode = mapper.readTree(responseStr)
call.fireHttpResponse(HttpStatusCode.OK, response)
}
}
}
class RouteTest {
#Mock
var service = MyService()
private val testEnv = createTestEnvironment {
config = HoconApplicationConfig(ConfigFactory.load("application.conf"))
}
#Test
fun test() = withApplication(testEnv) {
withTestApplication({ configureRouting(service) }) {
runBlocking {
// Your test...
}
}
Im trying to add the HostnameVerifier
HostnameVerifier { _, _ -> true }
to my kotlin multiplatform ktor client and I couldn't be able to understand how to do that:
#KtorExperimentalAPI actual val httpClient = HttpClient(OkHttp) {
engine {
// how to set HostnameVerifier?
val loggingInterceptor = HttpLoggingInterceptor()
loggingInterceptor.level = HttpLoggingInterceptor.Level.BODY
addInterceptor(loggingInterceptor)
}
}
Something like that:
actual val httpClient = HttpClient(OkHttp) {
engine {
// https://square.github.io/okhttp/3.x/okhttp/okhttp3/OkHttpClient.Builder.html
config { // this: OkHttpClient.Builder ->
// ...
hostnameVerifier {
_, _ -> true
}
}
val loggingInterceptor = HttpLoggingInterceptor()
loggingInterceptor.level = HttpLoggingInterceptor.Level.BODY
addInterceptor(loggingInterceptor)
}
}
I'm trying to create a Ktor (1.3.1) HttpClient Mock with JsonFeature like this:
#Test
fun mockFailure() = runBlocking {
val mock = MockEngine { call ->
respond("{}",
HttpStatusCode.OK,
headersOf("Content-Type", ContentType.Application.Json.toString()))
}
val client = HttpClient(mock) {
install(JsonFeature) {
serializer = KotlinxSerializer()
}
}
val resp = client.get<JsonObject>("dsf")
}
It appears to process it correctly, but then I get this error:
io.ktor.client.call.NoTransformationFoundException: No transformation found: class kotlinx.coroutines.io.ByteBufferChannel -> class kotlinx.serialization.json.JsonObject
with response from http://localhost/dsf:
status: 200 OK
response headers:
Content-Type: application/json
at io.ktor.client.call.HttpClientCall.receive(HttpClientCall.kt:79)
Try installing client content negotiation plugin in the HttpClient block:
val client = HttpClient(mock) {
install(JsonFeature) {
serializer = KotlinxSerializer()
}
//Here
install(ContentNegotiation) {
json(Json {
prettyPrint = true
isLenient = true
})
}
}
Required dependencies implementation("io.ktor:ktor-client-content-negotiation:$ktor_version")
Json Serialization implementation("io.ktor:ktor-serialization-kotlinx-json:$ktor_version")
Also change the last line to val resp = client.get<JsonObject>("dsf").body()
I tried to do unit test with Hoverfly to mock external API.
companion object {
#ClassRule #JvmField
val hoverflyRule: HoverflyRule = HoverflyRule.inSimulationMode(dsl(
service("people.zoho.com")
.get("/people/api/forms/P_EmployeeView/records").queryParam("authtoken","TOKEN")
.willReturn(success("{test:test}", "application/json"))
))
}
When I use the Apache client with ktor, that doesn't work. But with another client like khttp, it works. Any ideas why?
You should setup default system proxy in Apache config:
http://hoverfly.readthedocs.io/projects/hoverfly-java/en/latest/pages/misc/misc.html
example with ktor(0.9.3-alpha-3):
class ApplicationMockupTest {
companion object {
#ClassRule
#JvmField
val hoverflyRule: HoverflyRule = HoverflyRule.inSimulationMode(
dsl(
service("people.zoho.com:443")
.get("/people/api/forms/P_EmployeeView/records")
.queryParam("authtoken", "TOKEN")
.willReturn(success("{j:gr}", "application/json"))
)
)
}
#Test
fun exampleTest() = runBlocking<Unit> {
val client = HttpClient(Apache.setupDefaultProxy())
val token = "TOKEN"
val url = "https://people.zoho.com/people/api/forms/P_EmployeeView/records?authtoken=$token"
val requestString = client.get<String>(url)
hoverflyRule.verifyAll()
Unit
}
fun HttpClientEngineFactory<ApacheEngineConfig>.setupDefaultProxy() = config {
customizeClient {
useSystemProperties()
}
}
}