Akka HTTP doesn't render collection when using custom ToEntityMarshaller - akka-http

I have defined a custom ToEntityMarshaller for type Organisation. When requesting localhost:8080/organisations it return an empty JSON array. Only when I remove the implicit def organisationMarshaller: ToEntityMarshaller[Organisation] it return the correct representation of the stream.
Anybody has an idea what is going on here?
import akka.NotUsed
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.common.{EntityStreamingSupport, JsonEntityStreamingSupport}
import akka.http.scaladsl.model.{HttpEntity, StatusCodes, _}
import akka.http.scaladsl.server.Directives._
import akka.stream.ActorMaterializer
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._
import akka.http.scaladsl.marshalling.{Marshaller, ToEntityMarshaller, ToResponseMarshaller}
import akka.http.scaladsl.model.TransferEncodings.gzip
import akka.http.scaladsl.model.headers.{HttpEncoding, HttpEncodings}
import akka.stream.scaladsl.{Flow, Source}
import akka.util.ByteString
import spray.json.DefaultJsonProtocol
import spray.json.DefaultJsonProtocol._
import scala.concurrent.Future
import scala.io.StdIn
import scala.util.Random
final case class Organisation(name: String, id: String)
trait Protocols extends DefaultJsonProtocol {
import spray.json._
implicit val organisationFormat = jsonFormat2(Organisation)
val `vnd.example.api.v1+json` =
MediaType.applicationWithFixedCharset("vnd.example.api.v1+json", HttpCharsets.`UTF-8`)
// -- WORKS AFTER REMOVING THIS DECLARATION --
implicit def organisationMarshaller: ToEntityMarshaller[Organisation] = Marshaller.oneOf(
Marshaller.withFixedContentType(`vnd.example.api.v1+json`) { organisation =>
HttpEntity(`vnd.example.api.v1+json`, organisation.toJson.compactPrint)
})
}
object Server extends App with Protocols {
implicit val system = ActorSystem("api")
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
implicit val jsonStreamingSupport: JsonEntityStreamingSupport = EntityStreamingSupport.json()
.withParallelMarshalling(parallelism = 10, unordered = false)
// (fake) async database query api
def dummyOrganisation(id: String) = Organisation(s"Organisation $id", id.toString)
def fetchOrganisation(id: String): Future[Option[Organisation]] = Future(Some(dummyOrganisation(id)))
def fetchOrganisations(): Source[Organisation, NotUsed] = Source.fromIterator(() => Iterator.fill(10000) {
val id = Random.nextInt()
dummyOrganisation(id.toString)
})
val route =
encodeResponse {
pathPrefix("organisations") {
get {
val organisations = fetchOrganisations()
complete(organisations)
}
}
}
val bindingFuture = Http().bindAndHandle(route, "localhost", 8080)
println(s"Server online at http://localhost:8080/\nPress RETURN to stop...")
StdIn.readLine()
bindingFuture.flatMap(_.unbind()).onComplete(_ => system.terminate())
}

Related

Add property path to custom validation in Micronaut

Given the below, working validator factory, is it possible somehow to manipulate propertyPath? When debugging, I can see propertyPath inside io.micronaut.validation.validator.DefaultValidator$DefaultConstraintValidatorContext.
Motivation for doing so, is that some fields are conditionally required, and I need to put validation annotation on class-level, but then propertyPath is set to class, not to the actual fields with validation errors. When returning validation errors to API-clients, it would be nice to have correct property name for validation errors.
import io.micronaut.context.annotation.Factory
import io.micronaut.validation.validator.constraints.ConstraintValidator
import jakarta.inject.Singleton
import no.mycompany.myapp.web.config.UiConfig
import no.mycompany.myapp.web.viewmodel.MyExampleFormVm
#Factory
class ExampleValidatorFactory(private val uiConfig: UiConfig) {
#Singleton
fun validForm(): ConstraintValidator<ValidForm, MyExampleFormVm> = ConstraintValidator { value, _, context ->
val formTypeFromConfig = uiConfig.formTypes.first { it.id == value.formType }
if (formTypeFromConfig.labelSubCompanyId != null && value.subCompanyId == null) {
context.messageTemplate("Form requires one or more sub-company-ids")
// TODO: set propertyPath to "subCompanyId"
return#ConstraintValidator false
}
if (formTypeFromConfig.labelCompanyId != null && value.companyId == null) {
context.messageTemplate("Form requires company-id")
// TODO: set propertyPath to "companyId"
return#ConstraintValidator false
}
true
}
}
Annotation
import javax.validation.Constraint
#Retention(AnnotationRetention.RUNTIME)
#Constraint(validatedBy = [])
annotation class ValidForm(
val message: String = "Invalid form ({validatedValue})"
)
Example bean to validate:
import io.micronaut.core.annotation.Introspected
import javax.validation.constraints.NotBlank
import javax.validation.constraints.Pattern
#Introspected
#ValidForm
data class MyExampleFormVm(
#field:NotBlank
val formType: String = "",
#field:Pattern(regexp = "[8|9]\\d{8}")
val companyId: String? = null,
#field:Pattern(regexp = "[8|9]\\d{8}")
val subCompanyId: String? = null
)
This is my custom ConstraintExceptionHandler. I want it.propertyPath.lastOrNull()?.name to return e.g. "subCompanyId".
import io.micronaut.context.annotation.Replaces
import io.micronaut.context.annotation.Requires
import io.micronaut.http.HttpRequest
import io.micronaut.http.HttpResponse
import io.micronaut.http.HttpStatus
import io.micronaut.http.annotation.Produces
import io.micronaut.http.server.exceptions.response.ErrorResponseProcessor
import io.micronaut.validation.exceptions.ConstraintExceptionHandler
import jakarta.inject.Singleton
import java.time.Instant
import java.util.*
import javax.validation.ConstraintViolationException
#Produces
#Singleton
#Replaces(ConstraintExceptionHandler::class)
#Requires(classes = [ConstraintViolationException::class, ConstraintExceptionHandler::class])
class CustomConstraintExceptionHandler(responseProcessor: ErrorResponseProcessor<*>) :
ConstraintExceptionHandler(responseProcessor) {
override fun handle(request: HttpRequest<*>, exception: ConstraintViolationException): HttpResponse<*> =
HttpResponse.badRequest(
ApiError(
timestamp = Instant.now().toEpochMilli(),
httpStatusCode = HttpStatus.BAD_REQUEST.code,
errorType = ApiErrorType.VALIDATION_ERROR,
url = request.path,
validationErrors = exception.constraintViolations
.associate { (it.propertyPath.lastOrNull()?.name ?: FALLBACK_PROPERTY_PATH) to it.message }
))
companion object {
const val FALLBACK_PROPERTY_PATH = "ROOT"
}
}
---
import com.fasterxml.jackson.annotation.JsonInclude
import io.micronaut.core.annotation.Introspected
#Introspected
#JsonInclude(value = JsonInclude.Include.NON_NULL)
data class ApiError(
val timestamp: Long,
val httpStatusCode: Int,
val errorType: ApiErrorType,
val url: String,
val validationErrors: Map<String, String>? = null
)
enum class ApiErrorType {
VALIDATION_ERROR,
SYSTEM_ERROR;
}
Question from #Bonatti
If it.propertyPath.lastOrNull() is null, on the Handler, then that is
weird.
If you do like this, resultMap will contain an entry with key="ROOT".
val exampleForm = MyExampleFormVm(
formType = "SomeValue",
companyId = "912345678",
subCompanyId = null
)
val constraintViolations: Set<ConstraintViolation<MyExampleFormVm>> = validator.validate(exampleForm)
val resultMap = constraintViolations.associate { (it.propertyPath.lastOrNull()?.name ?: "ROOT") to it.message }
Environment: Micronaut 3.7.4

How do I provide my own DateTimeProvider with Spring Data

I am trying to provide a transactionally consistent set of datetimes
import org.springframework.beans.factory.ObjectFactory
import org.springframework.beans.factory.config.BeanFactoryPostProcessor
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.context.annotation.Scope
import org.springframework.data.auditing.DateTimeProvider
import org.springframework.transaction.support.SimpleTransactionScope
import java.time.Instant
import java.time.OffsetDateTime
import java.time.ZoneOffset
import java.util.Optional
#Configuration
open class TransactionScopeTimeConfiguration {
#Bean
open fun transactionBFPP(): BeanFactoryPostProcessor =
BeanFactoryPostProcessor { it.registerScope("transaction", SimpleTransactionScope()) }
#Bean
#Scope("transaction")
open fun nowInstant(): Instant = Instant.now()
#Bean
#Scope("transaction")
open fun nowOffsetDateTime(nowInstant: Instant): OffsetDateTime = nowInstant.atOffset(ZoneOffset.UTC)
#Bean
open fun transactionDateTimeProvider(factory: ObjectFactory<OffsetDateTime>): DateTimeProvider =
DateTimeProvider { Optional.of(factory.`object`) }
}
However, when debugging my test I note that the function inside of transactionDateTimeProvider is never called (the creation of the Bean is)
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.Test
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest
#DataJpaTest
#Import(TransactionScopeTimeConfiguration)
internal open class ExceptionDaoTest {
#Test
fun save(#Autowired exceptionDao: ExceptionJpaDao) {
val toCreate = ExceptionEntity("someid")
val saved = exceptionDao.save(toCreate)
assertThat(saved).isInstanceOf(ExceptionEntity::class.java)
.extracting({ it.id }, { it.businessDivisionId })
.containsExactly(toCreate.id, "someid")
.doesNotContainNull()
assertThat(saved.lastModifiedOn).isSameAs(saved.createdOn)
}
}
The test actually passes, but I haven't actually exercised this functionality in this test. It's important that any other datetimes are transactionally consistent with the audit traits.

How to pass current ClassLoader to KotlinToJVMBytecodeCompiler for dynamic (runtime) compilation kotlin code programmatically?

I created simple utility for runtime compilation kotlin code:
package com.example
import org.jetbrains.kotlin.cli.common.CLIConfigurationKeys
import org.jetbrains.kotlin.cli.common.config.addKotlinSourceRoot
import org.jetbrains.kotlin.cli.common.messages.MessageRenderer
import org.jetbrains.kotlin.cli.common.messages.PrintingMessageCollector
import org.jetbrains.kotlin.cli.jvm.compiler.EnvironmentConfigFiles
import org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment
import org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler
import org.jetbrains.kotlin.cli.jvm.config.addJvmClasspathRoots
import org.jetbrains.kotlin.codegen.state.GenerationState
import org.jetbrains.kotlin.com.intellij.openapi.Disposable
import org.jetbrains.kotlin.config.CommonConfigurationKeys
import org.jetbrains.kotlin.config.CompilerConfiguration
import org.jetbrains.kotlin.config.JVMConfigurationKeys
import org.jetbrains.kotlin.config.JvmTarget
import java.io.File
import kotlin.script.experimental.jvm.util.KotlinJars
class KotlinDynamicCompiler {
fun compileScript(moduleName: String,
sourcePath: String,
saveClassesDir: File
): GenerationState {
val stubDisposable = StubDisposable();
val configuration = CompilerConfiguration()
configuration.put(CommonConfigurationKeys.MODULE_NAME, moduleName)
configuration.put(CLIConfigurationKeys.MESSAGE_COLLECTOR_KEY, PrintingMessageCollector(System.out, MessageRenderer.PLAIN_FULL_PATHS, true))
configuration.put(JVMConfigurationKeys.OUTPUT_DIRECTORY, saveClassesDir)
configuration.put(JVMConfigurationKeys.JVM_TARGET, JvmTarget.JVM_1_8)
configuration.addKotlinSourceRoot(sourcePath)
configuration.addJvmClasspathRoots(listOf(KotlinJars.stdlib))
val env = KotlinCoreEnvironment.createForProduction(stubDisposable, configuration, EnvironmentConfigFiles.JVM_CONFIG_FILES)
return KotlinToJVMBytecodeCompiler.analyzeAndGenerate(env)!!;
}
inner class StubDisposable : Disposable {
#Volatile
var isDisposed: Boolean = false
private set
override fun dispose() {
isDisposed = true
}
};
}
And it works for code as
package com.example.kt
class SimpleClass(val str:String){
fun test(){
}
}
class UsedSimpleClass(val simpleClass: SimpleClass, val file: java.io.File) {
}
But it not works if I want to use no-base package classes as:
package com.example.kt
import com.example.pojo.TestPojo //class have in project that call runtime compilation
class SimpleClass(val str:TestPojo){
}
or:
package com.example.kt
import com.fasterxml.jackson.databind.ObjectMapper //class have in project classpath where called runtime compilation
class SimpleClass(val str:ObjectMapper){
}
How to pass current ClassLoader to KotlinToJVMBytecodeCompiler for dynamic (runtime) compilation kotlin code programmatically?
More details:
Test project on github with crashed test: https://github.com/nekkiy/dynamic-kotlin
Cause:
We need use codegeneration and would like to test generated code. But I don't understand how to pass current classes environment.
Thanks for attention.
Solution:
I have used method fun classpathFromClassloader(currentClassLoader: ClassLoader, unpackJarCollections: Boolean = false): List<File>? from kotlin.script.experimental.jvm.util.jvmClasspathUtil.kt and it works.
Result dynamic compiller:
package com.example
import org.jetbrains.kotlin.cli.common.CLIConfigurationKeys
import org.jetbrains.kotlin.cli.common.config.addKotlinSourceRoots
import org.jetbrains.kotlin.cli.common.messages.MessageRenderer
import org.jetbrains.kotlin.cli.common.messages.PrintingMessageCollector
import org.jetbrains.kotlin.cli.jvm.compiler.EnvironmentConfigFiles
import org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment
import org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler
import org.jetbrains.kotlin.cli.jvm.config.addJvmClasspathRoots
import org.jetbrains.kotlin.codegen.state.GenerationState
import org.jetbrains.kotlin.com.intellij.openapi.Disposable
import org.jetbrains.kotlin.config.CommonConfigurationKeys
import org.jetbrains.kotlin.config.CompilerConfiguration
import org.jetbrains.kotlin.config.JVMConfigurationKeys
import org.jetbrains.kotlin.config.JvmTarget
import java.io.ByteArrayOutputStream
import java.io.File
import java.io.PrintStream
import kotlin.script.experimental.jvm.util.KotlinJars
import kotlin.script.experimental.jvm.util.classpathFromClassloader
class KotlinDynamicCompiler {
fun compileModule(moduleName: String,
sourcePath: List<String>,
saveClassesDir: File,
classLoader: ClassLoader? = null,
forcedAddKotlinStd: Boolean = true
): GenerationState {
val stubDisposable = StubDisposable();
val configuration = CompilerConfiguration()
configuration.put(CommonConfigurationKeys.MODULE_NAME, moduleName)
val baos = ByteArrayOutputStream()
val ps: PrintStream = PrintStream(baos)
configuration.put(CLIConfigurationKeys.MESSAGE_COLLECTOR_KEY, PrintingMessageCollector(ps, MessageRenderer.PLAIN_FULL_PATHS, true))
configuration.put(JVMConfigurationKeys.OUTPUT_DIRECTORY, saveClassesDir)
// configuration.put(JVMConfigurationKeys.RETAIN_OUTPUT_IN_MEMORY, true)
configuration.put(JVMConfigurationKeys.JVM_TARGET, JvmTarget.JVM_1_8)
val classPath = mutableSetOf<File>()
if (classLoader != null) {
classPath.addAll(classpathFromClassloader(classLoader)!!);
}
if (forcedAddKotlinStd) {
classPath.add(KotlinJars.stdlib)
}
configuration.addJvmClasspathRoots(classPath.toList())
configuration.addKotlinSourceRoots(sourcePath)
val env = KotlinCoreEnvironment.createForProduction(stubDisposable, configuration, EnvironmentConfigFiles.JVM_CONFIG_FILES)
val result = KotlinToJVMBytecodeCompiler.analyzeAndGenerate(env);
ps.flush();
if (result != null) {
return result
} else {
throw IllegalStateException("Compilation error. Details:\n$baos")
}
}
inner class StubDisposable : Disposable {
#Volatile
var isDisposed: Boolean = false
private set
override fun dispose() {
isDisposed = true
}
};
}
Note: This function is contained in experimental package.
P.S. I also updated github-project.

I can't generate a class using Kotlin processor

I'm implementing a processor to generate kotlin code using custom annotations. The problem is that I cannot find a way to relate the annotation to the field it was declared for, and I cannot find a way to understand if a field is of a nullable type. The processor doesn't succeed to generate the code because the getAnnotationsByType doesn't return the annotations for the current field (the list it's empty). Not even the order is good, fields are passed first and the annotations after all the fields.
package it.kfi.xml.binding.processor
import com.google.auto.service.AutoService
import com.squareup.kotlinpoet.*
import it.kfi.xml.binding.annotations.XmlClass
import it.kfi.xml.binding.annotations.XmlProperty
import java.io.File
import java.lang.reflect.Type
import javax.annotation.Nullable
import javax.annotation.processing.AbstractProcessor
import javax.annotation.processing.Processor
import javax.annotation.processing.RoundEnvironment
import javax.lang.model.SourceVersion
import javax.lang.model.element.Element
import javax.lang.model.element.ElementKind
import javax.lang.model.element.TypeElement
import javax.lang.model.element.VariableElement
import javax.lang.model.type.NullType
import javax.lang.model.type.TypeMirror
import javax.print.DocFlavor
import javax.tools.Diagnostic
import kotlin.reflect.KClass
import kotlin.reflect.full.createType
#AutoService(Processor::class)
class XmlBinder : AbstractProcessor() {
companion object {
const val KAPT_KOTLIN_GENERATED_OPTION_NAME = "kapt.kotlin.generated"
}
override fun getSupportedAnnotationTypes(): MutableSet<String> {
return mutableSetOf(XmlClass::class.java.name)
}
override fun getSupportedSourceVersion(): SourceVersion = SourceVersion.latest()
override fun process(annotations: MutableSet<out TypeElement>?, roundEnv: RoundEnvironment): Boolean {
roundEnv.getElementsAnnotatedWith(XmlClass::class.java)
.forEach {
if (it.kind != ElementKind.CLASS) {
processingEnv.messager.printMessage(Diagnostic.Kind.ERROR, "Only classes can be annotated")
return true
}
processClass(it)
}
return false
}
private fun processClass(element: Element) {
val className = element.simpleName.toString() + "Model"
val packageName = processingEnv.elementUtils.getPackageOf(element).toString()
val classBuilder = TypeSpec.classBuilder(className)
classBuilder.addModifiers(KModifier.PUBLIC)
val initFromXml = FunSpec.builder("initFromXml")
initFromXml.addModifiers(KModifier.PUBLIC)
initFromXml.addParameter(ParameterSpec.builder("xml", String::class).build())
val properties = element.enclosedElements
var x: Int = 1
//Look for elements annotated with XmlField and add those elements to the generated class
for (property in properties) {
val annotation = property.getAnnotationsByType(XmlProperty::class.java)
val v = 10
classBuilder.addProperty(PropertySpec.varBuilder(property.simpleName.toString(), String::class, KModifier.PUBLIC).initializer(v.toString()).build())
initFromXml.addStatement("this.${property.simpleName} = \"${v.toString()}\"")
}
classBuilder.addFunction(initFromXml.build())
val fileName = "kfi_generated_$className"
val file = FileSpec.builder(packageName, fileName).addType(classBuilder.build()).build()
val kaptKotlinGeneratedDir = processingEnv.options[KAPT_KOTLIN_GENERATED_OPTION_NAME]
file.writeTo(File(kaptKotlinGeneratedDir))
}
}
Can anyone help me found a way to relate annotations to their fields or properties ?

Apply Play framework's downs evolution after test

I try to test my play 2 application with scalatest.
To test the model I use test database. Each time I run tests the data inserts again. So I need to delete data after test or even drop all the tables.
My code is:
import models.User
import org.joda.time.DateTime
import org.scalatest.BeforeAndAfterAll
import org.scalatestplus.play.{OneAppPerSuite, PlaySpec}
import org.squeryl.adapters.PostgreSqlAdapter
import org.squeryl.{Session, SessionFactory}
import play.api.db.DB
import play.api.{Application, GlobalSettings}
import play.api.test.FakeApplication
class UserSpec extends PlaySpec with OneAppPerSuite with BeforeAndAfterAll {
implicit override lazy val app: FakeApplication = {
FakeApplication(
additionalConfiguration = Map(
"db.default.driver" -> "org.postgresql.Driver",
"db.default.url" -> "jdbc:postgresql://localhost:5432/test",
"db.default.user" -> "test",
"db.default.password" -> "test"),
withGlobal = Some(new GlobalSettings {
override def onStart(app: Application) = {
SessionFactory.concreteFactory = Some(() => Session.create(DB.getConnection()(app), new PostgreSqlAdapter))
}}
)
)}
override protected def afterAll(): Unit = {
}
"User" should {
"be creatable and saved to DB" in {
val user = User.createUser(
User("example1#example.com",
"John",
"Doe",
new DateTime(),
"afc677037be3d92324fa6597d6c1506b534e306b" // sha1("123456aA")
))
user.isPersisted mustBe true
}
}
}
Is it possible to apply Downs evolution after the tests?