I use the default Exposed framework configuration, which has the built-in logging of SQL statements that the framework creates for the database calls.
As a result, I see SQL statements in the logs in the following format:
[...] DEBUG Exposed - INSERT INTO sensitive_table (column1, column2) VALUES ('PII1', 'PII2')
Is it possible to configure logging in Exposed to hide (e.g. replace with '?') the sensitive information that can be present in the SQL statement parameters?
[...] DEBUG Exposed - INSERT INTO sensitive_table (column1, column2) VALUES (?, ?)
I solved this problem using a custom SqlLogger that logs SQL without injecting parameters values.
object SafeSqlLogger : SqlLogger {
private val log: Logger = LoggerFactory.getLogger(SafeSqlLogger::class.java)
override fun log(context: StatementContext, transaction: Transaction) {
log.debug(context.sql(TransactionManager.current()))
}
}
I disabled the Exposed logger in the logback config.
<logger name="Exposed" level="OFF"/>
And added the logger to the transactions that I wanted to log.
transaction {
addLogger(SafeSqlLogger)
// query the database
}
As a result, I got the following log statements:
[...] DEBUG SafeSqlLogger - INSERT INTO sensitive_table (column1, column2) VALUES (?, ?)
And finally wrote a function that can be used instead of transaction for logged transactions.
fun <T> loggedTransaction(db: Database? = null, statement: Transaction.() -> T): T {
return transaction(db.transactionManager.defaultIsolationLevel,
db.transactionManager.defaultRepetitionAttempts,
db
) {
addLogger(SafeSqlLogger)
statement.invoke(this)
}
}
Hope this will be helpful for anyone having the same problem as me.
I had the same problem with logging huge ByteArray values. And I came up with another solution: we create our own custom type:
object BasicBinaryColumnTypeCustomLogging : BasicBinaryColumnType() {
override fun valueToString(value: Any?): String {
return "bytes:${(value as ByteArray).size}"
}
}
And then in our Table object we use it like:
object Images : Table("image") {
// val file = binary("file")
val file = binaryCustomLogging("file")
private fun binaryCustomLogging(name: String): Column<ByteArray> = registerColumn(name, BasicBinaryColumnTypeCustomLogging)
}
So in your case you can create your own type with custom
Related
You know that Array and List only store the same data struction.
I run the Code A and get the Result A.
It seems that the Flow can emit both Int value and String value, why?
Code A
import kotlinx.coroutines.*
import kotlinx.coroutines.flow.*
suspend fun performRequest(request: Int): Int {
delay(1000) // imitate long-running asynchronous work
return request
}
fun main() = runBlocking<Unit> {
(1..3).asFlow() // a flow of requests
.transform { request ->
emit("Making request $request")
if (request >1) {
emit(performRequest(request))
}
}
.collect { response -> println(response) }
}
Result A
Making request 1
Making request 2
2
Making request 3
3
This is not a question of Flow but Java/Kotling generics and type safety.
The type this flow returns is Comperable<*>
val flow: Flow<Comparable<*>> = (1..3).asFlow() // a flow of requests
.transform { request ->
emit("Making request $request")
if (request > 1) {
emit(performRequest(request))
}
If you explicitly specify which value you want to return Flow you can restrict the types.
About generics you can refer here or check any document about generics in java/kotlin, type safety you can refer this question
Also when you are in doubt what your specified type is use alt + enter with Android Studio to see avaialble options and select Specify type explicitly.
Disregarding the nature of this request, you can have the functionality you want by making your flow emit instances of some algebraic data type that is basically a "sum" (from the type-theoretic POV) of your constituent types:
sealed interface Record
data class IntData(val get: Int) : Record
data class Metadata(val get: String) : Record
// somewhere later (flow is of type Flow<Record>)
fun main() = runBlocking<Unit> {
(1..3).asFlow() // a flow of requests
.transform { request ->
emit(Metadata("Making request $request"))
if (request > 1) {
emit(IntData(performRequest(request)))
}
// probably want to handle the `else` case too
}
.collect { response -> println(response) }
}
This would be a good solution since it's extendable (i.e. you can add the other cases later on if you need to).
In your specific case though, since you just want to debug the flow, you might not want to actually emit the "metadata" and just go for the tests of your code directly.
I have read the set-based consistency validation blog and I want to validate through a dispatch interceptor. I follow the example, but I use reactive repository and it doesn't really work for me. I have tried both block and not block. with block it throws error, but without block it doesn't execute anything. here is my code.
class SubnetCommandInterceptor : MessageDispatchInterceptor<CommandMessage<*>> {
#Autowired
private lateinit var privateNetworkRepository: PrivateNetworkRepository
override fun handle(messages: List<CommandMessage<*>?>): BiFunction<Int, CommandMessage<*>, CommandMessage<*>> {
return BiFunction<Int, CommandMessage<*>, CommandMessage<*>> { index: Int?, command: CommandMessage<*> ->
if (CreateSubnetCommand::class.simpleName == (command.payloadType.simpleName)){
val interceptCommand = command.payload as CreateSubnetCommand
privateNetworkRepository
.findById(interceptCommand.privateNetworkId)
// ..some validation logic here ex.
// .filter { network -> network.isSubnetOverlap() }
.switchIfEmpty(Mono.error(IllegalArgumentException("Requested subnet is overlap with the previous subnet.")))
// .block() also doesn't work here it throws error
// block()/blockFirst()/blockLast() are blocking, which is not supported in thread reactor-
}
command
}
}
}
Subscribing to a reactive repository inside a message dispatcher is not really recommended and might lead to weird behavior as underling ThreadLocal (used by Axox) is not adapted to be used in reactive programing
Instead, check out Axon's Reactive Extension and reactive interceptors section.
For example what you might do:
reactiveCommandGateway.registerDispatchInterceptor(
cmdMono -> cmdMono.flatMap(cmd->privateNetworkRepository
.findById(cmd.privateNetworkId))
.switchIfEmpty(
Mono.error(IllegalArgumentException("Requested subnet is overlap with the previous subnet."))
.then(cmdMono)));
We have a project where we use a Postgres sequence for generating an increasing number, but I cannot figure out how to actually use the sequence in kotlin exposed.
I see there is a Sequence class and a NextVal class encapsulating a sequence but those cannot be used by its own as far as I can see. I thought I could use Sequence.nextLongVal() but this one returns the NextVal class, no way to get the through value out of this one.
So how can I get the value of the nextVal() execution?
We stumbled across the same problem trying to utilize Sequence.nextLongVal() directly using Postgre and exposed. We found the following workaround.
A solution using exec
Assuming we have defined and created a sequence in our datasource:
val sequence = Sequence(/* our sequence's parameters */)
...
transaction {
SchemaUtils.createSequence(sequence)
}
We suggest to define a helper function to retrieve the next value of a given sequence using exposed's exec.
fun Transaction.nextValueOf(sequence: Sequence): Long = exec("SELECT nextval('${sequence.identifier}');") { resultSet ->
if (resultSet.next().not()) {
throw Error("Missing nextValue in resultSet of sequence '${sequence.identifier}'")
}
else {
resultSet.getLong(1)
}
} ?: throw Error("Unable to get nextValue of sequence '${sequence.identifier}'")
Now, we can use this function in a transaction as shown here:
transaction {
...
val nextValue = nextValueOf(sequence)
...
}
I understand that in Kotlin there is no such thing as "Non-local variables" or "Global Variables" I am looking for a way to modify variables in another "Scope" in Kotlin by using the function below:
class Listres(){
var listsize = 0
fun gatherlistresult(){
var listallinfo = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
listallinfo.addOnSuccessListener {
listResult -> listsize += listResult.items.size
}
}
}
the value of listsize is always 0 (logging the result from inside of the .addOnSuccessListener scope returns 8) so clearly the listsize variable isn't being modified. I have seen many different posts about this topic on other sites , but none fit my usecase.
I simply want to modify listsize inside of the .addOnSuccessListener callback
This method will always be returned 0 as the addOnSuccessListener() listener will be invoked after the method execution completed. The addOnSuccessListener() is a callback method for asynchronous operation and you will get the value if it gives success only.
You can get the value by changing the code as below:
class Demo {
fun registerListResult() {
var listallinfo = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
listallinfo.addOnSuccessListener {
listResult -> listsize += listResult.items.size
processResult(listsize)
}
listallinfo.addOnFailureListener {
// Uh-oh, an error occurred!
}
}
fun processResult(listsize: Int) {
print(listResult+"") // you will get the 8 here as you said
}
}
What you're looking for is a way to bridge some asynchronous processing into a synchronous context. If possible it's usually better (in my opinion) to stick to one model (sync or async) throughout your code base.
That being said, sometimes these circumstances are out of our control. One approach I've used in similar situations involves introducing a BlockingQueue as a data pipe to transfer data from the async context to the sync context. In your case, that might look something like this:
class Demo {
var listSize = 0
fun registerListResult() {
val listAll = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
val dataQueue = ArrayBlockingQueue<Int>(1)
listAll.addOnSuccessListener { dataQueue.put(it.items.size) }
listSize = dataQueue.take()
}
}
The key points are:
there is a blocking variant of the Queue interface that will be used to pipe data from the async context (listener) into the sync context (calling code)
data is put() on the queue within the OnSuccessListener
the calling code invokes the queue's take() method, which will cause that thread to block until a value is available
If that doesn't work for you, hopefully it will at least inspire some new thoughts!
I would like to use a Flow as a return type for all functions in my repository. For ex:
suspend fun create(item:T): Flow<Result<T>>
This function should call 2 data sources: remote(to save data on the server) and local(to save returned data from the server locally). The question is how I can implement this scenario:
try to save data with RemoteDataSource
if 1. fails - try it N times with M timeout
if data has finally returned from the server - same them locally with LocalDataSource
return flow with locally saved data
RemoteDataSource and LocalDataSource both have fun create with the same signature:
suspend fun create(item:T): Flow<Result<T>>
So they both return flow of data. If you have any ideas about how to implement it, I will be grateful.
------ Update #1 ------
a part of a possible solution:
suspend fun create(item:T): Flow<T> {
// save item remotely
return remoteDataSource.create(item)
// todo: call retry if fails
// save to local a merge two flows in one
.flatMapConcat { remoteData ->
localDataSource.create(remoteData)
}
.map {
// other mapping
}
}
Is it a working idea?
I think you have the right idea but you are trying to do everything at once.
What I found works best (and easily) is to have:
an exposed flow of data coming from your local datasource (easy with Room)
one or more exposed suspend functions like create or refresh that operate on the remote data source and save to the local one (if there is no error)
For ex I have a repository that fetches vehicles in my project (the isCurrent info is only local and isLeft/isRight is because I use Either but any error handling applies):
class VehicleRepositoryImpl(
private val localDataSource: LocalVehiclesDataSource,
private val remoteDataSource: RemoteVehiclesDataSource
) : VehicleRepository {
override val vehiclesFlow = localDataSource.vehicleListFlow
override val currentVehicleFlow = localDataSource.currentVehicleFLow
override suspend fun refresh() {
remoteDataSource.getVehicles()
.fold(
ifLeft = { /* handle errors, retry, ... */ },
ifRight = { reset(it) }
)
}
private suspend fun reset(vehicles: List<VehicleEntity>) {
val current = currentVehicleFlow.first()
localDataSource.reset(vehicles)
if (current != null) localDataSource.setCurrentVehicle(current)
}
override suspend fun setCurrentVehicle(vehicle: VehicleEntity) =
localDataSource.setCurrentVehicle(vehicle)
override suspend fun clear() = localDataSource.clear()
}
Hope this helps and you can adapt it to your case :)