i'm trying to move from imperative try-with-resources to reactive try-with-resources without success. I have the following piece of code i would like to move.
private final AmazonS3 amazonS3;
private final String bucket;
#Override
public Mono<String> getTemplate(String templateId) {
return Mono.fromCallable(() -> {
S3Object s3Object = amazonS3.getObject(bucket, templateId);
try (s3Object) {
return IOUtils.toString(s3Object.getObjectContent());
}
}).subscribeOn(Schedulers.boundedElastic());
}
I would like to rewrite using reactive try-with-resources construct.
My first try was using Flux.using
Flux.using(amazonS3.getObject(bucket, templateId),
s3Object -> Flux.just(IOUtils.toString(s3Object.getObjectContent())),
S3Object::close);
the s3Object is not being casted as an S3Object so getObjectContent doesn't exist.
Then i had a look athttps://projectreactor.io/docs/core/release/reference/ and i guess that i might use Disposable, however i'm not sure how to wrap S3Object with a disposable object.
Does anyone know how can i make it work?
Thanks
You can't achieve this with the approach you're taking. It's literally impossible to take a blocking API like the one you see here (AWS SDK v1) and somehow wrap it to make it reactive.
You can however use the AWS SDK v2 (you should be using this anyway for new development), which has an asynchronous S3 client (S3AsyncClient) that you can use to return a CompleteableFuture<String>:
CompletableFuture<String> contents = s3AsyncClient
.getObject(GetObjectRequest.builder().build(), new ByteArrayAsyncResponseTransformer<>())
.thenApplyAsync(rb -> rb.asUtf8String());
You can then use Mono.fromFuture(contents) to obtain a Mono<String> from the above CompleteableFuture.
Related
I'd like to be able to use Micronaut's declarative client to hit an a different endpoint based on whether I'm in a local development environment vs a production environment.
I'm setting my client's base uri in application.dev.yml:
myserviceclient:
baseUri: http://localhost:1080/endpoint
Reading the docs from Micronaut, they have the developer jumping through quite a few hoops to get a dynamic value piped into the actual client. They're actually quite confusing. So I've created a configuration like this:
#ConfigurationProperties(PREFIX)
class MyServiceClientConfig {
companion object {
const val PREFIX = "myserviceclient"
const val BASE_URL = "http://localhost:1080/endpoint"
}
var baseUri: String? = null
fun toMap(): MutableMap<String, Any> {
val m = HashMap<String, Any>()
if (baseUri != null) {
m["baseUri"] = baseUri!!
}
return m
}
}
But as you can see, that's not actually reading any values from application.yml, it's simply setting a const value as a static on the class. I'd like that BASE_URL value to be dynamic based on which environment I'm in.
To use this class, I've created a declarative client like this:
#Client(MyServiceClientConfig.BASE_URL)
interface MyServiceClient {
#Post("/user/kfc")
#Produces("application/json")
fun sendUserKfc(transactionDto: TransactionDto)
}
The docs show an example where they're interpolating values from the config map that's built like this:
#Get("/api/\${bintray.apiversion}/repos/\${bintray.organization}/\${bintray.repository}/packages")
But how would I make this work in the #Client() annotation?
Nowhere in that example do they show how bintray is getting defined/injected/etc. This appears to be the same syntax that's used with the #Value() annotation. I've tried using that as well, but every value I try to use ends up being null.
This is very frustrating, but I'm sure I'm missing a key piece that will make this all work.
I'm setting my client's base uri in application.dev.yml
You probably want application-dev.yml.
But how would I make this work in the #Client() annotation?
You can put a config key in the #Client value using something like #Client("${myserviceclient.baseUri}").
If you want the url somewhere in your code use this:
#Value("${micronaut.http.services.occupancy.urls}")
private String occupancyUrl;
I am using jdbcTemplate to query hive then writing the results to a .csv file. I basically just generate a list of objects then steam the list to write each record to the file.
I will like to stream the results as they coming back from hive and write it to the file instead of wait to get the whole thing then processing it. Can anyone pointing me to the right direction? Thanks!
private List<Avs> queryAvsData(String asSql) {
List<Avs> llistAvs = new ArrayList<Avs>();
List<Map<String, Object>> rows = hiveJdbcTemplate.queryForList(asSql);
Iterator<Map<String, Object>> it = rows.iterator();
while (it.hasNext()) {
Map<String, Object> row = it.next();
Avs laAvs = Avs.builder()
.make((String) row.get("make"))
.model((String) row.get("model"))
.build();
llistAvs.add(laAvs);
}
return llistAvs;
}
It doesn't look like there's a built-in solution, but you can do it. Basically, you wrap the existing functionality in an iterator, and use a spliterator to turn it into a stream. Here's a blog post on the subject:
The code implements Spring’s ResultSetExtractor interface, which is a Single Abstract Method (SAM) interface, allowing the use of a lambda expression to implement it.
The implementation wraps the SQL ResultSet in an iterator, constructs a stream using the Spliterators and StreamSupport utility classes, and applies that to a Function taking a stream of row sets and returning a generic result.
It's possible to stream values from JdbcTemplate. The following example is a service based on Spring Boot 2.4.8.
As, I run into problems (connection leak) using queryForStream then I will put a demo code here just to know that stream must be closed after usage.
import lombok.RequiredArgsConstructor;
import org.springframework.jdbc.core.SingleColumnRowMapper;
import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
import org.springframework.stereotype.Service;
import java.util.Map;
import java.util.stream.Stream;
#Service
#RequiredArgsConstructor
public class DataCleaningService {
private final NamedParameterJdbcTemplate jdbcTemplate;
public void doSomeStreaming() {
String nativeQuery = "SELECT string_value FROM my_table WHERE column = :valueToFiler";
Map<String, Object> queryParameters = Map.of("valueToFiler", "my value");
SingleColumnRowMapper<String> stringRowMapper = SingleColumnRowMapper.newInstance(String.class);
try (Stream<String> stringValueStream = jdbcTemplate.queryForStream(nativeQuery, queryParameters, stringRowMapper)) {
stringValueStream.forEach(stringValue -> {
// do the needed action with the value
//..
System.out.printf("My cool value: %s", stringValue);
});
}
}
}
In the Pivotal Native Client I've setup a method to read and write a Geode cache region as follows:
public void GeodePut(string region, string key, string value)
{
CacheFactory cF = CacheFactory.CreateCacheFactory();
Cache c cF.Create();
RegionFactory rF = c.CreateRegionFactory(RegionShortcut.CACHING_PROXY);
IRegion<string, string> r = rF.Create<string, string>(region);
r[key] = value;
cache.Close();
}
when I call this multiple times I get RegionExistsException how do I get around that? Thanks
Solution is easy.
Add a try-catch block to catch the RegionExistsException, then in the catch segment replace the 'create' method with 'get'.
Change this: rF.Create
for this: rf.get
This works pretty well using Java, i would post the exact signature of the method you needed but im not using .Net native client.
Hope it helps :)
It's to do with the cache.Close() command. I no longer use cache.Close()
I'm using NEST to index my objects and I'm running into a Newtonsoft error on serialization. One of my objects has a self referencing loop. Would there be a way for me to access the JsonSerializer and change how it handles self-references without having to modify the source code?
You can register custom converters on your client:
public void AddConverter(JsonConverter converter)
{
this.IndexSerializationSettings.Converters.Add(converter);
this.SerializationSettings.Converters.Add(converter);
}
This might be of help.
There is no direct way to alter the JsonSerializerSettings used in the client though.
There is a new api now, take a look at:
var cs2 = new ConnectionSettings(new Uri("http://localhost:9200"))
.SetJsonSerializerSettingsModifier(settings => settings.TypeNameHandling = TypeNameHandling.None)
.EnableTrace();
Thanks for adding the support!
Using Restlet 2.1 for Java EE, I am discovering an interesting problem with its ability to handle attributes.
Suppose you have code like the following:
cmp.getDefaultHost().attach("/testpath/{attr}",SomeServerResource.class);
and on your browser you provide the following URL:
http://localhost:8100/testpath/command
then, of course, the attr attribute gets set to "command".
Unfortunately, suppose you want the attribute to be something like command/test, as in the following URL:
http://localhost:8100/testpath/command/test
or if you want to dynamically add things with different levels, like:
http://localhost:800/testpath/command/test/subsystems/network/security
in both cases the attr attribute is still set to "command"!
Is there some way in a restlet application to make an attribute that can retain the "slash", so that one can, for example, make the attr attribute be set to "command/test"? I would like to be able to just grab everything after testpath and have the entire string be the attribute.
Is this possible? Someone please advise.
For the same case I usually change the type of the variable :
Route route = cmp.getDefaultHost().attach("/testpath/{attr}",SomeServerResource.class);
route.getTemplate().getVariables().get("attr") = new Variable(Variable.TYPE_URI_PATH);
You can do this by using url encoding.
I made the following attachment in my router:
router.attach("/test/{cmd}", TestResource.class);
My test resource class looks like this, with a little help from Apache Commons Codec URLCodec
#Override
protected Representation get() {
try {
String raw = ResourceWrapper.get(this, "cmd");
String decoded = new String(URLCodec.decodeUrl(raw.getBytes()));
return ResourceWrapper.wrap(raw + " " + decoded);
} catch(Exception e) { throw new RuntimeException(e); }
}
Note my resource wrapper class is simply utility methods. The get returns the string of the url param, and the wrap returns a StringRepresentation.
Now if I do something like this:
http://127.0.0.1/test/haha/awesome
I get a 404.
Instead, I do this:
http://127.0.0.1/test/haha%2fawesome
I have URLEncoded the folder path. This results in my browser saying:
haha%2fawesome haha/awesome
The first is the raw string, the second is the result. I don't know if this is suitable for your needs as it's a simplistic example, but as long as you URLEncode your attribute, you can decode it on the other end.