I am using below code to get the latest measurement API details for specific device but its not returning the data in descending order:
import com.cumulocity.sdk.client.measurement.MeasurementFilter;
import com.cumulocity.sdk.client.Platform;
import com.cumulocity.rest.representation.measurement.MeasurementRepresentation;
#Autowired
private Platform platform;
MeasurementFilter filter = new MeasurementFilter().byType("type").bySource("deviceId").byDate(fromDate,dateTo);
Iterable<MeasurementRepresentation> mRep = platform.getMeasurementApi().getMeasurementsByFilter(filter).get().elements(1);
List<MeasurementRepresentation> mRepList = StreamSupport.stream(mRep.spliterator(), false).collect(Collectors.toList());
...
MeasurementFilter api
we can get the latest data using 'revert=true' in Http REST url call..
../measurement/measurements?source={deviceId}&type={type}&dateTo=xxx&dateFrom=xxx&revert=true
How we can use 'revert=true' or other way to get measurement details in order using Cumulocity Java SDK? appreciate your help here.
The SDK currently has no out-of-the-box QueryParam for revert parameter so you have to create it yourself:
import com.cumulocity.sdk.client.Param;
public class RevertParam implements Param {
#Override
public String getName() {
return "revert";
}
}
And then you can combine it with your query. Therefore you to include your Query Param when you use the get() on the MeasurementCollection. You are currently not passing anything but you can pass pageSize and an arbitrary number of QueryParam.
private Iterable<MeasurementRepresentation> getMeasurementByFilterAndQuery(int pageSize, MeasurementFilter filter, QueryParam... queryParam) {
MeasurementCollection collection = measurementApi.getMeasurementByFilter(filter);
Iterable<MeasurementRepresentation> iterable = collection.get(pageSize, queryParam).allPages();
return iterable;
}
private Optional<MeasurementRepresentation> getLastMeasurement(GId source) {
QueryParam revertQueryParam = new QueryParam(new RevertParam(), "true");
MeasurementFilter filter = new MeasurementFilter()
.bySource(source)
.byFromDate(new DateTime(0).toDate());
Iterable<MeasurementRepresentation> iterable = measurementRepository.getMeasurementByFilterAndQuery(1, filter, revertQueryParam);
if (iterable.iterator().hasNext()) {
return Optional.of(iterable.iterator().next());
} else {
return Optional.absent();
}
}
Extending your code it could look like this:
QueryParam revertQueryParam = new QueryParam(new RevertParam(), "true");
MeasurementFilter filter = new MeasurementFilter().byType("type").bySource("deviceId").byDate(fromDate,dateTo);
Iterable<MeasurementRepresentation> mRep = platform.getMeasurementApi().getMeasurementsByFilter(filter).get(1, revertQueryParam);
List<MeasurementRepresentation> mRepList = StreamSupport.stream(mRep.spliterator(), false).collect(Collectors.toList());
What you did with elements is not incorrect but it is not limiting the API call to just return one value. It would query with defaultPageSize (=5) and then on Iterable level limit it to only return one. The elements() function is more for usage when you need more elements than the maxPageSize (=2000). Then it will handle automatic requesting for additional pages and you can just loop through the Iterable.
Related
Hi I am trying with Spring WebFlux and reactor.
What I want to achieve is to aggregate some requests into one call to a third party API. And then return the results. My code looks like this:
#RestController
#RequiredArgsConstructor
#RequestMapping(value = "/aggregate", produces = MediaType.APPLICATION_JSON_VALUE)
public class AggregationController {
private final WebClient externalApiClient;
//sink that stores the query params
private Sinks.Many<String> sink = Sinks.many().multicast().onBackpressureBuffer();
#GetMapping
public Mono<Mono<Map<String, BigDecimal>>> aggregate(#RequestParam(value = "pricing", required = false) List<String> countryCodes) {
//store the query params in the sink
countryCodes
.forEach(sink::tryEmitNext);
return Mono.from(
sink.asFlux()
.distinct()
.bufferTimeout(5, Duration.ofSeconds(5))
)
.map(countries -> String.join(",", countries))
.map(concatenatedCountries -> invokeExternalPricingApi(concatenatedCountries))
.doOnSuccess(s -> sink.emitComplete((signalType, emitResult) -> emitResult.isSuccess()));
}
private Mono<Map<String, BigDecimal>> invokeExternalPricingApi(String countryCodes) {
return externalApiClient.get().uri(uriBuilder -> uriBuilder
.path("/pricing")
.queryParam("q", countryCodes)
.build())
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(new ParameterizedTypeReference<>() {
});
}
}
I have the Sinks.Many<String> sink to accumulate the query params from the callers of the API. When there are 5 items or 5 seconds expired, I want to call the third party API. The issue is that when I do two requests like:
GET localhost:8081/aggregate?pricing=CH,EU
GET localhost:8081/aggregate?pricing=AU,NZ,PT
On one request I get the response and on the other I get null. Also, after the first interaction the system stops working as if the sink was broken.
What am I missing here?
Is there any API using which I can get graphTraversalName to graphName mapping defined in the script?
I am using the below messy code but it's error-prone if both graphs are using the same underlying storage.
Map<String, String> graphTraversalToNameMap = new ConcurrentHashMap<String, String>();
while(traversalSourceIterator.hasNext()){
String traversalSource = traversalSourceIterator.next();
String currentGraphString = ( (GraphTraversalSource) graphManager.getAsBindings().get(traversalSource)).getGraph().toString();
graphNameTraversalMap.put(currentGraphString, traversalSource);
}
Iterator<String> graphNamesIterator = graphManager.getGraphNames().iterator();
while(graphNamesIterator.hasNext()){
String graphName = graphNamesIterator.next();
String currentGraphString = graphManager.getGraph(graphName).toString();
String traversalSource = graphNameTraversalMap.get(currentGraphString);
graphTraversalToNameMap.put(traversalSource, graphName);
}
Does gremlinExecutor.getScriptEngineManager().getBindings().entrySet() provide order guarantee? I can iterate over this and populate my map
Is there any API using which I can get graphTraversalName to graphName mapping defined in the script?
No. They share the same namespace in Gremlin Server so the relationship gets lost programmatically. You would need to do something like what you are doing but I wouldn't rely on toString() of a Graph for equality. Perhaps use the Graph instance itself? Although that might not work either depending on your situation and what you want for equality as you could have two different Graph configurations pointed at the same data and want to resolve those as the same graph. I'm also not sure that any approach will work generally for all graph systems. Anyway, I think I'd experiment with using Map<Graph, String> graphTraversalToNameMap for your case and see how that goes.
Does gremlinExecutor.getScriptEngineManager().getBindings().entrySet() provide order guarantee?
No as it is backed by a ConcurrentHashMap. You would have to provide your own order.
Underlying storage details can be obtained from the configuration object and can be used for the mapping, sample code:
public class GraphTraversalMappingUtil {
public static void populateGraphTraversalToNameMapping(GraphManager graphManager){
if(graphTraversalToNameMap.size() != 0){
return;
}
Iterator<String> traversalSourceIterator = graphManager.getTraversalSourceNames().iterator();
Map<StorageBackendKey, String> storageKeyToTraversalMap = new HashMap<StorageBackendKey, String>();
while(traversalSourceIterator.hasNext()){
String traversalSource = traversalSourceIterator.next();
StorageBackendKey key = new StorageBackendKey(
graphManager.getTraversalSource(traversalSource).getGraph().configuration());
storageKeyToTraversalMap.put(key, traversalSource);
}
Iterator<String> graphNamesIterator = graphManager.getGraphNames().iterator();
while(graphNamesIterator.hasNext()) {
String graphName = graphNamesIterator.next();
StorageBackendKey key = new StorageBackendKey(
graphManager.getGraph(graphName).configuration());
graphTraversalToNameMap.put(storageKeyToTraversalMap.get(key), graphName);
}
}
}
For full code, refer: https://pastebin.com/7m8hi53p
I am using jdbcTemplate to query hive then writing the results to a .csv file. I basically just generate a list of objects then steam the list to write each record to the file.
I will like to stream the results as they coming back from hive and write it to the file instead of wait to get the whole thing then processing it. Can anyone pointing me to the right direction? Thanks!
private List<Avs> queryAvsData(String asSql) {
List<Avs> llistAvs = new ArrayList<Avs>();
List<Map<String, Object>> rows = hiveJdbcTemplate.queryForList(asSql);
Iterator<Map<String, Object>> it = rows.iterator();
while (it.hasNext()) {
Map<String, Object> row = it.next();
Avs laAvs = Avs.builder()
.make((String) row.get("make"))
.model((String) row.get("model"))
.build();
llistAvs.add(laAvs);
}
return llistAvs;
}
It doesn't look like there's a built-in solution, but you can do it. Basically, you wrap the existing functionality in an iterator, and use a spliterator to turn it into a stream. Here's a blog post on the subject:
The code implements Spring’s ResultSetExtractor interface, which is a Single Abstract Method (SAM) interface, allowing the use of a lambda expression to implement it.
The implementation wraps the SQL ResultSet in an iterator, constructs a stream using the Spliterators and StreamSupport utility classes, and applies that to a Function taking a stream of row sets and returning a generic result.
It's possible to stream values from JdbcTemplate. The following example is a service based on Spring Boot 2.4.8.
As, I run into problems (connection leak) using queryForStream then I will put a demo code here just to know that stream must be closed after usage.
import lombok.RequiredArgsConstructor;
import org.springframework.jdbc.core.SingleColumnRowMapper;
import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
import org.springframework.stereotype.Service;
import java.util.Map;
import java.util.stream.Stream;
#Service
#RequiredArgsConstructor
public class DataCleaningService {
private final NamedParameterJdbcTemplate jdbcTemplate;
public void doSomeStreaming() {
String nativeQuery = "SELECT string_value FROM my_table WHERE column = :valueToFiler";
Map<String, Object> queryParameters = Map.of("valueToFiler", "my value");
SingleColumnRowMapper<String> stringRowMapper = SingleColumnRowMapper.newInstance(String.class);
try (Stream<String> stringValueStream = jdbcTemplate.queryForStream(nativeQuery, queryParameters, stringRowMapper)) {
stringValueStream.forEach(stringValue -> {
// do the needed action with the value
//..
System.out.printf("My cool value: %s", stringValue);
});
}
}
}
Is there a way to add a response header to spring webflux controller endpoint? for example to the following method I have to add a custom header say 'x-my-header'
#GetMapping(value = "/search/{text}")
#ResponseStatus(value = HttpStatus.OK)
public Flux<SearchResult> search(#PathVariable(
value = "text") String text){
return searchService().find(text);
}
In the functional API, this is really easy; the ServerResponse builder has builders for almost everything you need.
With the annotated controllers; you can return an ResponseEntity<Flux<T>> and set the headers:
#GetMapping(value = "/search/{text}")
public ResponseEntity<Flux<SearchResult>> search(#PathVariable(
value = "text") String text) {
Flux<SearchResult> results = searchService().find(text);
return ResponseEntity.ok()
.header("headername", "headervalue")
.body(results);
}
Note that the updated code doesn't need the #ResponseStatus annotation now.
UPDATE:
Apparently the solution above works; unless you have spring-cloud-starter-netflix-hystrix-dashboard dependency. In that case you can use the following code:
#GetMapping(value = "/search/{text}")
public Mono<ResponseEntity<List<SearchResult>>> search(#PathVariable(
value = "text") String text) {
return searchService().find(text)
.collectList()
.map(list -> ResponseEntity.ok()
.header("Header-Name", "headervalue")
.body(list));
}
A couple of things to note:
Outer type should be Mono<ResponseEntity<T>>: There is one response for request. If you declare it to be a Flux, Spring will try to deserialize the ResponseEntity as if it was a POJO.
You need to use an operator to transform the Flux into a Mono: collectList() or single() will do the job for you.
Checked with Spring Boot 2.0.3.RELEASE
I'm trying to download a csv file after applying filters to the DataProvider.
For some reason the filtered results are shown in the Grid, but the downloaded csv file still contains all data.
#AutoView
class FinancialTransactionsView : VerticalLayout(), View {
private val grid: Grid<FinancialTransaction>
private val yearField: ComboBox<Int>
private val dataProvider = DataProvider.ofCollection(FinancialTransaction.findAll())
private val fileDownloader: FileDownloader
init {
label("Financial Transactions") {
styleName = ValoTheme.LABEL_H1
}
yearField = comboBox("Select Year") {
setItems(listOf(2016, 2017, 2018))
addSelectionListener {
// Filter the data based on the selected year
if (it.value != it.oldValue) setDataProvider()
}
}
// Create FileDownloader and initialize with all contents in the DataProvider
fileDownloader = FileDownloader(createCsvResource())
val downloadButton = button("Download csv") {
styleName = ValoTheme.BUTTON_PRIMARY
onLeftClick {
// The idea here is to assign values from the filtered DataProvider to the FileDownloader
fileDownloader.fileDownloadResource = createCsvResource()
}
}
fileDownloader.extend(downloadButton)
fileDownloader.fileDownloadResource = createCsvResource()
grid = grid(dataProvider = dataProvider) {
expandRatio = 1f
setSizeFull()
addColumnFor(FinancialTransaction::companyId)
addColumnFor(FinancialTransaction::fiscalYear)
addColumnFor(FinancialTransaction::fiscalPeriod)
addColumnFor(FinancialTransaction::currency)
addColumnFor(FinancialTransaction::finalizedDebitAmountInCurrency)
addColumnFor(FinancialTransaction::finalizedCreditAmountInCurrency)
appendHeaderRow().generateFilterComponents(this, FinancialTransaction::class)
}
}
private fun createCsvResource(): StreamResource {
return StreamResource(StreamResource.StreamSource {
val csv = dataProvider.items.toList().toCsv()
try {
return#StreamSource csv.byteInputStream()
} catch (e: IOException) {
e.printStackTrace()
return#StreamSource null
}
}, "financial_transactions.csv")
}
private fun setDataProvider() {
dataProvider.clearFilters()
if (!yearField.isEmpty)
dataProvider.setFilterByValue(FinancialTransaction::fiscalYear, yearField.value)
}
}
toCsv() is an extension function List<FinancialTransaction> which returns a string containing csv data.
What can I do to get the filtered results in my csv file?
val csv = dataProvider.items.toList().toCsv()
I am not Kotlin guy, but I assume dataProvider.items is a shorthand to dataProvider.getItems() in Java, i.e. this method (and you use ListDataProvider)
https://vaadin.com/download/release/8.4/8.4.1/docs/api/com/vaadin/data/provider/ListDataProvider.html#getItems--
In Vaadin getItems() returns all items by passing all filters.
So instead you should do either of the following
dataProvider.fetch(..)
https://vaadin.com/download/release/8.4/8.4.1/docs/api/com/vaadin/data/provider/DataProvider.html#fetch-com.vaadin.data.provider.Query-
Where you give the filters you want to apply in the query, or
grid.getDataCommunicator.fetchItemsWithRange(..)
https://vaadin.com/download/release/8.4/8.4.1/docs/api/com/vaadin/data/provider/DataCommunicator.html#fetchItemsWithRange-int-int-
Which returns list of items with filters you have set applied, which I think is ideal for you
Thank you for using Vaadin-on-Kotlin!
I've just updated the Databases Guide which should hopefully answer all of your questions. If not, just let me know and I'll update the guides accordingly.
The ListDataProvider.items will not apply any filters and will always return all items.
You need to use the getAll() extension function in order to obey the filters set by the Grid.
This is now explained in the Exporting data from DataProviders chapter of the Databases Guide.
In your code, both the grid and the yearField will set the filter to the same data provider,
thus overwriting values set by each other. Please read the Chaining Data Providers chapter in the Databases Guide to learn how to AND multiple filters set by multiple components.
When you use private val dataProvider = DataProvider.ofCollection(FinancialTransaction.findAll()), that will load all transactions from the database in-memory. You can use a more memory-efficient way: private val dataProvider = FinancialTransaction.dataProvider (given that FinancialTransaction is an Entity)
Please let me know if this answers your questions. Thanks!