I'm asking for your help,I'm facing a problem with the tJavaFlex component.
editorName ;ProductName ;end_date_resorption_versions;end_date_supported_versions ;end_date_recommended_versions
EditorA;PN_A;31/03/2017,31/03/2017,31/03/2017,31/03/2017,31/03/2017,31/03/2017;null;null
EditorA;PN_A;30/06/2024;null;30/06/2024
EditorA;PN_A;30/11/2020,30/06/2017;null;null
EditorA;PN_A;null;30/06/2024;null
EditorA;PN_A ;null;null;null
EditorA;PN_A;30/11/2020,30/11/2020;null;null
EditorB;PN_B;18/05/2017,31/03/2017,31/01/20;null;null
EditorB;PN_B;03/06/2024;01/02/2020;30/06/2024
EditorB;PN_B;23/12/2014 ;null;null
EditorB;PN_B;null;01/02/2020;30/06/2020
EditorB;PN_B;null;null;null
EditorB;PN_B;12/12/2012,31/12/2020;null;13/01/2020
As u can see there are list of date (string format) in same column.
What i want to do ==> it's to find the min date for each column (not row)
My approach is this :
By column, store all values in an array
convert ListString to ListDate
Find Date min in each ListDate
I thought the best way to do that was to use the tJavaFlex component.
----------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------
**Start Code :**
SimpleDateFormat sdf = new SimpleDateFormat("dd/MM/yyyy");
// Array for end_date_resorption_versions
List<String> myStringList_edrv = new ArrayList<>();
List<Date> dates = new ArrayList<>();
// Array for end_date_supported_versions
List<String> myStringList_edsv = new ArrayList<>();
List<Date> dates_edsv = new ArrayList<>();
// Array for end_date_recommended_versions
List<String> myStringList_edrev = new ArrayList<>();
List<Date> dates_edrev = new ArrayList<>();
----------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------
**main code :**
if (row4.end_date_resorption_versions == null ){
row4.end_date_resorption_versions = "31/12/2099";
}
if (row4.end_date_supported_versions == null ){
row4.end_date_supported_versions = "31/12/2099";
}
if (row4.end_date_recommended_versions == null) {
row4.end_date_recommended_versions = "31/12/2099";
}
// populating data :
myStringList_edrv.addAll(Arrays.asList(row4.end_date_resorption_versions.split(",")));
myStringList_edsv.addAll(Arrays.asList(row4.end_date_supported_versions.split(",")));
myStringList_edrev.addAll(Arrays.asList(row4.end_date_recommended_versions.split(",")));
----------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------
**end code :**
// transform ListString to ListDate : end_date_resorption_versions
for (int i = 0 ; i < myStringList_edrv.size(); i++) {
dates.add(sdf.parse(myStringList_edrv.get(i)));
}
// transform ListString to ListDate : end_date_supported_versions
for (int i = 0 ; i < myStringList_edsv.size(); i++) {
dates_edsv.add(sdf.parse(myStringList_edsv.get(i)));
}
// transform ListString to ListDate : end_date_recommended_versions
for (int i = 0 ; i < myStringList_edrev.size(); i++) {
dates_edrev.add(sdf.parse(myStringList_edrev.get(i)));
}
// getMinDate : end_date_resorption_versions
row6.out_date_edrv = sdf.format(Collections.min(dates));
// getMinDate : end_date_supported_versions
row6.out_date_edsv = sdf.format(Collections.min(dates_edsv));
// getMinDate : end_date_recommended_versions
row6.out_date_edrev = sdf.format(Collections.min(dates_edrev));
row6.out_editor_name = row4.editor_name;
row6.out_product_name = row4.product_name;
System.out.println("out_date_edrv => " + row6.out_date_edrv);
System.out.println("out_date_edsv => " + row6.out_date_edsv);
System.out.println("out_date_edrev => " + row6.out_date_edrev);
all my values are null, while the results present in the system.out.println are good
my job design is :
tPostgresqlInput----row4(Main)----tJavaFlex_1----row6(Main)----tLogRow
some help would really be appreciated.
You don't need all that java code. Here's a solution using only Talend components which has the advantage of being easier to maintain should your requirement change.
I start by normalizing your date columns; if only end_date_resorption_versions can contain a list of dates, then you can skip tNormalize_2 and tNormalize_3 which normalize end_date_supported_versions and end_date_recommended_versions respectively.
the tMap_1 is probably not needed, I use it to convert the "null" literal to null, since I copy pasted your sample data in a file for my test, otherwise the following conversion would fail.
tConvertType_1 then converts the date string to Date type, by checking the option "Auto cast", and setting the schema as follows:
Finally, tAggregateRow_1 will group on the editorName and ProductName columns and get the minimum date from each date column:
No explanation, just a nasty hack : Try to insert a tFlowToIterate before your tJavaFlex and feed the tJavaFlex with an iterate flow instead of a main flow (See pictures below).
Here, for some reason my tJavaFlex only reads null values from the input main flow
But it manages to read the exact same data when provided through an iterate flow.
Note that as long as your input data flow isn't renamed (row4 in my example), you won't have to touch the code in your tJavaFlex.
I have absolutely no clue on why nor when tJavaFlex wouldn't cope well with flows of type main, and I find that rather frustrating. I would surely appreciate if a talend guru could give me an explanation !
Related
In my ASP.NET Core 1.1 app I'm using EPPlus.Core to export data to Excel. Some columns exported have mostly numbers but rarely a text (e.g. N/A). These columns in generated Excel are showing (as expected) a green triangle on the top left corner of their cells. I want to get rid of those warnings.
Question: What's a good way of getting rid of those triangles when Excel is generated? I tried setting the format of these columns to text as follows but it did not work. I guess we need to set the format of these columns to General but I can't figure out how:
workSheet.Cells["D1:P1"].Style.Numberformat.Format = "General";
UPDATE:
Per a user's request, the code looks similar to following.
Error at inner loop for (var j = 4; j < testlist[i].Count(); j++){...}: MyViewModel does not contain a definition of Count()
Error at line if (testlist[i][j] is string){...}: cannot apply indexing with [] to an extension of type MyViewModel
Controller:
....
....
var testlist = (qry to load a MyViewModel).ToList();
using (ExcelPackage pkg= new ExcelPackage())
{
var ws = excel.Workbook.Worksheets.Add("TestWorkSheet");
ws.Cells[1, 1].LoadFromCollection(rc_excel, true);
//I'm starting from 2nd row and 5th column
for (var i = 1; i < testlist.Count; i++)
{
for (var j = 4; j < testlist[i].Count(); j++)
{
if (testlist[i][j] is string)
{
....
....
}
}
pkg.Save();
return(....);
}
So I have a standard service reference proxy calss for MS CRM 2013 (i.e. right-click add reference etc...) I then found the limitation that CRM data calls limit to 50 results and I wanted to get the full list of results. I found two methods, one looks more correct, but doesn't seem to work. I was wondering why it didn't and/or if there was something I'm doing incorrectly.
Basic setup and process
crmService = new CrmServiceReference.MyContext(new Uri(crmWebServicesUrl));
crmService.Credentials = System.Net.CredentialCache.DefaultCredentials;
var accountAnnotations = crmService.AccountSet.Where(a => a.AccountNumber = accountNumber).Select(a => a.Account_Annotation).FirstOrDefault();
Using Continuation (something I want to work, but looks like it doesn't)
while (accountAnnotations.Continuation != null)
{
accountAnnotations.Load(crmService.Execute<Annotation>(accountAnnotations.Continuation.NextLinkUri));
}
using that method .Continuation is always null and accountAnnotations.Count is always 50 (but there are more than 50 records)
After struggling with .Continutation for a while I've come up with the following alternative method (but it seems "not good")
var accountAnnotationData = accountAnnotations.ToList();
var accountAnnotationFinal = accountAnnotations.ToList();
var index = 1;
while (accountAnnotationData.Count == 50)
{
accountAnnotationData = (from a in crmService.AnnotationSet
where a.ObjectId.Id == accountAnnotationData.First().ObjectId.Id
select a).Skip(50 * index).ToList();
accountAnnotationFinal = accountAnnotationFinal.Union(accountAnnotationData).ToList();
index++;
}
So the second method seems to work, but for any number of reasons it doesn't seem like the best. Is there a reason .Continuation is always null? Is there some setup step I'm missing or some nice way to do this?
The way to get the records from CRM is to use paging here is an example with a query expression but you can also use fetchXML if you want
// Query using the paging cookie.
// Define the paging attributes.
// The number of records per page to retrieve.
int fetchCount = 3;
// Initialize the page number.
int pageNumber = 1;
// Initialize the number of records.
int recordCount = 0;
// Define the condition expression for retrieving records.
ConditionExpression pagecondition = new ConditionExpression();
pagecondition.AttributeName = "address1_stateorprovince";
pagecondition.Operator = ConditionOperator.Equal;
pagecondition.Values.Add("WA");
// Define the order expression to retrieve the records.
OrderExpression order = new OrderExpression();
order.AttributeName = "name";
order.OrderType = OrderType.Ascending;
// Create the query expression and add condition.
QueryExpression pagequery = new QueryExpression();
pagequery.EntityName = "account";
pagequery.Criteria.AddCondition(pagecondition);
pagequery.Orders.Add(order);
pagequery.ColumnSet.AddColumns("name", "address1_stateorprovince", "emailaddress1", "accountid");
// Assign the pageinfo properties to the query expression.
pagequery.PageInfo = new PagingInfo();
pagequery.PageInfo.Count = fetchCount;
pagequery.PageInfo.PageNumber = pageNumber;
// The current paging cookie. When retrieving the first page,
// pagingCookie should be null.
pagequery.PageInfo.PagingCookie = null;
Console.WriteLine("#\tAccount Name\t\t\tEmail Address");while (true)
{
// Retrieve the page.
EntityCollection results = _serviceProxy.RetrieveMultiple(pagequery);
if (results.Entities != null)
{
// Retrieve all records from the result set.
foreach (Account acct in results.Entities)
{
Console.WriteLine("{0}.\t{1}\t\t{2}",
++recordCount,
acct.EMailAddress1,
acct.Name);
}
}
// Check for more records, if it returns true.
if (results.MoreRecords)
{
// Increment the page number to retrieve the next page.
pagequery.PageInfo.PageNumber++;
// Set the paging cookie to the paging cookie returned from current results.
pagequery.PageInfo.PagingCookie = results.PagingCookie;
}
else
{
// If no more records are in the result nodes, exit the loop.
break;
}
}
Everybody!
I need to read and write a history data array.My data does loose data while page refresh. Thanks!
I resolved this problem by using LocalStore HTML 5.
With a some simple data, you can use:
var data = {};// data is a object just include text and value.
data = {"key1": value1, "key2": value2};// Example, i have a this object.
//===> We save a object "history" by a string of "data" var.
localStorage.setItem('history', JSON.stringify(data));
// ===> And get it:
var retrievedHistory = localStorage.getItem('history');
// In Sencha Touch, you can parse this oject to your model. and Syns it.
var history = Ext.getStore("HistoryTracking");
history.removeAll();
var num = Object.keys(data).length;
for (i = 0; i < num; i++)
{
var string = '' + data["trackID" + i];
history.add(string);
}
history.sync();
Hope my sharing help you have same problem!
So I want to allow the user to conditionally turn columns on/off in a Cardboard app I built. I have two problems.
I tried using the 'columns' attribute in the config but I can't seem to find a default value for it that would allow ALL columns to display(All check boxes checked) based on the attribute, ie. the default behavior if I don't include 'columns' in the config object at all (tried null, [] but that displays NO columns).
So that gets to my second problem, if there is no default value is there a simple way to only change that value in the config object or do I have to encapsulate the entire variable in 'if-else' statements?
Finally if I have to manually build the string I need to parse the values of an existing custom attribute (a drop list) we have on the portfolio object. I can't seem to get the rally.forEach loop syntax right. Does someone have a simple example?
Thanks
Dax - Autodesk
I found a example in the online SDK from Rally that I could modify to answer the second part (This assumes a custom attribute on Portfolio item called "ADSK Kanban State" and will output values to console) :
var showAttributeValues = function(results) {
for (var property in results) {
for (var i=0 ; i < results[property].length ; i++) {
console.log("Attribute Value : " + results[property][i]);
}
}
};
var queryConfig = [];
queryConfig[0] = {
type: 'Portfolio Item',
key : 'eKanbanState',
attribute: 'ADSK Kanban State'
};
rallyDataSource.findAll(queryConfig, showAttributeValues);
rally.forEach loops over each key in the first argument and will execute the function passed as the second argument each time.
It will work with either objects or arrays.
For an array:
var array = [1];
rally.forEach(array, function(value, i) {
//value = 1
//i = 0
});
For an object:
var obj = {
foo: 'bar'
};
rally.forEach(obj, function(value, key) {
//value = 'bar'
//key = 'foo'
});
I think that the code to dynamically build a config using the "results" collection created by your query above and passed to your sample showAttributeValues callback, is going to look a lot like the example of dynamically building a set of Table columns as shown in:
Rally App SDK: Is there a way to have variable columns for table?
I'm envisioning something like the following:
// Dynamically build column config array for cardboard config
var columnsArray = new Array();
for (var property in results) {
for (var i=0 ; i < results[property].length ; i++) {
columnsArray.push("'" + results[property][i] + "'");
}
}
var cardboardConfig = {
{
attribute: 'eKanbanState',
columns: columnsArray,
// .. rest of config here
}
// .. (re)-construct cardboard...
Sounds like you're building a neat board. You'll have to provide the board with the list of columns to show each time (destroying the old board and creating a new one).
Example config:
{
attribute: 'ScheduleState'
columns: [
'In-Progress',
'Completed'
]
}
I am trying to perform a "translation" of sorts of a stream of text. More specifically, I need to tokenize the input stream, look up every term in a specialized dictionary and output the corresponding "translation" of the token. However, i also want to preserve all the original whitespaces, stopwords etc from the input so that the output is formatted in the same way as the input instead of ended up being a stream of translations. So if my input is
Term1: Term2 Stopword! Term3
Term4
then I want the output to look like
Term1': Term2' Stopword! Term3'
Term4'
(where Termi' is translation of Termi) instead of simply
Term1' Term2' Term3' Term4'
Currently I am doing the following:
PatternAnalyzer pa = new PatternAnalyzer(Version.LUCENE_31,
PatternAnalyzer.WHITESPACE_PATTERN,
false,
WordlistLoader.getWordSet(new File(stopWordFilePath)));
TokenStream ts = pa.tokenStream(null, in);
CharTermAttribute charTermAttribute = ts.getAttribute(CharTermAttribute.class);
while (ts.incrementToken()) { // loop over tokens
String termIn = charTermAttribute.toString();
...
}
but this, of course, loses all the whitespaces etc. How can I modify this to be able to re-insert them into the output? thanks much!
============ UPDATE!
I tried splitting the original stream into "words" and "non-words". It seems to work fine. Not sure whether it's the most efficient way, though:
public ArrayList splitToWords(String sIn)
{
if (sIn == null || sIn.length() == 0) {
return null;
}
char[] c = sIn.toCharArray();
ArrayList<Token> list = new ArrayList<Token>();
int tokenStart = 0;
boolean curIsLetter = Character.isLetter(c[tokenStart]);
for (int pos = tokenStart + 1; pos < c.length; pos++) {
boolean newIsLetter = Character.isLetter(c[pos]);
if (newIsLetter == curIsLetter) {
continue;
}
TokenType type = TokenType.NONWORD;
if (curIsLetter == true)
{
type = TokenType.WORD;
}
list.add(new Token(new String(c, tokenStart, pos - tokenStart),type));
tokenStart = pos;
curIsLetter = newIsLetter;
}
TokenType type = TokenType.NONWORD;
if (curIsLetter == true)
{
type = TokenType.WORD;
}
list.add(new Token(new String(c, tokenStart, c.length - tokenStart),type));
return list;
}
Well it doesn't really lose whitespace, you still have your original text :)
So I think you should make use of OffsetAttribute, which contains startOffset() and endOffset() of each term into your original text. This is what lucene uses, for example, to highlight snippets of search results from the original text.
I wrote up a quick test (uses EnglishAnalyzer) to demonstrate:
The input is:
Just a test of some ideas. Let's see if it works.
The output is:
just a test of some idea. let see if it work.
// just for example purposes, not necessarily the most performant.
public void testString() throws Exception {
String input = "Just a test of some ideas. Let's see if it works.";
EnglishAnalyzer analyzer = new EnglishAnalyzer(Version.LUCENE_35);
StringBuilder output = new StringBuilder(input);
// in some cases, the analyzer will make terms longer or shorter.
// because of this we must track how much we have adjusted the text so far
// so that the offsets returned will still work for us via replace()
int delta = 0;
TokenStream ts = analyzer.tokenStream("bogus", new StringReader(input));
CharTermAttribute termAtt = ts.addAttribute(CharTermAttribute.class);
OffsetAttribute offsetAtt = ts.addAttribute(OffsetAttribute.class);
ts.reset();
while (ts.incrementToken()) {
String term = termAtt.toString();
int start = offsetAtt.startOffset();
int end = offsetAtt.endOffset();
output.replace(delta + start, delta + end, term);
delta += (term.length() - (end - start));
}
ts.close();
System.out.println(output.toString());
}