I am using a data table in Powershell ISE on 2 large csv files. The second file is used to lookup various facilities by ID number found in File1. So, File2 has either 1 or many facilities depending on where they go. I'm trying to do a standard foreach loop, but not sure how to populate where the number variable would go, since it's SQL syntax (is my understanding). $Script:MappingTable2 has at least one matching ID, but sometimes many. I have loaded these into data tables and can't figure out how to add the PowerShell equivalent of $_ for lookup. So, where 2017 will pull all the facilities that have 2017 as the ID, I would like to look these up per $Code in File1. $Script:MappingTable2.select("PRACT_ID = '$Code.ID'") won't work, but if just the number is there (like below) it grabs all the rows related to the ID. Or just one if only a one to one match. Hoping to add the array as a single combined string to a new 3rd csv cell in a new column, which is supposed to be a merging of the 2 files. Thx
foreach ($Code in $File1) {
$Script:MappingTable2.select("PRACT_ID = '2017'")}
I figured it out! The $Code.gettype() was BaseType - System.Object Name - DataRow. When I set it to a [string] it works. So this works and is very fast.
foreach ($Code in $File1) {
[string]$ID = $Code.PRACT_ID
$Script:MappingTable2.select("PRACT_ID = '$ID'")}
Related
I am working with 350 datasets. I want to automate naming the final datasets with values from the dataset.
For example, if ID is abc and year is 2010. There are two columns in the dataset with those values. I want to pull that information out and use in the file name. and the name would look like abc_2010.dta in this case.
So basically I want to do
foreach file in `files' {
**calculation codes**
** construct the file name as three digit ID_year.dta **
}
I have already done the calculation part. I need some help with the naming of the files.
If I understand what you are trying to do, I believe you should be able to do this:
foreach file in `files' {
**calculation codes**
** construct the file name as three digit ID_year.dta **
local fname:di "`=id[1]'_`=year[1]'"
save `fname', replace
}
Note that this assume that after the calculation of the current iteration of the loop through files, the value of id in the first row holds the three digit code and the value of year in the first row holds the year.
Does anyone know of a method that allows you to search a string through a text file using StreamReader that allows you to account for multiple instances of finding the results. Basically I am creating a booking application and each time a customer books a seat, their PrimaryKey, FirstName, LastName and the co-ordinates of the seat on a data grid (which I have used as a method to book seats) are generated then saved to a text file.
I want the ability to be able to read multiple instances of a PrimaryKey then find the seat co-ordinates of each line that this PrimaryKey is listed on and repopulate another similar datagridview with these co-ordinates which is all going to be driven by a combobox index change.
It seems a bit complicated to understand but if anyone can help then please let me know.
I just need the knowhow of how to search multiple instances, so after its found the string once then look through the rest of the file to find another instance, I can do the rest by myself.
I'm coding using Visual Basic.Net
Yes it's possible to search multiple times through a file, but you'd either have to reopen the file, or rewind the stream (FileStream.Seek).
Wildly inefficient though.
If it has to remain an unsorted and unstructured file, build an in memory index to it.
If your key is an integer, create a Dictionary<int,int> of Key and Position in the stream.
Then when you want find key X you use FileStream.Seek to move to it, and read a line to get the data. If you find yourself grouping by say aeroplaneID, build a Dictionary<Int, List<Int,Int>>
where the key is the aeroplane id and the list is a list of primary keys and positions in the file.
You could push all that off to a background thread. You could try and get really clever and build them up as you need them. Personally though I'd be trying to move my storage to a more suitable format. You aren't struggling to do this because you've missed a class, you are struggling because you shouldn't.
Something like
Dictionary<int, int> _fileIndex = new Dictionary<int,int>();
using(FileStream fs = new FileStream(DataFileName,FileMode.Open,FileAccess.Read))
{
StreamReader reader = new StreamReader(fs);
int lastPosition = 0;
string currentLine = null;
while(currentLine = reader.ReadLine() != null)
{
String[] data = currentLine.Split(new char[] {','});
int key = int.Parse(data[0]);
fileIndex.Add(key,lastPosition);
lastPosition = fs.Position;
}
}
NB didn't test the above and there should be a bit more error checking in it. If there's alot of data in the line, then might be better off not suing split and just pulling out everything up to the correct delimiter. Also be careful how many indexes you keep live, wouldn't be long before they used up more space than just reading the entire thing in to memory.
Then you could create a class or structure to implement a line in the file, and write a bit of code
to use FileStream.Seek) to get there. If you wanted to load up a bunch of 'em it would make sense to get your list of positions of each one in the file and then sort them in order, then you could rip through the file in it's 'order' picking them out.
Currently I need to create tests for my application. I used "dbunit" to achieve that and now need to compare 2 datasets:
1) The records from the database I get with QueryDataSet
2) The expected results are written in the appropriate FlatXML in a file which I read in as a dataset as well
Basically 2 datasets can be compared this way.
Now the problem are columns with a Timestamp. They will never fit together with the expected dataset. I really would like to ignore them when comparing them, but it doesn't work the way I want it.
It does work, when I compare each table for its own with adding a column filter and ignoreColumns. However, this approch is very cumbersome, as many tables are used in that comparison, and forces one to add so much code, it eventually gets bloated.
The same applies for fields which have null-values
A probable solution would also be, if I had the chance to only compare the very first column of all tables - and not by naming it with its column name, but only with its column index. But there's nothing I can find.
Maybe I am missing something, or maybe it just doesn't work any other way than comparing each table for its own?
For the sake of completion some additional information must be posted. Actually my previously posted solution will not work at all as the process reading data from the database got me trapped.
The process using "QueryDataset" did read the data from the database and save it as a dataset, but the data couldn't be accessed from this dataset anymore (although I could see the data in debug mode)!
Instead the whole operation failed with an UnsupportedOperationException at org.dbunit.database.ForwardOnlyResultSetTable.getRowCount(ForwardOnlyResultSetTable.java:73)
Example code to produce failure:
QueryDataSet qds = new QueryDataSet(connection);
qds.addTable(“specificTable”);
qds.getTable(„specificTable“).getRowCount();
Even if you try it this way it fails:
IDataSet tmpDataset = connection.createDataSet(tablenames);
tmpDataset.getTable("specificTable").getRowCount();
In order to make extraction work you need to add this line (the second one):
IDataSet tmpDataset = connection.createDataSet(tablenames);
IDataSet actualDataset = new CachedDataSet(tmpDataset);
Great, that this was nowhere documented...
But that is not all: now you'd certainly think that one could add this line after doing a "QueryDataSet" as well... but no! This still doesn't work! It will still throw the same Exception! It doesn't make any sense to me and I wasted so much time with it...
It should be noted that extracting data from a dataset which was read in from an xml file does work without any problem. This annoyance just happens when trying to get a dataset directly from the database.
If you have done the above you can then continue as below which compares only the columns you got in the expected xml file:
// put in here some code to read in the dataset from the xml file...
// and name it "expectedDataset"
// then get the tablenames from it...
String[] tablenames = expectedDataset.getTableNames();
// read dataset from database table using the same tables as from the xml
IDataSet tmpDataset = connection.createDataSet(tablenames);
IDataSet actualDataset = new CachedDataSet(tmpDataset);
for(int i=0;i<tablenames.length;i++)
{
ITable expectedTable = expectedDataset.getTable(tablenames[i]);
ITable actualTable = actualDataset.getTable(tablenames[i]);
ITable filteredActualTable = DefaultColumnFilter.includedColumnsTable(actualTable, expectedTable.getTableMetaData().getColumns());
Assertion.assertEquals(expectedTable,filteredActualTable);
}
You can also use this format:
// Assert actual database table match expected table
String[] columnsToIgnore = {"CONTACT_TITLE","POSTAL_CODE"};
Assertion.assertEqualsIgnoreCols(expectedTable, actualTable, columnsToIgnore);
I'm using the Dynamic LINQ library code sample to dynamically return some data. The number of columns I'm returning is variable depending on user input. I'm enumerating the IQueryable into an IList.
What I need to do is step through the Ilist one field at a time. I can get one row at a time by iterating through the IList's rows but I can't for the life of me pull one field out of the collection.
For example, here I'm returning two columns (hard coded for testing but in prod it will be variable depending on what fields the user chooses):
Dim Dynq = dc.dt _
.Where("RUN_ID = """ & runNumber & """ and Upper_Pressure > 95") _
.OrderBy("Upper_Pressure") _
.Select(" new (Run_ID,Process)")
Dim something = DirectCast(Activator.CreateInstance(GetType(List(Of )).MakeGenericType(Dynq.ElementType), Dynq), IList)
Now I can pull a field out of the Ilist if I know the column name with something like:
something.Run_ID.ToString
but I wont know the columns I'm using until runtime and dynamically inserting it in a variable that's set at runtime doesn't work.
So in Summary I have a Ilist that looks something like this
1 | Auto
2 | Auto
3 | Manual
4 | Manual
and I'd like a way to return
1
and then return
Auto
and then
2
etc...
I would greatly appreciate the help of those more learned than I in this.
Your question is a little confusing, but I think you want to flatten your return set to return specific columns as elements in order by column then row...
One method, you could use is the SelectMany operator.
For example (sorry in C# as my brain is turning one-tracked!):
// Find flattened list of some explicitly specified property values...
var flattened = something.SelectMany(e => new [] { e.Run_ID.ToString(), e.Process.ToString() });
Not sure if that's what your after, but it could be a step in the right direction.
myRs=myStmt.executeQuery("select i_col,col_name from tab_col")
i=0
while (myRs.next()):
list= myRs.getString("I_COL")+','+myRs.getString("COL_NAME")
i have a jython code to run a sql statement i want to store all the row of the sql into a single variable. I used to list to store the value but its always storing only the single line , so is there is way to append all the rows and keep adding to single variable.
Thanks so much for your help.
With that code you overwrite the "list" variable in every iteration of the while loop (= is an assignment), try something like this (I used rs rather than list to avoid a name clash with the builtin function list()):
myRs=myStmt.executeQuery("select i_col,col_name from tab_col")
rs=[]
i=0
while (myRs.next()):
rs.append(myRs.getString("I_COL")+','+myRs.getString("COL_NAME"))