Use pandas to append a .xslx file is overwriting it? - pandas

I have a very simple code and I cannot figure out why it isn't working. Consider the dataframes below,
test1 = pd.DataFrame({'A':[1,2],'B':[3,4]})
test2 = test1+3
I would like to export these into two separate sheets in an excel spreadsheet so I wrote the code
with pd.ExcelWriter(path+'.xlsx',mode='W') as writer:
test1.to_excel(writer,sheet_name='Sheet1')
with pd.ExcelWriter(path+'.xlsx',mode='A') as writer:
test2.to_excel(writer,sheet_name='Sheet2')
where the path is just a string to the folder. Running this returns a single spreadsheet labeled 'Sheet2' and has the data from table2. Evidently it overwrote it and didn't append it. Why? How can I solve this?
Of course I can write
with pd.ExcelWriter(path+'.xlsx',mode='W') as writer:
test1.to_excel(writer,sheet_name='Sheet1')
test2.to_excel(writer,sheet_name='Sheet2')
which will generate a spreadsheet with both sheet1 and sheet2 and the data stored properly, but I would like to get this append to work because I am trying to generalize to many more dataframes.

Related

Pass file locations to Macro in SAS

I'm at the start of learning/using SAS so I'm hoping this is easy.. I just can't find any solutions, certainly not simple enough for me to understand at this stage!
I'm looking to pass the file names(full path) to a Macro I've already got working.
I can get the file names into a dataset, but is there a way to pass these names into the Macro so that it runs it for each file one at a time? Looking to 'process' each csv in a folder into the Macro individually.
My code is pretty simple:
Info I want to pass to the Macro-
Data fName;
Run;
(A single column of 10 filepaths for example)
The Macro I want to 'push' these to is-
%processFiles(FileToProcess= '//example/test/file01.csv')
The simple data set works, the Macro works.. just can't figure out how to 'trigger' it for each file in that dataset.
Hopefully it's something easy.
Thanks
Tried several approaches; loops, passing as a list, nesting macros, etc.
Use the dataset to generate the calls to the macro.
So if the dataset is named FILES and variable with the file name is called FNAME then you just need to run this data step to call the macro once for each file in the dataset.
data _null_;
set files ;
call execute('%nrstr(%processFiles)(FileToProcess='||quote(trim(fname),"'")||')');
run;

Export values to specific cell in an excel file (using PDI)

I have in table input result as below:
I would like pass this result to excel file in specific row.
For example number_record I want to write to column J5, amount_ma to column K2 and so on. Can something like this be done in pentaho?
I have prepare a solution for you HERE. Using this solution, you can specifically tell which field you want to insert in which position in Excel. Like in my solution i told to insert field [NUMBER_RECORD to C5, AMOUNT_MA to D7,AMOUNT_WD to B5] Please run transformation "getData.ktr".
Add a new sheet to this Excel file, and output a line with the result to the new sheet. And on the old sheet in the cells you need, add formulas with a link to the first row in the new sheet

Pandas dataframe to extract data from csv file seperated by markers and save into excel sheets

I have a csv file which contains a time-based data, and I use a column "Marker" to help identify the idle time (value: 0) and a set of useful data collection steps(value: 1 and 2 represents step-1 and step-2), and the time-based data repeats the idle and data collection steps multiple times. **I would like to loop the csv file by checking the value of "Marker" column and separate the useful data steps and save each set of the data into separate excel sheet.
I have the following code in mind:
n=len(df)
i=0
newdf=[]
for i in range(0,n):
if df.MARKER[i]==1:
newdf.append(df.iloc[:,i])
if df.MARKER[i]==0:
end
return newdf
I have no thought about the remaining part of the codes yet since this is not able to proceed.
It looks like you're just trying to filter to elements where MARKER == 1. If that's the case, you could just do newdf = df[df['MARKER'] == 1]

Export specific data from a selected cell's row into a CSV file

I have been searching for a solution to this, but it seems like I cannot find it.
So basically, I want to select e.g. H2 and then run the macro.
Then it should copy/paste some specific cells into a new CSV file, e.g. O2 and F2. I also want a fixed value that should always be there, called "No".
The first row of the CSV file should be "UTF-8". The next (2nd row in the CSV) should be some headers that is fixed, just as the UTF-8.
Could a solution be to copy all the relevant data into another sheet with the proper format, and then just export that sheet as a CSV?
Illustration:
"UTF-8"
"Name","ID","Email","Customer"
"H2","O2","F2","No"
Solution ended up being exporting the correct data into another sheet with the proper setup, using the following.
Sheets("Sheet1").Range("XX:XX").Copy Destination:=Sheets("Sheet2").Range("XX")

excel macro to read text file and find matches in cells

I really could use some help
I have two .txt/csv files that I need to read from into my excel file.
In my excel file I have a whole column, each cell containing string of characters and I need to write a script to be able find matches and and copy an adjacent column from that txt file.
An example of a single row on my txt file is shown below:
"AB101AA","AB10 1AA","AB101A","AB10 1A","AB101","AB10 1","AB10","AB10","AB","10",394251,806376,,
"AB101AF","AB10 1AF","ABERDEEN","ABERDEENSHIRE",,"ABERDEEN, CITY OF"
My excel file would have a cell which probably say "AB101AF" and i want the corresponding cell to run through a million rows and find the match and then find the corresponding nth cell on the txt file and return it on the excel spreadsheet example "ABERDEEN, CITY OF".
I know I havent been helpful in explaining the issue. But any help would be appreciated.
Thank you
Depending upon the size of your text file you could import the file using the GetExternalData option in Excel. This would allow you to load your data into a different Sheet and then use a lookup to your data from the main Sheet. Using Match and/or vlookup should help here.
You could also add a workbook connection to the text file and search using the connection.