Python 3 copy data as is with xlrd and xlwt - xlrd

I'm new to programming, teaching myself Python3. Wife asked me to make her a script that will read data from excel, from one column, and copy every other row into another column.
She has an excel of 12k rows and it's in form of: row1=string, row2=(int/float/date/time), row3=string, row4=(int/float/date/time)...
what i did is this:
import xlrd
import xlwt
workbook = xlrd.open_workbook("MyExcel.xls")
sheet = workbook.sheet_by_index(0)
wb = xlwt.Workbook()
ws = wb.add_sheet("Sheet1")
i = 0
data = []
for i in range(sheet.nrows):
if i % 2 == 0:
value = sheet.cell(i, 0).value
data.append(value)
j = 0
for j in range(len(data)):
ws.write(j, 0, data[j])
j += 1
wb.save("MyOutput.xls")
It works fine, but the issue is that it works fine with strings and integers, not with dates or time, it converts those into floats.
I just need it to copy the value of each cell "as is" no formatting or anything, just copy paste. If a cell has a value of: "monkey/73.0:blah" i just want it to copy it as "monkey/73.0:blah" into the new excel.
Any idea how to achieve this?
ps: I know it can be achieved within excel itself, without using python(with INDEX and ROWS), but i'm curious as to how to transfer data in this way as normal copy paste, without it for example dividing my dates into a float if its 1/1/2016.
Thx

You can't do exactly what you want to, because you don't understand what Excel stores in the file.
Dates and times are stored as floating-point values. Time values are between 0 and 1.0, and date/time values are stored as larger numbers (negative dates are not handled correctly by many versions of Excel). You can't just copy the cell values "as-is" (i.e., without formatting) because formatting is the only way a cell is known to be a date or time!
To read the dates & times, you could replace your block of code that builds the data list with this:
cv = lambda v, t: (v if t != 3
else datetime(*(xlrd.xldate_as_tuple(v, workbook.datemode))))
data = [ cv(sheet.cell(i, 0).value, sheet.cell_type(i, 0))
for i in range(1, sheet.nrows, 2) ]
This will give you a list of string, float, and datetime objects you can write to your output file (writing dates using xlrw is left as an exercise for the reader because I haven't done it, and it's late here & now).

Related

Setting number format using Excel object quickly

I am writing 12,000 records with 46 columns, as a production report, to an excel file. The worksheet is not displayed while it is filled with data.
Previous StackOverflow information taught me to use arrays of objects to put values in ranges for speed. I had hoped this worked for formatting the values as well.
Code snip:
objExcel.Calculation = XlCalculation.xlCalculationManual
objExcel.ScreenUpdating = False
dcel = objWS.Range(objWS.Cells(rowdatastart, 1), objWS.Cells(rowdataend, nProdReportCol.ProdReportColCount - 2))
dcel.Value = aobj
dcel.NumberFormat = bobj
objExcel.ScreenUpdating = True
objExcel.Calculation = XlCalculation.xlCalculationAutomatic
aobj and bobj are object(,) arrays that fit the range. bobj contains strings such as "h:mma/p" to display time as "12:23a", and "0.00" to show numbers as "53/25".
The "dcel.value = aobj" takes half a second.
The "dcel.NumberFormat = bobj" takes 38 seconds.
Any suggestion for what I've missed/misunderstood? I'd rather a 7 second report (start excel, write, save, close excel) not take 45 seconds because I want the numbers/dates/times to appear as I choose.
If each column has its own format then try formatting per whole column. Also for each column do not use an array, if the formats are identical then you can just use a single string.
Do so for each column.
Please attempt and provide feedback.
After some more experimentation, I found the same solution as S Meaden. As only 14 of the 46 columns are not "General", I collected the column numbers for dates, times and 2-decimal numeric. Looping through each of these, gen a range for the 12000 records and 1 column, set the format. This takes about half a second for all the columns.
Odd that setting a row of 46 cells using a 46 object array takes so much longer than 12000 cells in a column, but there you are.
Thanks everyone.

Excel Summing over Rows with for loop - Type mismatch error

so I am currently working on an Excel sheet where I have to calculate confidence intervals. Long story short, I think the only way I can do this automatically, is to write vba code. The first step would be to calculate the average of the cells in a column for several columns in the sheet. What I did:
Dim temp As Double
temp = 0
Dim it_row As Long
for it_row = 1 to 100
if IsBlank(Sheet.Cells(it_row,it_col)) then
temp = temp + 0
else
temp = temp + Sheet.Cells(it_row,it_col).Value
end if
next it_row
Dim Average As Double
Average = temp/100
'writing average in another cell
This code does not work, as the compiler returns Type missmatch, error code 13
in the line
temp = temp + Sheet.Cells(it_row,it_col).Value
I tried to do a CDouble(Sheet.Cells(it_row,it_col).Value) but that did not work.
Any help is appreciated, as I am quite desperate because googling did not really help me.
I should mention that I do have to use vba and this code because this is part of a bigger automated process and my supervisor said I must use vba for automation in the next step.
The Average and Sum Excel Functions ignore text, boolean, and empty values:
Average = Application.Average(Sheet.Cells(1, it_col).Resize(100))
Check
Sheet.Cells(it_row,it_col).Value
...with
if isnumeric(Sheet.Cells(it_row,it_col).Value)
...before adding it to a double type value. If that check fails, then you can chose to skip it or treat it as sth. else.
I would've added this as a comment, but I don't have enough rep. to add comments.

How to apply bitwise formula to a range using VBA

I am looking for a way to apply a bitwise AND on a value in excel and apply it on range. Here is a picture to illustrate :
I want to apply a bitwise AND on the Column "Status" with a fixed value, lets say (1001b).
If Status & 1001b = 1001b, then report the "Value" field in "Output". Else, put 0.
Both Excel and VBA can be used, but as the table can be huge, the solution may not use loops (Too slow, time efficient solution would be great). I am using excel 2010, and therefore the excel BITAND function is not available for me (only available since Excel 2013 version)
I search on SO and internet and didn't find a correct approach to a similar problem.
I try to optimize my initial solution which use recursivity, to be more time efficient, here is what I have done :
For cptDonnee_s32 = 0 To JeuDonnee_rs.RecordCount - 2 ' Loop through all my data
If (Value And 8) = 8) And (Value And 1) = 1) Then ' Compare bitfields one by one
Output = Value ' Replace output by value here
Else
Output = 0 ' put 0 in output there
End If
next
You could use a simple UDF:
Function BitAnd(dIn As Double, lMatch As Long) As Boolean
BitAnd = (dIn And lMatch) = lMatch
End Function
and enter:
=if(BitAnd(K2,9),L2,0)
and copy down.

VBA - draw in numbers from one sheet and get data from another

I am trying to write a VBA to draw data from one sheet to another, but am stuck on something.
I only need some of the data in the original sheet (let's call it s1), in particular, I need data between two rows.
I have these rows written down in another sheet (s2), so I know exactly between which rows I need the data from. As you may expect there are multiple rows between which I need the data.
The problem is now that I am trying to write a VBA that is able to look up these rows in my row sheet (s2), and then goes to the sheet in which all my original data is contained (s1), and then draws out the data between the two rows into a third sheet (s3).
I have not been able to make it draw in the numbers from s2 (can't seem to work out how to tell it that it is these two rows between which I need the data, but from another sheet), and currently have to input the row numbers myself, which is really tedious, since the dataset is large!
Any help would be much appreciated!
Thank you!
Have you tried the .find option.
You could use something like this:
findrow = w1.Cells.Find(what:="Content", MatchCase:=False).Row
An more extensive example would help
Just break down your problem into smaller steps.
You can assign a value on a sheet using cell references:
Sheets("S1").Cells(iRow, iCol) = Value
'or
Sheets("S3").Cells(2, 10) = Sheets("S2").Cells(2, 10)
You can use Sheets("S1").Range("J" & iRow) or Range("J2").
If there are conditions that need to be met for something to happen, use If statements.
If (Value 1 < Value 2) Then
Sheets("S3").Cells(2, 10) = Sheets("S2").Cells(2, 10)
Else
Sheets("S3").Cells(2, 10) = Sheets("S1").Cells(2, 10)
End If
Without more information about the specific choices you are using to determine which rows to copy and from which sheet, it's hard to say. But you can do things in smaller batches or copy whole ranges, by looping.
Dim iRow As Integer
'This would copy all the data from Sheet("S1").Range("J1:J20) to Sheet "S3".
For iRow = 1 to 20
Sheets("S3").Cells(iRow, 10) = Sheets("S1").Cells(iRow, 10)
Next iRow
You could insert an If statement into each row, inside the loop to see if a certain criteria is met, and then decide which sheet in which to copy the data.

Excel VBA creating a new column with formula

I have an excel file with a column which has date data. I want the user to input a date of their choosing and then I want to create a new column that lists the difference in days between the two dates. The Macro that I have is working but I have a few questions and I would like to make it better. Link to MWE small data file is here.
The user input date was 9/30/2013, which I stored in H20
Macro:
Sub Date_play()
Dim x As Date
Dim x2 As Date
Dim y As Variant
x = InputBox(Prompt:="Please enter the Folder Report Date. The following formats are acceptable: 4 1 2013 or April 1 2013 or 4/1/2013")
x2 = Range("E2")
y = DateDiff("D", x2, x)
MsgBox y
'Used DateDiff above and it works but I don't know how to use it to fill a column or indeed a cell.
Range("H20").FormulaR1C1 = x
Range("H1").FormulaR1C1 = "Diff"
Range("H2").Formula = "=DATEDIF(E2,$H$20,""D"")"
Range("H2").AutoFill Destination:=Range("H2:H17")
Range("H2:H17").Select
End Sub
Now, could I have done this without storing the user input date in a particular cell? I would've preferred to use the variable "x" in the formula but it wasn't working for me. I had to store the user input in H20 and then use $H$20.
What's the difference between the function Datedif and the procedure DateDiff? I am able to use the procedure DateDiff in my macro but I don't know how to use it to fill out my column. Is one method better than the other?
Is there a better way to add columns to the existing sheet, where the columns include some calculations involving existing data on the sheet and some user inputs? There are tons of more complicated calculations I want to do next.
Thanks
Q1. Now, could I have done this without storing the user input date in a particular cell? I would've preferred to use the variable "x" in the formula but it wasn't working for me. I had to store the user input in H20 and then use $H$20.
Try this (UNTESTED)
Replace
Range("H20").FormulaR1C1 = x
Range("H1").FormulaR1C1 = "Diff"
Range("H2").Formula = "=DATEDIF(E2,$H$20,""D"")"
Range("H2").AutoFill Destination:=Range("H2:H17")
Range("H2:H17").Select
by
Range("H2:H17").Formula = "=DATEDIF(E2," & datevalue(x) & ",""D"")"
The above will fill all the cells with the formula in one go. You do not need the Autofill to do the job. Also Inputbox is the worst choice to accept dates. You might want to see THIS
Q2. What's the difference between the function Datedif and the procedure DateDiff? I am able to use the procedure DateDiff in my macro but I don't know how to use it to fill out my column. Is one method better than the other?
DATEDIF is a worksheet function and DateDiff is a VBA Function.