Status Bar in for loop - sql

I was trying to create a status bar in my for loop, but the status remains at 0%. I'm not sure what the issue could be.
n <- length(unique(mu_ut$AnimalID))
pb <- txtProgressBar(min = 0, max = n, style = 3)
for(i in unique(mu_ut$AnimalID)){
AnID <- paste("AnID", i, sep = ".")
assign(AnID, mu_ut[mu_ut$AnimalID == i,])
dbWriteTable(conn = db, name = AnID, value = mu_ut[mu_ut$AnimalID == i,],
field.types = c(DateAndTime = "DATETIME",
AnimalID = "CHAR",
Species = "CHAR",
Sex = "CHAR",
CurrentCohort = "CHAR",
BirthYear = "DATE",
CaptureUnit = "CHAR",
CaptureSubunit = "CHAR",
CaptureArea = "CHAR"), overwrite = TRUE)
Sys.sleep(1)
setTxtProgressBar(pb, i)
}
Is it a problem with the "i" argument in the setTxtProgressBar function?
Thank you in advance.

The issue is that setTxtProgressBar needs its value to be in the sequence min=0 to max=n, as set in your txtProgressBar. Since i is really one of the unique values within mu_ut$AnimalID, even if it is an integer, it is not clear that it is also in the range 0:n.
Try this.
ids <- unique(mu_ut$AnimalID)
pb <- txtProgressBar(min = 0, max = length(ids), style = 3)
for (ind in seq_along(ids)) {
id <- ids[ind]
AnID <- paste("AnID", id, sep = ".")
assign(AnID, mu_ut[mu_ut$AnimalID == id,])
dbWriteTable(conn = db, name = AnID, value = mu_ut[mu_ut$AnimalID == id,],
field.types = c(DateAndTime = "DATETIME",
AnimalID = "CHAR",
Species = "CHAR",
Sex = "CHAR",
CurrentCohort = "CHAR",
BirthYear = "DATE",
CaptureUnit = "CHAR",
CaptureSubunit = "CHAR",
CaptureArea = "CHAR"), overwrite = TRUE)
Sys.sleep(1)
setTxtProgressBar(pb, ind)
}
(This is very similar to #CareyCaginalp's comment.)

Related

Force single line of string in VObject

I am trying to create vCards (Email contacts) unsing the vobject library and pandas for python.
When serializing the values I get new lines in the "notes" of my output(no new lines in source). In every new line, created by ".serialize()", there is also a space in the beginning. I would need to get rid of both.
Example of output:
BEGIN:VCARD
VERSION:3.0
EMAIL;TYPE=INTERNET:test#example.at
FN:Valentina test
N:Valentina;test;;;
NOTE:PelletiererIn Mitglieder 2 Preiserhebung Aussendung 2 Pressespiegelver
sand 2 GeschäftsführerIn PPA_PelletiererInnen GeschäftsführerIn_Pellet
iererIn
ORG:Test Company
TEL;TYPE=CELL:
TEL;TYPE=CELL:
TEL;TYPE=CELL:
END:VCARD
Is there a way that I can force the output in a single line?
output = ""
for _,row in df.iterrows():
j = vobject.vCard()
j.add('n')
j.n.value = vobject.vcard.Name(row["First Name"],row["Last Name"])
j.add('fn')
j.fn.value = (str(row["First Name"]) + " " + row["Last Name"])
o = j.add("email")
o.value = str((row["E-mail Address"]))
o.type_param = "INTERNET"
#o = j.add("email")
#o.value = str((row["E-mail 2 Address"]))
#o.type_param = "INTERNET"
j.add('org')
j.org.value = [row["Organization"]]
k = j.add("tel")
k.value = str(row["Home Phone"])
k.type_param = "CELL"
k = j.add("tel")
k.value = str(row["Business Phone"])
k.type_param = "CELL"
k = j.add("tel")
k.value = str(row["Mobile Phone"])
k.type_param = "CELL"
j.add("note")
j.note.value = row["Notiz für Kontaktexport"]
output += j.serialize()
print(output)

Creating Multiple SQL Tables based on the factors from a column

I am trying to create a SQLite table for each factor that I have in a R dataframe. I have created a for loop in an attempt to accomplish this, but when I include the dWriteTable() function it gives me an error. I believe it might have something to do with the "argument" but I can't say for certain.
Here is the code I currently have with the for loop:
# Connects to the database##
mydb <- dbConnect(RSQLite::SQLite(), "../Output/all_data.sqlite")
#Reads the selected table in database
mu_ut <- dbReadTable(mydb, "mu_ut")
for(i in unique(mu_ut$AnimalID)){
AnID <- paste("AnID", i, sep = ".")
dfname <- assign(AnID, mu_ut[mu_ut$AnimalID == i,])
dbWriteTable(conn = mydb, name = dfname, value = dat_csv,
field.types = c(DateAndTime = "DATETIME",
AnimalID = "CHAR",
Species = "CHAR",
Sex = "CHAR",
CurrentCohort = "CHAR",
BirthYear = "DATE",
CaptureUnit = "CHAR",
CaptureSubunit = "CHAR",
CaptureArea = "CHAR"))
}
I get the error message when I run it:
Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘dbWriteTable’ for signature ‘"SQLiteConnection", "data.frame", "data.frame"’
Any help would be appreciated!
Thank you!
When using dbWriteTable :
name is a character string giving the name of the table.
value is a dataframe with the values to write into the table
Try:
dbWriteTable(conn = mydb, name = AnID, value = mu_ut[mu_ut$AnimalID == i,],
field.types = c(DateAndTime = "DATETIME",
AnimalID = "CHAR",
Species = "CHAR",
Sex = "CHAR",
CurrentCohort = "CHAR",
BirthYear = "DATE",
CaptureUnit = "CHAR",
CaptureSubunit = "CHAR",
CaptureArea = "CHAR"))

Fuzzy matching and iteration through DataFrame

I have these two DataFrames: I want to fuzzy match the Surname strings to the corresponding Names
dico = {'Name': ['Arthur','Henri','Lisiane','Patrice'],
"Age": ["20","18","62","73"],
"Studies": ['Economics','Maths','Psychology','Medical']
}
dico2 = {'Surname': ['Henri2','Arthur1','Patrice4','Lisiane3']}
dico = pd.DataFrame.from_dict(dico)
dico2 = pd.DataFrame.from_dict(dico2)
I want to fuzzy match the Surname strings to the corresponding Names to have an output as follows
Name Surname Age Studies
0 Arthur Arthur1 20 Economics
1 Henri Henri2 18 Maths
2 Lisiane Lisiane3 62 Psychology
3 Patrice Patrice4 73 Medical
and here is my code so far:
dico['Surname'] = []
for i in dico2:
lst = [0, 0, 0]
for j in dico:
if lst[0] < fuzz.ratio(i,j):
lst[0] = fuzz.ratio(i,j)
lst[1] = i
lst[2] = j
dico['Surname'].append(i)
but i get a ValueError: Length of values (0) does not match length of index (4), which I don't get why. Thanks !
dico = {'Name': ['Arthur','Henri','Lisiane','Patrice'],
"Age": ["20","18","62","73"],
"Studies": ['Economics','Maths','Psychology','Medical']
}
dico2 = {'Surname': ['Henri2','Arthur1','Patrice4','Lisiane3']}
dico = pd.DataFrame.from_dict(dico)
dico2 = pd.DataFrame.from_dict(dico2)
temp = pd.DataFrame()
for x in range (0, len(dico.Name)):
name_str = dico.Name[x]
temp = pd.concat([temp, dico2[dico2.Surname.str.contains(name_str)].Surname])
temp.columns=['Surname']
temp = temp.reset_index(drop = True)
dico = pd.concat([dico, temp], axis=1)
Solution
map_list = []
for name in dico['Name']:
best_ratio = None
for idx, surname in enumerate(dico2['Surname']):
if best_ratio == None:
best_ratio = fuzz.ratio(name, surname)
best_idx = 0
else:
ratio = fuzz.ratio(name, surname)
if ratio > best_ratio:
best_ratio = ratio
best_idx = idx
map_list.append(dico2['Surname'][best_idx]) # obtain surname
dico['Surname'] = pd.Series(map_list) # add column
dico = dico[["Name", "Surname", "Age", "Studies"]] # reorder columns
Result
The error originates from
dico['Surname'] = []
dico['Surname'] is length 4, while [] is length 0. You can instead collect your surnames in a list and then add the surnames to the dataframe in one go after the loop.
You also need to tell the outer loop to iterate over dico2['Surname'] instead of the entire dataframe.
surnames = []
for i in dico2['Surname']:
lst = [0, 0, 0]
for j in dico:
if lst[0] < fuzz.ratio(i,j):
lst[0] = fuzz.ratio(i,j)
lst[1] = i
lst[2] = j
surnames.append(i)
dico['Surname'] = surnames
EDIT: only fixed the error in question. Also see maxbachmann's advise on not calling fuzz.ratio twice.

Too many iif statements Access SQL

I have a large nested iif statement. I have 16 different fields that I need to assign either a value of 1 or 0, depending on their initial value. Here it is:
select
iif(overlimitexception = "Yes",0,1) as OverLimit,
iif(CashOutageIdentified = "Yes",0,1) AS CashOutage,
iif(MissingAuditIdentified = "Yes",0,1) AS MissingAudit,
iif(NightDropObserved = "Yes",0,1) AS NightDrop,
iif(SecurityTestComplete = "No",0,1) AS SecurityTest,
iif(CashPatternIdentified = "Yes",0,1) AS CashPattern,
iif(AllClearChange = "No",0,1) AS AllClear,
iif(RobberyKitReview = "No",0,1),
iif(EvacPlanReview = "No",0,1),
iif(KeyComboIssue = "Yes",0,1) AS KeyCombo,
iif(DualControlIssue = "Yes",0,1) AS DualControl,
iif(TaIssue = "Yes", 0,1) AS TA,
iif(CleanDeskIssue = "Yes",0,1) AS CleanDesk,
iif(MonthlyOpsIssue = "Yes",0,1) AS MonthlyOps,
iif(OverShortIssue = "Yes",0,1) AS OverShort,
iif(CashTargetIssue = "Yes",0,1) AS CashTarget
From [ROM Acknowledgement]
I could easily work around this if I was dealing with just one field, but I am dealing with 16. Is there a better way to do this? Thanks for any help!
If you're allergic to iif, the following presents a possible alternative:
select
1+(overlimitexception = "Yes") as OverLimit,
1+(CashOutageIdentified = "Yes") AS CashOutage,
1+(MissingAuditIdentified = "Yes") AS MissingAudit,
1+(NightDropObserved = "Yes") AS NightDrop,
1+(SecurityTestComplete = "No") AS SecurityTest,
1+(CashPatternIdentified = "Yes") AS CashPattern,
1+(AllClearChange = "No") AS AllClear,
1+(RobberyKitReview = "No"),
1+(EvacPlanReview = "No"),
1+(KeyComboIssue = "Yes") AS KeyCombo,
1+(DualControlIssue = "Yes") AS DualControl,
1+(TaIssue = "Yes") AS TA,
1+(CleanDeskIssue = "Yes") AS CleanDesk,
1+(MonthlyOpsIssue = "Yes") AS MonthlyOps,
1+(OverShortIssue = "Yes") AS OverShort,
1+(CashTargetIssue = "Yes") AS CashTarget
from
[ROM Acknowledgement]
Assume there is only "No" and "Yes" values in mentioned fields, you can write this
1 - eval(overlimitexception) as OverLimit,
-eval(SecurityTestComplete) AS SecurityTest,
Or a bit faster
3 - len(overlimitexception) as OverLimit,
-2 + len(SecurityTestComplete) AS SecurityTest,

How to increment a variable name by 1 in Lua, and then set that as the variable name?

My current code is:
obj1 = object:new{x = math.random(1,92), y = math.random(1,92), roomx = 0, roomy = 0, symbol = "t", name = "Tree"}
obj2 = object:new{x = math.random(1,92), y = math.random(1,92), roomx = 0, roomy = 0, symbol = "t", name = "Tree"}
obj3 = object:new{x = math.random(1,92), y = math.random(1,92), roomx = 0, roomy = 0, symbol = "t", name = "Tree"}
And so on. Since they are all the same thing, I'd like to be able to generate a variable that I can increment the name by one, and then use a for loop to create a lot of them. Is there a way I can do that in Lua? Thanks!
You can use a table to hold the objects and add them using a for loop.
local objects = {}
--> This will create 20 objects
for i=1, 20 do
objects[i] = object:new{x = math.random(1,92), y = math.random(1,92), roomx = 0, roomy = 0, symbol = "t", name = "Tree"}
end