Im am 100% new to Lua and need a way to check my etc/hosts file for a string. The other posts I found were dealing with searching in strings and and reading files line by line.
Here is some of my script which is a combination of some of the examples I found on here:
file = io.open("C:\Windows\System32\drivers\etc\hosts", "a")
function check(file)
if file.match(str, "nbs") then
file:close()
Screen.Next();
else
file:write("\n", %UserIP%, " nbs document")
file:close()
Screen.Next();
end
end;
As you can see Im searching the file hosts for the sting nbs. If it exists I want to move on. If it does not, I plan to append the file with a new line.
The above seems to do nothing when I enter it into my Lua Shell.
EDIT 1: adding full script; this is the full original script + my additions
-- These actions are performed when the Next button is clicked.
-- from _SUF70_Global_Functions.lua:
-- is the "Name:" field empty?
if(g_EditFieldIsEmpty(CTRL_EDIT_01)) then
-- the name field is empty...tell the user that it's a required field
-- and remain on this screen (don't advance to the next one)
-- "Invalid Entry"
local strTitle = SetupData.GetLocalizedString("MSG_INVALID_ENTRY");
-- get the label for the "Name:" edit field (since it may have been translated)
local strFieldName = DlgStaticText.GetProperties(CTRL_STATICTEXT_LABEL_01).Text;
-- strip off the trailing ":" (if present)
strFieldName = String.TrimRight(strFieldName, ":");
-- "The <fieldname> field cannot be empty."
local strPrompt = SetupData.GetLocalizedString("MSG_THE")
..strFieldName
..SetupData.GetLocalizedString("MSG_FIELD_CANNOT_BE_EMPTY");
Dialog.Message(strTitle, strPrompt, MB_OK, MB_ICONEXCLAMATION, MB_DEFBUTTON1);
else
--andrew you added the lines below
file = io.open("C:\Windows\System32\drivers\etc\hosts", "a")
function check(file)
if file.match(str, "test") then
file:close()
Screen.Next()
else
file:write("\n", "test")
file:close()
Screen.Next();
end
end;
-- the "Name:" field isn't empty...so
-- advance to the next screen
-- Screen.Next();
end;
This ended up being the solution I was looking for:
ip = SessionVar.Expand("%UserIP%")
result = TextFile.ReadToString(SessionVar.Expand("%SystemFolder%\\drivers\\etc\\hosts"));
if string.match(result, "nbs") then
Screen.Next()
else
file = io.open("C:\\Windows\\System32\\drivers\\etc\\hosts", "a")
file:write("\n", ip, " nbs document")
file:close()
Screen.Next()
end
In this case I was using the applications built in functions. These seem to make use of C as well so they would not work in a Lua shell.
Your code creates a file handle and defines a function check().
If it is supposed to do more you have to have to add more content. Like a function call to check.
Once you call check you'll most likely face a script error because you call a function file.match which does not exist in native Lua.
file = io.open("C:\Windows\System32\drivers\etc\hosts", "a")
-- this loop will give you each line of the file as a string and you can use string.match
for line in file:lines() do
if line:match("nbs") then
print("I found one!!")
end
end
I'm sure you can take it from here. Please refer to https://www.lua.org/manual/5.3/manual.html#6.8
This is my sample file
#%cty_id1,#%ccy_id2,#%cty_src,#%cty_cd3,#%cty_nm4,#%cty_reg5,#%cty_natnl6,#%cty_bus7,#%cty_data8
690,ALL2,,AL,ALBALODMNIA,,,,
90,ALL2,,,AQ,AKNTARLDKCTICA,,,
161,IDR2,,AZ,AZLKFMERBALFKIJAN,,,,
252,LTL2,,BJ,BENLFMIN,,,,
206,CVE2,,BL,SAILFKNT BAFSDRTHLEMY,,,,
360,,,BW2,BOPSLFTSWLSOANA,,,,
The problem is for #%cty_cd3 is a standard column(NOT NULL) with length 2 letters only, but in sql server the record shifts to the other column,(due to a extra comma in btw)how do i validate a csv file,to make sure that
when there's a 2 character word need to be only in 4 column?
there are around 10000 records ?
Set of rules Defined !
Should have a standard set of delimiters for eachrow
if not
Check for NOT NULL values having Null values
If found Null
remove delimiter at the pointer
The 3 ,,, are not replaced with 2 ,,
#UPDATED : Can i know if this can be done using a script ?
Updated i need only a function That operates on records like
90,ALL2,,,AQ,AKNTARLDKCTICA,,, correct them using a Regex or any other method and put back into the source file !
Your best bet here may be to use the tSchemaComplianceCheck component in Talend.
If you read the file in with a tFileInputDelimited component and then check it with the tSchemaComplianceCheck where you set cty_cd to not nullable then it will reject your Antarctica row simply for the null where you expect no nulls.
From here you can use a tMap and simply map the fields to the one above.
You should be able to easily tweak this as necessary, potentially with further tSchemaComplianceChecks down the reject lines and mapping to suit. This method is a lot more self explanatory and you don't have to deal with complicated regex's that need complicated management when you want to accommodate different variations of your file structure with the benefit that you will always capture all of the well formatted rows.
You could try to delete the empty field in column 4, if column no. 4 is not a two-character field, as follows:
awk 'BEGIN {FS=OFS=","}
{
for (i=1; i<=NF; i++) {
if (!(i==4 && length($4)!=4))
printf "%s%s",$i,(i<NF)?OFS:ORS
}
}' file.csv
Output:
"id","cty_ccy_id","cty_src","cty_nm","cty_region","cty_natnl","cty_bus_load","cty_data_load"
6,"ALL",,"AL","ALBANIA",,,,
9,"ALL",,"AQ","ANTARCTICA",,,
16,"IDR",,"AZ","AZERBAIJAN",,,,
25,"LTL",,"BJ","BENIN",,,,
26,"CVE",,"BL","SAINT BARTH�LEMY",,,,
36,,,"BW","BOTSWANA",,,,
41,"BNS",,"CF","CENTRAL AFRICAN REPUBLIC",,,,
47,"CVE",,"CL","CHILE",,,,
50,"IDR",,"CO","COLOMBIA",,,,
61,"BNS",,"DK","DENMARK",,,,
Note:
We use length($4)!=4 since we assume two characters in column 4, but we also have to add two extra characters for the double quotes..
The solution is to use a look-ahead regex, as suggested before. To reproduce your issue I used this:
"\\,\\,\\,(?=\\\"[A-Z]{2}\\\")"
which matches three commas followed by two quoted uppercase letters, but not including these in the match. Ofc you could need to adjust it a bit for your needs (ie. an arbitrary numbers of commas rather than exactly three).
But you cannot use it in Talend directly without tons of errors. Here's how to design your job:
In other words, you need to read the file line by line, no fields yet. Then, inside the tMap, do the match&replace, like:
row1.line.replaceAll("\\,\\,\\,(?=\\\"[A-Z]{2}\\\")", ",,")
and finally tokenize the line using "," as separator to get your final schema. You probably need to manually trim out the quotes here and there, since tExtractDelimitedFields won't.
Here's an output example (needs some cleaning, ofc):
You don't need to entry the schema for tExtractDelimitedFields by hand. Use the wizard to record a DelimitedFile Schema into the metadata repository, as you probably already did. You can use this schema as a Generic Schema, too, fitting it to the outgoing connection of tExtractDelimitedField. Not something the purists hang around, but it works and saves time.
About your UI problems, they are often related to file encodings and locale settings. Don't worry too much, they (usually) won't affect the job execution.
EDIT: here's a sample TOS job which shows the solution, just import in your project: TOS job archive
EDIT2: added some screenshots
Coming to the party late with a VBA based approach. An alternative way to regex is to to parse the file and remove a comma when the 4th field is empty. Using microsoft scripting runtime this can be acheived the code opens a the file then reads each line, copying it to a new temporary file. If the 4 element is empty, if it is it writes a line with the extra comma removed. The cleaned data is then copied to the origonal file and the temporary file is deleted. It seems a bit of a long way round, but it when I tested it on a file of 14000 rows based on your sample it took under 2 seconds to complete.
Sub Remove4thFieldIfEmpty()
Const iNUMBER_OF_FIELDS As Integer = 9
Dim str As String
Dim fileHandleInput As Scripting.TextStream
Dim fileHandleCleaned As Scripting.TextStream
Dim fsoObject As Scripting.FileSystemObject
Dim sPath As String
Dim sFilenameCleaned As String
Dim sFilenameInput As String
Dim vFields As Variant
Dim iCounter As Integer
Dim sNewString As String
sFilenameInput = "Regex.CSV"
sFilenameCleaned = "Cleaned.CSV"
Set fsoObject = New FileSystemObject
sPath = ThisWorkbook.Path & "\"
Set fileHandleInput = fsoObject.OpenTextFile(sPath & sFilenameInput)
If fsoObject.FileExists(sPath & sFilenameCleaned) Then
Set fileHandleCleaned = fsoObject.OpenTextFile(sPath & sFilenameCleaned, ForWriting)
Else
Set fileHandleCleaned = fsoObject.CreateTextFile((sPath & sFilenameCleaned), True)
End If
Do While Not fileHandleInput.AtEndOfStream
str = fileHandleInput.ReadLine
vFields = Split(str, ",")
If vFields(3) = "" Then
sNewString = vFields(0)
For iCounter = 1 To UBound(vFields)
If iCounter <> 3 Then sNewString = sNewString & "," & vFields(iCounter)
Next iCounter
str = sNewString
End If
fileHandleCleaned.WriteLine (str)
Loop
fileHandleInput.Close
fileHandleCleaned.Close
Set fileHandleInput = fsoObject.OpenTextFile(sPath & sFilenameInput, ForWriting)
Set fileHandleCleaned = fsoObject.OpenTextFile(sPath & sFilenameCleaned)
Do While Not fileHandleCleaned.AtEndOfStream
fileHandleInput.WriteLine (fileHandleCleaned.ReadLine)
Loop
fileHandleInput.Close
fileHandleCleaned.Close
Set fileHandleCleaned = Nothing
Set fileHandleInput = Nothing
KillFile (sPath & sFilenameCleaned)
Set fsoObject = Nothing
End Sub
If that's the only problem (and if you never have a comma in the field bt_cty_ccy_id), then you could remove such an extra comma by loading your file into an editor that supports regexes and have it replace
^([^,]*,[^,]*,[^,]*,),(?="[A-Z]{2}")
with \1.
i would question the source system which is sending you this file as to why this extra comma in between for some rows? I guess you would be using comma as a delimeter for importing this .csv file into talend.
(or another suggestion would be to ask for semi colon as column separator in the input file)
9,"ALL",,,"AQ","ANTARCTICA",,,,
will be
9;"ALL";,;"AQ";"ANTARCTICA";;;;
I am looking for some expert insight concerning recursion within vbscript.
From various examples found online I created the following code, which works by the way.
http://saltwetbytes.wordpress.com/2010/05/04/vbscript-grabbing-subfolders-recursively/
http://technet.microsoft.com/en-us/library/ee198872.aspx
Function GetAllSubFolders(RootFolder, ByRef pSubfoldersList)
Dim fso, SubFolder, root
Set fso = CreateObject("scripting.filesystemobject")
set root = fso.getfolder(RootFolder)
For Each Subfolder in root.SubFolders
If pSubFoldersList = "" Then
pSubFoldersList = Subfolder.Path
Else
pSubFoldersList = pSubFoldersList & "|" & Subfolder.Path
End If
GetAllSubFolders Subfolder, pSubFoldersList
Next
GetAllSubFolders = pSubFoldersList
End Function
My question is: Is this a good aproach when it comes to creating a recursive function (using a parameter for storing previous results)?
I prefer putting this in a (self-contained) "function", so the procedure returns the subsubfolders as the result. But most of the examples found use a "sub" I always get confused when it comes to "sub" vs "function" (I understand when you want a procedure that needs to return something you use a function, imho this seems to be the case in this example)
But I could also use a "sub" and just simple reference the output parameter (ByRef pSubfoldersList)
So what is the best practise or is it better to use a whole different approach all together?
(the function is this examples is also very slow compared to [shell.exec "cmd /c dir RootFolder /s /b /a:d"], I guess this is a side effect from the recursion or maybe the FSO is just really slow?)
whether it is good practice to pass the result in a recursive function, i don't really know, you can test this out by doing it this way and the other and comparing the time and memory taken. Haven't tried this with your version cause i get the error "Microsoft VBScript runtime error: Permission denied" if I start from the root of the c:
The real problem with your solution is the concatenation, that takes time because the in your case BIG variable gets created every time. Better were to store the result in an array or in the case of VBscript in a dictionary. I'll post an example.
What the difference between sub and function concerns: you are right about the main difference, the returning of a result but that is optional so I always use functions, the only drawback is that if you don't assign the value to a variable and you use more than 2 parameters you have to use "call". When you use your approach with ByRef you could also define the var in the main global context, it's perhaps less encapsulated but more readable and you can more easily reuse or debug the results.
What the speed concerns: vbscript is VERY slow in file handling, if you used WMI perhaps you could speed up a bit but not much, indeed for some operations it is better to shell out and let the OS take care of it. I now program in Ruby and there most jobs like this you can write in one line of code and it is much faster.
Speaking about fast, if your only purpose is to have a list of your files, get to know the tool "search everything", in less than a second you can search millions of files, if you don't know it check it out !
Here is an example using the dictionary
set fso = CreateObject("Scripting.FileSystemObject")
set filelist = CreateObject("Scripting.Dictionary")
iCount = 0
ShowSubfolders fso.GetFolder("C:\Documents and Settings\peter")
PrintFilelist(filelist)
'--- ---
Function ShowSubFolders(Folder)
For Each Subfolder in Folder.SubFolders
on error resume next
wscript.echo Subfolder.Path 'show some progress
Set fFolder = fso.GetFolder(Subfolder.Path)
if err.number <> 0 then wscript.echo err.description
For Each File in fFolder.Files
iCount = iCount+1
filelist.add iCount, File.Path
Next
ShowSubFolders Subfolder
Next
End Function
'--- ---'
Function PrintFilelist(ByRef dic)
Dim index, allKeys, allItems, msg
allKeys = dic.Keys
' allKeys is an array to all the keys
allItems = dic.Items
' allItems is an array to all the items
wscript.echo "There are " & dic.Count & " number of files in the dictionary"
For index = 0 To dic.Count-1
' Notice, dictionary range goes from 0 to count-1
wscript.echo "Key=" & allKeys(index) & " Filename=" & allItems(index)
Next
End Function