VBA - Handling folder name with utf8 characters - vba

I'm using Application.FileDialog(msoFileDialogFolderPicker) to pick a folder and it handles well folders with utf8 names.
But when I try to Debug.Print the result of SelectedItems(1) or save it to a config file or do anything, I loose the accents of the folder.
For example:
Original folder:
"D:\Śākta"
'Debug.Print' or saving into an utf8 file result saves:
"D:\Sakta" (removed all the accents)
The problem is that I try to save the selected folder to a config file and when I try to load it, next time it will of course won't recognize as a real folder because of the missing accents.
How to get the real name of the folder with the accents to be able to save it after, not this "representation" of it?
Update:
Just checked, and even the InputBox kills the accents!

#John Coleman's answer solved the issue switching the file saving to 'ADODB.Stream'
Here is an example of reading and writing config file supporting UTF8:
Public Function fileExists(ByVal fullFilename As String) As Boolean
fileExists = CreateObject("Scripting.FileSystemObject").fileExists(fullFilename)
End Function
Public Function ReadTextFile(ByVal sPath As String) As String
If fileExists(sPath) Then
Dim fsT As Object
Set fsT = CreateObject("ADODB.Stream")
fsT.Mode = adModeRead
fsT.Type = 2 'Specify stream type - we want To save text/string data.
fsT.Charset = "utf-8" 'Specify charset For the source text data.
fsT.Open 'Open the stream And write binary data To the object
fsT.LoadFromFile (sPath)
ReadTextFile = fsT.ReadText
fsT.Close
Set fsT = Nothing
Else
ReadTextFile = ""
End If
End Function
Public Function WriteTextFile(ByVal s As String, ByVal sPath As String) As Boolean
Dim objStreamUTF8NoBOM: Set objStreamUTF8NoBOM = CreateObject("ADODB.Stream")
Dim fsT As Object
Set fsT = CreateObject("ADODB.Stream")
fsT.Type = 2 'Specify stream type - we want To save text/string data.
fsT.Charset = "utf-8" 'Specify charset For the source text data.
fsT.Open 'Open the stream And write binary data To the object
fsT.WriteText s
fsT.Position = 0
fsT.SaveToFile sPath, 2 'Save binary data To disk
fsT.Position = 3
With objStreamUTF8NoBOM
.Type = 1
.Open
fsT.CopyTo objStreamUTF8NoBOM
.SaveToFile sPath, 2
Close
End With
fsT.Close
Set fsT = Nothing
Set objStreamUTF8NoBOM = Nothing
End Function
Function SetSettings(ByVal Keyname As String, ByVal Wstr As String) As String
Dim settingsFileContent
settingsFileContent = ReadTextFile(IniFileName)
Set RE = CreateObject("VBScript.RegExp")
RE.Pattern = Keyname + "=.*"
RE.MultiLine = 1
If RE.Test(settingsFileContent) Then
settingsFileContent = RE.Replace(settingsFileContent, Keyname + "=" + Wstr)
Else
settingsFileContent = settingsFileContent + IIf(Len(settingsFileContent) = 0, "", vbNewLine) + Keyname + "=" + Wstr
End If
WriteTextFile settingsFileContent, IniFileName
SetSettings = Wstr
End Function
Private Function GetSettings(ByVal Keyname As String) As String
Dim settingsFileContent As String
settingsFileContent = ReadTextFile(IniFileName)
Set RE = CreateObject("VBScript.RegExp")
RE.MultiLine = 1
RE.Global = 1
RE.Pattern = "\r"
settingsFileContent = RE.Replace(settingsFileContent, "")
RE.Global = 0
RE.Pattern = "^" + Keyname + "=(.*)"
Set allMatches = RE.Execute(settingsFileContent)
If allMatches.Count <> 0 Then
Debug.Print (Keyname + ": """ + allMatches.Item(0).SubMatches.Item(0) + """")
GetSettings = allMatches.Item(0).SubMatches.Item(0)
Else
GetSettings = ""
End If
End Function

Related

Classic ASP file upload issue

I am trying to upload a csv file then read it line by line and pass it to my Database. I have a issue on file upload I could see that whenever I upload a file an extra number gets added to a column in my csv and due to this my code fails. I could see that the file gets saved in the expected location and when I opened it I could see an extra number which I never added. What could be the reason for this?
Here is my code for uploading the CSV file and reading it and using a loop I'm iterating the values in the csv file and storing the values in my variables which is then passed to a method to store in DB.
<HTML>
<HEAD>
<!--#include file="clsUpload.asp"-->
<!-- #include file = "Bin/includes/GenFunctions.asp" -->
<!-- #include file = "Bin/includes/HeaderFooter.asp" -->
</HEAD>
<title>CMTL Entry File Upload</title>
<%showHeader%>
<BODY text="#000000" leftmargin="0" topmargin="0" marginwidth="0" marginheight="0">
<FORM ACTION = "clsUploadtest.asp" ENCTYPE="multipart/form-data" METHOD="POST">
File Name: <INPUT TYPE=FILE NAME="txtFile"><P>
<INPUT TYPE = "SUBMIT" NAME="cmdSubmit" VALUE="SUBMIT">
</FORM><P>
<%
'Declare the form variables
Dim intRecordId
Dim strRecordId
Dim strAction
Dim strInputDate
Dim strPayer
Dim strAmount
Dim strTransType
Dim strPayee
Dim strAddRemarks
Dim strComments
Dim strAccNo
Dim strPolicyNo
Dim strBranchConfDate
Dim strConfBy
Dim strSheetUpdDate
Dim strMailUpdDate
Dim strUploadTime
Dim strUploadId
Dim strStatus
Dim strLastMdfdDate
Dim strLastMdfdId
Dim lineData
Dim code
Dim name
Dim mail
Dim id
Dim price
Dim amount
Dim MyArray
Dim i
i = 0
set obj = Server.CreateObject("AC_FI.mMTL")
If err.number <> 0 Then
RaiseGenError objCMTL,"Error while creating the object "
Response.End
end if
set o = new clsUpload
if o.Exists("cmdSubmit") then
'get client file name without path
sFileSplit = split(o.FileNameOf("txtFile"), "\")
sFile = sFileSplit(Ubound(sFileSplit))
o.FileInputName = "txtFile"
o.FileFullPath = Server.MapPath(".") & "\entry\" & sFile
o.save
if o.Error = "" then
response.write "Success. File saved to " & o.FileFullPath
else
response.write "Failed due to the following error: " & o.Error
end if
set fso = Server.CreateObject("Scripting.FileSystemObject")
set fs = fso.OpenTextFile(Server.MapPath("entry/" & sFile), 1, False)
Do While Not fs.AtEndOfStream
lineData = fs.ReadLine
'lineData = replace(lineData,chr(13),",")
'lineData = replace(lineData,chr(34),"")
MyArray = Split(lineData , ",")
strRecordId=MyArray(0)
strInputDate=MyArray(1)
strPayer=MyArray(2)
strAmount=MyArray(3)
strTransType=MyArray(4)
strPayee=MyArray(5)
strAddRemarks=MyArray(6)
strComments=MyArray(7)
strAccNo=MyArray(8)
strPolicyNo=MyArray(9)
strBranchConfDate=MyArray(10)
strConfBy=MyArray(11)
strSheetUpdDate=MyArray(12)
strMailUpdDate=MyArray(13)
strUploadTime=MyArray(14)
strUploadId=MyArray(15)
strStatus=MyArray(16)
strRecordId = convertNullString(strRecordId)
strAction = convertNullString(strAction)
strInputDate = convertNullDate(strInputDate)
strPayer = convertNullString(strPayer)
strAmount = convertNullString(strAmount)
strTransType = convertNullString(strTransType)
strPayee = convertNullString(strPayee)
strAddRemarks = convertNullString(strAddRemarks)
strComments = convertNullString(strComments)
strAccNo = convertNullString(strAccNo)
strPolicyNo = convertNullString(strPolicyNo)
strBranchConfDate = convertNullDate(strBranchConfDate)
strConfBy = convertNullString(strConfBy)
strSheetUpdDate = convertNullDate(strSheetUpdDate)
strMailUpdDate = convertNullDate(strMailUpdDate)
strUploadTime = convertNullDate(strUploadTime)
strUploadId = convertNullString(strUploadId)
strStatus = convertNullString(strStatus)
intRecordId = obj.InsertData(strInputDate,strPayer,strAmount,strTransType,strPayee,strAddRemarks,_
strComments,strAccNo,strPolicyNo,strBranchConfDate,strConfBy,strSheetUpdDate,_
strMailUpdDate,strUploadTime,strUploadId,strStatus, pErrorcode)
i=i+1
Loop
fs.close()
set fs = nothing
set fso = nothing
end if
set o = nothing
%>
<%showFooter%>
</BODY>
</HTML>
this is the clsupload.asp file that I have used for file upload
<%
' ------------------------------------------------------------------------------
' Container of Field Properties
Class clsField
Public FileName
Public ContentType
Public Value
Public FieldName
Public Length
Public BinaryData
End Class
' ------------------------------------------------------------------------------
Class clsUpload
' ------------------------------------------------------------------------------
Private nFieldCount
Private oFields()
Private psFileFullPath
Private psError
Private psFileInputName
' ------------------------------------------------------------------------------
Public Property Get Count()
Count = nFieldCount
End Property
' ------------------------------------------------------------------------------
Public Default Property Get Field(ByRef asFieldName)
Dim lnLength
Dim lnIndex
lnLength = UBound(oFields)
If IsNumeric(asFieldName) Then
If lnLength >= asFieldName And asFieldName > -1 Then
Set Field = oFields(asFieldName)
Else
Set Field = New clsField
End If
Else
For lnIndex = 0 To lnLength
If LCase(oFields(lnIndex).FieldName) = LCase(asFieldName) Then
Set Field = oFields(lnIndex)
Exit Property
End If
Next
Set Field = New clsField
End If
End Property
' ------------------------------------------------------------------------------
Public Function Exists(ByRef avKeyIndex)
Exists = Not IndexOf(avKeyIndex) = -1
End Function
' ------------------------------------------------------------------------------
Public Property Get ValueOf(ByRef avKeyIndex)
Dim lnIndex
lnIndex = IndexOf(avKeyIndex)
if lnIndex = -1 Then Exit Property
ValueOf = oFields(lnIndex).Value
End Property
' ------------------------------------------------------------------------------
Public Property Get FileNameOf(ByRef avKeyIndex)
Dim lnIndex
lnIndex = IndexOf(avKeyIndex)
if lnIndex = -1 Then Exit Property
FileNameOf = oFields(lnIndex).FileName
End Property
' ------------------------------------------------------------------------------
Public Property Get LengthOf(ByRef avKeyIndex)
Dim lnIndex
lnIndex = IndexOf(avKeyIndex)
if lnIndex = -1 Then Exit Property
LengthOf = oFields(lnIndex).Length
End Property
' ------------------------------------------------------------------------------
Public Property Get BinaryDataOf(ByRef avKeyIndex)
Dim lnIndex
lnIndex = IndexOf(avKeyIndex)
if lnIndex = -1 Then Exit Property
BinaryDataOf = oFields(lnIndex).BinaryData
End Property
' ------------------------------------------------------------------------------
Private Function IndexOf(ByVal avKeyIndex)
Dim lnIndex
If avKeyIndex = "" Then
IndexOf = -1
ElseIf IsNumeric(avKeyIndex) Then
avKeyIndex = CLng(avKeyIndex)
If nFieldCount > avKeyIndex And avKeyIndex > -1 Then
IndexOf = avKeyIndex
Else
IndexOf = -1
End If
Else
For lnIndex = 0 To nFieldCount - 1
If LCase(oFields(lnIndex).FieldName) = LCase(avKeyIndex) Then
IndexOf = lnIndex
Exit Function
End If
Next
IndexOf = -1
End If
End Function
' ------------------------------------------------------------------------------
Public Property Let FileFullPath(sValue)
psFileFullPath = sValue
End Property
'___________________________________________________________________________________
Public Property Get FileFullPath()
FileFullPath = psFileFullPath
End Property
' ------------------------------------------------------------------------------
Public Property Let FileInputName(sValue)
psFileInputName = sValue
End Property
' -------------------- ----------------------------------------------------------
Public Function Save()
if psFileFullPath <> "" and psFileInputName <> "" then
'Save to connectionless client side recordset, write to stream,
'and persist stream.
'would think you should be able to write directly to
'stream without recordset, but I could not get that to work
On error resume next
binData = o.BinaryDataOf(psFileInputName)
set rs = server.createobject("ADODB.RECORDSET")
rs.fields.append "FileName", 205, LenB(binData)
rs.open
rs.addnew
rs.fields(0).AppendChunk binData
if err.number = 0 then
set objStream = Server.CreateObject("ADODB.Stream")
objStream.Type = 1
objStream.Open
objStream.Write rs.fields("FileName").value
objStream.SaveToFile psFileFullPath, 2
objStream.close
set objStream = Nothing
ENd if
rs.close
set rs = nothing
psError = Err.Description
else
psError = "One or more required properties (FileFullPath and/or FileInputName) not set"
End If
End Function
Public Property Get Error()
Error = psError
End Property
' ------------------------------------------------------------------------------
Public Property Get ContentTypeOf(ByRef avKeyIndex)
Dim lnIndex
lnIndex = IndexOf(avKeyIndex)
if lnIndex = -1 Then Exit Property
ContentTypeOf = oFields(lnIndex).ContentType
End Property
' ------------------------------------------------------------------------------
Private Sub Class_Terminate()
Dim lnIndex
For lnIndex = 0 To nFieldCount - 1
Set oFields(0) = Nothing
Next
End Sub
' ------------------------------------------------------------------------------
Private Sub Class_Initialize()
Dim lnBytes ' Bytes received from the client
Dim lnByteCount ' Number of bytes received
Dim lnStartPosition ' Position at which content begins
Dim lnEndPosition ' Position at which content ends
Dim loDic ' Contains properties of each
' specific field
' Local dictionary object(s)
' to be appended to class-scope
' dictionary object.
Dim lnBoundaryBytes ' Bytes contained within the current boundary
Dim lnBoundaryStart ' Position at which the current boundary begins
' within the lnBytes binary data.
Dim lnBoundaryEnd ' Position at which the current boundary ends
' within the lnBytes binary data.
Dim lnDispositionPosition
Dim lsFieldName ' Name of the current field being parsed from
' Binary Data
Dim lsFileName ' Name of the file within the current boundary
Dim lnFileNamePosition ' Location of file name within current boundary
Dim loField ' clsField Object
Dim lsValue ' Value of the current field
Dim lsContentType ' ContentType of the binary file (MIME Type)
' Initialize Fields
nFieldCount = 0
ReDim oFields(-1)
' Read the bytes (binary data) into memory
lnByteCount = Request.TotalBytes
lnBytes = Request.BinaryRead(lnByteCount)
'Get the lnBoundaryBytes
lnStartPosition = 1
lnEndPosition = InstrB(lnStartPosition, lnBytes, CStrB(vbCr))
If lnEndPosition >= lnStartPosition Then
lnBoundaryBytes = MidB(lnBytes, lnStartPosition, lnEndPosition - lnStartPosition)
End If
lnBoundaryStart = InstrB(1, lnBytes, lnBoundaryBytes)
' Loop until the BoundaryBytes begin with "--"
Do Until (lnBoundaryStart = InstrB(lnBytes, lnBoundaryBytes & CStrB("--")))
' All data within this boundary is stored within a local dictionary
' to be appended to the class-scope dictionary.
ReDim Preserve oFields(nFieldCount)
nFieldCount = nFieldCount + 1
Set loField = New clsField
lnDispositionPosition = InstrB(lnBoundaryStart, lnBytes, CStrB("Content-Disposition"))
' Get an object name
lnStartPosition = InstrB(lnDispositionPosition, lnBytes, CStrB("name=")) + 6
lnEndPosition = InstrB(lnStartPosition, lnBytes, CStrB(""""))
lsFieldName = CStrU(MidB(lnBytes, lnStartPosition, lnEndPosition - lnStartPosition))
loField.FieldName = lsFieldName
' Get the location fo the file name.
lnFileNamePosition = InstrB(lnBoundaryStart, lnBytes, CStrB("filename="))
lnBoundaryEnd = InstrB(lnEndPosition, lnBytes, lnBoundaryBytes)
'Test if object is a file
If Not lnFileNamePosition = 0 And lnFileNamePosition < lnBoundaryEnd Then
' Parse Filename
lnStartPosition = lnFileNamePosition + 10
lnEndPosition = InstrB(lnStartPosition, lnBytes, CStrB(""""))
lsFileName = CStrU(MidB(lnBytes,lnStartPosition,lnEndPosition-lnStartPosition))
loField.FileName = lsFileName
' Parse Content-Type
lnStartPosition = InstrB(lnEndPosition,lnBytes,CStrB("Content-Type:")) + 14
lnEndPosition = InstrB(lnStartPosition,lnBytes,CStrB(vbCr))
lsContentType = CStrU(MidB(lnBytes,lnStartPosition,lnEndPosition-lnStartPosition))
loField.ContentType = lsContentType
' Parse Content
lnStartPosition = lnEndPosition + 4
lnEndPosition = InstrB(lnStartPosition,lnBytes,lnBoundaryBytes)-2
lsValue = MidB(lnBytes,lnStartPosition,lnEndPosition-lnStartPosition)
loField.BinaryData = lsValue & CStrB(vbNull)
loField.Length = LenB(lsValue)
Else
' Parse Content
lnStartPosition = InstrB(lnDispositionPosition, lnBytes, CStrB(vbCr)) + 4
lnEndPosition = InstrB(lnStartPosition, lnBytes, lnBoundaryBytes) - 2
lsValue = CStrU(MidB(lnBytes,lnStartPosition,lnEndPosition-lnStartPosition))
loField.Value = lsValue
loField.Length = Len(lsValue)
End If
Set oFields(UBound(oFields)) = loField
'Loop to next object
lnBoundaryStart = InstrB(lnBoundaryStart + LenB(lnBoundaryBytes), lnBytes, lnBoundaryBytes)
Set loField = Nothing
Loop
End Sub
' ------------------------------------------------------------------------------
Private Function CStrU(ByRef psByteString)
Dim lnLength
Dim lnPosition
lnLength = LenB(psByteString)
For lnPosition = 1 To lnLength
CStrU = CStrU & Chr(AscB(MidB(psByteString, lnPosition, 1)))
Next
End Function
' ------------------------------------------------------------------------------
Private Function CStrB(ByRef psUnicodeString)
Dim lnLength
Dim lnPosition
lnLength = Len(psUnicodeString)
For lnPosition = 1 To lnLength
CStrB = CStrB & ChrB(AscB(Mid(psUnicodeString, lnPosition, 1)))
Next
End Function
' ------------------------------------------------------------------------------
End Class
' ------------------------------------------------------------------------------
%>
You can see that the file has an extra value added in it and code fails. I have no idea from where this value comes in each time I upload the file. The file in my local machine doesn't have that value. Can anyone help?

Efficiently append .csv files together (VB.NET)

I have a question regarding a piece of coding I would like to make more efficient in Visual Basic. What I am trying to do is the following:
I have a folder containing 100 .csv files (comma delimited), these files have around 5000 lines and around 200 columns. The order of columns may vary from one file to another, and some column are missing in some files.
My goal is to create one big .csv file that combines all the 100 .csv files, with a selection of column that I specify in advance.
Here is how I proceed:
Create an array to store the name of the columns I want in the final “big .csv”
Loop through all the files in the folder. For each file,
For each line in the file, use the Split function, to create an array containing all the values for a given line.
create a mapping array, that store the position of the column in the file for each column name chosen in the first step (do this only for the first line of each file)
Write in a file (“the big .csv”) the header (do this only once)
Write in the same big file, for each line of each file, the data based on the position of the column.
So that process works well, I get the outcome that I want but it is very slow… (It takes ~40min on my computer for ~200 files which once appended contains 500,000 lines and 200 columns. A colleague has managed to do a similar process, appending all the files, using the data.table package in R, and he is able to perform the same appending with the same .csv tables in 5-10min on the same computer)
I was wondering if there is a better alternative than going through the file “cell by cell”? Could I identify the column I don’t want from the source file and delete them entirely? Is there a function to append files together rather than reading each cell and then write them back?
Edit: Alternatively, is there another programming language that is significantly more efficient (Python? Power-Shell?) to do this kind of files manipulation?
Edit2: More details about why I consider it slow.
Edit3: The piece of code relevant to my question as requested in the comments:
Public Module Public_Variables
'Initializr technical parameters
Public Enable_SQL_Upload As String = "Yes"
Public Enable_CSV_Output As String = "Yes"
Public Enable_Runlog As String = "Yes"
Public MPF_Type As String
'Initialize Path and Folder location
Public Path As String '= "L:\Prophet\1902_Analysis\results\RUN_200" ' "S:\Users\Jonathan\12_Project_Space\Input" '"K:\Prophet\1809\Model_B2_LS_Analysis\results\RUN_31"
'Initialize parameters for Actuarial SQL database connection
Public SQLServer As String '= "MELAIPWBAT01"
Public SQLDataBase As String '= "VALDATA"
Public SQLTableName As String '= "RPT_1902_" & Right(Path, 7) '"aMPF_Retail_1808"
Public HeaderScopeFileName As String
Public ValidFilesFileName As String = "S:\Users\Jonathan\12_Project_Space\Tables\ValidFiles.txt"
Public InputFileName As String = "S:\Users\Jonathan\12_Project_Space\Tables\Input.txt"
Public tsrl As String = Now.Year.ToString() + Now.Month.ToString() + Now.Day.ToString() + "_" + Now.Hour.ToString() + "h" + Now.Minute.ToString() + "m" + Now.Second.ToString() + "s"
Public RunLogFileName As String = "\\Melaipwbat01\c$\Users\AACT064\Desktop\SQL_CSV_BULK_INSERT\RunLog" & tsrl & ".csv"
Public RunLogFile As IO.StreamWriter = My.Computer.FileSystem.OpenTextFileWriter(RunLogFileName, False)
Public HeaderFile1 As String '= "K:\Prophet\1903\MPF\MPF_SNAPSHOT\C_TROC.rpt" ' "K:\Prophet\1903\Model_B2_LS_Analysis\results\RUN_200\C_TROC.rpt" '"K:\Prophet\1809\Model_B2_LS_Analysis\results\RUN_31\C_DIO0.rpt"
Public HeaderFile2 As String '= "K:\Prophet\1901\Model_GRP\results\RUN_16\CORS_0.rpt" '"K:\Prophet\1809\Model_B2_LS_Analysis\results\RUN_31\C_DIO0.rpt"
Public ValidFilesFile As New System.IO.StreamReader(ValidFilesFileName)
'UserForm Design
'Public UserFormHeight_Start As Integer = 800
'Public UserFormWidth_Start As Integer = 800
Public UserFormHeight_ProgressBarExtension As Integer = 0
Public UserFormWidth_ProgressBarExtension As Integer = 0
'Initialize Misc.
Public Input_Array(,) As String
Public TextLine As String
Public TextLineSplit() As String
Public ValidFilesArray(,) As String
Public RecordCount As Integer = 0
Public BodyString As String = ""
Public SqlCommandText1 As String = ""
Public SqlCommandText2 As String = ""
'Initialize IS variables
Public Is_RPT_Name As Integer = 1
Public Is_RPT_Type As Integer = 2
Public Is_RPT_Valid As Integer = 3
Public Is_Name As Integer = 1
Public Is_Type As String = 2
Public Is_Not_found As Integer = -1
Public Is_RetailDCS As Integer = 0
Public Is_GroupDCS As Integer = 1
Public Is_MPF_Type As Integer
Public Is_Input_Header As Integer = 1
Public Is_Input_Path As Integer = 2
Public Is_Input_Server As Integer = 3
Public Is_Input_Database As Integer = 4
Public Is_Input_TableName As Integer = 5
Public Is_Input_HeaderFileRetailDCS As Integer = 6
Public Is_Input_HeaderFileGroupDCS As Integer = 7
'Initialize temp variables
Public temp_Valid As Integer
'Initialize the header of the SQL table that is created from that application
Public HeaderScope(,) As String
Public HeaderMapId(,) As String
Public HeaderStringSQL As String = ""
Public HeaderStringCSV As String = ""
Public MappingFound As Boolean
'Initialization for the files looping
Public temp_file As Integer = 1
Public temp_line As Integer
Public temp_headerfile As String
Public FileSize As Integer
Public TimerCounter As Integer = 0
End Module
Public Class UF_UserForm
Private Sub UF_UserForm_Load(sender As Object, e As EventArgs) Handles MyBase.Load
'Me.Height = UserFormHeight_Start
'Me.Width = UserFormWidth_Start
TB_Input.Text = InputFileName
CB_SQLUpload.Text = Enable_SQL_Upload
CB_CSVOutput.Text = Enable_CSV_Output
CB_Runlog.Text = Enable_Runlog
GB_Progress.Visible = False
End Sub
Private Sub B_Run_Click_1(sender As Object, e As EventArgs) Handles B_Run.Click
'Disable the button, switch to 'Progress' tab
B_Run.Enabled = False
TabControl1.SelectedIndex = 1
'Start the Timer
Timer1.Interval = 1000
TimerCounter = 0
Timer1.Start()
'Initilialize the parameters with the Text Box values
TB_Server.Enabled = False
TB_Database.Enabled = False
TB_TableName.Enabled = False
TB_Path.Enabled = False
TB_FileType.Enabled = False
CB_SQLUpload.Enabled = False
CB_CSVOutput.Enabled = False
CB_Runlog.Enabled = False
Enable_SQL_Upload = CB_SQLUpload.Text
Enable_CSV_Output = CB_CSVOutput.Text
Enable_Runlog = CB_Runlog.Text
'Extract the inputs from the input.txt file
Dim temp_Input As Integer
Dim InputFirstCol As String
Dim InputSecCol As String
Dim Nb_Of_Runs As Integer
Dim EndOfLoop As Boolean
InputFileName = TB_Input.Text
Dim InputFile As New System.IO.StreamReader(InputFileName)
temp_Input = 0
EndOfLoop = False
Do While InputFile.Peek() <> -1 And EndOfLoop = False
TextLine = InputFile.ReadLine()
TextLineSplit = TextLine.Split(",")
InputFirstCol = TextLineSplit(0)
If InputFirstCol <> "#" And InputFirstCol <> "" And InputFirstCol <> "--End--" Then
InputSecCol = TextLineSplit(1)
Else
InputSecCol = ""
End If
If InputFirstCol = "--End--" Then
EndOfLoop = True
Else
If InputFirstCol = "#" Then
temp_Input = temp_Input + 1
ElseIf InputFirstCol = "HeaderScope" Then
ReDim Preserve Input_Array(7, temp_Input)
Input_Array(Is_Input_Header, temp_Input) = InputSecCol
ElseIf InputFirstCol = "Path" Then
Input_Array(Is_Input_Path, temp_Input) = InputSecCol
ElseIf InputFirstCol = "Server" Then
Input_Array(Is_Input_Server, temp_Input) = InputSecCol
ElseIf InputFirstCol = "Database" Then
Input_Array(Is_Input_Database, temp_Input) = InputSecCol
ElseIf InputFirstCol = "TableName" Then
Input_Array(Is_Input_TableName, temp_Input) = InputSecCol
ElseIf InputFirstCol = "HeaderFileRetailDCS" Then
Input_Array(Is_Input_HeaderFileRetailDCS, temp_Input) = InputSecCol
ElseIf InputFirstCol = "HeaderFileGroupDCS" Then
Input_Array(Is_Input_HeaderFileGroupDCS, temp_Input) = InputSecCol
End If
End If
Loop
Nb_Of_Runs = temp_Input
'Create an array to store the timer per run
Dim Timer_Array(Nb_Of_Runs) As String
'Let's start the loop for each run
For temp_Run = 1 To Nb_Of_Runs
'Initialize date stamp variables and create the date stamp (called ts)
Dim now As DateTime = DateTime.Now
Dim ts As String = now.Year.ToString() + now.Month.ToString() + now.Day.ToString() + "_" + now.Hour.ToString() + "h" + now.Minute.ToString() + "m" + now.Second.ToString() + "s"
Dim temp_full_count As Integer = 0
Dim File_Count As Integer = 0
Dim temp_count As Integer = 0
'Open the.csv file
Dim OutputCSVFileName As String = "\\Melaipwbat01\c$\Users\AACT064\Desktop\SQL_CSV_BULK_INSERT\SQL_Upload" & ts & ".csv" ' "S:\Users\Jonathan\12_Project_Space\Output\RPT_Files" & ts & ".csv"
Dim SQLUploadFileName As String = Replace(OutputCSVFileName, "\\Melaipwbat01\c$", "C:")
Dim SQLQueriesFileName As String = "\\Melaipwbat01\c$\Users\AACT064\Desktop\SQL_CSV_BULK_INSERT\SQL_Queries" & ts & ".csv"
Dim outFile As IO.StreamWriter = My.Computer.FileSystem.OpenTextFileWriter(OutputCSVFileName, False)
Dim SQLQueriesFile As IO.StreamWriter = My.Computer.FileSystem.OpenTextFileWriter(SQLQueriesFileName, False)
'Store the inputs from the Input_Array
SQLServer = Input_Array(Is_Input_Server, temp_Run)
TB_Server.Text = Input_Array(Is_Input_Server, temp_Run)
SQLDataBase = Input_Array(Is_Input_Database, temp_Run)
TB_Database.Text = Input_Array(Is_Input_Database, temp_Run)
SQLTableName = Input_Array(Is_Input_TableName, temp_Run)
TB_TableName.Text = Input_Array(Is_Input_TableName, temp_Run)
Path = Input_Array(Is_Input_Path, temp_Run)
TB_Path.Text = Input_Array(Is_Input_Path, temp_Run)
HeaderScopeFileName = Input_Array(Is_Input_Header, temp_Run)
TB_FileType.Text = Input_Array(Is_Input_Header, temp_Run)
HeaderFile1 = Input_Array(Is_Input_HeaderFileRetailDCS, temp_Run)
TB_HeaderRetailDCS.Text = Input_Array(Is_Input_HeaderFileRetailDCS, temp_Run)
HeaderFile2 = Input_Array(Is_Input_HeaderFileGroupDCS, temp_Run)
TB_HeaderGroupDCS.Text = Input_Array(Is_Input_HeaderFileGroupDCS, temp_Run)
'Open the folder location and store all the files objetcs into files()
Dim files() As String = IO.Directory.GetFiles(Path)
'Open the Header scope .txt file
Dim HeaderScopeFile As New System.IO.StreamReader(HeaderScopeFileName)
'Initialize the variable ValidFilesArray
temp_Valid = 1
Do While ValidFilesFile.Peek() <> -1
TextLine = ValidFilesFile.ReadLine()
TextLineSplit = TextLine.Split(", ")
ReDim Preserve ValidFilesArray(3, temp_Valid)
ValidFilesArray(Is_RPT_Name, temp_Valid) = TextLineSplit(Is_RPT_Name - 1)
ValidFilesArray(Is_RPT_Type, temp_Valid) = TextLineSplit(Is_RPT_Type - 1)
ValidFilesArray(Is_RPT_Valid, temp_Valid) = TextLineSplit(Is_RPT_Valid - 1)
temp_Valid = temp_Valid + 1
Loop
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'' Display the Progress Group Box ''''''''''''''''''''''''''''''''''''''''''''''''
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
L_ProgressPC.Text = "Initialisation"
ProgressBar.Value = 0
GB_Progress.Visible = True
'Me.Height = UserFormHeight_Start + UserFormHeight_ProgressBarExtension
'Me.Width = UserFormWidth_Start + UserFormWidth_ProgressBarExtension
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'' Nb of Files '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' The goal of this piece of code aims at calculating the number of .rpt files
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
TB_Runlog.Text = "Checking number of .rpt files..." & Environment.NewLine & TB_Runlog.Text
For Each file As String In files
If CheckValidRPTFile(file, False) = True Then
File_Count = File_Count + 1
L_NbOfFiles.Text = File_Count
L_NbOfRuns.Text = temp_Run & "/" & Nb_Of_Runs
End If
Application.DoEvents()
Next
TB_Runlog.Text = "... " & " Run number " & temp_Run & Environment.NewLine & TB_Runlog.Text
TB_Runlog.Text = "... " & File_Count & " rpt files founds" & Environment.NewLine & TB_Runlog.Text
TB_Runlog.Text = "---------------------------------------" & Environment.NewLine & TB_Runlog.Text
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' The key is to define the header with the field names and the field types
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Dim HeaderNbOfField As Integer = 0
Do While HeaderScopeFile.Peek() <> -1
HeaderNbOfField = HeaderNbOfField + 1
'We split the libe into an array using the comma delimiter
TextLine = HeaderScopeFile.ReadLine()
TextLineSplit = TextLine.Split(", ")
ReDim Preserve HeaderScope(2, HeaderNbOfField)
HeaderScope(Is_Name, HeaderNbOfField) = TextLineSplit(0)
HeaderScope(Is_Type, HeaderNbOfField) = TextLineSplit(1)
Loop
'That array stores the position of a given field in the file
'It is important to initialise the array to -1
'When further down we assign HeaderMapId, if a value remains "-1" il will mean the field was not assigned thus not found.
'We will then assign a default value for those not found fields
ReDim HeaderMapId(1, HeaderNbOfField)
For temp_headermapini = 0 To HeaderNbOfField
HeaderMapId(Is_RetailDCS, temp_headermapini) = Is_Not_found
HeaderMapId(Is_GroupDCS, temp_headermapini) = Is_Not_found
Next
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'' Header ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' The goal of this piece of code is to populate the variable HeaderMapId
' HeaderMapId stores the position of each field define is HeaderScope variable
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
L_ProgressPC.Text = "Find position of each field in the flat file"
'We use HeaderFile as the base to define the position of each field
'The application will function properly only if every .rpt file in the folder have same header as HeaderFile
For temp_header = 0 To 1
'We loop through 2 different type of MPFs
'Typicaly coming from Retail DCS and Group DCS
If temp_header = Is_RetailDCS Then
temp_headerfile = HeaderFile1
ElseIf temp_header = Is_GroupDCS Then
temp_headerfile = HeaderFile2
Else
temp_headerfile = "" 'Error
End If
Dim objReaderHeader As New System.IO.StreamReader(temp_headerfile)
'We read line by line until the end of the file
Do While objReaderHeader.Peek() <> -1
TextLine = objReaderHeader.ReadLine()
'We only care about the header, which starts with the character "!" in prophet .rpt files
If Strings.Left(TextLine, 1) = "!" Then
Dim temp_scope As Integer
Dim temp_scope_id As Integer
'We split the line in an array delimited by comma
TextLineSplit = TextLine.Split(", ")
'We loop through the array
'Once a field match one of the field define in HeaderScope, we store the position of that field in HeaderMapId
temp_scope_id = 1
For Each s As String In TextLineSplit
For temp_scope = 1 To UBound(HeaderScope, 2)
If HeaderScope(Is_Name, temp_scope) = s Then
HeaderMapId(temp_header, temp_scope) = temp_scope_id
End If
Next
temp_scope_id = temp_scope_id + 1
Next
End If
Loop
Next
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'' Header''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'Create the query for SQL table creation
Dim temp_field As Integer
For temp_field = 1 To HeaderNbOfField
If temp_field = 1 Then
HeaderStringSQL = "(Prophet_Name varchar(255),"
HeaderStringCSV = "Prophet_Name,"
End If
'We replace the bracket by underscore to avoid crashes when creating the SQL table
HeaderStringSQL = HeaderStringSQL & Replace(Replace(HeaderScope(Is_Name, temp_field), "(", "_"), ")", "_") & " " & HeaderScope(Is_Type, temp_field)
HeaderStringCSV = HeaderStringCSV & Replace(Replace(HeaderScope(Is_Name, temp_field), "(", "_"), ")", "_")
If temp_field <> HeaderNbOfField Then
HeaderStringSQL = HeaderStringSQL & ","
HeaderStringCSV = HeaderStringCSV & ","
Else
HeaderStringSQL = HeaderStringSQL & ")"
HeaderStringCSV = HeaderStringCSV & ","
End If
Next
If Enable_CSV_Output = "Yes" Then
'Remove braquets and single quotes
outFile.WriteLine(Replace(Replace(Replace(HeaderStringCSV, ")", ""), "(", ""), "'", ""))
End If
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'' Body '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' We loop through all the files and pick the information we need based on HeaderMapId
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'We loop through each file in the folder location
For Each file As String In files
Dim objReader As New System.IO.StreamReader(file)
Dim fileInfo As New IO.FileInfo(file)
'We only loop through the valid .rpt files
If CheckValidRPTFile(file, True) = True Then
'Count the number of files we go through
temp_count = temp_count + 1
'Count the number of lines in the file (called FileSize)
'Dim objReaderLineCOunt As New System.IO.StreamReader(file)
'FileSize = 0
'Do While objReaderLineCOunt.Peek() <> -1
'TextLine = objReaderLineCOunt.ReadLine()
'FileSize = FileSize + 1
'Loop
FileSize = 1000000
'temp_line = 1
''We loop through line by line for a given file
'Do While objReader.Peek() <> -1
Dim TextLines() As String = System.IO.File.ReadAllLines(file)
For Each TextLine2 In TextLines
'Update the Progress Bar
temp_full_count = temp_full_count + 1
If temp_full_count Mod 200 = 0 Then
ProgressBar.Value = Int(100 * temp_count / File_Count)
L_ProgressPC.Text = "Processing file " & fileInfo.Name & " " & temp_count & "/" & File_Count & " - line " & temp_line & "/" & FileSize
L_RecordsProcessed.Text = temp_full_count
Application.DoEvents()
End If
'We split the libe into an array using the comma delimiter
'TextLine = objReader.ReadLine()
TextLineSplit = TextLine2.Split(", ")
'Skip line that are not actual prophet records (skip header and first few lines)
If Strings.Left(TextLine2, 1) = "*" Then
'We loop through the number of field we wish to extract for the file
For temp_field = 1 To HeaderNbOfField
If temp_field = 1 Then
BodyString = "('" & Strings.Left(fileInfo.Name, Len(fileInfo.Name) - 4) & "',"
End If
If MPF_Type = "RetailDCS" Then
Is_MPF_Type = Is_RetailDCS
ElseIf MPF_Type = "GroupDCS" Then
Is_MPF_Type = Is_GroupDCS
Else
MsgBox("Is_MPF_Type value is nor recognized")
End
End If
'The array HeaderMapId tells us where to pick the information from the file
'This assumes that each file in the folder have same header as the 'HeaderFile'
If HeaderMapId(Is_MPF_Type, temp_field) = Is_Not_found Then
BodyString = BodyString & "98766789"
Else
BodyString = BodyString & TextLineSplit(HeaderMapId(Is_MPF_Type, temp_field) - 1)
End If
If temp_field <> HeaderNbOfField Then
BodyString = BodyString & ","
Else
BodyString = BodyString & ")"
End If
Next
'We replace double quotes with single quotes
BodyString = Replace(BodyString, """", "'")
'This Line is to add records to the .csv file
If Enable_CSV_Output = "Yes" Then
'Remove braquets and single quotes
outFile.WriteLine(Replace(Replace(Replace(BodyString, ")", ""), "(", ""), "'", ""))
End If
End If
temp_line = temp_line + 1
Next
TB_Runlog.Text = "Completed: " & fileInfo.Name & Environment.NewLine & TB_Runlog.Text
temp_file = temp_file + 1
End If
Next
outFile.Close()
ProgressBar.Value = Int(100 * temp_count / File_Count)
L_RecordsProcessed.Text = temp_full_count
Application.DoEvents()
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'' Upload to SQL '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' The goal of this code is to create the SQL table
' And push the .csv created into the SQL table using BULK INSERT function
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
L_ProgressPC.Text = "Creating SQL Table"
Application.DoEvents()
If Enable_SQL_Upload = "Yes" Then
'First query is to create the table
SqlCommandText1 = "CREATE TABLE [" & SQLDataBase & "].[analysis]." & SQLTableName & "_" & ts & " " & HeaderStringSQL
'Second query is to populate the data
SqlCommandText2 = "BULK INSERT [" & SQLDataBase & "].[analysis]." & SQLTableName & "_" & ts & " " & " FROM '" & SQLUploadFileName & "' WITH ( FIELDTERMINATOR = ',', ROWTERMINATOR = '\n' , FIRSTROW=2)"
SQLQueriesFile.WriteLine(SqlCommandText1)
SQLQueriesFile.WriteLine(SqlCommandText2)
SQLQueriesFile.Close()
Using connection As New SqlConnection("Data Source=" & SQLServer & ";Integrated Security=True;Connection Timeout=2000;Initial Catalog=" & SQLDataBase & ";")
connection.Open()
Dim command As New SqlCommand(SqlCommandText1, connection)
command.ExecuteNonQuery()
Dim command2 = New SqlCommand(SqlCommandText2, connection)
command2.CommandTimeout = 2000
command2.ExecuteNonQuery()
connection.Close()
End Using
End If
Timer_Array(temp_Run) = L_Timer.Text
Next 'temp_Run
RunLogFile.Close()
Timer1.Stop()
L_ProgressPC.Text = "Job completed"
One way to speed up your program is to mininize the number of access to disk. Right now, you are reading each file twice, line-by-line. Each file most likely fits in memory. So, what you can do, is read all lines of a file in memory, and then process its lines. This will be much faster.
Something like:
'We only loop through the valid .rpt files
If CheckValidRPTFile(file, True) = True Then
''Count the number of files we go through
'temp_count = temp_count + 1
''Count the number of lines in the file (called FileSize)
'Dim objReaderLineCOunt As New System.IO.StreamReader(file)
'FileSize = 0
'Do While objReaderLineCOunt.Peek() <> -1
' TextLine = objReaderLineCOunt.ReadLine()
' FileSize = FileSize + 1
'Loop
'temp_line = 1
''We loop through line by line for a given file
'Do While objReader.Peek() <> -1
Dim TextLines() As String = System.IO.File.ReadAllLines(file)
For Each TextLine In TextLines
'We split into an array using the comma delimiter
'TextLine = objReader.ReadLine()
TextLineSplit = TextLine.Split(", ")
I looked at your updated code. You have a few additional performance issues in your code.
Reading and writing to network shared folder files is not very efficient especially when there is a lot of back and forth access to the file because everything goes through the network rather than direct local drive access.
Probably the most inefficient part of your program is iterating through your 200 fields for each line of the files. Let's say we have 5000 lines per file on average, then this means 200 x 5000 x 284 = 284 millions iterations!
The efficient method for accessing network shared files is to read an entire file in memory using System.IO.ReadAllLines() or System.IO.File.ReadAllText(), and then process its contents. Similarly, writing to a network shared file should consist of building the file contents in memory (if possible) using StringBuilder or a List(Of String), and then write the entire file to the network share with System.IO.File.WriteAllText() of System.IO.File.WriteAllLines. This should be the preferred way of accessing network shared files for best performance.
For the second performance issue, the loop through fields can be simplified as follows.
Dim BodyStringBuilder As New StringBuilder("")
BodyStringBuilder.Append("('" & Strings.Left(fileInfo.Name, Len(fileInfo.Name) - 4) & "',")
If MPF_Type = "RetailDCS" Then
Is_MPF_Type = Is_RetailDCS
ElseIf MPF_Type = "GroupDCS" Then
Is_MPF_Type = Is_GroupDCS
Else
MsgBox("Is_MPF_Type value is nor recognized")
End
End If
'Skip line that are not actual prophet records (skip header and first few lines)
If Strings.Left(TextLine2, 1) = "*" Then
'We loop through the number of field we wish to extract for the file
For temp_field = 1 To HeaderNbOfField
'The array HeaderMapId tells us where to pick the information from the file
'This assumes that each file in the folder have same header as the 'HeaderFile'
If HeaderMapId(Is_MPF_Type, temp_field) = Is_Not_found Then
BodyStringBuilder.Append("98766789,")
Else
BodyStringBuilder.Append(TextLineSplit(HeaderMapId(Is_MPF_Type, temp_field) - 1) & ",")
End If
Next
' Replace last "," by ")".
BodyStringBuilder.Remove(BodyStringBuilder.Length - 1, 1).Append(")")
'We replace double quotes with single quotes
BodyString = Replace(BodyString.ToString, """", "'")

Read Line-By-Line without using StreamReader

I have been doing A-Level Computing for a couple of months now and I have decided to make a phonebook relevant to the methods used in class. For my exam I am not allowed to use StreamReader etc... However, I can use methods such as FileOpen(1, "FILE NAME", OpenMode.Binary).
I want to store usernames and passwords in a text file (I know its not a good method but it links to my lessons). My .txt file is formatted like:
Username, Password.
Username2, Password2.
What would i be able to do to make it store each Username and Password in a list?
Here's what I've got... (don't know why it doesn't work)
Private Sub CheckAccount()
Dim Username_File As New ArrayList
Dim Username_Temp As String = ""
Dim Password_File As New ArrayList
Dim Password_Temp As String = ""
FileOpen(1, "Users.txt", OpenMode.Binary)
Do While Not EOF(1)
Dim PosCount_Start As Integer = 0
Dim PosCount_End As Integer = 0
Dim TempChar As Char = ""
Do
FileGet(1, TempChar)
If TempChar = "," Then Exit Do
Username_Temp = Username_Temp + TempChar
PosCount_End = PosCount_End + 1
Loop
Username_Temp = Username_Temp.Substring(PosCount_Start, (PosCount_End - PosCount_Start))
Username_File.Add(Username_Temp)
PosCount_End = 0
Do
FileGet(1, TempChar)
If TempChar = "." Then Exit Do
Password_Temp = Password_Temp + TempChar
PosCount_End = PosCount_End + 1
Loop
Password_Temp = Password_Temp.Substring(0, PosCount_End)
Password_Temp = Password_Temp.Trim
Password_File.Add(Password_Temp)
PosCount_End = 0
Loop
FileClose(1)
For i = 0 To Username_File.Count - 1
Dim Temp As String
Temp = Username_File(i)
If Temp.ToLower = Username.ToLower Then
If Password = Password_File(i) Then
LoggedIn = True
End If
End If
Next
End Sub

Base64 Encode .JPG from request in classic ASP/VBScript

I've been stuck on this one for a few days with no progress. Using classic ASP, I need to take an uploaded .JPG file (from html form input type='file') and base64 encode it so that I can send it to a Java web service. The Java web service is simply storing the image (in a SQL image field) in a database. I figured converting to BASE64 was the best way to transfer the parameter in the xml. Here's what I have so far:
HTML:
<label>Upload Picture</label>
<input name="file" id="file" type="file" size=40 /> <br />
ASP:
Dim load
Set load = new Loader
load.initialize
Dim fileData
fileData = load.getFileData("file")
Dim fileName
fileName = LCase(load.getFileName("file"))
Dim fileSize
fileSize = load.getFileSize("file")
Dim objXML
Dim objNode
Dim strB64
Set objXML = Server.CreateObject("MSXML2.DomDocument.3.0")
Set objNode = objXML.createElement("base64")
objNode.dataType = "bin.base64" 'stores binary as base64 string
objNode.nodeTypedValue = fileData 'binary value
strB64 = objNode.Text
LOADER:
the loader is a lot of code (I can copy all of it here if needed). It essentially gets all the bytes from the request and parses them into a dictionary object. Here's how it gets the data:
Class Loader
Private dict
Private Sub Class_Initialize
Set dict = Server.CreateObject("Scripting.Dictionary")
End Sub
Public Sub Initialize
If Request.TotalBytes > 0 Then
Dim binData
binData = Request.BinaryRead(Request.TotalBytes)
getData binData
End If
End Sub
Public Function getFileData(name) '"file"
If dict.Exists(name) Then
getFileData = dict(name).Item("Value")
Else
getFileData = ""
End If
End Function
I can post the sub that parses and stores the binary in a dictionary if needed.
The above code gives this error:
msxml3.dll error '80004005'
Error parsing '????' as bin.base64 datatype.
on this line:
objNode.nodeTypedValue = fileData 'binary value
UPDATE:
Here's where the data is loaded to the dictionary:
Private Sub getData(rawData)
Dim separator
separator = MidB(rawData, 1, InstrB(1, rawData, ChrB(13)) - 1)
Dim lenSeparator
lenSeparator = LenB(separator)
Dim currentPos
currentPos = 1
Dim inStrByte
inStrByte = 1
Dim value, mValue
Dim tempValue
tempValue = ""
While inStrByte > 0
inStrByte = InStrB(currentPos, rawData, separator)
mValue = inStrByte - currentPos
If mValue > 1 Then
value = MidB(rawData, currentPos, mValue)
Dim begPos, endPos, midValue, nValue
Dim intDict
Set intDict = Server.CreateObject("Scripting.Dictionary")
begPos = 1 + InStrB(1, value, ChrB(34))
endPos = InStrB(begPos + 1, value, ChrB(34))
nValue = endPos
Dim nameN
nameN = MidB(value, begPos, endPos - begPos)
Dim nameValue, isValid
isValid = True
If InStrB(1, value, stringToByte("Content-Type")) > 1 Then
begPos = 1 + InStrB(endPos + 1, value, ChrB(34))
endPos = InStrB(begPos + 1, value, ChrB(34))
If endPos = 0 Then
endPos = begPos + 1
isValid = False
End If
midValue = MidB(value, begPos, endPos - begPos)
intDict.Add "FileName", trim(byteToString(midValue))
begPos = 14 + InStrB(endPos + 1, value, stringToByte("Content-Type:"))
endPos = InStrB(begPos, value, ChrB(13))
midValue = MidB(value, begPos, endPos - begPos)
intDict.Add "ContentType", trim(byteToString(midValue))
begPos = endPos + 4
endPos = LenB(value)
nameValue = MidB(value, begPos, ((endPos - begPos) - 1))
Else
nameValue = trim(byteToString(MidB(value, nValue + 5)))
End If
If isValid = True Then
intDict.Add "Value", nameValue
intDict.Add "Name", nameN
dict.Add byteToString(nameN), intDict
End If
End If
currentPos = lenSeparator + inStrByte
Wend
End Sub
Private Function stringToByte(toConv)
Dim tempChar, i
For i = 1 to Len(toConv)
tempChar = Mid(toConv, i, 1)
stringToByte = stringToByte & chrB(AscB(tempChar))
Next
End Function
Private Function byteToString(toConv)
dim i
For i = 1 to LenB(toConv)
byteToString = byteToString & chr(AscB(MidB(toConv,i,1)))
Next
End Function
I finally found something that works! Here's the link:
http://www.motobit.com/tips/detpg_binarytostring/
The issue I was having was that the data from the BinaryRead was multibyte data (which ASP doesn't play well with). MultiByte data must be converted To VT_UI1 | VT_ARRAY in order for the base64 encoding via the Dom Document Object (or nearly any other function - including loading to a stream). This can be achieved using an ADO recordset object as such:
Function MultiByteToBinary(MultiByte)
' 2000 Antonin Foller, http://www.motobit.com
' MultiByteToBinary converts multibyte string To real binary data (VT_UI1 | VT_ARRAY)
' Using recordset
Dim RS, LMultiByte, Binary
Const adLongVarBinary = 205
Set RS = CreateObject("ADODB.Recordset")
LMultiByte = LenB(MultiByte)
If LMultiByte>0 Then
RS.Fields.Append "mBinary", adLongVarBinary, LMultiByte
RS.Open
RS.AddNew
RS("mBinary").AppendChunk MultiByte & ChrB(0)
RS.Update
Binary = RS("mBinary").GetChunk(LMultiByte)
End If
MultiByteToBinary = Binary
End Function
Hopefully this will help someone else as well.
Let ADODB.Stream do your heavy lifting, this assumes you already have saved the file to disk. These are the functions for BASE64 I use, I'm not 100% sure where I originally got them.
private function readBytes(file)
dim inStream
' ADODB stream object used
set inStream = WScript.CreateObject("ADODB.Stream")
' open with no arguments makes the stream an empty container
inStream.Open
inStream.type= TypeBinary
inStream.LoadFromFile(file)
readBytes = inStream.Read()
end function
private function encodeBase64(bytes)
dim DM, EL
Set DM = CreateObject("Microsoft.XMLDOM")
' Create temporary node with Base64 data type
Set EL = DM.createElement("tmp")
EL.DataType = "bin.base64"
' Set bytes, get encoded String
EL.NodeTypedValue = bytes
encodeBase64 = EL.Text
end function
private function decodeBase64(base64)
dim DM, EL
Set DM = CreateObject("Microsoft.XMLDOM")
' Create temporary node with Base64 data type
Set EL = DM.createElement("tmp")
EL.DataType = "bin.base64"
' Set encoded String, get bytes
EL.Text = base64
decodeBase64 = EL.NodeTypedValue
end function
private Sub writeBytes(file, bytes)
Dim binaryStream
Set binaryStream = CreateObject("ADODB.Stream")
binaryStream.Type = adTypeBinary
'Open the stream and write binary data
binaryStream.Open
binaryStream.Write bytes
'Save binary data to disk
binaryStream.SaveToFile file, adSaveCreateOverWrite
End Sub

Updating the Northwind Refresh Table Links in Access

There is this database program that I am working on. For some reason the boss purchased all 64bit 2010 Office Suites so I am updating the program to work on the 64bit Office.
In this section I have a problem with trying to figure out the way to make this work on 64bit Access. I can't seem to get a straight answer about msaof, nor can I find any work that has the updated code. Its part of the Northwind Refresh Table Link which can be found on the internet but code only works in 32bit.
Private Sub MSAOF_to_OF(msaof As MSA_OPENFILENAME, of As OPENFILENAME)
' This sub converts from the friendly MSAccess structure to the win32 structure.
Dim strFile As String * 512
' Initialize some parts of the structure.
of.hwndOwner = Application.hWndAccessApp
of.hInstance = 0
of.lpstrCustomFilter = 0
of.nMaxCustrFilter = 0
of.lpfnHook = 0
of.lpTemplateName = 0
of.lCustrData = 0
If msaof.strFilter = "" Then
of.lpstrFilter = MSA_CreateFilterString(ALLFILES)
Else
of.lpstrFilter = msaof.strFilter
End If
of.nFilterIndex = msaof.lngFilterIndex
of.lpstrFile = msaof.strInitialFile & String$(512 - Len(msaof.strInitialFile), 0)
of.nMaxFile = 511
of.lpstrFileTitle = String$(512, 0)
of.nMaxFileTitle = 511
of.lpstrTitle = msaof.strDialogTitle
of.lpstrInitialDir = msaof.strInitialDir
of.lpstrDefExt = msaof.strDefaultExtension
of.flags = msaof.lngFlags
of.lStructSize = Len(of)
End Sub
One of thing is that I get the error "of.nMaxCustrFilter = 0" does not exist but when I comment it out the debugger still points to it and Highlights the entire first line.
Update:This is the entire code
Option Explicit ' Require variables to be declared before being used.
Option Compare Database ' Use database order for string comparisons.
Private Declare PtrSafe Function GetOpenFileName Lib "comdlg32.dll" Alias "GetOpenFileNameA" (pOpenfilename As OPENFILENAME) As Boolean
Declare PtrSafe Function GetSaveFileName Lib "comdlg32.dll" Alias "GetSaveFileNameA" (pOpenfilename As OPENFILENAME) As Boolean
Type MSA_OPENFILENAME
' Filter string used for the File Open dialog filters.
' Use MSA_CreateFilterString() to create this.
' Default = All Files, *.*
strFilter As String
' Initial Filter to display.
' Default = 1.
lngFilterIndex As Long
' Initial directory for the dialog to open in.
' Default = Current working directory.
strInitialDir As String
' Initial file name to populate the dialog with.
' Default = "".
strInitialFile As String
strDialogTitle As String
' Default extension to append to file if user didn't specify one.
' Default = System Values (Open File, Save File).
strDefaultExtension As String
' Flags (see constant list) to be used.
' Default = no flags.
lngFlags As Long
' Full path of file picked. On OpenFile, if the user picks a
' nonexistent file, only the text in the "File Name" box is returned.
strFullPathReturned As String
' File name of file picked.
strFileNameReturned As String
' Offset in full path (strFullPathReturned) where the file name
' (strFileNameReturned) begins.
intFileOffset As Integer
' Offset in full path (strFullPathReturned) where the file extension begins.
intFileExtension As Integer
End Type
Const ALLFILES = "All Files"
Type OPENFILENAME
lStructSize As Long
hwndOwner As LongPtr
hInstance As LongPtr
lpstrFilter As String
lpstrCustomFilter As String
nMaxCustFilter As Long
nFilterIndex As Long
lpstrFile As String
nMaxFile As Long
lpstrFileTitle As String
nMaxFileTitle As Long
lpstrInitialDir As String
lpstrTitle As String
flags As Long
nFileOffset As Integer
nFileExtension As Integer
lpstrDefExt As String
lCustData As Long
lpfnHook As LongPtr
lpTemplateName As String
End Type
Const OFN_ALLOWMULTISELECT = &H200
Const OFN_CREATEPROMPT = &H2000
Const OFN_EXPLORER = &H80000
Const OFN_FILEMUSTEXIST = &H1000
Const OFN_HIDEREADONLY = &H4
Const OFN_NOCHANGEDIR = &H8
Const OFN_NODEREFERENCELINKS = &H100000
Const OFN_NONETWORKBUTTON = &H20000
Const OFN_NOREADONLYRETURN = &H8000
Const OFN_NOVALIDATE = &H100
Const OFN_OVERWRITEPROMPT = &H2
Const OFN_PATHMUSTEXIST = &H800
Const OFN_READONLY = &H1
Const OFN_SHOWHELP = &H10
Function FindNorthwind(strSearchPath) As String
' Displays the open file dialog box for the user to locate
' the ElectricData database. Returns the full path to ElectricData.
Dim msaof As MSA_OPENFILENAME
' Set options for the dialog box.
msaof.strDialogTitle = "Where Is ElectricData.accdb?"
msaof.strInitialDir = strSearchPath
msaof.strFilter = MSA_CreateFilterString("Databases", "**.accdb")
' Call the Open File dialog routine.
MSA_GetOpenFileName msaof
' Return the path and file name.
FindNorthwind = Trim(msaof.strFullPathReturned)
End Function
Function MSA_CreateFilterString(ParamArray varFilt() As Variant) As String
' Creates a filter string from the passed in arguments.
' Returns "" if no args are passed in.
' Expects an even number of args (filter name, extension), but
' if an odd number is passed in, it appends *.*
Dim strFilter As String
Dim intRet As Integer
Dim intNum As Integer
intNum = UBound(varFilt)
If (intNum <> -1) Then
For intRet = 0 To intNum
strFilter = strFilter & varFilt(intRet) & vbNullChar
Next
If intNum Mod 2 = 0 Then
strFilter = strFilter & "*.*" & vbNullChar
End If
strFilter = strFilter & vbNullChar
Else
strFilter = ""
End If
MSA_CreateFilterString = strFilter
End Function
Function MSA_ConvertFilterString(strFilterIn As String) As String
' Creates a filter string from a bar ("|") separated string.
' The string should pairs of filter|extension strings, i.e. "Access Databases|**.accdb|All Files|*.*"
' If no extensions exists for the last filter pair, *.* is added.
' This code will ignore any empty strings, i.e. "||" pairs.
' Returns "" if the strings passed in is empty.
Dim strFilter As String
Dim intNum As Integer, intPos As Integer, intLastPos As Integer
strFilter = ""
intNum = 0
intPos = 1
intLastPos = 1
' Add strings as long as we find bars.
' Ignore any empty strings (not allowed).
Do
intPos = InStr(intLastPos, strFilterIn, "|")
If (intPos > intLastPos) Then
strFilter = strFilter & Mid$(strFilterIn, intLastPos, intPos - intLastPos) & vbNullChar
intNum = intNum + 1
intLastPos = intPos + 1
ElseIf (intPos = intLastPos) Then
intLastPos = intPos + 1
End If
Loop Until (intPos = 0)
' Get last string if it exists (assuming strFilterIn was not bar terminated).
intPos = Len(strFilterIn)
If (intPos >= intLastPos) Then
strFilter = strFilter & Mid$(strFilterIn, intLastPos, intPos - intLastPos + 1) & vbNullChar
intNum = intNum + 1
End If
' Add *.* if there's no extension for the last string.
If intNum Mod 2 = 1 Then
strFilter = strFilter & "*.*" & vbNullChar
End If
' Add terminating NULL if we have any filter.
If strFilter <> "" Then
strFilter = strFilter & vbNullChar
End If
MSA_ConvertFilterString = strFilter
End Function
Private Function MSA_GetSaveFileName(msaof As MSA_OPENFILENAME) As Integer
' Opens the file save dialog.
Dim of As OPENFILENAME
Dim intRet As Integer
MSAOF_to_OF msaof, of
of.flags = of.flags Or OFN_HIDEREADONLY
intRet = GetSaveFileName(of)
If intRet Then
OF_to_MSAOF of, msaof
End If
MSA_GetSaveFileName = intRet
End Function
Function MSA_SimpleGetSaveFileName() As String
' Opens the file save dialog with default values.
Dim msaof As MSA_OPENFILENAME
Dim intRet As Integer
Dim strRet As String
intRet = MSA_GetSaveFileName(msaof)
If intRet Then
strRet = msaof.strFullPathReturned
End If
MSA_SimpleGetSaveFileName = strRet
End Function
Private Function MSA_GetOpenFileName(msaof As MSA_OPENFILENAME) As Integer
' Opens the file open dialog.
Dim of As OPENFILENAME
Dim intRet As Integer
MSAOF_to_OF msaof, of
intRet = GetOpenFileName(of)
If intRet Then
OF_to_MSAOF of, msaof
End If
MSA_GetOpenFileName = intRet
End Function
Function MSA_SimpleGetOpenFileName() As String
' Opens the file open dialog with default values.
Dim msaof As MSA_OPENFILENAME
Dim intRet As Integer
Dim strRet As String
intRet = MSA_GetOpenFileName(msaof)
If intRet Then
strRet = msaof.strFullPathReturned
End If
MSA_SimpleGetOpenFileName = strRet
End Function
Public Function CheckLinks() As Boolean
' Check links to the ElectricData database; returns true if links are OK.
Dim dbs As Database, rst As DAO.Recordset
Set dbs = CurrentDb()
' Open linked table to see if connection information is correct.
On Error Resume Next
Set rst = dbs.OpenRecordset("lstPartClasses")
' If there's no error, return True.
If Err = 0 Then
CheckLinks = True
Else
CheckLinks = False
End If
End Function
Private Sub OF_to_MSAOF(of As OPENFILENAME, msaof As MSA_OPENFILENAME)
' This sub converts from the win32 structure to the friendly MSAccess structure.
msaof.strFullPathReturned = Left$(of.lpstrFile, InStr(of.lpstrFile, vbNullChar) - 1)
msaof.strFileNameReturned = of.lpstrFileTitle
msaof.intFileOffset = of.nFileOffset
msaof.intFileExtension = of.nFileExtension
End Sub
Private Sub MSAOF_to_OF(msaof As MSA_OPENFILENAME, of As OPENFILENAME)
' This sub converts from the friendly MSAccess structure to the win32 structure.
Dim strFile As String * 512
' Initialize some parts of the structure.
of.hwndOwner = Application.hWndAccessApp
of.hInstance = 0
of.lpstrCustomFilter = 0
of.nMaxCustrFilter = 0
of.lpfnHook = 0
of.lpTemplateName = 0
of.lCustrData = 0
If msaof.strFilter = "" Then
of.lpstrFilter = MSA_CreateFilterString(ALLFILES)
Else
of.lpstrFilter = msaof.strFilter
End If
of.nFilterIndex = msaof.lngFilterIndex
of.lpstrFile = msaof.strInitialFile & String$(512 - Len(msaof.strInitialFile), 0)
of.nMaxFile = 511
of.lpstrFileTitle = String$(512, 0)
of.nMaxFileTitle = 511
of.lpstrTitle = msaof.strDialogTitle
of.lpstrInitialDir = msaof.strInitialDir
of.lpstrDefExt = msaof.strDefaultExtension
of.flags = msaof.lngFlags
of.lStructSize = Len(of)
End Sub
Private Function RefreshLinks(strFilename As String) As Boolean
' Refresh links to the supplied database. Return True if successful.
Dim dbs As Database
Dim intCount As Integer
Dim tdf As TableDef
' Loop through all tables in the database.
Set dbs = CurrentDb
For intCount = 0 To dbs.TableDefs.Count - 1
Set tdf = dbs.TableDefs(intCount)
' If the table has a connect string, it's a linked table.
If Len(tdf.Connect) > 0 Then
tdf.Connect = ";DATABASE=" & strFilename
' Debug.Print tdf.Connect
' Debug.Print tdf.SourceTableName
Err = 0
On Error Resume Next
tdf.RefreshLink ' Relink the table.
If Err <> 0 Then
RefreshLinks = False
Exit Function
End If
End If
Next intCount
RefreshLinks = True ' Relinking complete.
End Function
Public Function RelinkTables() As Boolean
' Tries to refresh the links to the American Campus IT Department database.
' Returns True if successful.
Const conMaxTables = 8
Const conNonExistentTable = 3011
Const conNotNorthwind = 3078
Const conNwindNotFound = 3024
Const conAccessDenied = 3051
Const conReadOnlyDatabase = 3027
Const conAppTitle = "Calvin's Electric - Bid/Job Program"
Dim strAccDir As String
Dim strSearchPath As String
Dim strFilename As String
Dim intError As Integer
Dim strError As String
' Get name of directory where Msaccess.exe is located.
strAccDir = SysCmd(acSysCmdAccessDir)
' Get the default sample database path.
If Dir(strAccDir & "\.") = "" Then
strSearchPath = strAccDir
Else
strSearchPath = strAccDir & "\"
End If
' Look for the ElectricData database.
If (Dir(strSearchPath & "ElectricData.accdb") <> "") Then
strFilename = strSearchPath & "ElectricData.accdb"
Else
' Can't find ElectricData, so display the File Open dialog.
MsgBox "Can't find linked tables in the Calvin's Electric Bid And Job Program. You must locate the ElectricData Database in order to use " _
& conAppTitle & ".", vbExclamation
strFilename = FindNorthwind(strSearchPath)
If strFilename = "" Then
strError = "Sorry, you must locate ElectricData.accdb to open " & conAppTitle & "."
GoTo Exit_Failed
End If
End If
' Fix the links.
If RefreshLinks(strFilename) Then ' It worked!
RelinkTables = True
Exit Function
End If
' If it failed, display an error.
Select Case Err
Case conNonExistentTable, conNotNorthwind
strError = "File '" & strFilename & "' does not contain the required ElectricData tables."
Case Err = conNwindNotFound
strError = "You can't run " & conAppTitle & " until you locate the ElectricData database."
Case Err = conAccessDenied
strError = "Couldn't open " & strFilename & " because it is read-only or located on a read-only share."
Case Err = conReadOnlyDatabase
strError = "Can't reattach tables because " & conAppTitle & " is read-only or is located on a read-only share."
Case Else
strError = Err.Description
End Select
Exit_Failed:
MsgBox strError, vbCritical
RelinkTables = False
End Function
As an alternative to messing around with the 32/64-bit API declarations, you could just use the Application.FileDialog method that is available in Access 2010. It works with both the 32-bit and 64-bit versions of Access.
It seems likely that you have Declare Function somewhere that need to read Declare PtrSafe Function. Then you will have to make sure that you have a 64 bit library for the DLL you are calling. It seems (not well tested) to work fine in my 64 bit application using the code here http://www.dbforums.com/microsoft-access/990945-building-database-help.html.