I am attempting to load the contents of a CSV file directly to an existing Google Sheet using the batch update method.
The problem is that my code is only uploading the last record, but from what I can tell the request object has all the data.
So far I am able to loop through the CSV file and cast each row to a Google Sheets ValueRange and add each ValueRange to a data object in order to apply it to the RequestBody required by the Google Sheets API.
This is all based on the API documentation available from Google.
Google Sheets API V4 Batch Update
I tossed in a few Message Boxes just to see what is happening in the background.
Below is the code I am using currently, and a sample set of data you can put into a csv file to run.
It appears that I am missing something with the upload request itself, if anyone has any input or experience with this I can't seem to get past this.
*Note: this code is simply a VB .Net winform app with a single button added to the form.
Imports System.Threading
Imports Google.Apis.Sheets.v4
Imports Google.Apis.Auth.OAuth2
Imports Google.Apis.Services
Imports Microsoft.VisualBasic.FileIO
Imports Google.Apis.Sheets.v4.Data
Imports Data = Google.Apis.Sheets.v4.Data
Public Class Form1
Dim ClientID As String = "<GOOGLE CLIENT>.apps.googleusercontent.com"
Dim ClientSecret As String = "<CLIENT SECRET>"
Dim Scopes As String() = {SheetsService.Scope.Spreadsheets}
Dim credential As UserCredential = GoogleWebAuthorizationBroker.AuthorizeAsync(New ClientSecrets() With {.ClientId = ClientID,
.ClientSecret = ClientSecret},
Scopes, "user",
CancellationToken.None).Result
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim r As Integer = 1
Dim oblist = New List(Of Object)
Dim sheetsService As SheetsService = New SheetsService(New BaseClientService.Initializer With {
.HttpClientInitializer = credential,
.ApplicationName = "Google-SheetsSample/0.1"
})
Dim spreadsheetId As String = "<SPREADSHEET ID>"
Dim valueInputOption As String = "RAW"
Dim valueRanges As ValueRange = New ValueRange()
Dim data As List(Of Data.ValueRange) = New List(Of Data.ValueRange)
Dim tfp As New TextFieldParser("<PATH TO CSV FILE I WANT TO UPLOAD>")
tfp.SetDelimiters(",")
tfp.TextFieldType = FieldType.Delimited
'tfp.ReadLine() ' skip header
While tfp.EndOfData = False
Dim fields = tfp.ReadFields()
Dim range2 As String = "A" & r ' range to update
valueRanges.MajorDimension = "ROWS" '"ROWS";//COLUMNS
valueRanges.Range = range2 ' apply range to ValueRnage
' My csv has 12 fields
oblist = New List(Of Object)() From {
fields(0), fields(1), fields(2), fields(3), fields(4), fields(5), fields(6), fields(7), fields(8), fields(9), fields(10), fields(11)
}
valueRanges.Values = New List(Of IList(Of Object)) From { ' add list of objects pulled from TextFieldParser to ValueRange
oblist
}
data.Add(valueRanges) ' Add valueRange to data for RequestBody
r += 1 ' Increment Range
End While
MsgBox("Count of value ranges in data object : " & data.Count)
Dim requestBody As Data.BatchUpdateValuesRequest = New Data.BatchUpdateValuesRequest()
requestBody.ValueInputOption = valueInputOption
requestBody.Data = data
MsgBox("Request body data : " & requestBody.Data.Count)
Dim request As SpreadsheetsResource.ValuesResource.BatchUpdateRequest = sheetsService.Spreadsheets.Values.BatchUpdate(requestBody, spreadsheetId)
Dim response As Data.BatchUpdateValuesResponse = request.Execute()
MsgBox("Count of Rows uploaded to google sheet : " & response.TotalUpdatedRows)
End Sub
End Class
CSV - EXAMPLE -
Column 1,Column 2,Column 3,Column 4,Column 5,Column 6,Column 7,Column 8,Column 9,Column 10,Column 11,Column 12
ex12,ex34,ex56,ex78,ex100,ex122,ex144,ex166,ex188,ex210,ex232,ex254
ex13,ex35,ex57,ex79,ex101,ex123,ex145,ex167,ex189,ex211,ex233,ex255
ex14,ex36,ex58,ex80,ex102,ex124,ex146,ex168,ex190,ex212,ex234,ex256
ex15,ex37,ex59,ex81,ex103,ex125,ex147,ex169,ex191,ex213,ex235,ex257
ex16,ex38,ex60,ex82,ex104,ex126,ex148,ex170,ex192,ex214,ex236,ex258
ex17,ex39,ex61,ex83,ex105,ex127,ex149,ex171,ex193,ex215,ex237,ex259
ex18,ex40,ex62,ex84,ex106,ex128,ex150,ex172,ex194,ex216,ex238,ex260
ex19,ex41,ex63,ex85,ex107,ex129,ex151,ex173,ex195,ex217,ex239,ex261
ex20,ex42,ex64,ex86,ex108,ex130,ex152,ex174,ex196,ex218,ex240,ex262
ex21,ex43,ex65,ex87,ex109,ex131,ex153,ex175,ex197,ex219,ex241,ex263
Here I create one code .. The difference is only you pass List to ValueRange whereas there should be IList ..
But for conversion from list to Ilist I used objList.ToList () and I don't know it's drawback you need to check
And also you have to create new object of valueRange within while loop
Dim sheetId = FileId
Dim service = GetGoogleAPPSheetService()
Dim val_range As ValueRange
Dim DataList As List(Of ValueRange) = New List(Of ValueRange)
Dim I As Integer = 0
Dim objList As List(Of Object)
Dim objMainList As List(Of IList(Of Object))
Dim objIlIst As IList(Of Object)
Dim objImainList As IList(Of IList(Of Object))
For Each dr As DataRow In dt.Rows
I += 1
val_range = New ValueRange
val_range.Range = "A" & I.ToString
objList = New List(Of Object) From {dr(0).ToString, dr(1).ToString}
objMainList = New List(Of IList(Of Object)) From {objList.ToList()}
val_range.Values = objMainList.ToList
DataList.Add(val_range)
Next
Dim req_body As BatchUpdateValuesRequest = New BatchUpdateValuesRequest
req_body.ValueInputOption = "RAW"
req_body.Data = DataList.ToList
Dim request As SpreadsheetsResource.ValuesResource.BatchUpdateRequest = service.Spreadsheets.Values.BatchUpdate(req_body, sheetId)
Dim response As BatchUpdateValuesResponse = request.Execute
MsgBox("Count of Rows uploaded to google sheet : " & response.TotalUpdatedRows)
Catch ex As Exception
Throw ex
End Try
I've been doing some testing with Microsoft Graph and I seem to have hit a brick wall; Wondering if anyone can give me a steer in the right direction.
The following code is from my test app (vb)...
Imports System.Net
Imports System.Text
Imports System.IO
Imports Newtonsoft.Json.Linq
Class MainWindow
Public Shared graph_url As String = "https://graph.microsoft.com/v1.0/"
Public Shared br As String = ControlChars.NewLine
Dim myscope As String = "https://graph.microsoft.com/.default"
Dim mysecret As String = "zzx...BX-"
Dim mytenantid As String = "7aa6d...d409199"
Dim myclientid As String = "4e37...5a59"
Dim myuri As String = "https://login.microsoftonline.com/" & mytenantid & "/oauth2/v2.0/token"
Dim mytoken As String = ""
Public Function HTTP_Post(ByVal url As String, ByVal postdata As String)
Try
Dim encoding As New UTF8Encoding
Dim postReq As HttpWebRequest = DirectCast(WebRequest.Create(url), HttpWebRequest)
postReq.Headers.Add("Authorization", "Bearer " & mytoken)
postReq.Method = "POST"
postReq.PreAuthenticate = True
Dim postreqstream As Stream = postReq.GetRequestStream()
If Not postdata = Nothing Then
Dim byteData As Byte() = encoding.GetBytes(postdata)
postreqstream.Write(byteData, 0, byteData.Length)
End If
postreqstream.Close()
request_header.Text = postReq.Headers.ToString
Dim postresponse As HttpWebResponse = DirectCast(postReq.GetResponse(), HttpWebResponse)
Dim postreqreader As New StreamReader(postresponse.GetResponseStream())
Dim response As String = postreqreader.ReadToEnd
Return (response)
Catch ex As Exception
user_results.Text = ex.Message.ToString
End Try
End Function
Public Function GetNewToken()
Console.WriteLine(myuri)
Dim post_data As String = "client_id=" & myclientid & "&client_secret=" & mysecret & "&scope=" & myscope & "&grant_type=client_credentials"
Dim token As String = HTTP_Post(myuri, post_data)
get_result.Text = JObject.Parse(token).SelectToken("access_token")
mytoken = JObject.Parse(token).SelectToken("access_token")
End Function
Private Sub Button_Click(sender As Object, e As RoutedEventArgs)
GetNewToken()
End Sub
Private Sub usrget_Copy_Click(sender As Object, e As RoutedEventArgs) Handles usrget_Copy.Click
Dim url As String = "https://graph.microsoft.com/v1.0/users/*{myUPN}*"
Dim post_data As String = Nothing
Dim return_data As String = HTTP_Post(url, post_data)
req_url.Text = url
End Sub
End Class
The GetNewToken() function works fine and I retrive a valid token without an issue.
When i try and use that token to query a user, i get 400 Bad request - no additional info as such; just that.
I've examined the headers of my POST request and tried it several different ways, but cannot seem to overcome this; As far as i can tell my request is formatted as specified in the documentation for Microsoft Graph.
Have also googled the hell out of this to see if there was something obvious or simple I have overlooked; I've seen a few posts on this topic where people suggested setting the Authorization header before anything else - I tried that and it made no difference.
A few people also suggected setting the Accept to 'application/json'; I also tried that and it made no difference.
I've outputted some of the details to the gui and grabbed a screenshot so you can see what I'm seeing...
Interestingly, If i try the 'Query users' before i grab a token, I get a 401 unauthorized which would suggest that the request itself is working correctly.
As i mentioned, this is a testing app whilst I get some functionality working (some data obscured for obvious reasons); I'm not concerned about the tidyness of code or anything like that at this point.
If anyone is able to help it would be very much appriciated.
Thanks in advance.
I am only doing this as a method to secure a third party product that does not have a native way of requesting and setting up 2FA. Essentially this creates a request that is sent to IT to have them manually add the secret key to a users profile when requested.
How should I generate a "secret" code for my two factor authentication system?
I'm using the QRCoder package to generate a nice displayable QR Code for my userbase. And it works great in the Microsoft Authenticator app, but both Authy and Google fail.
I suppose my random secret generator function is to blame?
Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
Dim generator As OneTimePassword = New OneTimePassword() With {
.Secret = GenerateRandomString(16),
.Issuer = "My Site",
.Label = "My Service",
.Type = OneTimePassword.OneTimePasswordAuthType.TOTP
}
Dim payload As String = generator.ToString()
Dim qrGenerator As QRCodeGenerator = New QRCodeGenerator()
Dim qrCodeData As QRCodeData = qrGenerator.CreateQrCode(payload, QRCodeGenerator.ECCLevel.Q)
Dim qrCode As QRCode = New QRCode(qrCodeData)
LiteralQRCode.Text = generator.Secret
Dim imgBarCode As New System.Web.UI.WebControls.Image()
imgBarCode.Height = 300
imgBarCode.Width = 300
Using bitMap As Bitmap = qrCode.GetGraphic(20)
Using ms As New MemoryStream()
bitMap.Save(ms, System.Drawing.Imaging.ImageFormat.Png)
Dim byteImage As Byte() = ms.ToArray()
imgBarCode.ImageUrl = "data:image/png;base64," + Convert.ToBase64String(byteImage)
End Using
plBarCode.Controls.Add(imgBarCode)
End Using
End Sub
Public Function GenerateRandomString(ByRef iLength As Integer) As String
Dim rdm As New Random()
Dim allowChrs() As Char = "ABCDEFGHIJKLOMNOPQRSTUVWXYZ0123456789".ToCharArray()
Dim sResult As String = ""
For i As Integer = 0 To iLength - 1
sResult += allowChrs(rdm.Next(0, allowChrs.Length))
Next
Return sResult
End Function
I ended up using OtpNet and using their Base32Encode function to get what I needed.
Hopefully this will help the next person who is attempting to work on a project that isn't exactly conventional.
Dim totp = KeyGeneration.GenerateRandomKey()
Dim generator As OneTimePassword = New OneTimePassword() With {
.Secret = Base32Encoding.ToString(totp),
.Issuer = "My Site",
.Label = "My Service",
.Type = OneTimePassword.OneTimePasswordAuthType.TOTP
}
I'm requesting remote SOAP web-service but all operation (from click search button to render interface with answer) took almost two minutes, it's too long. So I wonder if there any possible way to improve performance of the current code.
Operation that parse xml and read data to database working quite well, problem only about reading answer from stream.
Public Shared Function CallWebService(ByVal an As String, ByVal xmlcommand As String) As String
Dim _url = "http://testapi.interface-xml.com/appservices/ws/FrontendService"
Dim soapEnvelopeXml As XmlDocument = CreateSoapEnvelope(xmlcommand)
Dim webRequest As HttpWebRequest = CreateWebRequest(_url, an)
webRequest.Proxy = System.Net.WebRequest.DefaultWebProxy
InsertSoapEnvelopeIntoWebRequest(soapEnvelopeXml, webRequest)
Dim asyncResult As IAsyncResult = webRequest.BeginGetResponse(Nothing, Nothing)
asyncResult.AsyncWaitHandle.WaitOne()
Dim soapResult As String
Using webResponse As WebResponse = webRequest.EndGetResponse(asyncResult)
Using bs As New BufferedStream(webResponse.GetResponseStream())
Using rd As New StreamReader(bs)
soapResult = rd.ReadLine()
Return soapResult
End Using
End Using
End Using
End Function
Here is solution!
Public Shared Function CallWebService(ByVal an As String, ByVal xmlcommand As String) As String
Dim _url = "http://testapi.interface-xml.com/appservices/ws/FrontendService"
Dim soapEnvelopeXml As XmlDocument = CreateSoapEnvelope(xmlcommand)
Dim webRequest As HttpWebRequest = CreateWebRequest(_url, an)
webRequest.Proxy = System.Net.WebRequest.DefaultWebProxy
webRequest.Headers.Add("Accept-Encoding", "gzip, deflate")
InsertSoapEnvelopeIntoWebRequest(soapEnvelopeXml, webRequest)
Dim asyncResult As IAsyncResult = webRequest.BeginGetResponse(Nothing, Nothing)
asyncResult.AsyncWaitHandle.WaitOne()
Dim soapResult As String
Using webResponse As WebResponse = webRequest.EndGetResponse(asyncResult)
Using bs As New BufferedStream(webResponse.GetResponseStream())
Using gz As New GZipStream(bs, CompressionMode.Decompress)
Using rd As New StreamReader(gz)
soapResult = rd.ReadLine()
Return soapResult
End Using
End Using
End Using
End Using
End Function
I have the following code, it connects to PHP server and retrieve data successfully, i'm not very good with VB, how can i read the JSON response text and extract it's elements?
Public Class Form1
Private Sub submit_Click(sender As System.Object, e As System.EventArgs) Handles submit.Click
Dim user As String
Dim pass As String
user = uname.Text
pass = passwd.Text
Dim request As WebRequest = WebRequest.Create("http://domain.com/test.php")
request.Method = "POST"
Dim postData As String
postData = "username=" & user & "&password=" & pass
Dim byteArray As Byte() = Encoding.UTF8.GetBytes(postData)
request.ContentType = "application/x-www-form-urlencoded"
request.ContentLength = byteArray.Length
Dim dataStream As Stream = request.GetRequestStream()
dataStream.Write(byteArray, 0, byteArray.Length)
dataStream.Close()
Dim response As WebResponse = request.GetResponse()
Console.WriteLine(CType(response, HttpWebResponse).StatusDescription)
dataStream = response.GetResponseStream()
Dim reader As New StreamReader(dataStream)
Dim responseFromServer As String = reader.ReadToEnd()
If responseFromServer = "0" Then
MsgBox("Login Failed")
Else
MsgBox("json data")
End If
reader.Close()
dataStream.Close()
response.Close()
End Sub
End Class
The JSON response would be something like:
{"comments": [
{
"comment" : "some text",
"date" : "some date",
"user" : "user name"
},
{
"comment" : "some text",
"date" : "some date",
"user" : "user name"
}
],
"messages": [ .... ]
}
How to output the json string into:
Comments
user date comment
-----------------------------------
user 1 date 1 comment 1
user 2 date 2 comment 2
Messages
user date message
-----------------------------------
user 1 date 1 message 1
user 2 date 2 message 2
After long research and many tests I found out a very nice extension called Newtonsoft.json, it's extremely simple and can be installed from package manager console like this:
install-package Newtonsoft.json
And include it like this:
Imports Newtonsoft.Json
Imports Newtonsoft.Json.Linq
Then all i needed to do is to declare the elements names and values like this:
Else
Dim json As String = responseFromServer
Dim ser As JObject = JObject.Parse(json)
Dim data As List(Of JToken) = ser.Children().ToList
Dim output As String = ""
For Each item As JProperty In data
item.CreateReader()
Select Case item.Name
Case "comments"
output += "Comments:" + vbCrLf
For Each comment As JObject In item.Values
Dim u As String = comment("user")
Dim d As String = comment("date")
Dim c As String = comment("comment")
output += u + vbTab + d + vbTab + c + vbCrLf
Next
Case "messages"
output += "Messages:" + vbCrLf
For Each msg As JObject In item.Values
Dim f As String = msg("from")
Dim t As String = msg("to")
Dim d As String = msg("date")
Dim m As String = msg("message")
Dim s As String = msg("status")
output += f + vbTab + t + vbTab + d + vbTab + m + vbTab + s + vbCrLf
Next
End Select
Next
MsgBox(output)
End If
hope someone will find this useful
#razzak is absolutely right to use the Json.Net NuGet package. Another option that would cut this down dramatically, is to use the built in DeserializeObject function. As long as you have a well define model, then you can deserialize the Json right into an instance of the object using something like this:
dim myObject as MyDefinedObject = JsonConvert.DeserializeObject(responseFromServer)
or this in C#
MyDefinedObject m = JsonConvert.DeserializeObject<MyDefinedObject>(responseFromServer);
Also, if you don't want to loop, you could also select tokens using something like this:
Dim d = ser.SelectToken("$..resources[?(#)].travelDistance")
This code above was used to locate the travelDistance between two points from the Bing API. If you have ever dealt with the Bing or Google Map REST APIs, then you know the JSon is generally too large to loop through the data when you are looking for very specific values.
The JSon.Net website has a blog page that goes through some additional examples:
http://james.newtonking.com/json
To use
Imports Newtonsoft.Json
Imports Newtonsoft.Json.Linq
'Json.Net' library should be installed.
Imports Newtonsoft.Json
Imports Newtonsoft.Json.Linq
This seems to cut it on VB.net for youtube API V.3
of course it depends on what you are trying to accomplish
but Youtube returns data as Json format