delete Sharepoint file using Script Task - vb.net

I need to delete a Sharepoint file using the Script Task from SSIS. In Visual Basic, I've tried using the SPListItemCollection with imports Microsoft.Sharepoint but it doesn't recognize the namespace. I didn't find lots of threads on this subject or what I've found wasn't related with script task, so any help will be really appreciated. Many thanks
Update based on #Hadi answer
Thanks Hadi for your answer. I've given up the idea of using SPListCollection as it seems too complicated. Instead I'm trying to delete the file after it is downloaded from Sharepoint to the local folder. I would need help at the line that actually deletes the file. Here is the code:
Public Sub Main()
Try
' get location of local folder
Dim dir As DirectoryInfo = New DirectoryInfo(Dts.Variables("DestP").Value.ToString())
If dir.Exists Then
' Create the filename for local storage
Dim file As FileInfo = New FileInfo(dir.FullName & "\" & Dts.Variables("FileName").Value.ToString())
If Not file.Exists Then
' get the path of the file to download
Dim fileUrl As String = Dts.Variables("SHP_URL").Value.ToString()
If fileUrl.Length <> 0 Then
Dim client As New WebClient()
If Left(fileUrl, 4).ToLower() = "http" Then
'download the file from SharePoint
client.Credentials = New System.Net.NetworkCredential(Dts.Variables("$Project::UserN").Value.ToString(), Dts.Variables("$Project::Passw").Value.ToString())
client.DownloadFile(fileUrl.ToString() & "/" & Dts.Variables("FileName").Value.ToString(), file.FullName)
Else
System.IO.File.Copy(fileUrl.ToString() & Dts.Variables("FileName").Value.ToString(), file.FullName)
End If
'delete file from Sharepoint
client.(fileUrl.ToString() & "/" & Dts.Variables("FileName").Value.ToString(), file.FullName).delete()
Else
Throw New ApplicationException("EncodedAbsUrl variable does not contain a value!")
End If
End If
Else
Throw New ApplicationException("No ImportFolder!")
End If
Catch ex As Exception
Dts.Events.FireError(0, String.Empty, ex.Message, String.Empty, 0)
Dts.TaskResult = ScriptResults.Failure
End Try
Dts.TaskResult = ScriptResults.Success
End Sub

Update 1 - Delete using FtpWebRequest
You cannot delete file using WebClient class. You can do that using FtpWebRequest class. And send a WebRequestMethods.Ftp.DeleteFile request as mentioned in the link below:
How To Delete a File From FTP Server in C#
It should work with Sharepoint also.
Here is the function in VB.NET
Private Function DeleteFile(ByVal fileName As String) As String
Dim request As FtpWebRequest = CType(WebRequest.Create(fileUrl.ToString() & "/" & fileName), FtpWebRequest)
request.Method = WebRequestMethods.Ftp.DeleteFile
request.Credentials = New NetworkCredential(Dts.Variables("$Project::UserN").Value.ToString(), Dts.Variables("$Project::Passw").Value.ToString())
Using response As FtpWebResponse = CType(request.GetResponse(), FtpWebResponse)
Return response.StatusDescription
End Using
End Function
You should replace the following line:
client.(fileUrl.ToString() & "/" & Dts.Variables("FileName").Value.ToString(), file.FullName).delete()
With
DeleteFile(Dts.Variables("FileName").Value.ToString())
Also you may use the following credentials:
request.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;
References
How to pass credentials to httpwebrequest for accessing SharePoint Library
How To Delete a File From FTP Server in C#
Downloading all files in a FTP folder and then deleting them
Deleting file from FTP in C#
How To Delete a File From FTP Server in C#
Initial Answer
I was searching for a similar issue from a while, it looks like you cannot delete a Sharepoint file in SSIS using a File System Task or Execute Process Task, the only way is using a Script Task. There are many links online describing this process such as:
how to delete or remove only text files from share point in C# or SSIS script?
Fastest way to delete all items with C#
Deleting files programatically
Deleting all the items from a large list in SharePoint
Concerning the problem that you have mentioned, i think you should make sure that Microsoft.Sharepoint.dll is added as a reference inside the Script Task. If so try using Microsoft.Sharepoint.SPListItemCollection instead of SPListItemCollection.

Thanks #Hadi for your help.
For me it didn't work with FTPWebResponse.
It worked with HttpWebRequest. Here is the script:
Dim request As System.Net.HttpWebRequest = CType(WebRequest.Create(fileUrl.ToString() & "/" & Dts.Variables("FileName").Value.ToString()), HttpWebRequest)
request.Credentials = New System.Net.NetworkCredential(Dts.Variables("$Project::UserN").Value.ToString(), Dts.Variables("$Project::Passw").Value.ToString())
request.Method = "DELETE"
Dim response As System.Net.HttpWebResponse = CType(request.GetResponse(), HttpWebResponse)

Related

ftp download stopping before end of list

I'm using the following code to download files from an FTP server. Before entering this code, I have created a list of filenames from files that are in the directory on the FTP server. There are over 2000 files in the list.
As I iterate through the list, the files download properly until I reach exactly 121 files downloaded. Then it starts giving me an error of "file not found, access denied." for every file after that. However the files are there. If I start the process over again it will pick up from where it left off and download another 121 files and continue until it errors again after the next 121 files.
Here is the code:
For Each file As String In dirlist
DownloadFile(local_path + "\" + filename, new_path + "/" + Trim(filename), client)
Next
Private Sub DownloadFile(ByVal localpath As String, ByVal ftpuri As String, client As String)
Dim request As New WebClient()
request.Credentials = New NetworkCredential(user_name, password)
Dim bytes() As Byte = request.DownloadData(ftpuri)
Try
Dim DownloadStream As FileStream = IO.File.Create(localpath)
DownloadStream.Write(bytes, 0, bytes.Length)
DownloadStream.Close()
Catch ex As Exception
add_to_log(log_window, ex.Message)
End Try
End Sub
I do not understand why it is stopping before completing the list.

Cannot upload the file to dropbox when published on server

I'm trying to upload a file to dropbox. The file gets uploaded and code works when i'm running locally. But file never gets uploaded when published on the server. I'm having the following error .
Could not load type 'System.Security.Authentication.SslProtocols' from assembly 'System.Net.Primitives, Version=4.0.0.0, Culture=neutral, PublicKeyToken=*********'.
I'm trying the below code sample to upload it
public sub uploadtodropbox()
Using httpclient As New HttpClient
httpclient.Timeout = TimeSpan.FromMinutes(20)
Dim _DropboxClient = New DropboxClient(ApiKeyDropBox, config)
Dim resultstring As String = UploadToDB(_DropboxClient, dropboxdir, docName, fileBytes)
_DropboxClient.Dispose()
If Not resultstring = "RanToCompletion" Then
ErrorMsg &= "The following error occured: " & resultstring
End If
End Using
End Sub
and this is the funtion that is uploading to Dropbox
Private Function UploadToDB(_DropboxClient As DropboxClient, Directory As String, Filename As String, Content As [Byte]()) As String
Try
Dim result As String = "Unknown"
Dim trycnt As String = 0
Dim tryLimit As String = 20
Dim respnse As Task(Of FileMetadata)
Using _mStream = New MemoryStream(Content)
respnse = _DropboxClient.Files.UploadAsync(Convert.ToString(Directory & Convert.ToString("/")) & Filename, WriteMode.Overwrite.Instance, body:=_mStream)
While Not respnse.IsCompleted And trycnt < tryLimit
Threading.Thread.Sleep(1000)
trycnt += 1
result = respnse.Status.ToString()
End While
UploadToDB = result
End Using
End try
END function.
I have tried without using the httpclient then i am getting this error :
The type initializer for 'Dropbox.Api.DropboxRequestHandlerOptions' threw an exception.
Thank you
I go the answer . I have system.Net.primitives assembly that is creating a problem uploading the file to Dropbox. All i need to do is delete the reference and also the System.Net.Http has versions have been set wrong in the web.config file. Once i set that in the configuration everything works fine.
For the certificate I have set the on post back to validate and allow the certificates generated in the event handler.

WebClient.UploadData "The underlying connection was closed"

I'm trying to upload a file from an FTP site to Basecamp using the Basecamp API. I'm using a simple console application. Here's my code:
Try
Dim accountID As String = ConfigurationManager.AppSettings("BaseCampID")
Dim projectID As Integer = 9999999
Dim folderName As String = "XXXXX/XXXXX"
Dim fileName As String = "XXX.zip"
'The URL to access the attachment method of the API
Dim apiURL = String.Format("https://basecamp.com/{0}/api/v1/projects/{1}/attachments.json", accountID, projectID)
'Get the file from the FTP server as a byte array
Dim fileBytes As Byte() = GetFileBytes(String.Format("{0}\\{1}", folderName, fileName))
'Initialize the WebClient object
Dim client As New WebClient()
client.Headers.Add("Content-Type", "application/zip")
'Need to provide a user-agent with a URL or email address
client.Headers.Add("User-Agent", "Basecamp Upload (email#email.com)")
'Keep the connection alive so it doesn't close
client.Headers.Add("Keep-Alive", "true")
'Provide the Basecamp credentials
client.Credentials = New NetworkCredential("username", "password")
'Upload the file as a byte array to the API, and get the response
Dim responseStr As Byte() = client.UploadData(apiURL, "POST", fileBytes)
'Convert the JSON response to a BaseCampAttachment object
Dim attachment As BaseCampAttachment
attachment = JSonHelper.FromJSon(Of BaseCampAttachment)(Encoding.Default.GetString(responseStr))
Catch ex As Exception
Console.WriteLine(ex.Message)
Finally
Console.ReadLine()
End Try
But whenever it calls client.UploadData, I get the error message "The underlying connection was closed: The connection was closed unexpectedly." I ran into this issue earlier and thought I solved it by adding the "Keep-Alive" header, but it's not working anymore. The API works if I upload a local file with client.UploadFile, but I'd like to just upload the file from they byte array from the FTP rather than downloading the file locally then uploading it to Basecamp.
Any thoughts would be greatly appreciated. Thanks!
I never figured out what was wrong with the WebClient call, but I ended up using a Basecamp API wrapper from https://basecampwrapper.codeplex.com. That wrapper uses HTTPRequest and HTTPResponse instead of WebClient.UploadData. It's also much easier to just use that wrapper than to try writing my own code from scratch.

How can I fetch the last-modified value of a remote file?

I'd like to know the last-modifed date of a remote file (defined via url).
And only download it, if it's newer than my locally stored one.
I managed to do that for local files, but can't find a solution to do that for remote files (without downloading them)
working:
Dim infoReader As System.IO.FileInfo = My.Computer.FileSystem.GetFileInfo("C:/test.txt")
MsgBox("File was last modified on " & infoReader.LastWriteTime)
not working:
Dim infoReader As System.IO.FileInfo = My.Computer.FileSystem.GetFileInfo("http://google.com/robots.txt")
MsgBox("File was last modified on " & infoReader.LastWriteTime)
I'd love to have a solution which will only have to download the headers of a file
You can use the System.Net.Http.HttpClient class to fetch the last modified date from the server. Because it's sending a HEAD request, it will not fetch the file contents:
Dim client = New HttpClient()
Dim msg = New HttpRequestMessage(HttpMethod.Head, "http://google.com/robots.txt")
Dim resp = client.SendAsync(msg).Result
Dim lastMod = resp.Content.Headers.LastModified
You could also use the If-Modified-Since request header with a GET request. This way the response should be 304 - Not Modified if the file has not been changed (no file content sent), or 200 - OK if the file has been changed (and the contents of the file will be sent in the response), although the server is not required to honor this header.
Dim client = New HttpClient()
Dim msg = New HttpRequestMessage(HttpMethod.Get, "http://google.com/robots.txt")
msg.Headers.IfModifiedSince = DateTimeOffset.UtcNow.AddDays(-1) ' use the date of your copy of the file
Dim resp = client.SendAsync(msg).Result
Select Case resp.StatusCode
Case HttpStatusCode.NotModified
' Your copy is up-to-date
Case HttpStatusCode.OK
' Your copy is out of date, so save it
File.WriteAllBytes("C:\robots.txt", resp.Content.ReadAsByteArrayAsync.Result)
End Select
Note the use of .Result, since I was testing in a console application - you should probably await instead.
If the server offers it, you can get it through the HTTP header Last-Modified property. But your still stuck at downloading the full file.
You could get it through FTP.
See if the server allows you to see the list of files in a folder.
If the website offer the date somewhere that you could pull through screen scrapping.
I know this is a little bit old question but, there's still a better answer.
Dim req As WebRequest = HttpWebRequest.Create("someurl")
req.Method = "HEAD"
Dim resp As WebResponse = req.GetResponse()
Dim remoteFileLastModified As String = resp.Headers.Get("Last-Modified")
Dim remoteFileLastModifiedDateTime As DateTime
If DateTime.TryParse(remoteFileLastModified, remoteFileLastModifiedDateTime) Then
MsgBox("Date Last Modified:" + remoteFileLastModifiedDateTime.ToString("d MMMM yyyy dddd HH:mm:ss"))
Else
MsgBox("could not determine")
End If

Closing an XML file after reading so it can be deleted

I have a problem deleting an XML file after loading it into .XMLDocument.
My code parses the XML file for specific nodes and allocates their values to variables.
Once complete the code processes data based on the values from the XML file.
This works fine until the end when i try to delete the XML file as it is still open and i then get a error "The process cannot access the file because it is being used by another process" which i guess is the XMLDocument reader.
Here is a section of the the XML processing code - this works fine.
`Dim xmlDoc As XmlDocument = New XmlDocument()
xmlDoc.Load(strFileName)
intPassed = xmlDoc.SelectSingleNode("//CalibrationPassed").InnerText
boolCheck = xmlDoc.SelectSingleNode("//ChecksComplete").InnerText
intCertRequired = xmlDoc.SelectSingleNode("//Schedule").InnerText
Console.WriteLine("Calibration Passed: " & intPassed)
Console.WriteLine("Checks Complete:" & boolCheck)
Console.WriteLine("Schedule: " & intCertRequired)
strFirstName = xmlDoc.SelectSingleNode("//FirstName").InnerText
strEMail = xmlDoc.SelectSingleNode("//Email").InnerText
strCusEmail = xmlDoc.SelectSingleNode("//CustomerEmail").InnerText
strCompanyName = xmlDoc.SelectSingleNode("//CompanyName").InnerText
strContractNumber = xmlDoc.SelectSingleNode("//ContractNo").InnerText
Console.WriteLine("First name: " & strFirstName)
Console.WriteLine("Email: " & strEMail)
Console.WriteLine("Customer EMail: " & strCusEmail)
Console.WriteLine("Company name: " & strCompanyName)
Console.WriteLine("Contract no: " & strContractNumber)
Console.WriteLine("XML Parsing Complete")
`
The code being used to delete the file is:
If System.IO.File.Exists(strFileName) = True Then
System.IO.File.Delete(strFileName)
Console.WriteLine("Deleted XML file")
End If
Any help on where I'm going wrong would be great-fully received.
Thanks
XmlDocument.Load uses a stream reader under the hood. There are two strategies for avoiding this:
1) A Using block will close/dispose your stream automatically and promptly
Using xmlDoc As XmlDocument = New XmlDocument()
xmlDoc.Load(strFileName)
'all of your copying stuff
End Using
'now delete your file
2) Load your XML and avoid using a reader:
Dim strXml as string
strXml = System.IO.File.ReadAllText(strFileName)
Dim xmlDoc As XmlDocument = New XmlDocument()
xmlDoc.LoadXml(strXml)
'and then the rest of your code
The downside to the 2nd approach is that my example doesn't consider any other encoding, but it should get you past your current problem. Dealing with various encoding options is a whole different matter.
If you're not using an xmlreader, then try this:
Dim xmlDoc = New XmlDocument()
doc.Load(strFileName)
//do all the reading stuff
Using writer = New StreamWriter(strFileName)
xmlDoc.Save(writer)
End Using
It will save your xmlDoc (in an attempt at disposing) then it should have unlocked the document which can then be deleted.
I haven't tested the code but give it a go
Thanks for all the help, it was being held open by streamreader which i had not considered to be a cause of the problem as I assumed it would have caused an error when XMLDocument used the file.
This worked for me. Setting the reference to nothing (null) to force a garbage collection.
xmlDoc = Nothing