Log exceptions from log4net to Azure table storage in Azure function - asp.net-core

I am referring to this link to log exceptions from log4net to Azure table storage.
Now I need to do the same in Azure functions in .net core, but since there is no config files in Azure functions, I am not able to use the same there.
What can I try to resolve this?

Now I need to do the same in Azure functions in .net core, but since there is no config files in Azure functions am not able to use the same there
It seems you are trying to read some property what we usually read from config file. Yes you can do it in Azure Function also. There is file name local.settings.json you can read your required property from here. See the example below:
local.settings.json:
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"TableName": "YourTableName",
"AccountName": "YourAccountName",
"AccountKey": "YourAccountKey"
}
}
Read Your Property On Azure Function From local.settings.json:
public static class AzureFunctionApp2Arunraj414CaseForGetConfigProperty
{
[FunctionName("AzureFunctionApp2Arunraj414CaseForGetConfigProperty")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
//Read Request Body
var content = await new StreamReader(req.Body).ReadToEndAsync();
//Extract Request Body and Parse To Class
Users objUsers = JsonConvert.DeserializeObject<Users>(content);
//You Can Read Your Desired Value from local.settings.json file Like Below
var yourTableNameFromLocalSettingsJson = Environment.GetEnvironmentVariable("TableName");
var yourAccountNameFromLocalSettingsJson = Environment.GetEnvironmentVariable("AccountName");
var yourAccountKeyFromLocalSettingsJson = Environment.GetEnvironmentVariable("AccountKey");
//I am returning all the property I got from local.settings.json
var result = new OkObjectResult(yourTableNameFromLocalSettingsJson +" & "+ yourAccountNameFromLocalSettingsJson + " & " + yourAccountKeyFromLocalSettingsJson+ " From local.settings.json");
return result;
}
}
Debug & Testing:
See the screen shot below:
Note: You even can set your property outside of values on local.settings.json file as separate property.

Related

Easy way to retrieve image source in abp

I'm pretty new to ABP Framework and probably this question has a really simple answer, but I haven't managed to find it. Images are an important part of any app and handling them the best way (size, caching) is mandatory.
Scenario
setup a File System Blob Storing provider. This means that the upload file will be stored in the file system as an image file
make a service that uses a Blob container to save and retrieve the image. So, after saving it, I use the unique file name as a blob name. This name is used to retrieve it back.
the user is logged in, so authorization is required
I can easily obtain the byte[]s of the image by calling blobContainer.GetAllBytesOrNullAsync(blobName)
I want to easily display the image in <img> or in datatable row directly.
So, here is my question: is there an easy way to use a blob stored image as src of a <img> directly in a razor page? What I've managed to achieve is setting in the model, a source as a string made from image type + bytes converted to base 64 string (as here) however in this case I need to do it in the model and also I don't know if caching is used by the browser. I don't see how caching would work in this case.
I am aware that this may be a question more related to asp.net core, but I was thinking that maybe in abp there is some way via a link to access the image.
If you have the ID of the blob then it is easy to do. Just create a Endpoint to get the Image based on the blob id.
Here is the sample AppService
public class DocumentAppService : FileUploadAppService
{
private readonly IBlobContainer<DocumentContainer> _blobContainer;
private readonly IRepository<Document, Guid> _repository;
public DocumentAppService(IRepository<Document, Guid> repository, IBlobContainer<DocumentContainer> blobContainer)
{
_repository = repository;
_blobContainer = blobContainer;
}
public async Task<List<DocumentDto>> Upload([FromForm] List<IFormFile> files)
{
var output = new List<DocumentDto>();
foreach (var file in files)
{
using var memoryStream = new MemoryStream();
await file.CopyToAsync(memoryStream).ConfigureAwait(false);
var id = Guid.NewGuid();
var newFile = new Document(id, file.Length, file.ContentType, CurrentTenant.Id);
var created = await _repository.InsertAsync(newFile);
await _blobContainer.SaveAsync(id.ToString(), memoryStream.ToArray()).ConfigureAwait(false);
output.Add(ObjectMapper.Map<Document, DocumentDto>(newFile));
}
return output;
}
public async Task<FileResult> Get(Guid id)
{
var currentFile = _repository.FirstOrDefault(x => x.Id == id);
if (currentFile != null)
{
var myfile = await _blobContainer.GetAllBytesOrNullAsync(id.ToString());
return new FileContentResult(myfile, currentFile.MimeType);
}
throw new FileNotFoundException();
}
}
Upload function will upload the files and Get function will get the file.
Now set the Get route as a src for the image.
Here is the blog post: https://blog.antosubash.com/posts/dotnet-file-upload-with-abp
Repo: https://github.com/antosubash/FileUpload

non-invocable member 'File' cannot be used like a method error message- what am I missing?

I have a Blazor Application which had files uploaded to a upload folder on the web server. I am in the process of trying to figure out the code to download an uploaded file in the browser for retrieval and viewing. Right now the code is as below (the download part from code examples on the internet)
public void FileDetailsToolbarClickHandler(Syncfusion.Blazor.Navigations.ClickEventArgs args)
{
string path = null;
string uploads = System.IO.Path.Combine(System.IO.Directory.GetCurrentDirectory(), "wwwroot\\uploads");
path = uploads + "\\" + SelectedFileName;
if (args.Item.Text == "Delete")
{
//Code for Deleting goes here
//UploadRef.Remove();
if (System.IO.File.Exists(path))
{
System.IO.File.Delete(path);
}
FileDetailsService.FileDetailsDelete(SelectedFileId); //NavigationManager.NavigateTo($"/ServiceRequestNotes/servicerequestnoteadd");
NavigationManager.NavigateTo($"/ServiceRequests/serviceRequestsaddedit2/{Id}", forceLoad: true);
}
else
{
// its a download
IFileProvider provider = new PhysicalFileProvider(uploads);
IFileInfo fileinfo = provider.GetFileInfo(path + SelectedFileName);
var readStream = fileinfo.CreateReadStream();
var mimeType = "application/pdf";
return File(readStream, mimeType, SelectedFileName);
}
}
On the last statement I am a getting the following error message
non-invocable member 'File' cannot be used like a method error message
What am I missing or do I need to change or add to have the output from the readstream render to the browser?
The blazor application is a blazor server app not WASM. It does not make use of API controllers.
Any advice?
This is a void method. You can't return anything at all. Also, if you're trying to instantiate a File object, you'd have to use the new keyword.

Copy Data From Azure Blob Storage to AWS S3

I am new to Azure Data Factory and have an interesting requirement.
I need to move files from Azure Blob storage to Amazon S3, ideally using Azure Data Factory.
However S3 isnt supported as a sink;
https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview
I also understand from a variety of comments i've read on here that you cannot directly copy from Blob Storage to S3 - you would need to download the file locally and then upload it to S3.
Does anyone know of any examples, in Data factory, SSIS or Azure Runbook that can do such a thing, I suppose an option would be to write an azure logic-app or function that is called from Data Factory.
Managed to get something working on this - it might be useful for someone else.
I decided to write an azure function that uses a HTTP request as a trigger.
These two posts helped me a lot;
How can I use NuGet packages in my Azure Functions?
Copy from Azure Blob to AWS S3 using C#
Please note my answer to the Nuget packages if you are using Azure functions 2.x.
Here is the code - you can modify the basis of this to your needs.
I return a JSON Serialized object because Azure Data Factory requires this as a response from a http request sent from a pipeline;
#r "Microsoft.WindowsAzure.Storage"
#r "Newtonsoft.Json"
#r "System.Net.Http"
using System.Net;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Primitives;
using Newtonsoft.Json;
using Microsoft.WindowsAzure.Storage.Blob;
using System.Net.Http;
using Amazon.S3;
using Amazon.S3.Model;
using Amazon.S3.Transfer;
using Amazon.S3.Util;
public static async Task<IActionResult> Run(HttpRequest req, ILogger log)
{
log.LogInformation("Example Function has recieved a HTTP Request");
// get Params from query string
string blobUri = req.Query["blobUri"];
string bucketName = req.Query["bucketName"];
// Validate query string
if (String.IsNullOrEmpty(blobUri) || String.IsNullOrEmpty(bucketName)) {
Result outcome = new Result("Invalid Parameters Passed to Function",false,"blobUri or bucketName is null or empty");
return new BadRequestObjectResult(outcome.ConvertResultToJson());
}
// cast the blob to its type
Uri blobAbsoluteUri = new Uri(blobUri);
CloudBlockBlob blob = new CloudBlockBlob(blobAbsoluteUri);
// Do the Copy
bool resultBool = await CopyBlob(blob, bucketName, log);
if (resultBool) {
Result outcome = new Result("Copy Completed",true,"Blob: " + blobUri + " Copied to Bucket: " + bucketName);
return (ActionResult)new OkObjectResult(outcome.ConvertResultToJson());
}
else {
Result outcome = new Result("ERROR",false,"Copy was not successful Please review Application Logs");
return new BadRequestObjectResult(outcome.ConvertResultToJson());
}
}
static async Task<bool> CopyBlob(CloudBlockBlob blob, string existingBucket, ILogger log) {
var accessKey = "myAwsKey";
var secretKey = "myAwsSecret";
var keyName = blob.Name;
// Make the client
AmazonS3Client myClient = new AmazonS3Client(accessKey, secretKey, Amazon.RegionEndpoint.EUWest1);
// Check the Target Bucket Exists;
bool bucketExists = await AmazonS3Util.DoesS3BucketExistAsync (myClient,existingBucket);
if (!bucketExists) {
log.LogInformation("Bucket: " + existingBucket + " does not exist or is inaccessible to the application");
return false;
}
// Set up the Transfer Utility
TransferUtility fileTransferUtility = new TransferUtility(myClient);
// Stream the file
try {
log.LogInformation("Starting Copy");
using (var stream = await blob.OpenReadAsync()) {
// Note: You need permissions to not be private on the source blob
log.LogInformation("Streaming");
await fileTransferUtility.UploadAsync(stream,existingBucket,keyName);
log.LogInformation("Streaming Done");
}
log.LogInformation("Copy completed");
}
catch (AmazonS3Exception e) {
log.LogInformation("Error encountered on server. Message:'{0}' when writing an object", e.Message);
}
catch (Exception e) {
log.LogInformation("Unknown encountered on server. Message:'{0}' when writing an object", e.Message);
return false;
}
return true;
}
public class Result {
public string result;
public bool outcome;
public string UTCtime;
public string details;
public Result(string msg, bool outcomeBool, string fullMsg){
result=msg;
UTCtime=DateTime.Now.ToString("yyyy-MM-dd h:mm:ss tt");
outcome=outcomeBool;
details=fullMsg;
}
public string ConvertResultToJson() {
return JsonConvert.SerializeObject(this);
}
}
You can use Skyplane to copy data across clouds (110X speedup over CLI tools, with automatic compression to save on egress). To transfer from Azure blob storage to S3 you can call one of the commands:
skyplane cp -r az://azure-bucket-name/ s3://aws-bucket-name/
skyplane sync -r az://azure-bucket-name/ s3://aws-bucket-name/
ADF now includes SFTP as a sink. From the same link provided in the question (supported as a sink is the far-right column):
Using the AWS Transfer family you can set up an SFTP server and add a user with an SSH public key, then use that configuration to set up an SFTP connection from ADF that will connect directly to an S3 bucket.
Download Files From Azure Storage using AzCopy into a temporal local repository
You can download the files from Azure Cloud storage to your local system, just follow the below command, use the recursive flag to copy all the files
azcopy /Source:[source_container_url] /Dest:[local_file_path] /Sourcekey:[source_storage_account_access_key] /s
Upload Local Files to Amazon S3 using aws s3 cp command
aws s3 cp local_file_path s3://my-bucket/ --recursive

Accessing the Request.Content in the new ASP.NET vnext web api way of doing things?

I have searched high and low for this one and can't seem to find a way of accessing the Request.Content in an MVC web api. I basically am trying to create a File Service to and from Azure Blob and Table storage (table for storing metadata about the file, blob for the actual file)....
I was converting the steps in the following link, but this is where I have come unstuck
the back end I have working but can't find a way of the new unified controller passing a fileobject from json post through to the service! Any ideas would be greatly appreciated as always... or am I just going about this the wrong way?
Article here....
UPDATE: so to clarify, what I am trying to do in the new MVC 6 (where you no longer have an apicontroller to inherit from) is to access a file that has been uploaded to the api from a JSON post. That is the long and short of what I am trying to achieve.
I am trying to use the article based on the old Web API which uses the Request.Content to access it, however even if I use the WebAPIShim which they provide I still come unstuck with other objects or properties that are no longer available so I'm wondering if I need to approach it a different way, but either way, all I am trying to do is to get a file from a JSON post to a MVC 6 Web api and pass that file to my back end service....
ANY IDEAS?
Here is an example without relying on model binding.
You can always find the request data in Request.Body, or use Request.Form to get the request body as a form.
[HttpPost]
public async Task<IActionResult> UploadFile()
{
if (Request.Form.Files != null && Request.Form.Files.Count > 0)
{
var file = Request.Form.Files[0];
var contentType = file.ContentType;
using (var fileStream = file.OpenReadStream())
{
using (var memoryStream = new MemoryStream())
{
await fileStream.CopyToAsync(memoryStream);
// do what you want with memoryStream.ToArray()
}
}
}
return new JsonResult(new { });
}
If the only thing in your request is a File you can use the IFormFile class in your action:
public FileDetails UploadSingle(IFormFile file)
{
FileDetails fileDetails;
using (var reader = new StreamReader(file.OpenReadStream()))
{
var fileContent = reader.ReadToEnd();
var parsedContentDisposition = ContentDispositionHeaderValue.Parse(file.ContentDisposition);
fileDetails = new FileDetails
{
Filename = parsedContentDisposition.FileName,
Content = fileContent
};
}
return fileDetails;
}

Returning Azure BLOB from WCF service as a Stream - Do we need to close it?

I have a simple WCF service that exposes a REST endpoint, and fetches files from a BLOB container. The service returns the file as a stream. i stumbled this post about closing the stream after the response has been made :
http://devdump.wordpress.com/2008/12/07/disposing-return-values/
This is my code:
public class FileService
{
[OperationContract]
[WebGet(UriTemplate = "{*url}")]
public Stream ServeHttpRequest(string url)
{
var fileDir = Path.GetDirectoryName(url);
var fileName = Path.GetFileName(url);
var blobName = Path.Combine(fileDir, fileName);
return getBlob(blobName);
}
private Stream getBlob(string blobName)
{
var account = CloudStorageAccount.FromConfigurationSetting("ConnectingString");
var client = account.CreateCloudBlobClient();
var container = client.GetContainerReference("data");
var blob = container.GetBlobReference(blobName);
MemoryStream ms = new MemoryStream();
blob.DownloadToStream(ms);
ms.Seek(0, SeekOrigin.Begin);
return ms;
}
}
So I have two question :
Should I follow the pattern mentioned in the post ?
If I change my return type to Byte[], what are Cons/Pros ?
( My client is Silverlight 4.0, just in case it has any effect )
I'd consider changing your return type to byte[]. It's tidier.
Stream implements IDisposable, so in theory the consumer of your method will need to call your code in a using block:
using (var receivedStream = new FileService().ServeHttpRequest(someUrl))
{
// do something with the stream
}
If your client definitely needs access to something that Stream provides, then by all means go ahead and return that, but by returning a byte[] you keep control of any unmanaged resources that are hidden under the covers.
OperationBehaviorAttribute.AutoDisposeParameters is set to TRUE by default which calls dispose on all the inputs/outputs that are disposable. So everything just works.
This link :
http://devdump.wordpress.com/2008/12/07/disposing-return-values/
explains how to manually control the process.