.NET Core 6 API Access SharePoint Online but Http Error 403 - asp.net-core

To download file from SharePoint Online, I tried to access it by .NET Core 6 Web API CSOM and here is my sample code.
using Microsoft.SharePoint.Client
private void DownloadFileFromSPO()
{
string site = "https://domain.sharepoint.com/sites/xxx";
string user = "userID";
SecureString securePwd = new SecureString();
string password = "passward";
foreach(char c in password.ToCharArray())
{
securePwd.AppendChar(c);
}
using (ClientContext context = new ClientContext(site))
{
context.Credentials = new NetworkCredential(user, securePwd);
Microsoft.SharePoint.Client.File file context.Web.GetFileByUrl(fileUrl);
context.Load(file);
context.ExecuteQuery();
//do something
}
}
}
Unfortunately, when I called this method but got a http error 403: FORBIDDEN. Refer to this article, error 403 means server telling you, “I’m sorry. I know who you are–I believe who you say you are–but you just don’t have permission to access this resource.
So I supposed my API was trying to access correct website with correct userID and passward, but do not have authentication to access it. I also found there were many articles suggest replacing NetworkCredential(user, securePwd); by SharePointOnlineCredentials (user, securePwd);. However, when I tried this solution, I found my installed CSOM package did not have this class.
I also found MS docs said SharePointOnlineCredentials is not available anymore, they recommend another "OAuth access token" way to get the authentication.
My question is, is that the only way to get the authentication to access SharePoint Online? Due to company policy, configuring an application in Azure AD process is bothering me. Is there another solution, just by code and package, in this scenario?

Related

Unable to set scope games::prime for Login with Amazon

I've been tryin to setup an external login with amazon to integrate prime gaming for my web app in asp net core 3.1. I've been following this Amazon Prime Gaming Integration. I have been successfully stored the amazon userId, its token, refresh token and expires in into my database. However, I'm unable to call a post API request to /api/account/link to sync up my account according to this document. I am getting an error with message: Not all permissions are authorized. I did a bit research and it mentioned in another document link (Tips-Troubleshoot) that I need to set both scopes to games::prime and profile (I only had profile set before).
However, when adding games::prime scope, I get an exception for invalid scope: invalid_scope;Description=An unknown scope was requested
I'm using AspNet.Security.OAuth.Amazon to help with the external login with amazon, and this is the code that I have in my Startup.cs file
services.AddAuthentication()
.AddAmazon(o =>
{
o.ClientId = clientId;
o.ClientSecret = clientSecret;
o.Scope.Add("profile");
o.Scope.Add("games::prime");
o.SaveTokens = true; // get access token and refresh token
o.AccessDeniedPath = new PathString("/Home/WebAppSetting");
})
If I remove games::prime from the scope, then everything works but then I will get an error message Not all permissions are authorized
Can anyone please tell me what I'm missing and not doing correct in here ? I have been searching for amazon documents on their sites, but seems like there is nothing useful there.
Thank you
It turned out that in order to use scope games::prime, you need to contact amazon team to have them unblock that for you. This is not mentioned at all in the push integration documentation.

CSOM backend with Modern Authentication

I've been developing some backend code which connects to Sharepoint Online using CSOM and .Net Framework.
_context = new ClientContext(_sharepointDocumentRootUrl);
SecureString securePassword = new SecureString();
foreach (char c in _sharepointPassword)
{
securePassword.AppendChar(c);
}
_context.Credentials = new SharePointOnlineCredentials(_sharepointUserName, securePassword);
This worked well until the auth method was changed from Legacy Auth to Modern Auth. Now we get a "The sign-in name or password does not match one in the Microsoft account system"
Of course, we've double/tripple-checked the username/password.
Since it's backend code/system account we can't use OfficeDevPnP. The system account has MFA turned off. The IT department wont let us use the SharePoint App-Only auth method. And they are very sceptical to allow legacy auth for or system account.
I've been googling around without any success. Any ideas?
//Thomas

Azure SQL authenticate as user via API with MS Identity Platform

I have an ASP.NET Core 3.1 Web App calling an ASP.NET Core 3.1 Web API, which in turn accesses an Azure SQL database. Authentication is provided via MSAL (Microsoft Identity Platform) - i.e. using the relatively new Microsoft.Identity.Web and Microsoft.Identity.Web.UI libraries.
The goal is to ensure that the user pulls data from SQL via the API under the context of his/her own login, thus enabling row-level security, access auditing and other good things.
I have succeeded in getting the sign-in process to work for the Web App - and through that it obtains a valid access token to access the API using a scope I created when registering the latter with AD.
When I run both the API and the App locally from Visual Studio everything works as expected - the correct access tokens are provided to the App to access the API, and the API to access SQL - in both cases under the user's (i.e. my) identity.
When I publish the API to App Services on Azure, however, and access it there either from a local version of the Web App or an App-Services hosted version of it, the access token that the API gets to access SQL contains the API's Application Identity (system-assigned managed identity), and not the user's identity. Although I can access SQL as the application, it's not what we need.
The Web App obtains its access token using the GetAccessTokenForUserAsync method of ITokenAcquisition - taking as a parameter the single scope I defined for the API.
The API gets its token (to access SQL) like so:
var token = await new AzureServiceTokenProvider().GetAccessTokenAsync("https://database.windows.net", _tenantId)
...where _tenantId is the tenant ID of the subscription.
I have added the SQL Azure Database "user_impersonation" API permission to the AD registration for the API - but that has not helped. As an aside, for some reason Azure gives the full name of this permission as https://sql.azuresynapse.usgovcloudapi.net/user_impersonation - which is slightly alarming as this is just a UK-based regular Azure account.
I have found a few similar posts to this, but mostly for older versions of the solution set. I'm hoping to avoid having to write my own code to post the token requests - this is supposed to be handled by the MSAL libraries.
Should I somehow be separately requesting a SQL access scope from the Web App after sign-in, or should the API be doing something different to get hold of a SQL access token that identifies the user? Why does it work perfectly when running locally?
It seems like this should be a very common use case (the most common?) but it is barely documented - most documentation I've found refers only to the application identity being used or doesn't tell you what to do for this particular tech stack.
Finally - success! In the end this was the critical piece of documentation: Microsoft identity platform and OAuth 2.0 On-Behalf-Of flow - the key points being:
The App only asks for a token to access the API.
The API then requests a token, on behalf of the user identified via the 1st token, to access SQL.
The key is that - since the API cannot trigger a consent window for the second step - I had to use the Enterprise Applications tab in the Azure portal to pre-grant the permissions for SQL.
So the good news is it does work: maybe it's obvious to some but IMO it took me far too long to find the answer to this. I will write up a fuller explanation of how to do this in due course as it can't only be me struggling with this one.
The bad news is that - in the course of my investigations - I found that Azure B2C (which is the next thing I need to add in) doesn't support this "On Behalf Of" flow - click here for details. That's a great shame as I think it's the most obvious use case for it! Oh well, back to the drawing board.
I'm currently working on a similar problem, using a Net5.0 Web app. The reason it appears to be working locally is you are signed into Visual Studio with a user who can access Azure SQL and those are the rights you get in the Db. The IDE is using those credentials in place of the Managed Service Identity, the latter gets used when you upload the app to Azure.
As you noted, in the App registration you need to grant permission to the App for Azure SQL Database user_impersonation.
In your code, you need to request a token from https://database.windows.net//.default (note the // as it's needed for v1 endpoints). By referencing /.default you are asking for all permissions you've selected for the app in the app registration portal.
https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-permissions-and-consent#the-default-scope
In Startup.cs you need to EnableTokenAcquisitionToCallDownstreamApi with the scope you require.
services.AddMicrosoftIdentityWebAppAuthentication(Configuration)
.EnableTokenAcquisitionToCallDownstreamApi(new[]
{"https://database.windows.net//.default"})
// Adds the User and App InMemory Token Cache
.AddInMemoryTokenCaches();
services.AddAuthorization(options =>
{
// By default, all incoming requests will be authorized according to the
// default policy
options.FallbackPolicy = options.DefaultPolicy;
});
services.AddDbContext<MyDatabaseContext>(options =>
options.UseSqlServer(
Configuration.GetConnectionString("MyAzureConnection")));
// The database interface
services.AddScoped<ITodos, TodoData>();
services.AddRazorPages()
.AddRazorRuntimeCompilation()
.AddMvcOptions(o =>
{
var policy = new AuthorizationPolicyBuilder()
.RequireAuthenticatedUser()
.Build();
o.Filters.Add(new AuthorizeFilter(policy));
})
.AddMicrosoftIdentityUI();
You also need to decorate your controllers with [AuthorizeForScopes(Scopes = new string[]{"https://database.windows.net//.default"}] and include the required scopes for that Controller. For Razor, it's at the top of the page model and requires a reference to `using Microsoft.Identity.Web;'
namespace ToDoApp.Pages.Todos
{
[AuthorizeForScopes(ScopeKeySection = "AzureSQL:BaseUrl")]
public class CreateModel : PageModel
I'm using a section in my appsettings.json for the scope and retrieving it using ScopeKeySection:
"AzureSQL": {
"BaseUrl": "https://database.windows.net//.default",
"Scopes": "user_impersonation"
}
This shows you where to include it for MVC, Razor and Blazor:
https://github.com/AzureAD/microsoft-identity-web/wiki/Managing-incremental-consent-and-conditional-access#in-mvc-controllers
Finally, your DbContext needs a token which you could pass to it from the client app (perhaps...).
This is how I am doing it at the moment
public class MyDatabaseContext : DbContext
{
private readonly ITokenAcquisition _tokenAcquisition;
public MyDatabaseContext (ITokenAcquisition tokenAcquisition,
DbContextOptions<MyDatabaseContext> options)
: base(options)
{
_tokenAcquisition = tokenAcquisition;
string[] scopes = new[]{"https://database.windows.net//.default"};
var result = _tokenAcquisition.GetAuthenticationResultForUserAsync(scopes)
.GetAwaiter()
.GetResult();
token = result.AccessToken;
var connection = (SqlConnection)Database.GetDbConnection();
connection.AccessToken = result.token;
}
This is a flawed solution. If I restart the app and try to access it again I get an error Microsoft.Identity.Web.MicrosoftIdentityWebChallengeUserException: IDW10502: An MsalUiRequiredException was thrown due to a challenge for the user
It seems to be related to the TokenCache. If I sign out and in again or clear my browser cache the error is resolved. I've a workaround that signs the app in on failure, but it's deficient since I'm using the app's credentials.
However, it successfully connects to the Azure SQL Db as the user instead of the App with the user's rights instead. When I do solve the error (or find one) I will update this answer.

OWIN/OAuth2 3rd party login: Authentication from Client App, Authorization from Web API

I am trying to create a Web API that allows the API's clients (native mobile apps) to login using a 3rd party cloud storage provider. I'm using the following general flow from Microsoft:
Here is what I am trying to achieve:
I am using the default ASP.NET Web API Visual Studio template with external authentication, along with the OWin.Security.Providers Nuget package for Dropbox login functionality, and the existing built-in login functionality for Google (Drive) and Microsoft (OneDrive).
The issue I'm having is that the built-in functionality all seems to do the authentication and authorization as part of one flow. For example, if I set up the following in Startup.Auth.cs:
DropboxAuthenticationOptions dropboxAuthOptions = new DropboxAuthenticationOptions
{
AppKey = _dropboxAppKey,
AppSecret = _dropboxAppSecret
};
app.UseDropboxAuthentication(dropboxAuthOptions);
... and navigate to this url from my web browser:
http://<api_base_url>/api/Account/ExternalLogin?provider=Dropbox&response_type=token&client_id=self&redirect_uri=<api_base_url>
I am successfully redirected to Dropbox to login:
https://www.dropbox.com/1/oauth2/authorize?response_type=code&client_id=<id>&redirect_uri=<redirect_uri>
... and then after I grant access, am redirected back to:
http://<api_base_url>/Help#access_token=<access_token>&token_type=bearer&expires_in=1209600
... as you can see the token is part of that, so could be extracted. The problem is that the client needs to be the one navigating to Dropbox and returning the authorization code back up to the Web API, and the Web API would send the authorization code back to the third party to get the token which would then be returned to the client... as shown in the diagram above. I need the ExternalLogin action in the AccountController to somehow retrieve the Dropbox url and return that to the client (it would just be a json response), but I don't see a way to retrieve that (it just returns a ChallengeResult, and the actual Dropbox url is buried somewhere). Also, I think I need a way to separately request the token from the third party based on the authorization code.
This post seems a little similar to what I am trying to do:
Registering Web API 2 external logins from multiple API clients with OWIN Identity
... but the solution there seems to require the client to be an MVC application, which is not necessarily the case for me. I want to keep this as simple as possible on the client side, follow the flow from my diagram above, but also not reinvent the wheel (reuse as much as possible of what already exists in the OWIN/OAuth2 implementation). Ideally I don't want the client to have to reference any of the OWIN/OAuth libraries since all I really need the client to do is access an external url provided by the API (Dropbox in my example), have the user input their credentials and give permission, and send the resulting authorization code back up to the api.
Conceptually this doesn't sound that hard but I have no idea how to implement it and still use as much of the existing OAuth code as possible. Please help!
To be clear, the sample I mentioned in the link you posted CAN be used with any OAuth2 client, using any supported flow (implicit, code or custom). When communicating with your own authorization server, you can of course use the implicit flow if you want to use JS or mobile apps: you just have to build an authorization request using response_type=token and extract the access token from the URI fragment on the JS side.
http://localhost:55985/connect/authorize?client_id=myClient&redirect_uri=http%3a%2f%2flocalhost%3a56854%2f&response_type=token
For reference, here's the sample: https://github.com/aspnet-security/AspNet.Security.OpenIdConnect.Server/tree/dev/samples/Mvc/Mvc.Server
In case you'd prefer a simpler approach (that would involve no custom OAuth2 authorization server), here's another option using the OAuth2 bearer authentication middleware and implementing a custom IAuthenticationTokenProvider to manually validate the opaque token issued by Dropbox. Unlike the mentioned sample (that acts like an authorization proxy server between Dropbox and the MVC client app), the JS app is directly registered with Dropbox.
You'll have to make a request against the Dropbox profile endpoint (https://api.dropbox.com/1/account/info) with the received token to validate it and build an adequate ClaimsIdentity instance for each request received by your API. Here's a sample (but please don't use it as-is, it hasn't been tested):
public sealed class DropboxAccessTokenProvider : AuthenticationTokenProvider {
public override async Task ReceiveAsync(AuthenticationTokenReceiveContext context) {
using (var client = new HttpClient()) {
var request = new HttpRequestMessage(HttpMethod.Get, "https://api.dropbox.com/1/account/info");
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", context.Token);
var response = await client.SendAsync(request);
if (response.StatusCode != HttpStatusCode.OK) {
return;
}
var payload = JObject.Parse(await response.Content.ReadAsStringAsync());
var identity = new ClaimsIdentity("Dropbox");
identity.AddClaim(new Claim(ClaimTypes.NameIdentifier, payload.Value<string>("uid")));
context.SetTicket(new AuthenticationTicket(identity, new AuthenticationProperties()));
}
}
}
You can easily plug it via the AccessTokenProvider property:
app.UseOAuthBearerAuthentication(new OAuthBearerAuthenticationOptions {
AccessTokenProvider = new DropboxAccessTokenProvider()
});
It has its own downsides: it requires caching to avoid flooding the Dropbox endpoint and is not the right way to go if you want to accept tokens issued by different providers (e.g Dropbox, Microsoft, Google, Facebook).
Not to mention that if offers a very low security level: since you can't verify the audience of the access token (i.e the party the token was issued to), you can't ensure that the access token was issued to a client application you fully trust, which allows any third party developer to use his own Dropbox tokens with your API without having to request user's consent.
This is - obviously - a major security concern and that's why you SHOULD prefer the approach used in the linked sample. You can read more about confused deputy attacks on this thread: https://stackoverflow.com/a/17439317/542757.
Good luck, and don't hesitate if you still need help.

NetworkCredentials and Authorization in WebApi

I am having a few problems trying to connect to a ASP.NET webapi service (which I am running myself) from a sample console app using WebClient. The webapi is the typical sample site from MVC4:
public HttpResponseMessage Get()
{
return Request.CreateResponse(HttpStatusCode.OK, new string[] { "value1", "value2" });
}
The Controller is decorated with a custom Authenticate attribute:
public override void OnAuthorization(System.Web.Http.Controllers.HttpActionContext actionContext)
{
if (actionContext.Request.Headers.Authorization == null)
{
var response = new HttpResponseMessage(HttpStatusCode.Unauthorized);
response.Headers.Add("WWW-Authenticate", "Basic realm=\"localhost\"");
actionContext.Response = response;
return;
}
}
The client code is the usual:
var wb = WebRequest.Create("http://localhost:64921/Values");
wb.Credentials = new NetworkCredential("xxx", "xxx");
var aaa = wb.GetResponse();
Console.WriteLine(aaa);
Console.ReadLine();
Now, I know that the WebClient or WebRequest are supposed to wait for a 401 before sending credentials and that is exactly what I am trying to do here.
Needless to say with the setup above nothing works. I have gone into the IIS express config and changed the following:
<basicAuthentication enabled="true" /> (in the security section)
<add name="BasicAuthenticationModule" lockItem="false" /> (in the modules section)
The problem that I am having is that the 401 gets returned even before the server code is actualy hit. I mean that if I stick a breakpoint into the Controller or the Attribute they are not hit. The details of the error are the usual long text about error 401.2 which I reckon is something to do with IIS configs, but using IIS express and not the nice IIS I do not have a nice GUI to fix this. Can anyone help?
Thanks a lot!
In the IIS config, you have enabled Basic auth processing, so IIS returns the 401 if there are no credentials or the credentials are invalid.
If you want your code to do the basic auth processing, then you need to tell IIS to allow anonymous access.
EDIT from comments
If you ask IIS to do basic auth it will check credentials against Windows accounts. This will act before the server code runs, so the Custom Auth Filter will not be hit. In this case the headers returned will be correct and you will see the WebClient performing the double request (one anonymous, one with credentials). If the WebClient does not use a computer or domain account (with read permissions on the folder where the site is located), the request will fail.
If you want to do authentication/authorization yourself, you need to tell IIS express not to do any auth and then do it all yourself... this basically means leaving everything as it is in the config (in your case reverting the pieces of config shown in the question) and sending the correct headers, which you already do. If you debug, you will see the Authenticate filter being hit twice, the first time it will be an anonymous that will go inside the if and generate your HTTP 401 Challenge response, the second time it will have credentials in the form of a standard Basic Authorization header: Basic <BASE64_ENCODED_CREDENTIALS>