Why do I see "invalid client" in the IdentityServer4 logs when using a Microsoft Account external login? - authentication

Question

I have an IdentityServer4 implementation that has a default internal OIDC OpenIdConnect middleware to provide access to local APIs and support for external login providers, starting with Microsoft. I've implemented a custom IClientStore and attach it to the middleware with
.AddClientStore<IS4ClientStore>();
When the internal OIDC implicit connection auths, IS4ClientStore.FindClientByIdAsync is called by BuildLoginViewModelAsync like the example with
_clientStore.FindEnabledClientByIdAsync(context.ClientId);
which returns the expected IdentityServer4.Client and everything authenticates, etc.
When I call the external provider (Microsoft OAuth), it goes through the auth process connecting to my supplied applicationid, goes through the claims process by Microsoft and then Microsoft redirects back to https://localhost:5001/signin-microsoft.
The asp.net core middleware from calling services.AddAuthentication().AddMicrosoftAccount(options) in startup isn't using my client store to look up the client id, it just reports the error:
Request starting HTTP/1.1 GET https://localhost:5001/signin-microsoft?code=M91ae2ef1-701d-ceb4-d479-a905d3d02a4d&state=CfDJ8GGNLFmYOI9KouhmbB3NzeJ5omhARPg-YVCPW7u1aCyhnGfOx9_Nj4VL8cMIxmO48nk_8UkfB9Pv7Q7tzZZb8nsq5y26giY9fXuVyRsn5qx8a1nSX8tKFWk1uo9ongL5V0MXY6sgU6eNUEzsxgyNFz_20QLVU20y9G7jRpmxoOcpQ1s1SJx0Tu2BBlRrI840-D-jUmg1ix7xDUfmXF_rPVp6e88rzIuCfbQO4otNq2fAsm4
info: Microsoft.AspNetCore.Authentication.MicrosoftAccount.MicrosoftAccountHandler[4]
Error from RemoteAuthentication: OAuth token endpoint failure: Status: BadRequest;Headers: Cache-Control: no-cache, no-store
Pragma: no-cache
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
x-ms-request-id: 759b3046-cbad-489d-98c1-6e83bb390b00
P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
Set-Cookie: fpc=AZFdz4kBW-ZHi5twlc6DCrARHQUSAQDQZKAdWIfWCA; expires=Sat, 02-Mar-2019 08:43:04 GMT; path=/; secure; HttpOnly, x-ms-gateway-slice=prod; path=/; secure; HttpOnly, stsservicecookie=ests; path=/; secure; HttpOnly
Date: Thu, 31 Jan 2019 08:43:04 GMT
;Body: {"error":"unauthorized_client","error_description":"AADSTS700016: Application with identifier '000000005D256100' was not found in the directory '9188040d-6c67-4c5b-b112-36a304b66dad'. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You may have sent your authentication request to the wrong tenant\r\nTrace ID: 759b3046-cbad-489d-98c1-6e83bb390b00\r\nCorrelation ID: 858c340d-7c0c-4fa9-86c2-f8417e5915f1\r\nTimestamp: 2019-01-31 08:43:04Z","error_codes":[700016],"timestamp":"2019-01-31 08:43:04Z","trace_id":"759b3046-cbad-489d-98c1-6e83bb390b00","correlation_id":"858c340d-7c0c-4fa9-86c2-f8417e5915f1"};.
fail: Microsoft.AspNetCore.Server.Kestrel[13]
Connection id "0HLK7ARJEPT98", Request id "0HLK7DAJEPT98:00000005": An unhandled exception was thrown by the application.
System.Exception: An error was encountered while handling the remote login. ---> System.Exception: OAuth token endpoint failure: Status: BadRequest;Headers: Cache-Control: no-cache, no-store
Pragma: no-cache
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
x-ms-request-id: 759b3046-cbad-489d-98c1-6e83bb390b00
P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
Set-Cookie: fpc=AZFdz4kBW-ZHi5twlc6DCrARHQUSAQDQZKAdWIfWCA; expires=Sat, 02-Mar-2019 08:43:04 GMT; path=/; secure; HttpOnly, x-ms-gateway-slice=prod; path=/; secure; HttpOnly, stsservicecookie=ests; path=/; secure; HttpOnly
Date: Thu, 31 Jan 2019 08:43:04 GMT
;Body: {"error":"unauthorized_client","error_description":"AADSTS700016: Application with identifier '000000005D256100' was not found in the directory '9188040d-6c67-4c5b-b112-36a304b66dad'. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You may have sent your authentication request to the wrong tenant\r\nTrace ID: 759b3046-cbad-489d-98c1-6e83bb390b00\r\nCorrelation ID: 858c340d-7c0c-4fa9-86c2-f8417e5915f1\r\nTimestamp: 2019-01-31 08:43:04Z","error_codes":[700016],"timestamp":"2019-01-31 08:43:04Z","trace_id":"759b3046-cbad-489d-98c1-6e83bb390b00","correlation_id":"858c340d-7c0c-4fa9-86c2-f8417e5915f1"};
--- End of inner exception stack trace ---
at Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler`1.HandleRequestAsync()
at IdentityServer4.Hosting.FederatedSignOut.AuthenticationRequestHandlerWrapper.HandleRequestAsync() in C:\local\identity\server4\IdentityServer4\src\Hosting\FederatedSignOut\AuthenticationRequestHandlerWrapper.cs:line 38
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at IdentityServer4.Hosting.BaseUrlMiddleware.Invoke(HttpContext context) in C:\local\identity\server4\IdentityServer4\src\Hosting\BaseUrlMiddleware.cs:line 36
at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
Request finished in 476.3734ms 500
Am I supposed to use an implementation of AddMicrosoftAccount from IdentityServer4?

Answer 1

Thanks to Jim & Mackie to point me in the right direction (I thought it was due to a custom store implementation issue on my side)
When I was verifying I had the right clientid and secret on the https://apps.dev.microsoft.com/#/appList page, I noticed a link to go to the azure portal to manage my apps. (https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/applicationsListBlade) When I went to that link, the previous apps I had didn't show, so I just created a new app and got a new ApplicationId (which was now a guid instead of in through the portal) and Secret, plugged that into the .AddMicrosoftAccount call and I was able to go through the Auth process through microsoft login and get a token to finish the auth on my server.
My guess is that some percentage of applications that were added through the apps.dev.microsoft.com site aren't in Microsoft's backing store when finishing the auth process, the applications may be in the front-end query and allow the auth pages to find your application and provide the consent page, but then when issuing the token, the Microsoft side is looking in a different store and not finding the application by the ApplicationId provided through the apps.dev.microsoft.com page. So the solution, at least in my case, was to create the application in the azure portal instead.