Authentication is not enable for Mesos APIs by default.
After Install DCOS I want config Mesos API Authentication on it.
I'm going to set authentication for mesos APIs like : register_frameworks, run_task,...
the problem is after my configuration, DCOS GUI and marathon dosent work correctly.
I configured DCOS as follow:
Mesos environment variable config:
path:/opt/mesosphere/etc/mesos-master
#Authentication part
MESOS_LOG_DIR=/var/log/mesos
#Framework authentication
MESOS_AUTHENTICATORS="crammd5"
MESOS_AUTHENTICATE_FRAMEWORKS=true
MESOS_AUTHENTICATE_HTTP_FRAMEWORKS=true
MESOS_HTTP_FRAMEWORK_AUTHENTICATORS="basic"
MESOS_ACLS=/opt/mesosphere/etc/acls
MESOS_AUTHENTICATE=true
MESOS_CREDENTIALS=/opt/mesosphere/etc/mesos_credentials_auth.json
MESOS_ROLE=foo
Marathon environment variable config:
path:/opt/mesosphere/marathon
#authentication section
MARATHON_MESOS_AUTHENTICATION=enabled
#MARATHON_HTTP_CREDENTIALS=marathon:123456
MARATHON_MESOS_AUTHENTICATION_PRINCIPAL=marathon
MARATHON_MESOS_ROLE=foo
MARATHON_MESOS_AUTHENTICATION_SECRET_file=/opt/mesosphere/etc/marathon.secret
Marathon environment variable config: path:/opt/mesosphere/metronome
METRONOME_MESOS_AUTHENTICATION_ENABLED=true
METRONOME_MESOS_AUTHENTICATION_PRINCIPAL=metronome
METRONOME_MESOS_ROLE=foo
METRONOME_MESOS_AUTHENTICATION_SECRET_FILE= /opt/mesosphere/etc/metronome.secret
/opt/mesosphere/etc/metronome.secret (contain metronome secret without new line)
123456
/opt/mesosphere/etc/marathon.secret (contain marathon secret without new line)
123456
/opt/mesosphere/etc/acls
{
"run_tasks": [
{
"principals": {
"type": "ANY"
},
"users": {
"type": "ANY"
}
}
],
"register_frameworks": [
{
"principals": {
"type": "ANY"
},
"roles": {
"type": "ANY"
}
}
]
}
/opt/mesosphere/etc/mesos_credentials_auth.json
{
"credentials" : [
{
"principal": "principal1",
"secret": "secret1"
},
{
"principal": "principal2",
"secret": "secret2"
},
{
"principal": "marathon",
"secret": "123456"
},
{
"principal": "metronome",
"secret": "123456"
}
]
}
When I enable this configuration and stop and start dcos-mesos-master:
systemctl stop dcos-mesos-master.service
systemctl start dcos-mesos-master.service
systemctl stop dcos-marathon.service
systemctl start dcos-marathon.service
systemctl stop dcos-metronome.service
systemctl start dcos-metronome.service
http://IP/services page in DCOS dosnt work. I think its beacuase authentication of marathon don't set correctly. bcs this address dosent work after enable authentication configuration:\
http://IP/service/marathon/v2/deployments?_timestamp=1560449507192
I got this errors in mesos log after enable metronome authentication:
I0613 17:35:12.176092 305 authenticator.cpp:98] Creating new server
SASL connection
I0613 17:35:12.177258 304 master.cpp:10255] Re-authenticating
scheduler-aca98ea7-be34-49d1-9200-5ef8c15da153#172.17.0.2:15201;
discarding outstanding authentication
I0613 17:35:12.177523 304 master.cpp:10285] Ignoring stale
authentication result of scheduler-aca98ea7-be34-49d1-9200-
5ef8c15da153#172.17.0.2:15201
I0613 17:35:12.177582 304 authenticator.cpp:98] Creating new server
SASL connection
I0613 17:35:12.178586 302 master.cpp:10255] Re-authenticating
scheduler-aca98ea7-be34-49d1-9200-5ef8c15da153#172.17.0.2:15201;
discarding outstanding authentication
I0613 17:35:12.178850 302 master.cpp:10285] Ignoring stale
authentication result of scheduler-aca98ea7-be34-49d1-9200-
5ef8c15da153#172.17.0.2:15201
After search, finally I got my answer:
These security features are only available on "DC/OS mesosphere Enterprise" and you cant config it in open source version.
also I opened github issue with more details:(I hope it will be usefull)
https://github.com/mesosphere/marathon/issues/6942
Related
I am using AspNetCoreRateLimit version 4.0.1 and I have done all the setup in .net core 6 web api. I can see rate limit is working when I send a call via postman.
However, when I add IpRateLimitPolicies with specific IP address, the settings won't be applied.
I use postman and this time in the proxy I added the ip address to 127.0.0.1. I can see the ip hitting the api is set correctly when I use Request.HttpContext.Connection.RemoteIpAddress;
I registered them as follow in program.cs:
_serviceCollection.AddOptions();
_serviceCollection.AddMemoryCache();
_serviceCollection.Configure<IpRateLimitOptions>(builder.Configuration.GetSection("IpRateLimiting"));
_serviceCollection.Configure<IpRateLimitPolicies>(builder.Configuration.GetSection("IpRateLimitPolicies"));
_serviceCollection.AddInMemoryRateLimiting();
_serviceCollection.AddSingleton<IIpPolicyStore, MemoryCacheIpPolicyStore>();
_serviceCollection.AddSingleton<IRateLimitCounterStore, MemoryCacheRateLimitCounterStore>();
_serviceCollection.AddSingleton<IHttpContextAccessor, HttpContextAccessor>();
_serviceCollection.AddSingleton<IRateLimitConfiguration, RateLimitConfiguration>();
Also added:
app.UseIpRateLimiting();
My appsettings also looks like:
{
"IpRateLimiting": {
"EnableEndpointRateLimiting": false,
"StackBlockedRequests": false,
"RealIPHeader": "X-Real-IP",
"ClientIdHeader": "X-ClientId",
"IpWhitelist": [ ],
"EndpointWhitelist": [],
"ClientWhitelist": [],
"HttpStatusCode": 429,
"GeneralRules": [
{
"Endpoint": "*",
"Period": "10s",
"Limit": 1
}
]
},
"IpRateLimitPolicies": {
"IpRules": [
{
"Ip": "127.0.0.1",
"Rules": [
{
"Endpoint": "*",
"Period": "20s",
"Limit": 2
}
]
}
]
}
But apparently the settings under IpRateLimitPolicies won't be applied.
I wonder if I have missed anything here?
Thank you
After testing, I think the AspNetCoreRateLimit package is not compatible in .net5 and .net6. Maybe the .net core3.1 version will be more stable.
You can submit issues on github.
Apparently I have missed some configuration in program.cs or startup.cs:
https://github.com/stefanprodan/AspNetCoreRateLimit/issues/305
Since we are using startup I have added the following in Configure method
var ipPolicyStore = app.ApplicationServices.GetRequiredService();
ipPolicyStore.SeedAsync().GetAwaiter().GetResult();
Hi I was trying to implement ocelot for our experimental tests on dev.
Here is end-point of api that I want to reach by via ocelot. using 443 port for both of project.
but getting 502 bad gateway all the time.
end point => https://localhost/document/api/v1/Documents/XYZ
"ReRoutes": [
{
"DownstreamPathTemplate": "/document/api/v1/Documents/{name}",
"DownstreamScheme": "https",
"DownstreamHostAndPorts": [
{
"Host": "localhost",
"Port": 443
}
],
"UpstreamPathTemplate": "/apigateway/{name}/document",
"UpstreamHttpMethod": [ "Post" ],
"Priority": 0
}
],
"GlobalConfiguration": {
"BaseUrl": "https://localhost:443"
}
}
Microgateway alias name =>"apigateway"
Api alias name => "document"
In addition this I was able to debug on visiual studio but whenever I host both app on my local IIS getting 502 bad gateway
It appears that the configuration you have used is redirecting the request to the gateway itself, resulting in a circle.
i.e. the upstream call to the base URL of "localhost:443" is redirecting to the downstream "localhost:443" - the same.
Furthermore, the later versions of Ocelot appear to look for Routes in the configuration instead of ReRoutes documentation
I started a Multi-Host Fabric Network usind docker swarm made up of 1 CA-server, 1 Orderer, 2 Peers (both in Org1, one on PC1 and one on PC2), 2 CouchBD (one for each Peer) with fabric-sdk-rest running on PC2.
Now if I disable TLS in the Fabric Network, everything works fine. But if i enable the TLS in the network, the SDK cannot connect to the peers failing to query.
Here I show the configuration of the network and the fabric-sdk-rest:
(crypto-config.yaml)
OrdererOrgs:
- Name: Orderer
Domain: example.com
Specs:
- Hostname: orderer
PeerOrgs:
- Name: Org1
Domain: org1.example.com
Template:
Count: 2
Users:
Count: 0
(datasources.json)
{
"db": {
"name": "db",
"connector": "memory"
},
"fabricDataSource": {
"name": "fabricDataSource",
"connector": "fabric",
"keyStoreFile": "/tmp/fabricSDKStore",
"fabricUser": {
"username": "Admin#org1.example.com",
"mspid": "Org1MSP",
"cryptoContent": {
"privateKey":"$HOME/mynetwork/crypto-config/peerOrganizations/org1.example.com/users/Admin#org1.example.com/msp/keystore/KEY_sk",
"signedCert":"$HOME/mynetwork/crypto-config/peerOrganizations/org1.example.com/users/Admin#org1.example.com/msp/signcerts/Admin#org1.example.com-cert.pem"
}
},
"COMMENT_orgs":"Referenced by peers to avoid having to configure the same file location multiple times. Change CACertFile locations for your fabric",
"orgs": [
{ "name":"org1", "CACertFile":"$HOME/mynetwork/crypto-config/peerOrganizations/org1.example.com/ca/ca.org1.example.com-cert.pem"}
],
"COMMENT_peers" : "Configured array is for use with the fabric-sample when running it in a local docker set up. eventURL and publicCertFile not currently used.",
"peers": [
{ "requestURL":"grpcs://peer1.org1.example.com:7051", "eventURL":"grpcs://peer1.org1.example.com:7053", "orgIndex":"0", "publicCertFile":"$HOME/mynetwork/crypto-config/peerOrganizations/org1.example.com/peers/peer1.org1.example.com/msp/signcerts/peer1.org1.example.com-cert.pem", "hostname":"peer1" }
],
"COMMENT_peers_secure" : "UNUSED. This is a copy of the above with grpcs URLs. Replace peers content with this if grpcs urls are needed.",
"peers-secure": [
{ "requestURL":"grpcs://peer1.org1.example.com:7051", "eventURL":"grpcs://peer1.org1.example.com:7053", "orgIndex":"0", "publicCertFile":"$HOME/mynetwork/crypto-config/peerOrganizations/org1.example.com/peers/peer1.org1.example.com/msp/signcerts/peer1.org1.example.com-cert.pem", "hostname":"peer1" }
],
"orderers": [
{ "url":"grpcs://orderer.example.com:7050", "CACertFile":"$HOME/mynetwork/crypto-config/ordererOrganizations/example.com/ca/ca.example.com-cert.pem", "publicCertFile": "$HOME/mynetwork/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/signcerts/orderer.example.com-cert.pem", "hostname":"orderer"}
],
"COMMENT_orderers_secure" : "UNUSED. This is a copy of the above with grpcs URLs. Replace orderers content with this if grpcs urls are needed.",
"orderers-secure": [
{ "url":"grpcs://orderer.example.com:7050", "CACertFile":"$HOME/mynetwork/crypto-config/ordererOrganizations/example.com/ca/ca.example.com-cert.pem", "publicCertFile": "$HOME/mynetwork/crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/signcerts/orderer.example.com-cert.pem", "hostname":"orderer"}
],
"COMMENT_channels":"fabric-sdk-node Client class requires channel information to be configured during bootstrap.",
"channels": [
{ "name":"mychannel", "peersIndex":[0], "orderersIndex":[0] }
],
"channels-first-network": [
{ "name":"mychannel", "peersIndex":[0,1,2,3], "orderersIndex":[0] }
]
}
}
Once started the Hyperledger Fabric SDK REST server at https://0.0.0.0:3000, when I try to make the GET channels query from the explorer, I get the following error:
error: [fabricconnector.js]: Failed to queryChannels: Error: 14 UNAVAILABLE: Connect Failed
Error not handled for the GET request /api/fabric/1_0/channels: Error: 14 UNAVAILABLE: Connect Failed
at Object.exports.createStatusError ($HOME/mynetwork/fabric-sdk-rest/packages/loopback-connector-fabric/node_modules/grpc/src/common.js:87:15)
at Object.onReceiveStatus ($HOME/mynetwork/fabric-sdk-rest/packages/loopback-connector-fabric/node_modules/grpc/src/client_interceptors.js:1214:28)
at InterceptingListener._callNext ($HOME/mynetwork/fabric-sdk-rest/packages/loopback-connector-fabric/node_modules/grpc/src/client_interceptors.js:590:42)
at InterceptingListener.onReceiveStatus ($HOME/mynetwork/fabric-sdk-rest/packages/loopback-connector-fabric/node_modules/grpc/src/client_interceptors.js:640:8)
at callback ($HOME/mynetwork/fabric-sdk-rest/packages/loopback-connector-fabric/node_modules/grpc/src/client_interceptors.js:867:24)
E0510 10:51:04.780559355 12247 ssl_transport_security.cc:989] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
Has anyone ever seen this error? Can anyone help me get through this, please?
I am trying to get started with a monitoring server solution. I got the Sensu Clients, RabbitMQ and Uchiwa configured but then I tried using Graphite but there were so many parts to configure I tried InfluxDB instead. I am stuck configuring Sensu to InfluxDB.
Is there a part missing in the below configuration?
Client [Sensu] > RabbitMQ <> Sensu Server <> InfluxDB <> Grafana
Any suggestions?
cat influx.json
{
"influxdb": {
"hosts" : ["192.168.1.1"],
"host" : "192.168.1.1",
"port" : "8086",
"database" : "sensumetrics",
"time_precision": "s",
"use_ssl" : false,
"verify_ssl" : false,
"initial_delay" : 0.01,
"max_delay" : 30,
"open_timeout" : 5,
"read_timeout" : 300,
"retry" : null,
"prefix" : "",
"denormalize" : true,
"status" : true
}
}
cat handler.json
{
"handlers": {
"influxdb": {
"type": "pipe",
"command": "/opt/sensu/embedded/bin/metrics-influxdb.rb"
}}}
checks1,
{
"checks": {
"check_memory_linux": {
"handlers": ["influxdb","default"],
"command": "/opt/sensu/embedded/bin/check-memory-percent.rb -w 90 -c 95",
"interval": 60,
"occurrences": 5,
"subscribers": [ "TEST" ]
}}}
checks2,
{
"checks": {
"check_cpu_linux-elkctrl-pipe": {
"type": "metric",
"command": "/opt/sensu/embedded/bin/check-cpu.rb -w 80 -c 90",
"subscribers": ["TEST"],
"interval": 10,
"handlers": ["debug","influxdb"]
}}}
To use InfluxDB to persist your data, you must have:
InfluxDB plugin installed (also, installation and usage instructions here)
Definitions for the plugin (an influxdb.json containin at least the host, port, user, password and database to be used by Sensu)
The definition, as other config files, must be in /etc/sensu/conf.d/
Handler configuration set properly (also in conf.d)
Mutator for InfluxDB (extensions)
Your checks must send results to the handler, so their definition must contain:
"handlers": [
"influxdb"
]
Or whatever name you gave your handler.
Case, if the influxdb config you provided above is the full extent of your configuration, it would seem to be missing the username/password attributes required by the influxdb configuration. If they're present, but not provided in the post, no big deal. However, I'd recommend doing the following for your Sensu logs:
grep -i influxdb /var/logs/sensu/sensu-server.log
And seeing if the check result is getting sent to your influxdb instance. If they are, you should be receiving an error that might be pointing a bit more to what's going on.
You can also check your influxdb logs to see if they're getting a post from your Sensu server:
journalctl -u influxdb.service -f
But yeah, if the username/password is missing from the configuration, that'd be the first place that I start.
Can’t find any resources that simply say here’s where your cert goes and here’s how to enable it. I have the cert there when I run gcloud compute ssl-certificates list. I have a cluster with kubernetes running and exposing http traffic via this service:
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "foo-frontend-service"
},
"spec": {
"selector": {
"app": "foo-frontend-rc"
},
"ports": [
{
"protocol": "TCP",
"port": 80,
"targetPort": 3009
}
]
}
}
Need to know how to put the cert in the right place to be utilized
Need to know how to reconfigure my service
Need to know what my new SSL endpoint will be. Is it the same?
K8s doesn't have special TLS support for the ordinary services. You need to use one of the following methods:
using Ingress: see http://kubernetes.io/docs/user-guide/ingress/#tls. You need to choose a Ingress controller which implements the Ingress functionalities, you can use GLBC if you are on GCE, or you can use the nginx one. Both of them supports TLS. Please note that the Ingress is still beta feature with limitations.
The service-loadbalancer in the contrib repo also supports tls: https://github.com/kubernetes/contrib/tree/master/service-loadbalancer#ssl-termination