I am trying to connect Knative with an existing RabbitMQ queue outside the Kubernetes cluster, but after testing and looking at the documentation I am not sure this can be done, does anyone know how to use Knative-eventing with an external RabbitMQ as a message source ?
Regards!
Hey #Juan you were absolutely right about our docs, here I'll leave a PR fixing part of them while it's being reviewed and merged, and with a clear example about using external RabbitMQ instances:
https://github.com/knative-sandbox/eventing-rabbitmq/pull/786/files#diff-4fdb9e4eb3a1c9da58e4445d94aa5ce4573b5c8d005f20c41c767b07c09a2418
Hope this helps =), and thanks for the feedback!
If you find anything wrong, you can comment on the PR or leave a reply over here!
As listed in the Knative Eventing sources catalog, there is a RabbitMQ source being developed and maintained by the community (largely contributors from VMware and the RabbitMQ team).
Documentation and install instructions are here; in the case of bugs, please report them in the associated repo.
thank you #sameer for your answer, I think I am not understanding you correctly.
After visiting the link you give me, I have launched:
kubectl apply --filename https://github.com/knative-sandbox/eventing-rabbitmq/releases/latest/download/rabbitmq-source.yaml
To install the rabbitmq source and generated the secret:
kubectl create secret generic rabbitmqc-default-user -n knative-eventing --from-literal=user=root --from-file=password=/tmp/password
And the following resource:
apiVersion: sources.knative.dev/v1alpha1
kind: RabbitmqSource
metadata:
name: rabbitmq-source
namespace: knative-eventing
spec:
broker: "host.external.dns:5671/"
connectionSecret:
name: "rabbitmqc-default-user"
user:
secretKeyRef:
name: "rabbitmqc-default-user"
key: "username"
password:
secretKeyRef:
name: "rabbitmqc-default-user"
key: "password"
exchangeConfig:
name: "logs"
type: "fanout"
durable: true
autoDelete: false
queueConfig:
name: "test"
routingKey: ""
durable: false
autoDelete: false
channelConfig:
parallelism: 10
sink:
ref:
apiVersion: v1
kind: Service
name: recorder
But after generating the resources I get these errors:
knative-sources/rabbitmq-controller-manager
│ W0524 11:45:04.108665 1 reflector.go:324] k8s.io/client-go#v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Binding: the server could not find the requested resource (get bindings.rabbitmq.com) │
│ E0524 11:45:04.108718 1 reflector.go:138] k8s.io/client-go#v0.23.5/tools/cache/reflector.go:167: Failed to watch *v1beta1.Binding: failed to list *v1beta1.Binding: the server could not find the requested resource (get bindings.rabbitmq.com) │
│ W0524 11:45:04.109915 1 reflector.go:324] k8s.io/client-go#v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com) │
│ E0524 11:45:04.109935 1 reflector.go:138] k8s.io/client-go#v0.23.5/tools/cache/reflector.go:167: Failed to watch *v1beta1.Queue: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com) │
│ W0524 11:45:04.109976 1 reflector.go:324] k8s.io/client-go#v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com) │
│ E0524 11:45:04.109985 1 reflector.go:138] k8s.io/client-go#v0.23.5/tools/cache/reflector.go:167: Failed to watch *v1beta1.Exchange: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com) │
│ W0524 11:45:05.253806 1 reflector.go:324] k8s.io/client-go#v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com) │
│ E0524 11:45:05.254035 1 reflector.go:138] k8s.io/client-go#v0.23.5/tools/cache/reflector.go:167: Failed to watch *v1beta1.Exchange: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com) │
│ W0524 11:45:05.443628 1 reflector.go:324] k8s.io/client-go#v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com)
The truth is that I have searched and searched and I have not found information on how to connect it to an external RabbitMQ to Kubernetes, maybe I am not understanding the documentation well.
Related
I have been trying to use the experimental_cache feature, specifically the external caching with redis.
In the documentation (Caching in the Apollo Router - Apollo GraphQL Docs) it states that “it can be tested by building a custom Router binary, with the Cargo feature experimental_cache”.
I have taken this to mean that it should be added to Cargo.toml like this:
[features]
experimental_cache = []
With configuration like this:
supergraph:
apq:
experimental_cache:
in_memory:
limit: 512
redis:
urls: ["redis://..."]
query_planning:
experimental_cache:
in_memory:
limit: 512
redis:
urls: ["redis://..."]
However, doing this gives me the error:
ERROR configuration had errors:
1. /supergraph/apq/experimental_cache
supergraph:
apq:
experimental_cache:
┌ in_memory:
| limit: 512
| redis:
| urls: ["redis://..."]
└-----> Additional properties are not allowed ('redis' was unexpected)
2. /supergraph/query_planning/experimental_cache
limit: 512
redis:
urls: ["redis://..."]
query_planning:
experimental_cache:
┌ in_memory:
| limit: 512
| redis:
| urls: ["redis://..."]
└-----> Additional properties are not allowed ('redis' was unexpected)
2023-01-19T14:03:27.064894Z ERROR no valid configuration was supplied
Error: no valid configuration was supplied
I’m pretty sure I need additional configuration in order to use the redis part (the experimental_cache works fine without it), but it is not described in the documentation.
If someone could point me in the right direction it would be greatly appreciated.
Router version is 1.6.0
I have a rancher 2.6.67 server and RKE2 downstream cluster. The cluster was created without authorized cluster endpoint. How to add an authorised cluster endpoint to a RKE2 cluster created by Rancher article describes how to add it in an existing cluster, however although the answer looks promising, I still must miss some detail, because it does not work for me.
Here is what I did:
Created /var/lib/rancher/rke2/kube-api-authn-webhook.yaml file with contents:
apiVersion: v1
kind: Config
clusters:
- name: Default
cluster:
insecure-skip-tls-verify: true
server: http://127.0.0.1:6440/v1/authenticate
users:
- name: Default
user:
insecure-skip-tls-verify: true
current-context: webhook
contexts:
- name: webhook
context:
user: Default
cluster: Default
and added
"kube-apiserver-arg": [
"authentication-token-webhook-config-file=/var/lib/rancher/rke2/kube-api-authn-webhook.yaml"
to the /etc/rancher/rke2/config.yaml.d/50-rancher.yaml file.
After restarting rke2-server I found the network configuration tab in Rancher and was able to enable authorized endpoint. Here is where my success ends.
I tried to create a serviceaccount and got the secret to have token authorization, but it failed when connecting directly to the api endpoint on the master.
kube-api-auth pod logs this:
time="2022-10-06T08:42:27Z" level=error msg="found 1 parts of token"
time="2022-10-06T08:42:27Z" level=info msg="Processing v1Authenticate request..."
Also the log is full of messages like this:
E1006 09:04:07.868108 1 reflector.go:139] pkg/mod/github.com/rancher/client-go#v1.22.3-rancher.1/tools/cache/reflector.go:168: Failed to watch *v3.ClusterAuthToken: failed to list *v3.ClusterAuthToken: the server could not find the requested resource (get clusterauthtokens.meta.k8s.io)
E1006 09:04:40.778350 1 reflector.go:139] pkg/mod/github.com/rancher/client-go#v1.22.3-rancher.1/tools/cache/reflector.go:168: Failed to watch *v3.ClusterAuthToken: failed to list *v3.ClusterAuthToken: the server could not find the requested resource (get clusterauthtokens.meta.k8s.io)
E1006 09:04:45.171554 1 reflector.go:139] pkg/mod/github.com/rancher/client-go#v1.22.3-rancher.1/tools/cache/reflector.go:168: Failed to watch *v3.ClusterUserAttribute: failed to list *v3.ClusterUserAttribute: the server could not find the requested resource (get clusteruserattributes.meta.k8s.io)
I found that SA tokens will not work this way so I tried to use a rancher user token, but that fails as well:
time="2022-10-06T08:37:34Z" level=info msg=" ...looking up token for kubeconfig-user-qq9nrc86vv"
time="2022-10-06T08:37:34Z" level=error msg="clusterauthtokens.cluster.cattle.io \"cattle-system/kubeconfig-user-qq9nrc86vv\" not found"
Checking the cattle-system namespace, there are no SA and secret entries corresponding to the users created in rancher, however I found SA and secret entries related in cattle-impersonation-system.
I tried creating a new user, but that too, only resulted in new entries in cattle-impersonation-system namespace, so I presume kube-api-auth wrongly assumes the location of the secrets to be cattle-system namespace.
Now the questions:
Can I authenticate with downstream RKE2 cluster using normal SA tokens (not ones created through Rancher server)? If so, how?
What did I do wrong about adding the webhook authentication configuration? How to make it work?
I noticed, that since I made the modifications described above, I cannot download the kubeconfig file from the rancher UI for this cluster. What went wrong there?
Thanks in advance for any advice.
I have a service running a task definition with three containers:
service itself
envoy
x-ray daemon
And I want to trace and monitor my services interacting with each other with x-ray.
But I don't see any data in x-ray.
I can see the request logs and everything in the envoy logs but there are no error messages about missing connection to the x-ray daemon.
Envoy container has three env variables:
APPMESH_VIRTUAL_NODE_NAME = mesh/mesh-name/virtualNode/service-virtual-node
ENABLE_ENVOY_XRAY_TRACING = 1
ENVOY_LOG_LEVEL = trace
The x-ray daemon is pretty plain and has just a name and an image (amazon/aws-xray-daemon:1).
But when looking in the logs of the x-ray dameon, there is only the following:
2022-05-31T14:48:05.042+02:00 2022-05-31T12:48:05Z [Info] Initializing AWS X-Ray daemon 3.0.0
2022-05-31T14:48:05.042+02:00 2022-05-31T12:48:05Z [Info] Using buffer memory limit of 76 MB
2022-05-31T14:48:05.042+02:00 2022-05-31T12:48:05Z [Info] 1216 segment buffers allocated
2022-05-31T14:48:05.051+02:00 2022-05-31T12:48:05Z [Info] Using region: eu-central-1
2022-05-31T14:48:05.788+02:00 2022-05-31T12:48:05Z [Error] Get instance id metadata failed: RequestError: send request failed
2022-05-31T14:48:05.788+02:00 caused by: Get http://169.254.169.254/latest/meta-data/instance-id: dial tcp xxx.xxx.xxx.254:80: connect: invalid argument
2022-05-31T14:48:05.789+02:00 2022-05-31T12:48:05Z [Info] Starting proxy http server on 127.0.0.1:2000
As far as I read, the error you can see in these logs doesn't affect the functionality (https://repost.aws/questions/QUr6JJxyeLRUK5M4tadg944w).
I'm pretty sure I'm missing a configuration or access right.
It's running already on staging but I set this up several weeks ago and I don't find any differences between the configurations.
Thanks in advance!
In my case, I made a copy-paste mistake by copying trailing line break into the name of the environment variable ENABLE_ENVOY_XRAY_TRACING which wasn't visible in the overview and only inside the text field.
I linked wirecloud and Idm recently. When i login into wirecloud and i land into my wirecloud i got the following error:
Sorry, but the requested page is unavailable due to a server hiccup.
Our engineers have been notified, so check back later.
My idm configuration is:
URL
http://151.80.41.166:50002
Callback URL
http://151.80.41.166:50002/complete/fiware/
I cant get more error info
Exception Type: AuthStateMissing
Exception Value: Session value state missing.
Exception Location: /usr/local/lib/python2.7/site-packages/social_core/backends/oauth.py in validate_state, line 90
Python Executable: /usr/local/bin/python
Python Version: 2.7.14
Python Path:
['/opt/wirecloud_instance',
'/usr/local/lib/python27.zip',
'/usr/local/lib/python2.7',
'/usr/local/lib/python2.7/plat-linux2',
'/usr/local/lib/python2.7/lib-tk',
'/usr/local/lib/python2.7/lib-old',
'/usr/local/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/site-packages']
The problem was i got in the same machine idm and Wirecloud and they use the same cookie.
I add the follow lines on settings.py
SESSION_COOKIE_NAME = "wcsessionid"
CSRF_COOKIE_NAME = "wccsrftoken"
I know its experimental, I am trying to setup use docker-compose to build Spinnaker. I am seeing an error when trying to browse localhost:9000. Its trying to redirect to this page.
http://localhost:8084/auth/redirectto=http%3A%2F%2Flocalhost%3A9000%2F%23%2Finfrastructure
Looks like either its a fiat or gate issue. Tried adding proxy to apache2.
Errors in Fiat:
RetrofitError: unexpected url: front50/serviceAccounts
2017-09-15 19:24:31.642 WARN 1 --- [ont50Service-10]
c.n.s.f.p.internal.Front50Service : [] Falling back to service
account cache. Cause: unexpected url: front50/serviceAccounts
2017-09-15 19:24:31.645 WARN 1 --- [ecutionAction-1]
c.n.s.fiat.roles.UserRolesSyncer : [] User permission sync
failed. Server status is DOWN. Trying again in 10000 ms. Cause:
(Provider: DefaultServiceAccountProvider) retrofit.RetrofitError:
unexpected url: front50/serviceAccounts
Errors in gate:
2017-09-15 19:18:19.386 ERROR 1 --- [ost-startStop-1]
o.s.b.b.PropertiesConfigurationFactory : Properties configuration
failed validation
2017-09-15 19:18:19.394 ERROR 1 --- [ost-startStop-1]
o.s.b.b.PropertiesConfigurationFactory : Field error in object
'target' on field 'services[ORCA_HOST]': rejected value [orca]; codes
For errors in fiat, you can configure the environment variable in docker compose to
environment:
- "SERVICES_CLOUDDRIVER_BASEURL=http://clouddriver:7002"
- "SERVICES_FRONT50_BASEURL=http://front50:8080"
Update: This workaround work for Gate
environment:
- "services.clouddriver.host=clouddriver"
- "services.echo.host=echo"
- "services.front50.host=front50"
- "services.igor.host=igor"
- "services.orca.host=orca"
- "services.rosco.host=rosco"