I'm trying to connect to a RabbitMQ instance using the ampq package on Elixir, but at times the RabbitMQ instance won't be available at the time that the Elixir server is running. I was wondering how I might be able to implement a simple retry mechanism. There's one strategy here but that seems more involved than I feel necessary especially since there's a mention of it on the README about more information being found on the official docs. I unfortunately couldn't find anything.
Edit: This will crash the application on start and exit.
My code for the module is as follows:
Server.Gen.Rabbit (child)
defmodule Server.Gen.Rabbit do
use GenServer
use AMQP
defmodule State do
#type t :: %{
id: String.t(),
chan: map()
}
defstruct id: "", chan: nil
end
def start_link(%{id: id}) do
GenServer.start_link(
__MODULE__,
%State{
id: id
},
name: :"#{id}:rabbit"
)
end
def init(opts) do
host = "amqp://guest:guest#localhost"
case Connection.open(host) do
{:ok, conn} ->
{:ok, chan} = Channel.open(conn)
setup_queue(opts.id, chan)
:ok = Basic.qos(chan, prefetch_count: 1)
queue_to_consume = #online_receive_queue <> opts.id
IO.puts("queue_to_consume_online: " <> queue_to_consume)
{:ok, _consumer_tag} = Basic.consume(chan, queue_to_consume, nil, no_ack: true)
{:ok, %State{chan: chan, id: opts.id}}
{:error, _} ->
IO.puts("[Rabbit] error on connecting to server: #{host}")
{:backoff, 5_000}
end
end
Server (parent)
defmodule Server do
use Application
def start(_type, _args) do
import Supervisor.Spec, warn: false
children = [
{
GenRegistry,
worker_module: Server.Gen.Rabbit
},
Plug.Cowboy.child_spec(
scheme: :http,
plug: Server.Router,
options: [
port: String.to_integer(System.get_env("PORT") || "3000"),
dispatch: dispatch(),
protocol_options: [idle_timeout: :infinity]
]
)
]
opts = [strategy: :one_for_one, name: Server.Supervisor]
Supervisor.start_link(children, opts)
end
end
Related
I am writing some infrastructure testing in Chef InSpec & am not sure how to go about testing that a url is not accessible publicly. I have the following code snippet which I am currently using
environments = {
:ops => "ops",
}
control "verify-not-accessible-publicly" do
impact 1.0
title "verify we are not publicly accessible"
environments.each do |_, env|
uri = "http://#{env}.internal.example.com"
begin
result = http(uri, ssl_verify: true, open_timeout: 2, read_timeout: 5, max_redirects: 0)
rescue => e
unless e.class == Faraday::ConnectionFailed
raise e
end
end
end
end
This isn't working quite like I expect. I don't think the http(uri,...) block is actually executed until it is passed into a describe function.
Thanks
you should use http resource with a describe block and matchers
describe http('url', auth: {user: 'user', pass: 'test'}, params: {params}, method: 'method', headers: {headers}, data: data, open_timeout: 60, read_timeout: 60, ssl_verify: true, max_redirects: 3) do
its('status') { should eq number }
its('body') { should eq 'body' }
its('headers.name') { should eq 'header' }
end
I'm currently working on my first big elixir project and wanted to properly utilize testing this time.
However, if I add my Modules to the "normal" supervisor, i cannot start them again with start_supervised! and all tests fail with Reason: already started: #PID<0.144.0>
Here is my code:
(application.ex)
defmodule Websocks.Application do
# See https://hexdocs.pm/elixir/Application.html
# for more information on OTP Applications
#moduledoc false
use Application
def start(_type, _args) do
children = [
{Websocks.PoolSupervisor, []},
{Websocks.PoolHandler, %{}}
# {Websocks.Worker, arg}
]
# See https://hexdocs.pm/elixir/Supervisor.html
# for other strategies and supported options
opts = [strategy: :one_for_one, name: Websocks.Supervisor]
Supervisor.start_link(children, opts)
end
end
Some of my tests:
defmodule PoolHandlerTest do
use ExUnit.Case, async: true
alias Websocks.PoolHandler
doctest PoolHandler
setup do
start_supervised!({PoolHandler, %{}})
%{}
end
test "adding two pools and checking if they are there" do
assert PoolHandler.add(:first) == :ok
assert PoolHandler.add(:second) == :ok
assert PoolHandler.get_pools() == {:ok,%{:first => nil, :second => nil}}
end
and the pool handler:
defmodule Websocks.PoolHandler do
use GenServer
# Client
def start_link(default) when is_map(default) do
GenServer.start_link(__MODULE__, default, name: __MODULE__)
end
# Server (callbacks)
#impl true
def init(arg) do
{:ok, arg}
end
end
(I cut out the stuff i think is not necessary, but the complete code is on github here: github)
Thanks in advance for any help i get!
As #Everett mentioned in the comment - your application will already be started for you when you mix test, so there is no need to start your GenServers again. It seems like you're interacting with the global instance in your test, so if that's what you want, then it should just work.
However, if you'd like to start a separate instance just for your test, you need to start an unnamed one. For example, you could add an optional pid argument to your wrapper functions:
defmodule Websocks.PoolHandler do
# ...
def add(server \\ __MODULE__, value) do
GenServer.call(server, {:add, value})
end
# ...
end
Then, instead of using using start_supervised! like you do, you can start an unnamed instance in your setup and use it in your tests like so:
setup do
{:ok, pid} = GenServer.start_link(PoolHandler, %{})
{:ok, %{handler: pid}}
end
test "adding two pools and checking if they are there", %{handler: handler} do
PoolHandler.add(handler, :first)
# ...
end
I have this redux-observable epic which does a POST ajax request using RxJS.ajax.post and I don't think it is hitting my Elixir router properly as nothing is happening on my elixir backend. I am doing get requests to get categories correctly and in the same manner so I am hitting other paths in my Elixir router correctly. I am expecting the issue to be with my backend Elixir code not my frontend. I might need to change my path in router.ex.
When I press a button on the frontend, this object is what gets sent to the elixir backend (it dispatches this action with a product as the payload and hits the redux-observable epic below):
onPress = {() => {
props.uploadProduct({
name: props.name,
brand: props.brand,
description: props.description,
image: props.image
})
The epic:
import { ajax } from 'rxjs/observable/dom/ajax'
import { Observable } from 'rxjs'
export const uploadProductEpic = action$ =>
action$.ofType(UPLOAD_PRODUCT)
.mergeMap(action => {
ajax.post('http://localhost:4000/products/', action.payload)
})
.map(response => uploadProductFulfilled(response))
.catch(error => Observable.of(
uploadProductRejected(error))
)
the elixir router:
defmodule Api.Router do
use Plug.Router
if Mix.env == :dev do
use Plug.Debugger
end
plug :match
plug :dispatch
get "/categories/" do
Api.Repo.getCategories(conn)
end
post "/products/:product" do
IO.puts inspect conn
Api.Repo.insertProduct(conn, product)
end
end
IO.puts inspect conn doesn't log anything. So My Elixir router path post "/products/:product" do is not being hit by my POST request. What am I doing wrong?
This is the elixir function in repo.ex that I HOPE will insert the product into my database:
def insertProduct(conn, product) do
product = %Api.Product{name: product.name, brand: product.brand, description: product.description, image: product.image, rating: 0, numberOfVotes: 0}
changeset = Api.Product.changeset(product)
errors = changeset.errors
valid = changeset.valid?
case insert(changeset) do
{:ok, product} ->
conn
|> put_resp_content_type("application/json")
|> send_resp(200, Poison.encode!(%{
successs: "success"
}))
{:error, changeset} ->
conn
|> put_resp_content_type("application/json")
|> send_resp(500, Poison.encode!(%{
failure: "Errors"
}))
end
end
I am a frontend developer just trying to get into Elixir so any guidance and patience is appreciated. Thanks.
Your data is sent in the body of the request, not in the URL, so the route should be:
post "/products"
You'll also need to plug in a JSON parser after plug :match and before plug :dispatch, as described in the Parameter Parsing section in the documentation of Plug.Router:
plug :match
plug Plug.Parsers, parsers: [:json],
pass: ["application/json"],
json_decoder: Poison
plug :dispatch
The JSON data will now be present in conn.body_params, which you can send to your function:
post "/products" do
Api.Repo.insertProduct(conn, conn.body_params)
end
And finally, the keys in the JSON would be strings, so you need to use the bracket syntax to access them instead of dots:
product = %Api.Product{name: product["name"], brand: product["brand"], description: product["description"], image: product["image"], rating: 0, numberOfVotes: 0}
I'm not sure how you've defined Api.Product.changeset, but the default Phoenix convention defines a 2 arity function which calls cast and extracts the data from a map itself. If you're doing the same, you can do this instead:
changeset = Api.Product.changeset(%Api.Product{}, product)
I have been following along the RabbitMQ Work Queues tutorial for Elixir (Elixir Work Queues) which works quite nicely. On top of that I am now trying to get multiple consumers started and monitored by a Supervisor.
This last portion is proving to be a bit tricky. If I run the below code in 2 separate iex sessions, both are getting & handling messages from RabbitMQ.
Client (consumer)
defmodule MT.Client do
require Logger
#host Application.get_env(:mt, :host)
#username Application.get_env(:mt, :username)
#password Application.get_env(:mt, :password)
#channel Application.get_env(:mt, :channel)
def start_link do
MT.Client.connect
end
def connect do
{:ok, connection} = AMQP.Connection.open(host: #host, username: #username, password: #password)
{:ok, channel} = AMQP.Channel.open(connection)
AMQP.Queue.declare(channel, #channel, durable: true)
AMQP.Basic.qos(channel, prefetch_count: 1)
AMQP.Basic.consume(channel, #channel)
Logger.info "[*] Waiting for messages"
MT.Client.loop(channel)
end
def loop(channel) do
receive do
{:basic_deliver, payload, meta} ->
Logger.info "[x] Received #{payload}"
payload
|> to_char_list
|> Enum.count(fn x -> x == ?. end)
|> Kernel.*(1000)
|> :timer.sleep
Logger.info "[x] Done."
AMQP.Basic.ack(channel, meta.delivery_tag)
MT.Client.loop(channel)
end
end
end
Supervisor
defmodule MT.Client.Supervisor do
use Supervisor
require Logger
#name MTClientSupervisor
def start_link do
Supervisor.start_link(__MODULE__, :ok, name: #name)
end
def init(:ok) do
children = [
worker(MT.Client, [], restart: :transient, id: "MTClient01"),
worker(MT.Client, [], restart: :transient, id: "MTClient02"),
worker(MT.Client, [], restart: :transient, id: "MTClient03")
]
supervise(children, strategy: :one_for_one)
end
end
When running that in an iex session:
iex -S mix
MT.Client.Supervisor.start_link
Following is logged:
08:46:50.746 [info] [*] Waiting for messages
08:46:50.746 [info] [x] Received {"job":"TestMessage","data":{"message":"message........"}}
08:46:58.747 [info] [x] Done.
08:46:58.748 [info] [x] Received {"job":"TestMessage","data":{"message":"last........"}}
08:47:06.749 [info] [x] Done.
So clearly there in only 1 consumer active, which is consuming the messages sequentially.
Running the below in 2 iex sessions:
MT.Client.start_link
I'm not adding the logs here, but in this case I get 2 consumes handling messages at the same time
I am sure that I am simply not grasping the required details for Agent/GenServer/Supervisor. Anyone can point out what needs to be changed to MT.Client & MT.Client.Supervisor above to achieve the idea of having multiple consumers active on the same channel?
Also; I'm been experimenting with spawning a Consumer agent and using the resulting pid in AMQP.Basic.consume(channel, #channel, pid) - but that's failing as well.
I'd like to find out the current user from the server side. The user_controller module shows how current_user can be returned in response to the client request. But I'm having trouble finding out how to do it from another module on the server.
Below is what I tried but failed with the following error. Basically I copied over the Guardian.Plug code block from the user_controller.ex.
What's the correct way to do it?
(CompileError) web/GraphQLSession.ex:9: undefined function put_status/2
(stdlib) lists.erl:1337: :lists.foreach/2
(stdlib) erl_eval.erl:669: :erl_eval.do_apply/6
.
defmodule App.GraphQLSession do
use Guardian.Hooks
def root_eval(_conn) do
current_user =
case Guardian.Plug.current_resource(_conn) do
nil ->
_conn
|> put_status(:not_found)
|> render(App.V1.SessionView, "error.json", error: "User not found")
user ->
_conn
|> put_status(:ok)
|> render("show.json", user: user)
end
%{author: current_user}
end
end
<<<<< router.ex >>>>
defmodule App.Router do
use App.Web, :router
...
pipeline :api do
plug :accepts, ["json"]
plug Guardian.Plug.VerifyHeader
plug Guardian.Plug.LoadResource
end
scope "/graphql" do
pipe_through :api
forward "/", GraphQL.Plug, [schema: {App.PublicSchema, :schema}, root_value: {App.GraphQLSession, :root_eval} ]
end
<<<< user_controller.ex >> .... This shows how client can retrieve current_user via controller.
defmodule App.V1.UserController do
use App.Web, :controller
alias App.User
plug Guardian.Plug.EnsureAuthenticated, on_failure: { App.V1.SessionController, :unauthenticated_api }
plug :scrub_params, "user" when action in [:create, :update]
def current_user(conn, %{"jwt" => jwt}) do
case Guardian.Plug.current_resource(conn) do
nil ->
conn
|> put_status(:not_found)
|> render(App.V1.SessionView, "error.json", error: "User not found")
user ->
conn
|> put_status(:ok)
|> render("show.json", user: user)
end
UPDATE: After importing Plug.Conn and changing _conn to conn, per David Sulc's advice, the next error I get is as follows:
[error] #PID<0.1026.0> running App.Endpoint terminated
Server: localhost:4000 (http)
Request: POST /graphql
** (exit) an exception was raised:
** (Protocol.UndefinedError) protocol Enumerable not implemented for %Plug.Conn{adapter: {Plug.Adapters.Cowboy.Conn, :...}, assigns: %{error: "User not found", guardian_default_resource: nil, layout:
false}, before_send: [#Function<1.34093945/1 in Plug.Logger.call/2>, #Function<0.30221299/1 in Phoenix.LiveReloader.before_send_inject_reloader/1>], body_params: %{"query" => "mutation CreateMutation(
$input_0:CreateInput!){createQ(input:$input_0){clientMutationId,...F3}} fragment F0 on ...}
}}, cookies: %Plug.Conn.Unfetched{aspect: :cookies}, halted: false, host: "localhost", method: "POST", owner: #PID<0.1026.0>, params: %{"query" => "mutation ....}}}, path_info: [], peer: {{1
27, 0, 0, 1}, 50944}, port: 4000, private: %{App.Router => {[], %{GraphQL.Plug => []}}, :phoenix_endpoint => App.Endpoint, :phoenix_format => "json", :phoenix_pipelines => [:api], :phoenix_route => #Funct
ion<0.58758354/1 in App.Router.match_route/4>, :phoenix_router => App.Router, :phoenix_template => "error.json", :phoenix_view => App.V1.SessionView, :plug_session_fetch => #Function<0.29336444/1 in Plug.
Session.fetch_session/1>}, query_params: %{}, query_string: "", remote_ip: {127, 0, 0, 1}, req_cookies: %Plug.Conn.Unfetched{aspect: :cookies}, req_headers: [{"host", "localhost:4000"}, {"connection", "ke
ep-alive"}, {"content-length", "632"}, {"accept", "*/*"}, {"origin", "http://localhost:4000"}, {"user-agent", "}, {"content-type", "application/json"}, {"referer", "http://localhost:4000/"}, {"accept-encoding", "gzip, deflate"}, {"accept-language", "en-US,en;q=0.8,ko;q=0.6,zh-CN;q=0.4"
}, {"alexatoolbar-alx_ns_ph", "AlexaToolbar/alx-4.0"}], request_path: "/graphql", resp_body: nil, resp_cookies: %{}, resp_headers: [{"content-type", "application/json; charset=utf-8"}, {"cache-control", "
max-age=0, private, must-revalidate"}, {"x-request-id", "ikh03v5kqightov3npgl8bv0do5rv77d"}, {"access-control-allow-origin", "*"}, {"access-control-expose-headers", ""}, {"access-control-allow-credentials
", "true"}], scheme: :http, script_name: ["graphql"], secret_key_base: "x4K=====-00-----lksMUX", state: :sent, status: 404}
(elixir) lib/enum.ex:1: Enumerable.impl_for!/1
(elixir) lib/enum.ex:116: Enumerable.reduce/3
(elixir) lib/enum.ex:1477: Enum.reduce/3
(elixir) lib/enum.ex:1092: Enum.map/2
(rethinkdb) lib/rethinkdb/query/macros.ex:93: RethinkDB.Query.Macros.wrap/1
(rethinkdb) lib/rethinkdb/query/macros.ex:94: anonymous fn/1 in RethinkDB.Query.Macros.wrap/1
(elixir) lib/enum.ex:1092: anonymous fn/3 in Enum.map/2
(stdlib) lists.erl:1262: :lists.foldl/3
I'm learning Elixir myself, but hopefully this will help you along...
Elixir is complaining because the controller imports Plug modules for you (via use App.Web, :controller). This isn't your case in the module you defined, so you need to import it yourself (see line 2).
Also, note that the naming convention of preceding a variable with an underscore if for variables that are to be ignored (and therefore not used). Since your code uses the conn param, it shouldn't be matched as _conn.
defmodule App.GraphQLSession do
import Plug.Conn, only: [put_status: 2]
def root_eval(conn) do
current_user =
case Guardian.Plug.current_resource(conn) do
nil ->
conn
|> put_status(:not_found)
|> render(App.V1.SessionView, "error.json", error: "User not found")
user ->
conn
|> put_status(:ok)
|> render("show.json", user: user)
end
%{author: current_user}
end
end
To answer your updated question there are a few things that you should notice:
Your main error is that you're trying to implement the Enumerable protocol for something that doesn't support it. See the line: ** (Protocol.UndefinedError) protocol Enumerable not implemented. Check any code that is using the Enum.* module. Make sure anything you're passing into the function is enumerable.
Also, Guardian can't find the user you want to retrieve. In the stack trace you can see this: assigns: %{error: "User not found", guardian_default_resource: nil. Try using the api_sign_in function provided by Guardian. You can find more information here. Then you should be able to use Guardian.Plug.current_resource(conn)