problems with Scraping a Website with Elixir - selenium

Question

I'm trying to get a simple hound test working with my app, I figured out its an error with selenium. This is the code:
In mix.exs:
defmodule Scraper.Mixfile do
use Mix.Project
def project do
[app: :scraper,
version: "0.0.1",
elixir: "~> 1.0",
build_embedded: Mix.env == :prod,
start_permanent: Mix.env == :prod,
deps: deps]
end
# Configuration for the OTP application
#
# Type `mix help compile.app` for more information
def application do
[applications: [:logger, :httpoison, :hound]]
end
# Dependencies can be Hex packages:
#
# {:mydep, "~> 0.3.0"}
#
# Or git/path repositories:
#
# {:mydep, git: "https://github.com/elixir-lang/mydep.git", tag: "0.1.0"}
#
# Type `mix help deps` for more examples and options
defp deps do
[
{:httpoison, "~> 0.7"},
{:floki, "~> 0.7"},
{:hound, "~> 0.7"}
]
end
end
In lib/scraper.ex
defmodule Example do
use Hound.Helpers
def run do
Hound.start_session
IO.inspect "Iniciando"
navigate_to "http://akash.im"
IO.inspect page_title()
Hound.end_session
end
end
In config/config.exs
# This file is responsible for configuring your application
# and its dependencies with the aid of the Mix.Config module.
use Mix.Config
# This configuration is loaded before any dependency and is restricted
# to this project. If another project depends on this project, this
# file won't be loaded nor affect the parent project. For this reason,
# if you want to provide default values for your application for third-
# party users, it should be done in your mix.exs file.
# Sample configuration:
#
# config :logger, :console,
# level: :info,
# format: "$date $time [$level] $metadata$message\n",
# metadata: [:user_id]
# It is also possible to import configuration files, relative to this
# directory. For example, you can emulate configuration per environment
# by uncommenting the line below and defining dev.exs, test.exs and such.
# Configuration from the imported file will override the ones defined
# here (which is why it is important to import them last).
#
# import_config "#{Mix.env}.exs"
# Define how long the application will wait between failed attempts (in miliseconds)
config :hound, retry_time: 100000
# Start with selenium driver (default)
config :hound, driver: "selenium"
Starting a webdriver server
java -jar selenium-server-standalone-2.45.0.jar
Run app:
/scraper$ iex -S mix
Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false]
Interactive Elixir (1.0.5) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Example.run
** (exit) exited in: :gen_server.call(Hound.SessionServer, {:find_or_create_session, #PID<0.148.0>}, 60000)
** (EXIT) an exception was raised:
** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{id: nil, reason: :timeout}}
(hound) lib/hound/request_utils.ex:43: Hound.RequestUtils.send_req/4
(hound) lib/hound/session_server.ex:22: Hound.SessionServer.handle_call/3
(stdlib) gen_server.erl:629: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:661: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
11:26:13.971 [error] GenServer Hound.SessionServer terminating
Last message: {:find_or_create_session, #PID<0.148.0>}
State: #HashDict<[]>
** (exit) an exception was raised:
** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{id: nil, reason: :timeout}}
(hound) lib/hound/request_utils.ex:43: Hound.RequestUtils.send_req/4
(hound) lib/hound/session_server.ex:22: Hound.SessionServer.handle_call/3
(stdlib) gen_server.erl:629: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:661: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
(stdlib) gen_server.erl:212: :gen_server.call/3
(scraper) lib/scraper.ex:37: Example.run/0
iex(1)>

Answer 1

The request timed out in this case, as can be seen from the line
** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{id: nil, reason: :timeout}}
If you look at the stack trace, it indicates the error is at
(hound) lib/hound/request_utils.ex:43: Hound.RequestUtils.send_req/4
And if you open up hound source, on line 43 of lib/hound/request_utils.ex you see
case type do
:get ->
{:ok, resp} = HTTPoison.get(url, headers, #http_options)
:post ->
{:ok, resp} = HTTPoison.post(url, body, headers, #http_options)
:delete ->
{:ok, resp} = HTTPoison.delete(url, headers, #http_options)
end
This code expects a response, and crashes otherwise. There's a timeout error in your case, causing the crash.
Please check if the website up and reachable when you run the test, and retry.