Uploading images to amazon S3 using Phoenix/elixir and Arc - amazon-s3

Hey I am having problems getting Arc to upload images to my amazon S3 account. I don't know if the problem is configuring the app to connect to the account properly or how to set the upload file correctly. I have tried a few ways of doing things and have not had success.
First, I tried hard setting my key ID and Secret access key in a secret.config.exs file like so:
config :ex_aws,
access_key_id: "My key ID was here",
secret_access_key: "My secret access key was here",
region: "us-west-2",
s3: [
scheme: "https://",
host: "s3.amazonaws.com",
region: "us-west-2"
]
But no success. I tried again to set them in the config file using system environment variables like this:
config :arc,
bucket: "callmemd"
config :ex_aws,
access_key_id: [{:system, "AWS_ACCESS_KEY_ID"}, :instance_role],
secret_access_key: [{:system, "AWS_SECRET_ACCESS_KEY"}, :instance_role]
But, I am not sure I did that correctly. This is the code in the controller I am trying to use to call the upload with the uploaded file being saved properly in a param named avatar as an upload plug:
def create(conn, %{"doctor" => doctor_params}) do
avatar = doctor_params["avatar"]
Myapp.Avatar.store({avatar.filename, avatar.path})
I have also tried the following different configurations and many other variations with no luck:
Callme.Avatar.store({doctor_params["avatar"], doctor_params["id"]})
Callme.Avatar.store({%Plug.Upload{doctor_params}, doctor_params["id"]})
Callme.Avatar.store({%Plug.Upload{}, doctor_params["id"]})
Callme.Avatar.store({doctor_params["avatar"], doctor_params["id"]})
Here are my dependencies
defp deps do
[{:phoenix, "~> 1.1.4"},
{:postgrex, ">= 0.0.0"},
{:phoenix_ecto, "~> 2.0"},
{:phoenix_html, "~> 2.4"},
{:phoenix_live_reload, "~> 1.0", only: :dev},
{:gettext, "~> 0.9"},
{:comeonin, "~> 1.0"},
{:mailgun, "~> 0.1.2"},
{:poison, "~> 2.1", override: true},
{:cowboy, "~> 1.0"},
{:arc, "~> 0.5.2"},
{:ex_aws, "~> 0.4.10"},
{:httpoison, "~> 0.7"}]
end
Mix file:
def application do
[mod: {Callme, []},
applications: [:phoenix, :phoenix_html, :cowboy, :ex_aws, :httpoison, :logger, :gettext, :phoenix_ecto, :postgrex]]
end
Log when Create action is invoked successfully:
[info] POST /doctors
[debug] SELECT u0."id", u0."name", u0."email", u0."crypted_password", u0."admin", u0."inserted_at", u0."updated_at" FROM "users" AS u0 WHERE (u0."id" = $1) [26] OK query=0.0ms
[debug] Processing by Callme.DoctorController.create/2
Parameters: %{"_csrf_token" => "XxEtCRshOjg5Wj1ZLwIzWmYzPwYBJgAAmtGqxfcHa3q1mOF8WfGDLQ==", "_utf8" => "✓", "doctor" => %{"avatar" => %Plug.Upload{content_type: "image/jpeg", filename: "beta.jpg", path: "C:\\Users\\Frank\\AppData\\Local\\Temp/plug-1469/multipart-622422-206000-2"}, "bio" => "bio", "hiddeninfo" => "hiddeninfo", "name" => "name", "picture" => "picture", "specialty" => "specialty"}}
Pipelines: [:browser]
[debug] BEGIN [] OK query=0.0ms
[debug] INSERT INTO "doctors" ("inserted_at", "updated_at", "bio", "hiddeninfo", "name", "picture", "specialty") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id" [{{2016, 7, 27}, {12, 27, 2, 0}}, {{2016, 7, 27}, {12, 27, 2, 0}}, "bio", "hiddeninfo", "name", "picture", "specialty"] OK query=0.0ms
[debug] COMMIT [] OK query=16.0ms
[info] Sent 302 in 32ms
[info] GET /doctors
[debug] SELECT u0."id", u0."name", u0."email", u0."crypted_password", u0."admin", u0."inserted_at", u0."updated_at" FROM "users" AS u0 WHERE (u0."id" = $1) [26] OK query=0.0ms
[debug] Processing by Callme.DoctorController.index/2
Parameters: %{}
Pipelines: [:browser]
[debug] SELECT d0."id", d0."name", d0."hiddeninfo", d0."bio", d0."specialty", d0."picture", d0."inserted_at", d0."updated_at" FROM "doctors" AS d0 [] OK query=0.0ms
I adjusted the code a little bit and now I am getting the error as follows
Request: POST /doctors
Jul 28 10:28:37 callmemd app/web.1: ** (exit) an exception was raised:
Jul 28 10:28:37 callmemd app/web.1: ** (KeyError) key :id not found in: %Plug.Upload{content_type: "image/jpeg", filename: "file000166349580.jpg", path: "/tmp/plug-1469/multipart-726916-266069-2"}
Jul 28 10:28:37 callmemd app/web.1: (callme) web/controllers/doctor_controller.ex:21: Callme.DoctorController.create/2
Jul 28 10:28:37 callmemd app/web.1: (callme) web/controllers/doctor_controller.ex:1: Callme.DoctorController.action/2
Jul 28 10:28:37 callmemd app/web.1: (callme) web/controllers/doctor_controller.ex:1: Callme.DoctorController.phoenix_controller_pipeline/2
Jul 28 10:28:37 callmemd app/web.1: (callme) lib/phoenix/router.ex:261: Callme.Router.dispatch/2
Jul 28 10:28:37 callmemd app/web.1: (callme) web/router.ex:1: Callme.Router.do_call/2
Jul 28 10:28:37 callmemd app/web.1: (callme) lib/callme/endpoint.ex:1: Callme.Endpoint.phoenix_pipeline/1
Jul 28 10:28:37 callmemd app/web.1: (callme) lib/phoenix/endpoint/render_errors.ex:34: Callme.Endpoint.call/2
One problem was the variable for the amazon ID was named incorrectly for my system variable which I changed. Now I am trying with the controller code being as follows:
def create(conn, %{"doctor" => doctor_params}) do
changeset = Doctor.changeset(%Doctor{}, doctor_params)
avatar = doctor_params["avatar"]
Callme.Avatar.store({%Plug.Upload{}, id: avatar.id})

Related

Codeception Acceptance Testing issue using session snapshot

Update 10 Jun, 2021
So when removing the populator from codeception.yml the session problem goes away.
BUT: In the dump.sql is nothing having influence on users or sessions or cookies. There are only a few tables with demo data, and they are needed!
The relevant part in the file is this:
codeception.yml
...
modules:
enabled: [Db]
config:
Db:
dsn: "mysql:host=%HOST%;dbname=%DBNAME%"
user: "root"
password: "root"
populate: true
cleanup: true
# populator: "mysql -u$user -p$password $dbname < tests/codeception/_data/dump.sql"
...
Original Post
I think i read almost all similar reccources considering this issue, but nothing helped so far.
I am moving our Codeception tests on Github Actions. The whole build process is running but the acceptance tests not, because the session snapshot can't be restored.
The same Workflow works on a local server where i use selenium webdriver. I tried to run selenium in Actions (commented in build.yml) but that made some port problems.
What i'm doing in this short example is installing Joomla (works) and after that creating a Content category.
The second step (creating a Content category) tries to pick up the created session from the first step.
It's very simple and no problem locally, but on Actions the created session cannot be read.
The relevant report part:
InstallCest: Install joomla
Signature: InstallCest:installJoomla
Test: tests/codeception/acceptance/install/InstallCest.php:installJoomla
... works
InstallCest: createCategory
Signature: InstallCest:createcategory
Test: tests/codeception/acceptance/install/InstallCest.php:createcategory
Scenario --
[Db] Executing Populator: `mysql -uroot -proot test < tests/codeception/_data/dump.sql`
[Db] Populator Finished.
I create category "test 123"
Category creation in /administrator/
I open Joomla Administrator Login Page
[GET] http://127.0.0.1:8000/administrator/index.php
[Cookies] [{"name":"9d4bb4a09f511681369671a08beff228","value":"fail5495jbd01q6dc2nm06i7gf","path":"/","domain":"127.0.0.1","expiry":1623346855,"secure":false,"httpOnly":false},{"name":"8b5558aac8008f05fd8f8e59a3244887","value":"irhlqlj8jabat2n5746ba0sb5r","path":"/","domain":"127.0.0.1","expiry":1623346855,"secure":false,"httpOnly":false}]
[Snapshot] Restored "admin" session snapshot
[GET] http://127.0.0.1:8000/administrator/index.php?option=com_categories
Screenshot and page source were saved into '/home/runner/work/project_b/project_b/tests/codeception/_output/' dir
ERROR
The report:
session not created: No matching capabilities found
The HTML Snapshot:
Warning: session_start(): Failed to read session data: user (path: /var/lib/php/sessions) in /home/runner/work/project_b/project_b/joomla/libraries/joomla/session/handler/native.php on line 260
Error: Failed to start application: Failed to start the session
The php.log part
[Wed Jun 9 18:24:13 2021] 127.0.0.1:41972 Accepted
[Wed Jun 9 18:24:13 2021] 127.0.0.1:41972 [200]: GET /media/jui/fonts/IcoMoon.woff
[Wed Jun 9 18:24:13 2021] 127.0.0.1:41972 Closing
[Wed Jun 9 18:24:16 2021] 127.0.0.1:41982 Accepted
[Wed Jun 9 18:24:16 2021] PHP Warning: session_start(): Failed to read session data: user (path: /var/lib/php/sessions) in /home/runner/work/project_b/project_b/joomla/libraries/joomla/session/handler/native.php on line 260
[Wed Jun 9 18:24:16 2021] 127.0.0.1:41982 [500]: GET /administrator/index.php
[Wed Jun 9 18:24:16 2021] 127.0.0.1:41982 Closing
[Wed Jun 9 18:24:16 2021] 127.0.0.1:41986 Accepted
[Wed Jun 9 18:24:16 2021] PHP Warning: session_start(): Failed to read session data: user (path: /var/lib/php/sessions) in /home/runner/work/project_b/project_b/joomla/libraries/joomla/session/handler/native.php on line 260
[Wed Jun 9 18:24:16 2021] 127.0.0.1:41986 [500]: GET /administrator/index.php?option=com_categories
[Wed Jun 9 18:24:16 2021] 127.0.0.1:41986 Closing
I tried change the session.save_path without effect.
Posting relevant pieces:
composer.json
{
"name": "company/tests",
"description": "Company Product",
"license": "GPL-2.0+",
"require": {},
"require-dev": {
"codeception/codeception": "^4",
"fzaninotto/faker": "^1.6",
"behat/gherkin": "^4.4.1",
"phing/phing": "2.*",
"codeception/module-asserts": "^1.3",
"codeception/module-webdriver": "^1.2",
"codeception/module-filesystem": "^1.0",
"codeception/module-db": "^1.1"
}
}
build.yml
name: Codeception Tests
on: [push]
jobs:
tests:
runs-on: ${{ matrix.operating-system }}
strategy:
fail-fast: false
matrix:
operating-system: [ubuntu-latest]
php: ["7.4"]
name: PHP ${{ matrix.php }} Test on ${{ matrix.operating-system }}
env:
php-ini-values: post_max_size=32M
DB_DATABASE: test
DB_NAME: test
DB_ADAPTER: mysql
DB_USERNAME: root
DB_PASSWORD: root
DB_HOST: 127.0.0.1
DB_PORT: 3306
APP_URL: http://127.0.0.1:8000
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Checkout Joomla 3
uses: actions/checkout#v2
with:
repository: joomla/joomla-cms
ref: "3.9.27"
path: joomla
- name: Setup PHP
uses: shivammathur/setup-php#v2
with:
php-version: ${{ matrix.php }}
# ini-values: session.save_path=/tmp
extensions: mbstring, intl, zip, json
tools: composer:v2
- name: Start MySQL
run: |
sudo /etc/init.d/mysql start
mysql -e 'CREATE DATABASE test;' -uroot -proot
mysql -e 'SHOW DATABASES;' -uroot -proot
# Composer stuff ...
- name: Run chromedriver
run: nohup $CHROMEWEBDRIVER/chromedriver --url-base=/wd/hub /dev/null 2>&1 &
# - name: Start ChromeDriver (was a try)
# run: |
# google-chrome --version
# xvfb-run --server-args="-screen 0, 1280x720x24" --auto-servernum \
# chromedriver --port=4444 --url-base=/wd/hub &> chromedriver.log &
- name: Run PHP webserver
run: |
php -S 127.0.0.1:8000 -t joomla/ &> php.log.txt &
sleep 1;
- name: Install Tests
run: |
php vendor/bin/codecept run "tests/codeception/acceptance/install/InstallCest.php" -vv --html
env:
DB_PORT: ${{ job.services.mysql.ports[3306] }}
- name: Upload Codeception output
if: ${{ always() }}
uses: actions/upload-artifact#v2
with:
name: codeception-results
# path: Tests/Acceptance/_output/
path: tests/codeception/_output/
- name: Upload PHP log
if: ${{ failure() }}
uses: actions/upload-artifact#v2
with:
name: php-log
path: php.log.txt
Acceptance Suite
class_name: AcceptanceTester
modules:
enabled:
- Asserts
- JoomlaBrowser
- Helper\Acceptance
- DbHelper
- Filesystem
config:
JoomlaBrowser:
url: "http://127.0.0.1:8000/"
browser: "chrome"
restart: true
clear_cookies: true
# window_size: 1280x1024
window_size: false
port: 9515
capabilities:
unexpectedAlertBehaviour: "accept"
chromeOptions:
args: ["--headless", "--disable-gpu"] # Run Chrome in headless mode
# prefs:
# download.default_directory: "..."
username: "admin" # UserName for the Administrator
password: "admin" # Password for the Administrator
database host: "127.0.0.1:3306" # place where the Application is Hosted #server Address
database user: "root" # MySQL Server user ID, usually root
database password: "root" # MySQL Server password, usually empty or root
database name: "test" # DB Name, at the Server
database type: "mysqli" # type in lowercase one of the options: MySQL\MySQLi\PDO
database prefix: "jos_" # DB Prefix for tables
install sample data: "no" # Do you want to Download the Sample Data Along with Joomla Installation, then keep it Yes
sample data: "Default English (GB) Sample Data" # Default Sample Data
admin email: "admin#mydomain.com" # email Id of the Admin
language: "English (United Kingdom)" # Language in which you want the Application to be Installed
Helper\Acceptance:
url: "http://127.0.0.1:8000/" # the url that points to the joomla installation at /tests/system/joomla-cms - we need it twice here
MicrosoftEdgeInsiders: false # set this to true, if you are on Windows Insiders
error_level: "E_ALL & ~E_STRICT & ~E_DEPRECATED"
InstallCest.php
<?php
/**
* Install Joomla and create Category
*
* #since 3.7.3
*/
class InstallCest
{
/**
* Install Joomla, disable statistics and enable Error Reporting
*
* #param AcceptanceTester $I The AcceptanceTester Object
*
* #since 3.7.3
*
* #return void
*
*/
public function installJoomla(\AcceptanceTester $I)
{
$I->am('Administrator');
$I->installJoomlaRemovingInstallationFolder();
$I->doAdministratorLogin();
$I->disableStatistics();
$I->setErrorReportingToDevelopment();
}
/**
* Just create Category
*
* #param AcceptanceTester $I The AcceptanceTester Object
*
* #since 3.7.3
*
* #return void
*
*/
public function createCategory(\AcceptanceTester $I)
{
$I->createCategory('test 123');
}
}

Rswag: Authorization header appeared in query parameter

Version: rswag (2.0.5), rspec (3.8.0)
Environment: Rails 5.2.3, Ruby 2.4.5
It is my first time to use it, was stuck in authorization header for a day.
Here is what I did:
# in spec/swagger_helper.rb
config.swagger_docs = {
'api/v1/swagger.json' => {
swagger: '2.0',
info: {
title: 'API V1',
version: 'v1'
},
paths: {},
securityDefinitions: {
JWT: {
description: 'the jwt for API auth',
type: :apiKey,
name: 'Authorization',
in: :header
}
}
}
}
# in spec/integration/api/v1/nodes_spec.rb
path '/api/v1/nodes' do
get 'Get all servers' do
tags TAGS_NODE
produces 'application/json'
security [JWT: {}]
parameter name: :searchString, in: :query, type: :string
parameter name: :searchColumn, in: :query, type: :string
#parameter name: 'Authorization', :in => :header, :type => :string
let(:nodes) { create_list(:node_list, 32) }
response '200', 'Servers found' do
let(:'Authorization') { "Bearer #{gen_jwt}" }
let(:searchString) { 'test' }
let(:searchColumn) { ';Name;' }
run_test! do |repsonse|
data = JSON.parse(response.body)
puts data
end
end
end
end
Expected: The 'Bearer ....' is set in 'Authorization' header
Actual: In the test log, I found:
[INFO] [2019-09-29 01:11:13 UTC] [anony] [no session] [no req] [other other]Started GET "/api/v1/nodes?searchString=test&searchColumn=;Name;&params&headers[HTTP_AUTHORIZATION]=Bearer+eyJhbGciOiJIUzI1NiJ9.eyJleHAiOjE1Njk3NjI2NzMsImVtYWlsIjoidGVzdEBpYm0uY29tIiwib3JnYW5pemF0aW9ucyI6WyJ0ZXN0X29yZzEiLCJvcmcyIl0sInJvbGVzIjpbImNjX2NvbnNvbGVfYWRtaW4iLCJyb2xlMiJdLCJpbnZlbnRvcnkiOlsidGVzdF9pbnYiXX0.eenTMWUy6kSHO58_kLKoKWmNjvQ9i5TU9ex4Ou-ausE&headers[HTTP_ACCEPT]=application%2Fjson" for 127.0.0.1 at 2019-09-29 01:11:13 +0000
[INFO] [2019-09-29 01:11:13 UTC] [anony] [no session] [no req] [other other]Processing by Api::V1::NodesController#index as HTML
......
[DEBUG] [2019-09-29 01:11:13 UTC] [anony] [no session] [no req] [other other]Auth by JWT token....
[ERROR] [2019-09-29 01:11:13 UTC] [anony] [no session] [no req] [other other]No JWT token included in request
As marked as bold in the log, the 'Authorization' as well as 'Accept' headers are appeared in query parameters, which are supposed to be http headers, so that no JWT token can be retrieved from header in code.
I also tried not to use securityDefinition, but specify a parameter in header as following: parameter name: 'Authorization', :in => :header, :type => :string. It did not work either.
Not sure any configuration I missed, or something wrong I did? Thanks!
Update: it seems to be related to other gems conflict? I had another try to create a new Rails 5 api only app, add rspec and rswag gems only, and run with a simple test case, it worked!
Here is my Gemfile:
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
ruby '2.4.5'
gem 'rails', '5.2.3'
gem 'puma', '3.11'
gem 'bootsnap', '1.1.0', require: false
group :development, :test do
# Call 'byebug' anywhere in the code to stop execution and get a debugger console
gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
end
group :development do
gem 'listen', '>= 3.0.5', '< 3.2'
end
group :test do
# Test framework
gem "rspec-rails"
gem "database_cleaner", '1.6.0'
gem "simplecov"
gem "simplecov-rcov"
gem "factory_bot_rails", '5.1.0'
gem "ci_reporter_rspec"
gem "faker"
end
# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]
gem 'pg', '1.1.4'
gem 'delayed_job_active_record', '4.1.3'
gem 'delayed_job_worker_pool', '0.2.3'
gem 'dalli', '2.7.8'
gem 'ruby-kafka', '0.7.5'
gem 'active_model_serializers', '0.10.10'
gem 'will_paginate', '3.1.7'
gem 'rest-client', '1.8.0'
gem 'symmetric-encryption', '4.3.0', require: false
gem 'unicorn', '5.2.0'
gem 'rubyzip', '1.2.2'
gem 'jwt', '2.2.1'
gem 'rubyXL', '3.3.30'
gem 'apartment', '2.2.1'
gem 'rswag', '2.0.5'
[Resolved]
Seems not working with Rack::Test::Methods
It worked after remove the line 'include Rack::Test::Methods" in a helper file, which was added previously to use 'get' to test the API.
Seems not working with Rack::Test::Methods
It worked after remove the line 'include Rack::Test::Methods" in a helper file, which was added previously to use 'get' to test the API.

phoenix module ExAws.S3.Client is not loaded and could not be found

I use the library https://github.com/CargoSense/ex_aws, and I faced such problem:
== Compilation error on file web/models/s3.ex ==
** (CompileError) web/models/s3.ex:2: module ExAws.S3.Client is not loaded and could not be found
(elixir) expanding macro: Kernel.use/2
web/models/s3.ex:2: Minion.S3 (module)
(elixir) lib/kernel/parallel_compiler.ex:116: anonymous fn/4 in Kernel.ParallelCompiler.spawn_compilers/1
in my mix.exs:
def application do
[mod: {Minion, []},
applications: [:phoenix, ... :ex_aws, :httpoison, :poison]]
end
...
defp deps do
[{:phoenix, "~> 1.2.1"},
...
{:ex_aws, "~> 1.0.0-beta0"},
{:poison, "~> 2.0"},
{:httpoison, "~> 0.8"}]
end
my config.exs:
config :minion, :ex_aws,
access_key_id: "...",
secret_access_key: "...",
region: "us-east-1",
s3: [
scheme: "http://",
host: "...",
region: "us-east-1"
]
my s3.ex:
defmodule Minion.S3 do
use ExAws.S3.Client, otp_app: :minion
end
I will be grateful for any help to fix this problem
:ex_awsin version >= 1.0.0 does not use the old (pre 1.0.0) style where you needed to do as in your module Minion.S3. That's why ExAws.S3.Client does not exists. Instead, try:
Remove your file s3.ex
In config.exs, change from config :minion, :ex_aws, to config :ex_aws,
Update your requests to the new style: S3.list_buckets |> ExAws.request #=> {:ok, response} instead of old style Minion.S3.list_buckets

Phoenix/Elixir - Invalid Parameter error with Arc Ecto cast_attachments

I am trying to make an image upload function with Phoenix/elixir, and I am getting an error on the create action with my console giving me the following error message:
[debug] Processing by Callme.DoctorController.create/2
Parameters: %{"_csrf_token" => "AnpLGQsANzU1YmxQAwA8Oy4CA3VzAAAA45djZauss3TelOpHInPFFQ==", "_utf8" => "✓", "doctor" => %{"avatar" => %Plug.Upload{content_type: "image/jpeg", filename: "file000166349580.jpg", path: "C:\\Users\\Frank\\AppData\\Local\\Temp/plug-1470/multipart-62273-279000-1"}, "bio" => "m", "hiddeninfo" => "m", "name" => "m", "picture" => "m", "specialty" => "m"}}
Pipelines: [:browser]
[error] Task #PID<0.406.0> started from #PID<0.403.0> terminating
** (ArgumentError) argument error
(crypto) :crypto.sha256_mac_nif(["AWS4", nil], "20160801", 32)
(crypto) crypto.erl:1055: :crypto.sha256_mac/3
(ex_aws) lib/ex_aws/auth.ex:90: ExAws.Auth.signing_key/3
(ex_aws) lib/ex_aws/auth.ex:53: ExAws.Auth.signature/7
(ex_aws) lib/ex_aws/auth.ex:42: ExAws.Auth.auth_header/7
(ex_aws) lib/ex_aws/auth.ex:15: ExAws.Auth.headers/6
(ex_aws) lib/ex_aws/request.ex:31: ExAws.Request.request_and_retry/7
lib/arc/storage/s3.ex:14: Arc.Storage.S3.put/3
(elixir) lib/task/supervised.ex:89: Task.Supervised.do_apply/2
(elixir) lib/task/supervised.ex:40: Task.Supervised.reply/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Function: #Function<2.133421496/0 in Arc.Actions.Store.async_put_version/3>
Args: []
[error] Task #PID<0.407.0> started from #PID<0.403.0> terminating
** (ArgumentError) argument error
(crypto) :crypto.sha256_mac_nif(["AWS4", nil], "20160801", 32)
(crypto) crypto.erl:1055: :crypto.sha256_mac/3
(ex_aws) lib/ex_aws/auth.ex:90: ExAws.Auth.signing_key/3
(ex_aws) lib/ex_aws/auth.ex:53: ExAws.Auth.signature/7
(ex_aws) lib/ex_aws/auth.ex:42: ExAws.Auth.auth_header/7
(ex_aws) lib/ex_aws/auth.ex:15: ExAws.Auth.headers/6
(ex_aws) lib/ex_aws/request.ex:31: ExAws.Request.request_and_retry/7
lib/arc/storage/s3.ex:14: Arc.Storage.S3.put/3
(elixir) lib/task/supervised.ex:89: Task.Supervised.do_apply/2
(elixir) lib/task/supervised.ex:40: Task.Supervised.reply/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Function: #Function<2.133421496/0 in Arc.Actions.Store.async_put_version/3>
Args: []
[error] Ranch protocol #PID<0.403.0> (:cowboy_protocol) of listener Callme.Endpoint.HTTP terminated
** (exit) an exception was raised:
** (ArgumentError) argument error
(crypto) :crypto.sha256_mac_nif(["AWS4", nil], "20160801", 32)
(crypto) crypto.erl:1055: :crypto.sha256_mac/3
(ex_aws) lib/ex_aws/auth.ex:90: ExAws.Auth.signing_key/3
(ex_aws) lib/ex_aws/auth.ex:53: ExAws.Auth.signature/7
(ex_aws) lib/ex_aws/auth.ex:42: ExAws.Auth.auth_header/7
(ex_aws) lib/ex_aws/auth.ex:15: ExAws.Auth.headers/6
(ex_aws) lib/ex_aws/request.ex:31: ExAws.Request.request_and_retry/7
lib/arc/storage/s3.ex:14: Arc.Storage.S3.put/3
(elixir) lib/task/supervised.ex:89: Task.Supervised.do_apply/2
(elixir) lib/task/supervised.ex:40: Task.Supervised.reply/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Here is the the create action:
def create(conn, %{"doctor" => doctor_params}) do
changeset = Doctor.changeset(%Doctor{}, doctor_params)
case Repo.insert(changeset) do
{:ok, _doctor} ->
conn
|> put_flash(:info, "Doctor created successfully.")
|> redirect(to: doctor_path(conn, :index))
{:error, changeset} ->
render(conn, "new.html", changeset: changeset)
end
And here is my doctor.ex file:
defmodule Callme.Doctor do
use Callme.Web, :model
use Arc.Ecto.Model
schema "doctors" do
field :name, :string
field :hiddeninfo, :string
field :bio, :string
field :specialty, :string
field :picture, :string
field :avatar, Callme.Avatar.Type
has_many :services, Callme.Service
timestamps
end
#required_fields ~w(name hiddeninfo bio specialty picture )
#optional_fields ~w()
#required_file_fields ~w()
#optional_file_fields ~w(avatar)
#doc """
Creates a changeset based on the `model` and `params`.
If no params are provided, an invalid changeset is returned
with no validation performed.
"""
def changeset(model, params \\ :empty) do
model
|> cast(params, #required_fields, #optional_fields)
|> cast_attachments(params, #required_file_fields, #optional_file_fields)
end
end
Here is the Avatar.ex file:
defmodule Callme.Avatar do
use Arc.Definition
use Arc.Ecto.Definition
# Include ecto support (requires package arc_ecto installed):
# use Arc.Ecto.Definition
#versions [:original, :thumb]
#extension_whitelist ~w(.jpg .jpeg .gif .png)
def acl(:thumb, _), do: :public_read
def validate({file, _}) do
file_extension = file.file_name |> Path.extname |> String.downcase
Enum.member?(#extension_whitelist, file_extension)
end
def transform(:thumb, _) do
{ :noaction }
end
def filename(version, _) do
version
end
def storage_dir(_, {file, user}) do
"uploads/avatars/#{user.id}"
end
def default_url(:thumb) do
"https://placehold.it/100x100"
end
end
My mix file:
defmodule Callme.Mixfile do
use Mix.Project
def project do
[app: :callme,
version: "0.0.1",
elixir: "~> 1.0",
elixirc_paths: elixirc_paths(Mix.env),
compilers: [:phoenix, :gettext] ++ Mix.compilers,
build_embedded: Mix.env == :prod,
start_permanent: Mix.env == :prod,
aliases: aliases,
deps: deps]
end
# Configuration for the OTP application.
#
# Type `mix help compile.app` for more information.
def application do
[mod: {Callme, []},
applications: [:phoenix, :phoenix_html, :cowboy, :ex_aws, :httpoison, :logger, :gettext,
:phoenix_ecto, :postgrex]]
end
# Specifies which paths to compile per environment.
defp elixirc_paths(:test), do: ["lib", "web", "test/support"]
defp elixirc_paths(_), do: ["lib", "web"]
# Specifies your project dependencies.
#
# Type `mix help deps` for examples and options.
defp deps do
[{:phoenix, "~> 1.1.4"},
{:postgrex, ">= 0.0.0"},
{:ecto, "~>1.1.2", override: true},
{:phoenix_ecto, "~> 2.0"},
{:phoenix_html, "~> 2.4"},
{:phoenix_live_reload, "~> 1.0", only: :dev},
{:gettext, "~> 0.9"},
{:comeonin, "~> 1.0"},
{:mailgun, "~> 0.1.2"},
{:poison, "~> 2.1", override: true},
{:cowboy, "~> 1.0"},
{:arc, "~> 0.5.2"},
{:ex_aws, "~> 0.4.10"},
{:arc_ecto, "~> 0.3.2"},
{:httpoison, "~> 0.7"}]
end
# Aliases are shortcut or tasks specific to the current project.
# For example, to create, migrate and run the seeds file at once:
# $ mix ecto.setup
#
# See the documentation for `Mix` for more info on aliases.
defp aliases do
["ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
"ecto.reset": ["ecto.drop", "ecto.setup"]]
end
end
and my config file:
use Mix.Config
# Configures the endpoint
config :callme, Callme.Endpoint,
url: [host: "localhost"],
root: Path.dirname(__DIR__),
secret_key_base: "jTrdsYnbNCqUuRuBqUIg0p6YKgUG3paPiZgB0ivCF45uWfUhZAd/zxuOiOe0GVKe",
render_errors: [accepts: ~w(html json)],
pubsub: [name: Callme.PubSub,
adapter: Phoenix.PubSub.PG2]
# Configures Elixir's Logger
config :logger, :console,
format: "$time $metadata[$level] $message\n",
metadata: [:request_id]
config :arc,
bucket: "callme"
config :ex_aws,
access_key_id: System.get_env("AWS_ACCESS_KEY"),
secret_access_key: System.get_env("AWS_SECRET_ACCESS_KEY")
#.env library
# Import environment specific config. This must remain at the bottom
# of this file so it overrides the configuration defined above.
import_config "#{Mix.env}.exs"
import_config "config.secret.exs"
# Configure phoenix generators
config :phoenix, :generators,
migration: true,
binary_id: false
config :callme, Callme.Repo,
adapter: Ecto.Adapters.Postgres,
username: "postgres",
password: "postgres",
database: "callme_dev",
hostname: "localhost",
pool_size: 10
And also, here is a link to the github repository because I can't think of any other files that might have errors on them.
I hope we can figure this out. This has been literally an entire week now of trying to figure what is wrong with this thing and I feel like I am taking crazy pills.

problems with Scraping a Website with Elixir

I'm trying to get a simple hound test working with my app, I figured out its an error with selenium. This is the code:
In mix.exs:
defmodule Scraper.Mixfile do
use Mix.Project
def project do
[app: :scraper,
version: "0.0.1",
elixir: "~> 1.0",
build_embedded: Mix.env == :prod,
start_permanent: Mix.env == :prod,
deps: deps]
end
# Configuration for the OTP application
#
# Type `mix help compile.app` for more information
def application do
[applications: [:logger, :httpoison, :hound]]
end
# Dependencies can be Hex packages:
#
# {:mydep, "~> 0.3.0"}
#
# Or git/path repositories:
#
# {:mydep, git: "https://github.com/elixir-lang/mydep.git", tag: "0.1.0"}
#
# Type `mix help deps` for more examples and options
defp deps do
[
{:httpoison, "~> 0.7"},
{:floki, "~> 0.7"},
{:hound, "~> 0.7"}
]
end
end
In lib/scraper.ex
defmodule Example do
use Hound.Helpers
def run do
Hound.start_session
IO.inspect "Iniciando"
navigate_to "http://akash.im"
IO.inspect page_title()
Hound.end_session
end
end
In config/config.exs
# This file is responsible for configuring your application
# and its dependencies with the aid of the Mix.Config module.
use Mix.Config
# This configuration is loaded before any dependency and is restricted
# to this project. If another project depends on this project, this
# file won't be loaded nor affect the parent project. For this reason,
# if you want to provide default values for your application for third-
# party users, it should be done in your mix.exs file.
# Sample configuration:
#
# config :logger, :console,
# level: :info,
# format: "$date $time [$level] $metadata$message\n",
# metadata: [:user_id]
# It is also possible to import configuration files, relative to this
# directory. For example, you can emulate configuration per environment
# by uncommenting the line below and defining dev.exs, test.exs and such.
# Configuration from the imported file will override the ones defined
# here (which is why it is important to import them last).
#
# import_config "#{Mix.env}.exs"
# Define how long the application will wait between failed attempts (in miliseconds)
config :hound, retry_time: 100000
# Start with selenium driver (default)
config :hound, driver: "selenium"
Starting a webdriver server
java -jar selenium-server-standalone-2.45.0.jar
Run app:
/scraper$ iex -S mix
Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false]
Interactive Elixir (1.0.5) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Example.run
** (exit) exited in: :gen_server.call(Hound.SessionServer, {:find_or_create_session, #PID<0.148.0>}, 60000)
** (EXIT) an exception was raised:
** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{id: nil, reason: :timeout}}
(hound) lib/hound/request_utils.ex:43: Hound.RequestUtils.send_req/4
(hound) lib/hound/session_server.ex:22: Hound.SessionServer.handle_call/3
(stdlib) gen_server.erl:629: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:661: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
11:26:13.971 [error] GenServer Hound.SessionServer terminating
Last message: {:find_or_create_session, #PID<0.148.0>}
State: #HashDict<[]>
** (exit) an exception was raised:
** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{id: nil, reason: :timeout}}
(hound) lib/hound/request_utils.ex:43: Hound.RequestUtils.send_req/4
(hound) lib/hound/session_server.ex:22: Hound.SessionServer.handle_call/3
(stdlib) gen_server.erl:629: :gen_server.try_handle_call/4
(stdlib) gen_server.erl:661: :gen_server.handle_msg/5
(stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
(stdlib) gen_server.erl:212: :gen_server.call/3
(scraper) lib/scraper.ex:37: Example.run/0
iex(1)>
The request timed out in this case, as can be seen from the line
** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{id: nil, reason: :timeout}}
If you look at the stack trace, it indicates the error is at
(hound) lib/hound/request_utils.ex:43: Hound.RequestUtils.send_req/4
And if you open up hound source, on line 43 of lib/hound/request_utils.ex you see
case type do
:get ->
{:ok, resp} = HTTPoison.get(url, headers, #http_options)
:post ->
{:ok, resp} = HTTPoison.post(url, body, headers, #http_options)
:delete ->
{:ok, resp} = HTTPoison.delete(url, headers, #http_options)
end
This code expects a response, and crashes otherwise. There's a timeout error in your case, causing the crash.
Please check if the website up and reachable when you run the test, and retry.