I'm using the request.el library (available via MELPA) to try and create a basic framework from which to start work in earnest on a Stack Exchange mode for Emacs. All I want to do is to be able to return the object as parsed by json-read to the calling function, but I can't even seem to make a connection.
I understand that to have my function return the object, the call must be made synchronously, so that is what :sync t is there for. I've considered making it an asynchronous call, but I don't think it would be beneficial considering its use-case.
At first, after I looked at the messages, I thought 'Maybe I don't have necessary binaries.' I did test request.el with some example calls that come with its documentation, and they work fine, so that's out.
I'm at a loss as to what is wrong. I don't yet have much experience, successful or otherwise, with anything dealing with a network, and don't fully understand the error message I'm getting. As far as I can tell, port 443 of the API is giving me the silent treatment, but I'm hesitant to think that's the case ;).
;; Works like a charm
(defun alist-to-json (alist)
"Converts the key-value pairs of `ALIST` into a JSON-friendly
string: \"key1=value1&key2=value2&\"."
(apply 'concat
(mapcar (lambda (kv)
(format "%s=%s&" (car kv)
(if (stringp (cdr kv))
(cdr kv)
(number-to-string (cdr kv)))))
alist)))
(defvar stack-api-root "https://api.stackexchange.com/2.1/")
(require 'json)
(require 'request)
(defun stack-api-request (call keys-alist)
"Makes the specified `CALL` to the Stack Exchange API with the
key-value pairs given `KEYS-ALIST`. For example,
(stack-api-request \"sites\" '((page . 2) (page_size . 25)))"
(let* ((base-call (concat stack-api-root call "?"))
(options (alist-to-json keys-alist)))
(request base-call
:params options
:parser 'json-read
:sync t)))
Backtrace
Debugger entered--Lisp error: (error "Could not create connection to api.stackexchange.com:443")
signal(error ("Could not create connection to api.stackexchange.com:443"))
error("Could not create connection to %s:%d" "api.stackexchange.com" 443)
url-http([cl-struct-url "https" nil nil "api.stackexchange.com" nil "/2.1/sites?&" nil nil t nil t] #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (nil) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "\n\n(fn &rest IGNORED)"] (nil))
url-https([cl-struct-url "https" nil nil "api.stackexchange.com" nil "/2.1/sites?&" nil nil t nil t] #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (nil) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "\n\n(fn &rest IGNORED)"] (nil))
url-retrieve-internal("https://api.stackexchange.com/2.1/sites?&" #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (nil) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "\n\n(fn &rest IGNORED)"] (nil) nil nil)
url-retrieve("https://api.stackexchange.com/2.1/sites?&" #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (nil) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "\n\n(fn &rest IGNORED)"])
url-retrieve-synchronously("https://api.stackexchange.com/2.1/sites?&")
request--url-retrieve-sync("https://api.stackexchange.com/2.1/sites?&" :params "page=2&page_size=25&" :parser json-read :sync t :error (closure (t) (&rest args) (apply (quote request-default-error-callback) (quote "https://api.stackexchange.com/2.1/sites?") args)) :url "https://api.stackexchange.com/2.1/sites?&" :response [cl-struct-request-response nil nil nil nil nil "https://api.stackexchange.com/2.1/sites?&" nil (:params "page=2&page_size=25&" :parser json-read :sync t :error (closure (t) (&rest args) (apply (quote request-default-error-callback) (quote "https://api.stackexchange.com/2.1/sites?") args)) :url "https://api.stackexchange.com/2.1/sites?&" :response #0) nil nil nil url-retrieve nil])
apply(request--url-retrieve-sync "https://api.stackexchange.com/2.1/sites?&" (:params "page=2&page_size=25&" :parser json-read :sync t :error (closure (t) (&rest args) (apply (quote request-default-error-callback) (quote "https://api.stackexchange.com/2.1/sites?") args)) :url "https://api.stackexchange.com/2.1/sites?&" :response [cl-struct-request-response nil nil nil nil nil "https://api.stackexchange.com/2.1/sites?&" nil #0 nil nil nil url-retrieve nil]))
request("https://api.stackexchange.com/2.1/sites?" :params "page=2&page_size=25&" :parser json-read :sync t)
(let* ((base-call (concat stack-api-root call "?")) (options (alist-to-json keys-alist))) (request base-call :params options :parser (quote json-read) :sync t))
stack-api-request("sites" ((page . 2) (page_size . 25)))
eval((stack-api-request "sites" (quote ((page . 2) (page_size . 25)))) nil)
eval-expression((stack-api-request "sites" (quote ((page . 2) (page_size . 25)))) nil)
call-interactively(eval-expression nil nil)
Messages:
Contacting host: api.stackexchange.com:443
Opening TLS connection to `api.stackexchange.com'...
Opening TLS connection with `gnutls-cli --insecure -p 443 api.stackexchange.com'...failed
Opening TLS connection with `gnutls-cli --insecure -p 443 api.stackexchange.com --protocols ssl3'...failed
Opening TLS connection with `openssl s_client -connect api.stackexchange.com:443 -no_ssl2 -ign_eof'...failed
Opening TLS connection to `api.stackexchange.com'...failed
I checked to make sure that this was not a problem with cURL. The call I use for curl is
curl api.stackexchange.com/2.1/sites --compressed
From the outside looking into the request repository, request is doing this as well. I don't know what could be going wrong.
I boiled down your example to the following snippet to reproduce your problem but it actually worked. Can you try this one?
(request
"http://api.stackexchange.com/2.1/sites"
:parser 'json-read
:params '((page . "2") (page_size . "25"))
:success (lambda (&rest args) (princ (plist-get args :data))))
It should print some data to *Messages* buffer and echo area.
EDIT: It seems that the problem in your example is that you are passing string to PARAMS which takes only alist. I will change code to raise error so that it is easier to debug.
Related
Please comment how to start backend and frontend with Calva Jack-in.
There is a clojurescript project for a web-application, I started REPL with calva jack-in (deps.edn + shadow-cljs), and tried to evaluate the function start!/stop! in the REPL. Even though there was no error message, I didn't find a corresponding response with "localhost:3000/", it was expected that "Hello, world" should be shown on browser.
To compile of both frontend and backend successfully:
$ npm install && npx shadow-cljs watch frontend backend
Start the server server with node.js successfully
$ node target/main.js
Clojurescript backend code for reference.
(ns mern.backend.core
(:require ["express" :as express]))
;; currently broken in shadow-cljs
(set! *warn-on-infer* true)
(defonce server (atom nil))
(def port 3000)
(defn start-server []
(println "Starting server")
(let [app (express)]
(.get app "/" (fn [_req res] (.send res "Hello, world")))
(.listen app port (fn [] (println "Example app listening on port 3000!")))))
(defn start! []
;; called by main and after reloading code
(reset! server (start-server)))
(defn stop! []
;; called before reloading code
(.close #server)
(reset! server nil))
(defn main []
;; executed once, on startup, can do one time setup here
(start!))
I am using SBCL, Emacs, Slime, and Dexador (a library for HTTP requests). I have this function that works:
(defun old-handle-response-and-status (final-url method &optional user-content)
(let ((status-code)
(response))
(cond ((equal method "get")
(multiple-value-bind (bresponse bstatus-code)
(handler-case (dex:get final-url)
(dex:http-request-bad-request ()
(values nil
"The server returned a failed request of 400 (bad request) status."))
(dex:http-request-failed (e)
(values nil
(format nil "The server returned a failed request of ~a status." (dex:response-status e)))))
(list (setf response bresponse)
(setf status-code bstatus-code))))
((equal method "post")
(multiple-value-bind (bresponse bstatus-code)
(handler-case (dex:post final-url
:content user-content)
(dex:http-request-bad-request ()
(values nil
"The server returned a failed request of 400 (bad request) status."))
(dex:http-request-failed (e)
(values nil
(format nil "The server returned a failed request of ~a status." (dex:response-status e)))))
(list (setf response bresponse)
(setf status-code bstatus-code)))))))
It works for GET, POST, and the error handling works as expected when a HTTP request faces errors. The iconic examples to show it working are:
CL-USER> (old-handle-response-and-status "http://www.paulgraham.com" "get")
("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">
<html>
(big HTML omitted)
</html>"
200)
CL-USER> (old-handle-response-and-status "https://httpbin.org/post" "post" '(("name" . "pedro")))
("{
medium JSON omitted
}
"
200)
CL-USER> (old-handle-response-and-status "https://httpstat.us/409" "get")
(NIL "The server returned a failed request of 409 status.")
Ok. While refactoring this code, I was trying to remove the cond clause. Thus I did a new shorter version:
(defun new-handle-response-and-status (method-call)
(let ((status-code)
(response))
(multiple-value-bind (bresponse bstatus-code)
(handler-case method-call
(dex:http-request-bad-request ()
(values nil
"The server returned a failed request of 400 (bad request) status."))
(dex:http-request-failed (e)
(values nil
(format nil "The server returned a failed request of ~a status." (dex:response-status e)))))
(list (setf response bresponse)
(setf status-code bstatus-code)))))
It majorly works, but only when the request is successful:
CL-USER> (new-handle-response-and-status (dex:get "http://www.paulgraham.com"))
("
HTML omitted
</html>"
NIL)
CL-USER> (new-handle-response-and-status (dex:post "https://httpbin.org/post" :content '(("name" . "pedro"))))
("{
medium JSON omitted
}
"
NIL)
When the request is a failed HTTP request, the refactoring does not work as expected!
When calling:
CL-USER> (new-handle-response-and-status (dex:get "https://httpstat.us/409"))
The Slime Debugger throws:
An HTTP request to "https://httpstat.us/409" returned 409 conflict.
I was expecting:
(NIL "The server returned a failed request of 409 status.")
I tried tweaking the input to be a quoted s-expression and inserting an eval:
(defun new-handle-response-and-status (method-call)
(let ((status-code)
(response))
(multiple-value-bind (bresponse bstatus-code)
(handler-case (eval method-call)
(dex:http-request-bad-request ()
(values nil
"The server returned a failed request of 400 (bad request) status."))
(dex:http-request-failed (e)
(values nil
(format nil "The server returned a failed request of ~a status." (dex:response-status e)))))
(list (setf response bresponse)
(setf status-code bstatus-code)))))
It works:
CL-USER> (new-handle-response-and-status '(dex:get "https://httpstat.us/409"))
(NIL "The server returned a failed request of 409 status.")
But, it feels as a bad practice - not really compatible with a refactoring effort. Is there a way to fix this without using eval?
Maybe using funcall?
The problem is that you're calling dex:get or dex:post before invoking the function, so the handler binding is not in effect.
You need to pass a function that calls it, and then call that function.
(defun new-handle-response-and-status (method-call)
(let ((status-code)
(response))
(multiple-value-bind (bresponse bstatus-code)
(handler-case (funcall method-call)
(dex:http-request-bad-request ()
(values nil
"The server returned a failed request of 400 (bad request) status."))
(dex:http-request-failed (e)
(values nil
(format nil "The server returned a failed request of ~a status." (dex:response-status e)))))
(list (setf response bresponse)
(setf status-code bstatus-code)))))
(new-handle-response-and-status (lambda () (dex:get "https://httpstat.us/409")))
Or you could convert it to a macro:
(defmacro new-handle-response-and-status (method-call)
`(let ((status-code)
(response))
(multiple-value-bind (bresponse bstatus-code)
(handler-case ,method-call
(dex:http-request-bad-request ()
(values nil
"The server returned a failed request of 400 (bad request) status."))
(dex:http-request-failed (e)
(values nil
(format nil "The server returned a failed request of ~a status." (dex:response-status e)))))
(list (setf response bresponse)
(setf status-code bstatus-code)))))
Remember evaluation rules.
CL-USER 36 > (defun foo (a)
(print 'foo1)
a
(print 'foo2)
'return-value)
FOO
CL-USER 37 > (foo (print 'bar1))
BAR1
FOO1
FOO2
RETURN-VALUE
(print 'bar1) is evaluated outside of foo.
I am just getting started with Clojurescript. I wrote some clojurescript code to use the shared aws credentials file to initialize S3 client and list buckets . However my code does not work.
(defn -main [arg1 arg2]
(println "hello")
(let[ creds (new AWS/SharedIniFileCredentials #js{:profile "superman"})
_ (AWS/config.update creds)
; dump out the accesskey to check if it has the correct profile
_ (AWS/config.getCredentials (fn [err] (if (nil? err) (do
(println "its good")
(println AWS/config.credentials.accessKeyId)))))
s3 (new (.-S3 AWS ))
] (.listBuckets s3 (fn[err buckets] (println "err: " err) (println buckets) )) ))
The AWS/config.getCredentials in the above does pick up the correct profile as seen from (println AWS/config.credentials.accessKeyId). The listbuckets code throws the following error:
#object[NodeError TypeError [ERR_INVALID_ARG_TYPE]: The "key" argument must be of type string or an instance of Buffer, TypedArray, DataView, or KeyObject. Received undefined]
I have Google AWS SDK S3 clojuresript AND is the only link I found . I used that to configure the S3 client but that does not seem to work
I would appreciate any help.
I checked it and the problem seems to be that SDK expects the credentials to be set before anything else, before instantiating the S3 client.
The following works for me on a minimal project with shadow-cljs:
(ns server.main
(:require ["aws-sdk" :as aws]))
(defn main! []
(println "App loaded...")
(let [creds (aws/SharedIniFileCredentials. #js {:profile "example-profile"})
_ (set! (.-credentials aws/config) creds)
s3 (aws/S3.)]
(.listBuckets s3 (fn [err data]
(if err
(println "ERROR:" err)
(println "OK:" data))))))
when I run it:
$ node target/main.js
App loaded...
OK: #js {:Buckets #js [#js {:Name dummy-test-bucket, :CreationDate #inst "2019-05-05T17:32:17.000-00:00"} #js {:Name mydomain.com, :CreationDate #inst "2018-06-19T04:16:10.000-00:00"}], :Owner #js {:DisplayName username, :ID f63f30bc25ab3609b8d3b5be6b3a872dd2c9f7947b2d509e2338357d93e74f2}}
The key was on this answer: https://stackoverflow.com/a/33175424/483566
Following this SO post, I would like to print the value of the preconditions in my function. However it fails for me in the following case (probably destructuring) :
I have a dir? helper function (feel free to skip this one) :
(defn dir? [s]
"returns true if the string passed is is an existing directory"
(->> (clojure.java.io/file s)
((juxt #(.exists %) #(.isDirectory %)))
(every? true?)))
It works just fine, and using the is macro, I get some nice error messages where I can see both the test and the parameters that were passed :
(is (dir? (io/file "resources/static"))) ;; => true
(is (dir? (io/file "resources/statice"))) ;; typo, error below
FAIL in clojure.lang.PersistentList$EmptyList#1
(boot.user4515592986834245937.clj:86) expected: (dir? (io/file
"resources/statice")) actual: (not (dir? #object[java.io.File
0x6730a420 "resources/statice"]))
However, when trying to use it in the precondition :pre, I get an ugly error :
(defn make-something
[&{:keys [dir]
:or {dir "default-dir"}}]
{:pre [(is (dir? (clojure.java.io/file dir)))]}
;;... do something with these
)
(make-something :dir "resources/statices") ;; doesn't exist
clojure.lang.Compiler$CompilerException: java.lang.AssertionError:
Assert failed: (is (dir? (io/file dir))),
compiling:(boot.user4515592986834245937.clj:80:12)
java.lang.AssertionError: Assert failed: (is (dir? (io/file dir)))
How can I get a nice error message in my function just like the one above ?
In case it matters, I am using Clojure 1.7.
You need to check your code (dir? function). Following snippet works for me:
(require '[clojure.java.io :as io])
(defn dir? [s]
(let [dir (io/file s)]
(and (.exists dir)
(.isDirectory dir))))
(defn make-something
[& {:keys [dir] :or {dir "default-dir"}}]
{:pre [(is (dir? dir))]}
(println dir))
(make-something :dir "/tmp")
out => /tmp
ret => nil
(make-something :dir "/xxxx")
FAIL in clojure.lang.PersistentList$EmptyList#1 (form-init3332271312167175068.clj:1)
expected: (dir? dir)
actual: (not (dir? "/xxxx"))
AssertionError Assert failed: (is (dir? dir)) user/make-sth (form-init3332271312167175068.clj:1)
Please bear with this contrived example but it was the simplest thing I could think of to recreate the issue.
(ns something.core)
(defn call-foo [something & args]
(let [a-foo (:foo (eval (:quux something)))]
(apply a-foo args)))
(def Something {
:foo (fn [& args] args)
:bar (fn [something] (call-foo something))
})
(defn make-something []
{:quux 'Something})
Running the following in the REPL or with lein run works well.
(let [subject (make-something)
actual (call-foo subject "hello" "greetings")]
(println actual))
;;=> (hello greetings)
The problem occurs only during this test and executing lein test:
(ns something.core-test
(:require [clojure.test :refer :all]
[something.core :refer :all]))
(deftest a-test
(let [subject (make-something)
actual (call-foo subject "hello" "greetings")]
(is (= ["hello" "greetings"] actual))))
This throws an error. An example output:
ERROR in (a-test) (Compiler.java:6464)
Uncaught exception, not in assertion.
expected: nil
actual: clojure.lang.Compiler$CompilerException: java.lang.RuntimeException: Unable to resolve symbol: Something in this context, compiling:(/private/var/folders/0n/c7q7860j34xfc2r1x4q51jrh0000gn/T/form-init9215140948330409114.clj:1:6436)
The line "Unable to resolve symbol: Something in this context" makes me think Something is not in context for some reason while I eval in call-foo. But why is this the case only in the test?
The problem is that eval does not see context. Your 'Something resolves in something.core and something.core-test since you have refered all. It won't resolve from whatever namespace where lein test runs its tests.
To fix the immediate problem change
'Something
to
`Something
so that it is namespace-qualified. The test will then run (and fail), but that's another issue (println returns nil for one thing).