I have been working with Go and the Bitmex API.
I have the following Bitmex api endpoint:
https://www.bitmex.com/api/v1/leaderboard
This returns an array of JSON objects structured as follows.
[
{
"name": "string",
"isRealName": true,
"profit": 0
}
]
However I get the following incorrect representation of the JSON when I marshal it.
[{0 false } {0 false } ... ]
I know my HTTP request is going through, as when I print the response.Body I get the following
[{"profit":256915996199,"isRealName":true,"name":"angelobtc"} ... ]
Here is the struct I am using to store the marshaled data.
type LeaderboardResponse struct {
profit float64 `json:"profit"`
isRealName bool `json:"isRealName"`
name string `json:"name"`
}
And my main method
func main() {
response, errorResponse := http.Get("https://www.bitmex.com/api/v1/leaderboard")
if response.Status == "200 OK"{
if errorResponse != nil {
fmt.Printf("The HTTP request failed with error %s\n",errorResponse)
} else {
body, errorBody := ioutil.ReadAll(response.Body)
if errorBody != nil {
fmt.Println("There was an error retrieving the body", errorBody)
} else {
leaderboard := []LeaderboardResponse{}
json.Unmarshal([]byte(body),&leaderboard)
if leaderboard != nil {
fmt.Println(leaderboard);
//The result of the statement above and the one below are different
fmt.Println(string(body))
} else {
fmt.Println("leaderboard array is undefined")
}
defer response.Body.Close()
}
}
} else {
fmt.Println("Response received with status ", string(response.Status))
}
}
It appears that the values of struct have not been modifies despite it being assigned the marshaled JSON body
How do I fix this issue?
Furthermore,
How do I add my API credentials to the HTTP request?
How do I access the response header and marshal it?
Related
There are no recipes for this which is surprising as it seems like a pretty basic use case.
If a request does not have a valid token, we want to return a 403 to the user. But trying to do that results in an empty response and "missing response context" error.
So this is currently the middleware:
func (auth \*Auth) Middleware() gin.HandlerFunc {
return func(c \*gin.Context) {
_, err := auth.Validate(c.Request.Context(), c.Request)
if (err != nil) {
c.AbortWithError(http.StatusForbidden, err)
return
}
c.Next()
}
}
Using it here in server.go
func Register(config \*viper.Viper, broker \*broker.Broker, metrics \*metrics.Metrics, clients *clients.Clients, auth \*auth.Auth) {
//...
r.Use(auth.Middleware())
//...
r.POST("/query", graphqlHandler(broker, clients, metrics))
r.GET("/query", graphqlHandler(broker, clients, metrics))
r.GET("/", playgroundHandler())
_ = r.Run()
}
There is no response returned in the event of an invalid token and you get:
missing response context
I was thinking I could use graphql.WithResponseContext but since that's immutable and we are aborting so there is no "next", there is no place for that context to go.
I am using ktor (2.1.0) to create a small API. While doing so, I am trying to leverage Ktor-server cool functionalities, including RequestValidation.
That being said, it is not working and I can't figure out why since it looks to me very close to the examples in the documentation.
This is my server config:
embeddedServer(Netty, port = 8080) {
routing {
post("/stock") {
val dto = call.receive<CreateStockRequest>()
call.respond(HttpStatusCode.NoContent)
}
}
install(ContentNegotiation) {
json(Json {
prettyPrint = true
isLenient = true
})
}
install(StatusPages) {
exception<RequestValidationException> { call, cause ->
call.respond(HttpStatusCode.BadRequest, cause.reasons.joinToString())
}
}
install(RequestValidation) {
validate<CreateStockRequest> { request ->
if (request.name.isBlank())
ValidationResult.Invalid("Stock must have a name")
if (request.symbol.isBlank())
ValidationResult.Invalid("Symbol must have a name")
ValidationResult.Valid
}
}
}
This is the request object being "received":
#Serializable
data class CreateStockRequest(val name: String, val symbol: String)
And this is the body being sent:
{
"name": "",
"symbol": "something"
}
I was expecting to get a BadRequest response, but I am getting a NoContent response as if everything was fine with the request.
Am I doing something wrong?
In your example, the server responds with BadRequest only if both "name" and "symbol" are blank. You can replace your validation logic with the when expression:
validate<CreateStockRequest> { request ->
when {
request.name.isBlank() -> ValidationResult.Invalid("Stock must have a name")
request.symbol.isBlank() -> ValidationResult.Invalid("Symbol must have a name")
else -> ValidationResult.Valid
}
}
I found the accepted answer to be a cleaner code solution, and that's why it is the accepted one.
That being said, the actual problem I was having is better explained by understanding that the compiler was getting confused with scopes.
On each "if" clause, I needed to return the validation result, but the compiler was not having it.
The reason for that was the fact that it got confused with the context to which my return statement referred.
My Solution was to use the '#' notation to specify the scope for the return:
validate<CreateStockRequest> { request ->
if (request.name.isBlank())
return#validate ValidationResult.Invalid("Stock must have a name")
if (request.symbol.isBlank())
return#validate ValidationResult.Invalid("Symbol must have a name")
return#validate ValidationResult.Valid
}
}
I have created a swagger specification which produces "application/zip"
/config:
get:
produces:
- application/zip
responses:
200: # OK
description: All config files
schema:
type: string
format: binary
I have implemented the handlers for this endpoint but I get this error
http: panic serving 127.0.0.1:20366: applicationZip producer has not yet been implemented
This error is coming from this code
func NewSampleAPI(spec *loads.Document) *SampleAPI {
return &SampleAPI{
...
ApplicationZipProducer: runtime.ProducerFunc(func(w io.Writer, data interface{}) error {
return errors.NotImplemented("applicationZip producer has not yet been implemented")
}),
After investigating this error my findings are that we need to implement something like this
api := operations.NewSampleAPI(swaggerSpec)
api.ApplicationZipProducer = func(w io.Writer, data interface{}) error {
...
}
So my question is that
what should we put in this Producer and why is it necessary to implement this because there is no implementation for "application/json" ?
Is "application/json" Producer is implemented by default and we need to implement other producers?
Note: I am using swagger "2.0" spec
Since you have used "application/zip" as response content then you might have implemented the backend code which might be returning io.ReadCloser.
Then your Producer will look like this
api.ApplicationZipProducer = runtime.ProducerFunc(func(w io.Writer, data interface{}) error {
if w == nil {
return errors.New("ApplicationZipProducer requires a writer") // early exit
}
if data == nil {
return errors.New("no data given to produce zip from")
}
if zp, ok := data.(io.ReadCloser); ok {
b, err := ioutil.ReadAll(zp)
if err != nil {
return fmt.Errorf("application zip producer: %v", err)
}
_, err = w.Write(b)
return nil
}
return fmt.Errorf("%v (%T) is not supported by the ApplicationZipProducer, %s", data, data)
})
This will parse the data interface into io.ReadCloser and read data from it and then it will fill into the io.Writer
Note: If your main purpose is just send file as attachment then you should use "application/octet-stream" and its producer is implemented by default
I have a RestController which looks like this
#RestController
#RequestMapping("/user")
class UserController(val userService: UserService) {
#PostMapping("/register")
fun register(#RequestBody body: UsernamePasswordResource): Mono<*> {
return userService.createUser(body.username, body.password)
.map {ResponseEntity(it, HttpStatus.CREATED)}
.doOnError {
if (it is DuplicateUserException){
ResponseEntity(ErrorResource(it.message!!), HttpStatus.BAD_REQUEST)
} else {
ResponseEntity(ErrorResource("Internal server error"), HttpStatus.INTERNAL_SERVER_ERROR)
}
}
}
}
When I create a using using this endpoint 200 OK in response and this as body:
{
"headers": {},
"body": {
"username": "test7",
"passwordEncoded": "4HxMMUI09pltEr9pyKIxsQ==",
"description": ""
},
"statusCode": "CREATED",
"statusCodeValue": 201
}
Why is the status of my ResponseEntity ignored and 200 is returned? How do i fix this? Thanks in advance.
Spring provides an HTTP 200 (OK) response by default when an endpoint returns successfully. To return a custom HTTP status, use the #ReponseStatus annotation above your function and pass your status code in the parenthesis. Your code snippet should look like:
#RestController
#RequestMapping("/user")
class UserController(val userService: UserService) {
#PostMapping("/register")
#ResponseStatus(HttpStatus.CREATED)
fun register(#RequestBody body: UsernamePasswordResource): Mono<*> {
return userService.createUser(body.username, body.password)
.map {ResponseEntity(it, HttpStatus.CREATED)}
.doOnError {
if (it is DuplicateUserException){
ResponseEntity(ErrorResource(it.message!!), HttpStatus.BAD_REQUEST)
} else {
ResponseEntity(ErrorResource("Internal server error"), HttpStatus.INTERNAL_SERVER_ERROR)
}
}
}
}
I figured it out. The endpoint was returning Mono<*>
After I changed my service to return AuthenticationResource instead of UserResource
I could also make the endpoint return Mono<ResponseEntity<AuthenticationResource>>.
After that the ResponseEntity was respected and used as normal.
Ok so I am Go Lang with the Echo framework, to try and build pdf which will load data from a database source - that bit will come later.
So this is how I am rendering my pdf html layout,
func (c *Controller) DataTest(ec echo.Context) error {
return ec.Render(http.StatusOK, "pdf.html", map[string]interface{}{
"name": "TEST",
"msg": "Hello, XXXX!",
})
}
The above function works fine, and renders the html (I built a temp path to the function). Now I want to use that function as my html template to build my pdf's.
So I am using wkhtmltopdf and the lib "github.com/SebastiaanKlippert/go-wkhtmltopdf"
This is how I would render the html in the pdf,
html, err := ioutil.ReadFile("./assets/pdf.html")
if err != nil {
return err
}
But I need to be able to update the template so that is why I am trying to render the page and take that into the pdf.
However, the Echo framework returns an error type and not of type bytes or strings and I am not sure how to update it so my rendered content is returned as bytes?
Thanks,
UPDATE
page := wkhtmltopdf.NewPageReader(bytes.NewReader(c.DataTest(data)))
This is how I am currently doing, the data is just a html string which is then turned into a slice of bytes for the NewReader.
This works fine, but I wanted to turn the DataTest function into a fully rendered html page by Echo. The problem with that is that when you return a rendered page, it is returned as an error type.
So I was trying to work out a why of updating it, so I could return the data as an html string, which would then be put in as a slice of bytes.
if you want rendered html, use echo' custom middleware. I hope it helps you.
main.go
package main
import (
"bufio"
"bytes"
"errors"
"fmt"
"html/template"
"io"
"net"
"net/http"
"github.com/labstack/echo"
)
type TemplateRegistry struct {
templates map[string]*template.Template
}
func (t *TemplateRegistry) Render(w io.Writer, name string, data interface{}, c echo.Context) error {
tmpl, ok := t.templates[name]
if !ok {
err := errors.New("Template not found -> " + name)
return err
}
return tmpl.ExecuteTemplate(w, "base.html", data)
}
func main() {
e := echo.New()
templates := make(map[string]*template.Template)
templates["about.html"] = template.Must(template.ParseFiles("view/about.html", "view/base.html"))
e.Renderer = &TemplateRegistry{
templates: templates,
}
// add custom middleware
// e.Use(PdfMiddleware)
// only AboutHandler for Pdf
e.GET("/about", PdfMiddleware(AboutHandler))
// Start the Echo server
e.Logger.Fatal(e.Start(":8080"))
}
// custom middleware
func PdfMiddleware(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) (err error) {
resBody := new(bytes.Buffer)
mw := io.MultiWriter(c.Response().Writer, resBody)
writer := &bodyDumpResponseWriter{Writer: mw, ResponseWriter: c.Response().Writer}
c.Response().Writer = writer
if err = next(c); err != nil {
c.Error(err)
}
// or use resBody.Bytes()
fmt.Println(resBody.String())
return
}
}
type bodyDumpResponseWriter struct {
io.Writer
http.ResponseWriter
}
func (w *bodyDumpResponseWriter) WriteHeader(code int) {
w.ResponseWriter.WriteHeader(code)
}
func (w *bodyDumpResponseWriter) Write(b []byte) (int, error) {
return w.Writer.Write(b)
}
func (w *bodyDumpResponseWriter) Flush() {
w.ResponseWriter.(http.Flusher).Flush()
}
func (w *bodyDumpResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
return w.ResponseWriter.(http.Hijacker).Hijack()
}
func AboutHandler(c echo.Context) error {
return c.Render(http.StatusOK, "about.html", map[string]interface{}{
"name": "About",
"msg": "All about Boatswain!",
})
}
view/about.html
{{define "title"}}
Boatswain Blog | {{index . "name"}}
{{end}}
{{define "body"}}
<h1>{{index . "msg"}}</h1>
<h2>This is the about page.</h2>
{{end}}
view/base.html
{{define "base.html"}}
<!DOCTYPE html>
<html>
<head>
<title>{{template "title" .}}</title>
</head>
<body>
{{template "body" .}}
</body>
</html>
{{end}}
So from what I understand you want to:
Render HTML from template
Convert HTML to PDF
Send it as an HTTP response? This part is unclear from your question, but it doesn't matter really.
So, the reason Echo returns error is because it actually not only does the template rendering, but also sending a response to client. If you want to do something else in-between, you can't use that method from echo.
Luckily, echo doesn't do anything magical there. You can just use the standard template package with the same result.
func GetHtml(filename string, data interface{}) (string, error) {
filedata, err := ioutil.ReadFile(filename)
if err != nil {
return "", err
}
asString := string(filedata)
t, err := template.New("any-name").Parse(asString)
if err != nil {
return "", err
}
var buffer bytes.Buffer
err = t.Execute(&buffer, data)
if err != nil {
return "", err
}
return buffer.String(), nil
}
There you have your function that returns a string. You can use buffer.Bytes() to have a byte array if that's preferrable.
After this you can do whatever you need to, like convert to PDF and write it back to clients using echoCtx.Response().Writer().
Hope that helps, but in future try asking more precise questions, then you are more likely to receive an accurate response.