How can I see or test graceful restarts in Go? - testing

I serve HTTP over gin's https://github.com/fvbock/endless. I would like to see the differences from the basic HTTP server.
I've sent syscall.SIGUSR1 signal with:
syscall.Kill(getPid(), syscall.SIGUSR1)
The app doesn't exit, but I cannot detect the restart.
What I have to do is initialise new configurations to the app when the toml config file changes.
My code is as follows:
package main
import (
"os"
"fmt"
"syscall"
"github.com/gin-gonic/gin"
"github.com/fvbock/endless"
"github.com/BurntSushi/toml"
)
type Config struct {
Age int
Cats []string
}
var cfg Config
func restart(c *gin.Context) {
syscall.Kill(os.Getpid(), syscall.SIGUSR1)
}
func init() {
toml.DecodeFile("config.toml", &cfg)
fmt.Println("Testing", cfg)
}
func main() {
router := gin.New()
router.GET("/restart", restart)
if err := endless.ListenAndServe("localhost:7777", router); err != nil {
panic(err)
}
}
When I hit the restart endpoint, I want the toml config printed out.

Updating answer based on the changes to your question. The endless library can allow you to handle that signal by default. You will need to register a hook. I've expanded on your example code below:
package main
import (
"os"
"fmt"
"syscall"
"github.com/gin-gonic/gin"
"github.com/fvbock/endless"
"github.com/BurntSushi/toml"
)
type Config struct {
Age int
Cats []string
}
var cfg Config
func restart(c *gin.Context) {
syscall.Kill(os.Getpid(), syscall.SIGUSR1)
}
func readConfig() {
toml.DecodeFile("config.toml", &cfg)
fmt.Println("Testing", cfg)
}
func main() {
readConfig()
router := gin.New()
router.GET("/restart", restart)
srv := endless.NewServer("localhost:7777", router)
srv.SignalHooks[endless.PRE_SIGNAL][syscall.SIGUSR1] = append(
srv.SignalHooks[endless.PRE_SIGNAL][syscall.SIGUSR1],
readConfig)
if err := srv.ListenAndServe(); err != nil {
panic(err)
}
}
Now when you call the restart endpoint, you should see the changes to config file refelcted in stdout. However in order to watch the file for changes you would need to use something like fsnotify

Related

Golang - Sending API POST Request - Not enough arguments error

The following code attempts to send a POST API request with a payload that is in RequestDetails.FormData. When I run main.go function, then I get the following errors.
go run main.go
# command-line-arguments
./main.go:53:17: not enough arguments in call to http.HandleFunc
./main.go:53:33: not enough arguments in call to reqDetails.Send
have ()
want (http.ResponseWriter, *http.Request)
./main.go:53:33: reqDetails.Send() used as value
The code is available below. Anybody knows what I could do wrong here? Thanks a lot for your help.
//main.go
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
"net/http"
)
// RequestDetails contains input data for Send
type RequestDetails struct {
EndPoint string
FormType string
FormData map[string]string
}
// Send sends API POST request to an endpoint
func (rd RequestDetails) Send(w http.ResponseWriter, r *http.Request) {
json_data, err := json.Marshal(rd.FormData)
if err != nil {
log.Fatal(err)
}
resp, err := http.Post(rd.EndPoint, rd.FormType, bytes.NewBuffer(json_data))
if err != nil {
log.Fatal(err)
}
fmt.Println(resp)
}
func main() {
m := map[string]string{
"AuthParamOne": "AP0000001",
"AuthParamTwo": "AP0000002",
"AuthParamThree": "AP0000003",
}
reqDetails := RequestDetails{
EndPoint: "https://httpbin.org/post",
FormType: "application/json",
FormData: m,
}
http.HandleFunc(reqDetails.Send())
}
you have to use HandleFunc in following below:
func HandleFunc(pattern string, handler func(ResponseWriter, *Request))
for code above follow this:
http.HandleFunc("/test",reqDetails.Send) //-> add reference instead of calling 'reqDetails.Send()'
reference: https://pkg.go.dev/net/http#HandleFunc
please vote up :)
In your Send method, you don't make use of w http.ResponseWriter, r *http.Request, So it seems you don't need them:
func (rd RequestDetails) Send() {...
Also in your last line, HandleFunc requires different arguments which once again is not necessary in your case. Just try to run the Send method:
reqDetails.Send()
The whole main.go file:
//main.go
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
"net/http"
)
// RequestDetails contains input data for Send
type RequestDetails struct {
EndPoint string
FormType string
FormData map[string]string
}
// Send sends API POST request to an endpoint
func (rd RequestDetails) Send() {
json_data, err := json.Marshal(rd.FormData)
if err != nil {
log.Fatal(err)
}
resp, err := http.Post(rd.EndPoint, rd.FormType, bytes.NewBuffer(json_data))
if err != nil {
log.Fatal(err)
}
fmt.Println(resp)
}
func main() {
m := map[string]string{
"AuthParamOne": "AP0000001",
"AuthParamTwo": "AP0000002",
"AuthParamThree": "AP0000003",
}
reqDetails := RequestDetails{
EndPoint: "https://httpbin.org/post",
FormType: "application/json",
FormData: m,
}
reqDetails.Send()
}
if your code like this
watcher := bufio.NewReader(os.Stdin)
input, _ := watcher.ReadString()
fmt.Println(input)
you needed this for reading line line
old -> input, _ := watcher.ReadString()
new -> input, _ := watcher.ReadString('\n')

Can't access mux params when testing API endpoint with custom handlers

I have a custom handler for my API endpoints like this:
type HTTPError struct {
Error error
Message string
Code int
}
type endpointREST func(http.ResponseWriter, *http.Request) *HTTPError
func (fn endpointREST) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if e := fn(w, r); e != nil {
http.Error(w, e.Message, e.Code)
}
}
my example route look like this:
func GetShare(w http.ResponseWriter, r *http.Request) *HTTPError {
vars := mux.Vars(r)
fmt.Println(r.URL) // http://127.0.0.1:36455/share/5713d228-a042-446d-a5e4-183b19fa832a
fmt.Println(vars) // -->> always empty map when testing
return nil
}
These routes work well (manual, using Postman) after setting them up with
router := mux.NewRouter().StrictSlash(true)
handler := cors.Default().Handler(router)
router.Handle("/share/{id}", endpointREST(GetShare)).Methods("GET")
log.Fatal(http.ListenAndServe(":6969", handler))
The Problem is, that i can't test the API this way, since mux.Vars(r) will always return an empty map in the testing environment.
This is my testing code:
func TestGetShare(t *testing.T) {
Reset()
router := mux.NewRouter()
ts := httptest.NewServer(router)
router.Handle("/share/{id}", endpointREST(GetShare)).Methods("GET")
defer ts.Close()
t.Run("unauthorized", func(t *testing.T) {
req, _ := http.NewRequest("GET", ts.URL + "/share/5713d228-a042-446d-a5e4-183b19fa832a", nil)
res, _ := http.DefaultClient.Do(req)
assert.Equal(t, http.StatusUnauthorized, res.StatusCode)
})
}
I suggest you tu use the SetURLVars test helper func:
// SetURLVars sets the URL variables for the given request, to be accessed via
// mux.Vars for testing route behaviour. Arguments are not modified, a shallow
// copy is returned.
//
// This API should only be used for testing purposes; it provides a way to
// inject variables into the request context. Alternatively, URL variables
// can be set by making a route that captures the required variables,
// starting a server and sending the request to that server.
func SetURLVars(r *http.Request, val map[string]string) *http.Request {
return requestWithVars(r, val)
}

Rest API in Go (Consume and host)

I am trying to request something, like a book by its id, and then host it locally so that if I write my local URL, like http://localhost:8080​/books?books=<book-id> it would show me the specific result.
To try to be concrete, I need to connect the two. Get the information from that URL, so "consume" and also host it locally, specifically by ID. I am not sure how to do both at once.
To create the paths, I've been using gorilla mux
So separately, I've used this, which would give me all the books at once (URL is not real).
func main() {
response, err := http.Get("https://bookibook.herokuapp.com/books/")
if err != nil {
fmt.Printf("there is no book with this ID %s\n", err)
} else {
data, _ := ioutil.ReadAll(response.Body)
fmt.Println(string(data))
}
}
and then this, which would create a local path for http://localhost:8080/books/ID
import (
"fmt"
"github.com/gorilla/mux"
"log"
"net/http"
)
func getID(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
fmt.Fprintf(w, "Get id %s\n", vars["id"])
}
func main() {
// Configure routes.
router := mux.NewRouter()
router.HandleFunc("/books/{id}/", getID).Methods(http.MethodGet)
// Start HTTP server.
if err := http.ListenAndServe(":8080", router); err != nil {
log.Fatal(err)
}
}

Go gRPC server make integration tests fail

I recently started to play again with go, and my task at the moment is to implement a service (that will run on k8s) that should act as a gRPC server.
As per requirements, at the moment, to satisfy the readinessProbe on k8s, I need to implement an healthcheck endpoint for my service, and this should be tested: what I've done is:
func main() {
server := startHTTPServer()
defer server.Close()
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, syscall.SIGTERM, syscall.SIGINT)
<-c
log.Println("Stopped")
}
func startHTTPServer() *http.Server {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
})
server := &http.Server{
Addr: net.JoinHostPort("", strconv.Itoa(livelinessPort)),
}
go server.ListenAndServe()
return server
}
and I tested in this way, following this article:
func TestMain(m *testing.M) {
flag.Parse()
wg := setup()
result := m.Run()
shutdown(wg)
os.Exit(result)
}
func shutdown(wg *sync.WaitGroup) {
syscall.Kill(syscall.Getpid(), syscall.SIGINT)
wg.Wait()
}
func setup() *sync.WaitGroup{
os.Setenv("APP_ENV", EnvTest)
wg := &sync.WaitGroup{}
startMainProcess(wg)
return wg
}
func startMainProcess(wg *sync.WaitGroup) {
go func() {
wg.Add(1)
defer wg.Done()
main()
}()
}
func TestK8SHealth(t *testing.T) {
res, err := http.Get("http://:8080/")
if err != nil {
t.Errorf("Unexpected API error: %s", err)
return
}
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
t.Errorf("Unexpected status code: %d", res.StatusCode)
return
}
}
and so far so good:
$ go test
PASS
2018/08/13 09:23:16 Stopped
ok github.com/... 0.015s
Te problem is when I try to add the gRPC server in the main application, following gPRC go examples. I've added this to my main file
func main() {
[...]
startChecksServiceServer()
[...]
}
func startChecksServiceServer() *grpc.Server {
flag.Parse()
lis, err := net.Listen("tcp", fmt.Sprintf(":%d", serverPort))
if err != nil {
log.Fatalf("Starting gPRC server error: failed to listen port %d: %v", serverPort, err)
}
grpcServer := grpc.NewServer()
pb.RegisterChecksServiceServer(grpcServer, &checksServiceServer{})
grpcServer.Serve(lis)
log.Println("Started grpc server")
return grpcServer
}
but now when I run the test, it fails after the actual test was run, when the interrupt signal is sent.
$ go test
PASS
signal: interrupt
FAIL github.com/... 0.016s
I made some tests and this is not depending by the actual test (as it pass with a test function that just return, and it's working while running the test suite using goLand:
GOROOT=/usr/local/go #gosetup
GOPATH=/Users/marco/Documents/projects/go #gosetup
/usr/local/go/bin/go test -c -i -o /private/var/folders/jh/znhv9f090yb7y390dckb23s00000gn/T/[...] #gosetup
/usr/local/go/bin/go tool test2json -t /private/var/folders/jh/znhv9f090yb7y390dckb23s00000gn/T/[...] -test.v -test.run
^TestK8SHealth$ #gosetup
=== RUN TestK8SHealth
--- PASS: TestK8SHealth (0.00s)
PASS
Process finished with exit code 1
--- EDIT
I found that the problem is around the listener. Having in the main function:
func main() {
net.Listen("tcp", fmt.Sprintf(":%d", serverPort))
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, syscall.SIGTERM, syscall.SIGINT)
<-c
log.Println("Stopped")
}
and a test with just a return (with the same setup of the previous one) will cause the test to fail:
$ go test -v
=== RUN Test2
--- PASS: Test2 (0.00s)
PASS
signal: interrupt
FAIL github.com/[...] 0.014s
grpcServer.Serve(lis) is a blocking call. So like what you did with HTTP server go server.ListenAndServe(), you need to run it in a goroutine. Could you please try if go grpcServer.Serve(lis) will make the test pass?
I am not sure about your new EDIT which indicates the problem is abut the listener. You can check whether net.Listen("tcp", fmt.Sprintf(":%d", serverPort)) returns or blocks.

How to use Golang's github.com/google/google-api-go-client/customsearch/v1

I've done the Oauth callback from which people said it's not needed, and just needs the cx code but I have yet to figure out how to add the cx parameter to the call.
package main
import (
"fmt"
"log"
"github.com/vinniyo/authCallback"
"github.com/google/google-api-go-client/customsearch/v1"
)
func main() {
client, err := authCallback.BuildOAuthHTTPClient()
if err != nil {
log.Fatalf("Error building OAuth client: %v", err)
}
service, err := customsearch.New(client)
if err != nil {
log.Fatalf("Error creating YouTube client: %v", err)
}
fmt.Println(service.Cse.List("bob").Do())
}
I know to upload a video to youtube you add parameters before do() but how do you figure out the formatting? eg:
upload := &youtube.Query{
Status: &youtube.VideoStatus{PrivacyStatus: *privacy},
}
The CseListCall struct has Cx method which lets you add that parameter: https://godoc.org/google.golang.org/api/customsearch/v1#CseListCall
fmt.Println(service.Cse.List("bob").Cx("my_cx_id").Do())