I'm using the Godog library to implement some cucumbers tests for my api code, right now I'm only testing one endpoint but I'm hitting an error where it looks like it's expecting to have a server open. I created a httptest server that listens to port 8080 but the tests are failing with a 404.
If I run my cucumber in debug mode they work but if I use the run test command they fail cos the expect an open port dial tcp localhost:8080. Could someone point me to the right direction since I quite don't know where I'm failing.
This is my godog_test
`
func mockServer() *httptest.Server {
router := mux.NewRouter()
u, _ := url.Parse("http://localhost:8080")
l, _ := net.Listen("tcp", u.Host)
server := httptest.NewUnstartedServer(router)
_ = server.Listener.Close()
server.Listener = l
server.Start()
return server
}
func killMockServer(server *httptest.Server) {
server.Close()
}
func TestFeatures(t *testing.T) {
suite := godog.TestSuite{
TestSuiteInitializer: InitializeTestSuite,
ScenarioInitializer: InitializeScenario,
Options: &godog.Options{
Format: "pretty",
Paths: []string{"features"},
TestingT: t,
},
}
if suite.Run() != 0 {
t.Fatal("non-zero status returned, failed to run feature tests")
}
}
func InitializeTestSuite(ctx *godog.TestSuiteContext) {
var server *httptest.Server
ctx.BeforeSuite(func() {
server = mockServer()
})
ctx.AfterSuite(func() {
fmt.Println("shutting down everything")
killMockServer(server)
})
}
`
Post step that I'm testing
`
func iCallPOSTTo(path string) error {
req, err := json.Marshal(reqBody)
if err != nil {
return err
}
request, err := http.NewRequest(
http.MethodPost,
endpoint+path,
bytes.NewReader(reqBody),
)
res, err := http.DefaultClient.Do(request)
if err != nil {
return err
}
resBody, err := io.ReadAll(res.Body)
if err != nil {
return err
}
res.Body.Close()
[REDACTED]
return nil
}
`
I tried using a mock server to open port 8080 since at first I was receiving a connection refused error, after that I'm getting a 404 which means that my test is not reaching my actual function that processes the post request. I'm not sure if the mock server is the correct approach on this case.
Related
Summary
It seems pretty simple to implement an exclusive consumer in RabbitMQ. You just need to enable the exclusive flag while starting the consumption or that's what I thought. I tried this approach but for some reason, it's creating multiple exclusive consumers which actually contradicts with it's own definition.
Setup
I am using following setup -
Exchange: default
Queue: quorum, durable
Messages: durable
Here is the full code that I am using to register exclusive consumer -
package main
import (
"fmt"
amqp "github.com/rabbitmq/amqp091-go"
"sync"
"time"
)
type Config struct {
Schema string
Host string
Port string
Username string
Password string
Vhost string
}
type Rabbit struct {
config Config
connection *amqp.Connection
lock sync.Mutex
}
// NewRabbit returns a Rabbit instance.
func NewRabbit() *Rabbit {
// setup appropriate values
config := Config{
Host: "",
Username: "",
Password: "",
Port: "",
Vhost: "",
Schema: "",
}
return &Rabbit{
config: config,
}
}
// Connect connects to RabbitMQ server.
func (r *Rabbit) Connect() error {
r.lock.Lock()
defer r.lock.Unlock()
// Check if connection is already available
if r.connection == nil || r.connection.IsClosed() {
// Try connecting
con, err := amqp.DialConfig(fmt.Sprintf(
"%s://%s:%s#%s:%s/%s",
r.config.Schema,
r.config.Username,
r.config.Password,
r.config.Host,
r.config.Port,
r.config.Vhost,
), amqp.Config{})
if err != nil {
return err
}
r.connection = con
}
return nil
}
func (r *Rabbit) StartConsumer(queueName string) error {
chn, err := r.connection.Channel()
if err != nil {
return err
}
// Make sure we process 1 message at a time
if err := chn.Qos(1, 0, false); err != nil {
return err
}
_, err = chn.QueueDeclare(
queueName,
true,
false,
false,
false,
amqp.Table{"x-queue-type": "quorum"}) // This will ensure that the created queue is quorum-queue
if err != nil {
fmt.Printf("Error creating queue with name: %s, err: %s", queueName, err.Error())
return err
}
messages, err := chn.Consume(
queueName,
queueName+"-consumer",
false,
true,
false,
false,
nil,
)
if err != nil {
fmt.Printf("Unable to start consumer for webhook queue: %s, err: %s", queueName, err.Error())
return err
}
go func() {
// This for-loop will wait indefinitely or until channel is closed
for msg := range messages {
fmt.Printf("Message: %v", msg.Body)
if err = msg.Ack(false); err != nil {
fmt.Printf("Unable to acknowledge the message, err: %s", err.Error())
}
}
}()
return nil
}
Here is the main.go
package main
import (
"fmt"
"os"
"os/signal"
"syscall"
)
func main() {
interrupt := make(chan os.Signal)
signal.Notify(interrupt, os.Interrupt, syscall.SIGINT, syscall.SIGTERM)
defer signal.Stop(interrupt)
rabbit := NewRabbit()
if err := rabbit.Connect(); err != nil {
fmt.Printf("Can't connect to RabbitMQ server, err: %s", err.Error())
} else {
fmt.Println("Successfully connected to RabbitMQ server")
}
err := rabbit.StartConsumer("test-queue")
if err != nil {
fmt.Printf("Error: %s", err.Error())
}
select {
case <-interrupt:
fmt.Println("Interrupt signal received")
break
}
fmt.Println("Application is about to close")
}
Here is go.mod
module ExclusiveRabbitMQConsumer
go 1.17
require github.com/rabbitmq/amqp091-go v1.3.4
And here is what I see after running 3 instances of this application -
RabbitMQ Management UI
All 3 consumers are shown as exclusive. The consumer tags are same however that doesn't matter according to RabbitMQ documentation.
I'm not sure what's wrong here.
Here api to go which should load the file when posting a request of the form
curl -X POST -d "url = http: //site.com/file.txt" http: // localhost: 8000 / submit
But 404 gets out, what's the reason?
Or how to download files via POST in API?
func downloadFile(url string) Task {
var task Task
resp, err := http.Get(url)
if err != nil {
fmt.Println("Error while downloading")
}
defer resp.Body.Close()
filename := strings.Split(url, "/")[len(strings.Split(url, "/"))-1]
fmt.Println(filename)
out, err := os.Create(filename)
if err != nil {
fmt.Println("Error while downloading")
}
defer out.Close()
_, err = io.Copy(out, resp.Body)
fmt.Println("Error while downloading")
}
func submit(c *gin.Context) {
c.Header("Content-Description", "File Transfer")
c.Header("Content-Transfer-Encoding", "binary")
url := c.Param("url")
fmt.Println("url " + url)
task := downloadFile(url)
hashFile(task.ID)
c.JSON(200, task.ID)
}
func main() {
router := gin.Default()
router.POST("/submit/:url", submit)
}
HTTP status 404 means the server couldn't find the requested URL. This appears to make perfect sense given your curl command. You appear to be requesting the URL http://localhost:8000/submit, but your application only has a single route:
router.POST("/submit/:url", submit)
This route requires a second URL segment after /submit, such as /submit/foo.
I am trying to authenticate a user by calling a REST api, before I proxy the request to a remote server.
However, I find that if I make the api call before proxy to remote server, the request will fail with the following error:
http: proxy error: http: ContentLength=139 with Body length 0.
If I remove the api call before proxy to remote server, the request can get through and return correct response.
My middleware is as following:
func AuthMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// the api call to external auth server
user_id, err := authenticate(r.FormValue("username"), r.FormValue("password"))
if err != nil {
http.Error(w, err.Error(), 401)
return
}
next.ServeHTTP(w, r)
})
}
My reverse proxy is as following:
func NewReverseProxy(target *url.URL) *httputil.ReverseProxy {
director := func(req *http.Request) {
req.URL.Scheme = target.Scheme
req.URL.Host = target.Host
req.URL.Path = target.Path
targetQuery := target.RawQuery
if targetQuery == "" || req.URL.RawQuery == "" {
req.URL.RawQuery = targetQuery + req.URL.RawQuery
} else {
req.URL.RawQuery = targetQuery + "&" + req.URL.RawQuery
}
if _, ok := req.Header["User-Agent"]; !ok {
// explicitly disable User-Agent so it's not set to default value
req.Header.Set("User-Agent", "")
}
}
return &httputil.ReverseProxy{Director: director}
}
And I am using Chi for routing
r.Use(AuthMiddleware)
r.Post("/", NewReverseProxy(targets).ServeHTTP)
What is the issue with this implementation?
If you do not care for the body anymore you can set the contentlength of the request to 0, reflecting the current state of the body:
func AuthMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// the api call to external auth server
user_id, err := authenticate(r.FormValue("username"), r.FormValue("password"))
if err != nil {
http.Error(w, err.Error(), 401)
return
}
r.ContentLength = 0
next.ServeHTTP(w, r)
})
}
In Go I'm using NewSingleHostReverseProxy to perform a reverse proxy, however I need to confirm the SSL certificates of the host site, to make sure I have the correct secure certificate... any ideas how I should do this? Should I be doing this with the handler or transport? I'm new to Go and still getting my head around it.
proxy := httputil.NewSingleHostReverseProxy(&url.URL{
Scheme: "https",
Host: "sha256.badssl.com",
})
http.ListenAndServe("127.0.0.1:80", proxy)
To access the certificate, you will have get access to the ConnectionState. The easiest way to do that is to provide your own version of DialTLS. In there you connect to the server using net.Dial, do the TLS handshake and then you are free to verify.
package main
import (
"crypto/tls"
"log"
"net"
"net/http"
"net/http/httputil"
"net/url"
)
func main() {
proxy := httputil.NewSingleHostReverseProxy(&url.URL{
Scheme: "https",
Host: "sha256.badssl.com",
})
// Set a custom DialTLS to access the TLS connection state
proxy.Transport = &http.Transport{DialTLS: dialTLS}
// Change req.Host so badssl.com host check is passed
director := proxy.Director
proxy.Director = func(req *http.Request) {
director(req)
req.Host = req.URL.Host
}
log.Fatal(http.ListenAndServe("127.0.0.1:3000", proxy))
}
func dialTLS(network, addr string) (net.Conn, error) {
conn, err := net.Dial(network, addr)
if err != nil {
return nil, err
}
host, _, err := net.SplitHostPort(addr)
if err != nil {
return nil, err
}
cfg := &tls.Config{ServerName: host}
tlsConn := tls.Client(conn, cfg)
if err := tlsConn.Handshake(); err != nil {
conn.Close()
return nil, err
}
cs := tlsConn.ConnectionState()
cert := cs.PeerCertificates[0]
// Verify here
cert.VerifyHostname(host)
log.Println(cert.Subject)
return tlsConn, nil
}
To tweak the SSL to the Reverse Host, it is possible to set the transport
options. So if you want to skip the verify you can set it like this.
proxy := httputil.NewSingleHostReverseProxy(&url.URL{
Scheme: "https",
Host: "sha256.badssl.com",
})
proxy.Transport = &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
http.ListenAndServe("127.0.0.1:80", proxy)
I'm trying to execute a command via session.Run() function over a ssh connection. So far I can successfully execute some commands but on others I keep getting the following error: "Process exited with: 1. Reason was: () exit status 1"
func (p *project) connect(config *ssh.ClientConfig) {
log.Printf("Trying connection...\n")
conn, err := ssh.Dial("tcp", fmt.Sprintf("%s:%s", p.hostname.name, p.port.name), config)
checkError("Failed to dial: ", err)
log.Printf("Connection established.\n")
for step := range p.typ.program.setup {
p.install(step, conn)
}
}
func (p *project) install(step int, conn *ssh.Client) {
session, err := conn.NewSession()
checkError("Failed to build session: ", err)
defer session.Close()
var stdoutBuf bytes.Buffer
session.Stdout = &stdoutBuf
log.Printf("Executing command: %s", p.typ.program.setup[step])
if err := session.Run(p.typ.program.setup[step]); err != nil {
log.Println(session.Stdout)
log.Fatal("Error on command execution", err.Error())
}
}
// That would be an example of a command which returns me an error
// "cd ~/www/www/ && git commit -m 'on the beginning was the commit'"
// That comes inside a slice on p.typ.program.setup accessed by the step(index).
The command output (session.Stdout) is the one i expect:
"# On branch master nothing to commit, working directory clean"
And just to note I already tried to execute the command directly on the console and it works just fine.
So, the code seems to be okay, the command ran on the remote but I still have an error no matter what.
Does anyone have a clue about why is that happening?
Thanks in advance.
Maybe my library will helps in your case: https://github.com/shagabutdinov/shell; it covers basic cases of running ssh commands in one session.
Try following:
handler := func(outputType int, message string) {
if(outputType == shell.Stdout) {
log.Println("stdout: ", message)
} else if(outputType == shell.Stdout) {
log.Println("stderr: ", message)
}
}
key, err := ssh.ParsePrivateKey([]byte(YOUR_PRIVATE_KEY))
if(err != nil) {
panic(err)
}
auth := []ssh.AuthMethod{ssh.PublicKeys(key)}
shell = shell.NewRemote(shell.RemoteConfig{
Host: "root#example.com:22",
Auth: auth,
})
if(err != nil) {
panic(err)
}
status, err := shell.Run("cd ~/www/www/", handler)
if(err != nil) {
panic(err)
}
status, err := shell.Run("git commit -m 'on the beginning was the commit'", handler)
if(err != nil) {
panic(err)
}
if(status == 0) {
console.log("command executed successfully")
} else {
console.log("command execution failed")
}