I'm trying to execute a command via session.Run() function over a ssh connection. So far I can successfully execute some commands but on others I keep getting the following error: "Process exited with: 1. Reason was: () exit status 1"
func (p *project) connect(config *ssh.ClientConfig) {
log.Printf("Trying connection...\n")
conn, err := ssh.Dial("tcp", fmt.Sprintf("%s:%s", p.hostname.name, p.port.name), config)
checkError("Failed to dial: ", err)
log.Printf("Connection established.\n")
for step := range p.typ.program.setup {
p.install(step, conn)
}
}
func (p *project) install(step int, conn *ssh.Client) {
session, err := conn.NewSession()
checkError("Failed to build session: ", err)
defer session.Close()
var stdoutBuf bytes.Buffer
session.Stdout = &stdoutBuf
log.Printf("Executing command: %s", p.typ.program.setup[step])
if err := session.Run(p.typ.program.setup[step]); err != nil {
log.Println(session.Stdout)
log.Fatal("Error on command execution", err.Error())
}
}
// That would be an example of a command which returns me an error
// "cd ~/www/www/ && git commit -m 'on the beginning was the commit'"
// That comes inside a slice on p.typ.program.setup accessed by the step(index).
The command output (session.Stdout) is the one i expect:
"# On branch master nothing to commit, working directory clean"
And just to note I already tried to execute the command directly on the console and it works just fine.
So, the code seems to be okay, the command ran on the remote but I still have an error no matter what.
Does anyone have a clue about why is that happening?
Thanks in advance.
Maybe my library will helps in your case: https://github.com/shagabutdinov/shell; it covers basic cases of running ssh commands in one session.
Try following:
handler := func(outputType int, message string) {
if(outputType == shell.Stdout) {
log.Println("stdout: ", message)
} else if(outputType == shell.Stdout) {
log.Println("stderr: ", message)
}
}
key, err := ssh.ParsePrivateKey([]byte(YOUR_PRIVATE_KEY))
if(err != nil) {
panic(err)
}
auth := []ssh.AuthMethod{ssh.PublicKeys(key)}
shell = shell.NewRemote(shell.RemoteConfig{
Host: "root#example.com:22",
Auth: auth,
})
if(err != nil) {
panic(err)
}
status, err := shell.Run("cd ~/www/www/", handler)
if(err != nil) {
panic(err)
}
status, err := shell.Run("git commit -m 'on the beginning was the commit'", handler)
if(err != nil) {
panic(err)
}
if(status == 0) {
console.log("command executed successfully")
} else {
console.log("command execution failed")
}
Related
I'm using the Godog library to implement some cucumbers tests for my api code, right now I'm only testing one endpoint but I'm hitting an error where it looks like it's expecting to have a server open. I created a httptest server that listens to port 8080 but the tests are failing with a 404.
If I run my cucumber in debug mode they work but if I use the run test command they fail cos the expect an open port dial tcp localhost:8080. Could someone point me to the right direction since I quite don't know where I'm failing.
This is my godog_test
`
func mockServer() *httptest.Server {
router := mux.NewRouter()
u, _ := url.Parse("http://localhost:8080")
l, _ := net.Listen("tcp", u.Host)
server := httptest.NewUnstartedServer(router)
_ = server.Listener.Close()
server.Listener = l
server.Start()
return server
}
func killMockServer(server *httptest.Server) {
server.Close()
}
func TestFeatures(t *testing.T) {
suite := godog.TestSuite{
TestSuiteInitializer: InitializeTestSuite,
ScenarioInitializer: InitializeScenario,
Options: &godog.Options{
Format: "pretty",
Paths: []string{"features"},
TestingT: t,
},
}
if suite.Run() != 0 {
t.Fatal("non-zero status returned, failed to run feature tests")
}
}
func InitializeTestSuite(ctx *godog.TestSuiteContext) {
var server *httptest.Server
ctx.BeforeSuite(func() {
server = mockServer()
})
ctx.AfterSuite(func() {
fmt.Println("shutting down everything")
killMockServer(server)
})
}
`
Post step that I'm testing
`
func iCallPOSTTo(path string) error {
req, err := json.Marshal(reqBody)
if err != nil {
return err
}
request, err := http.NewRequest(
http.MethodPost,
endpoint+path,
bytes.NewReader(reqBody),
)
res, err := http.DefaultClient.Do(request)
if err != nil {
return err
}
resBody, err := io.ReadAll(res.Body)
if err != nil {
return err
}
res.Body.Close()
[REDACTED]
return nil
}
`
I tried using a mock server to open port 8080 since at first I was receiving a connection refused error, after that I'm getting a 404 which means that my test is not reaching my actual function that processes the post request. I'm not sure if the mock server is the correct approach on this case.
Summary
It seems pretty simple to implement an exclusive consumer in RabbitMQ. You just need to enable the exclusive flag while starting the consumption or that's what I thought. I tried this approach but for some reason, it's creating multiple exclusive consumers which actually contradicts with it's own definition.
Setup
I am using following setup -
Exchange: default
Queue: quorum, durable
Messages: durable
Here is the full code that I am using to register exclusive consumer -
package main
import (
"fmt"
amqp "github.com/rabbitmq/amqp091-go"
"sync"
"time"
)
type Config struct {
Schema string
Host string
Port string
Username string
Password string
Vhost string
}
type Rabbit struct {
config Config
connection *amqp.Connection
lock sync.Mutex
}
// NewRabbit returns a Rabbit instance.
func NewRabbit() *Rabbit {
// setup appropriate values
config := Config{
Host: "",
Username: "",
Password: "",
Port: "",
Vhost: "",
Schema: "",
}
return &Rabbit{
config: config,
}
}
// Connect connects to RabbitMQ server.
func (r *Rabbit) Connect() error {
r.lock.Lock()
defer r.lock.Unlock()
// Check if connection is already available
if r.connection == nil || r.connection.IsClosed() {
// Try connecting
con, err := amqp.DialConfig(fmt.Sprintf(
"%s://%s:%s#%s:%s/%s",
r.config.Schema,
r.config.Username,
r.config.Password,
r.config.Host,
r.config.Port,
r.config.Vhost,
), amqp.Config{})
if err != nil {
return err
}
r.connection = con
}
return nil
}
func (r *Rabbit) StartConsumer(queueName string) error {
chn, err := r.connection.Channel()
if err != nil {
return err
}
// Make sure we process 1 message at a time
if err := chn.Qos(1, 0, false); err != nil {
return err
}
_, err = chn.QueueDeclare(
queueName,
true,
false,
false,
false,
amqp.Table{"x-queue-type": "quorum"}) // This will ensure that the created queue is quorum-queue
if err != nil {
fmt.Printf("Error creating queue with name: %s, err: %s", queueName, err.Error())
return err
}
messages, err := chn.Consume(
queueName,
queueName+"-consumer",
false,
true,
false,
false,
nil,
)
if err != nil {
fmt.Printf("Unable to start consumer for webhook queue: %s, err: %s", queueName, err.Error())
return err
}
go func() {
// This for-loop will wait indefinitely or until channel is closed
for msg := range messages {
fmt.Printf("Message: %v", msg.Body)
if err = msg.Ack(false); err != nil {
fmt.Printf("Unable to acknowledge the message, err: %s", err.Error())
}
}
}()
return nil
}
Here is the main.go
package main
import (
"fmt"
"os"
"os/signal"
"syscall"
)
func main() {
interrupt := make(chan os.Signal)
signal.Notify(interrupt, os.Interrupt, syscall.SIGINT, syscall.SIGTERM)
defer signal.Stop(interrupt)
rabbit := NewRabbit()
if err := rabbit.Connect(); err != nil {
fmt.Printf("Can't connect to RabbitMQ server, err: %s", err.Error())
} else {
fmt.Println("Successfully connected to RabbitMQ server")
}
err := rabbit.StartConsumer("test-queue")
if err != nil {
fmt.Printf("Error: %s", err.Error())
}
select {
case <-interrupt:
fmt.Println("Interrupt signal received")
break
}
fmt.Println("Application is about to close")
}
Here is go.mod
module ExclusiveRabbitMQConsumer
go 1.17
require github.com/rabbitmq/amqp091-go v1.3.4
And here is what I see after running 3 instances of this application -
RabbitMQ Management UI
All 3 consumers are shown as exclusive. The consumer tags are same however that doesn't matter according to RabbitMQ documentation.
I'm not sure what's wrong here.
I am trying to spin up a Postgres container via the Docker Go SDK. I can get the container started, and I copy my SQL file into the container and verified the file is there.
I cannot run this file, which right now just contains CREATE TABLE tester;
Here is my Go code that I am trying to use:
package gosqlcontainer
import (
"archive/tar"
"bufio"
"bytes"
"context"
"fmt"
"io"
"log"
"os"
"os/exec"
"time"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/client"
)
func StartContainer() string {
ctx := context.Background()
cli, err := client.NewClientWithOpts(client.WithAPIVersionNegotiation())
if err != nil {
panic(err)
}
imageName := "postgres"
out, err := cli.ImagePull(ctx, imageName, types.ImagePullOptions{})
if err != nil {
panic(err)
}
defer out.Close()
io.Copy(os.Stdout, out)
container, err := cli.ContainerCreate(ctx, &container.Config{
Image: imageName, Env: []string{"POSTGRES_PASSWORD=password"},
}, nil, nil, nil, "")
if err != nil {
panic(err)
}
if err := cli.ContainerStart(ctx, container.ID, types.ContainerStartOptions{}); err != nil {
panic(err)
}
tarFile := TarFile("test.sql")
if err = cli.CopyToContainer(ctx, container.ID, "/home/", tarFile, types.CopyToContainerOptions{AllowOverwriteDirWithFile: true}); err != nil {
fmt.Println("error copying to container", err)
}
time.Sleep(1 * time.Second)
execCommand, err := cli.ContainerExecCreate(ctx, container.ID, types.ExecConfig{AttachStdin: true, AttachStderr: true, AttachStdout: true, Cmd: []string{"psql", "-U postgres -f /home/test.sql"}})
if err != nil {
fmt.Println("exec err is", err)
}
if err := cli.ContainerExecStart(ctx, execCommand.ID, types.ExecStartCheck{}); err != nil {
fmt.Println("Err Start", err)
}
return container.ID
}
When I try to do this from CLI, I can run that file and verify that the DB is created. However, programmatically it does not work, and I do not see any errors returned.
The goal is for this to be a library I can reference to spin up a container, import data, and used for integration tests. I am currently calling it from a basic client locally, using go run main.go I am working on macOS Monterey v12.4
I am trying to write a simple admission controller for pod naming (validation) but for some reason I am generating a wrong response.
Here is my code:
package main
import (
"fmt"
"encoding/json"
"io/ioutil"
"net/http"
"github.com/golang/glog"
// for Kubernetes
"k8s.io/api/admission/v1beta1"
"k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"regexp"
)
type myValidServerhandler struct {
}
// this is the handler fuction from the HTTP server
func (gs *myValidServerhandler) serve(w http.ResponseWriter, r *http.Request) {
var Body []byte
if r.Body != nil {
if data , err := ioutil.ReadAll(r.Body); err == nil {
Body = data
}
}
if len(Body) == 0 {
glog.Error("Unable to retrive Body from API")
http.Error(w,"Empty Body", http.StatusBadRequest)
return
}
glog.Info("Received Request")
// this is where I make sure the request is for the validation prefix
if r.URL.Path != "/validate" {
glog.Error("Not a Validataion String")
http.Error(w,"Not a Validataion String", http.StatusBadRequest)
return
}
// in this part the function takes the AdmissionReivew and make sure in is in the right
// JSON format
arRequest := &v1beta1.AdmissionReview{}
if err := json.Unmarshal(Body, arRequest); err != nil {
glog.Error("incorrect Body")
http.Error(w, "incorrect Body", http.StatusBadRequest)
return
}
raw := arRequest.Request.Object.Raw
pod := v1.Pod{}
if err := json.Unmarshal(raw, &pod); err != nil {
glog.Error("Error Deserializing Pod")
return
}
// this is where I make sure the pod name contains the kuku string
podnamingReg := regexp.MustCompile(`kuku`)
if podnamingReg.MatchString(string(pod.Name)) {
return
} else {
glog.Error("the pod does not contain \"kuku\"")
http.Error(w, "the pod does not contain \"kuku\"", http.StatusBadRequest)
return
}
// I think the main problem is with this part of the code because the
// error from the events I getting in the Kubernetes namespace is that
// I am sending 200 without a body response
arResponse := v1beta1.AdmissionReview{
Response: &v1beta1.AdmissionResponse{
Result: &metav1.Status{},
Allowed: true,
},
}
// generating the JSON response after the validation
resp, err := json.Marshal(arResponse)
if err != nil {
glog.Error("Can't encode response:", err)
http.Error(w, fmt.Sprintf("couldn't encode response: %v", err), http.StatusInternalServerError)
}
glog.Infof("Ready to write response ...")
if _, err := w.Write(resp); err != nil {
glog.Error("Can't write response", err)
http.Error(w, fmt.Sprintf("cloud not write response: %v", err), http.StatusInternalServerError)
}
}
The code is working as expected except for a positive output (where the pod name meets the criteria)
there is another file with a main just grabbing the TLS files and starting the HTTP service.
so after a few digging I found what was wrong with my code
first this part
if podnamingReg.MatchString(string(pod.Name)) {
return
} else {
glog.Error("the pod does not contain \"kuku\"")
http.Error(w, "the pod does not contain \"kuku\"", http.StatusBadRequest)
return
}
by writing "return" twice I discarded the rest of the code and more so I haven't attached the request UID to the response UID and because I am using the v1 and not the v1beta1 I needed to adding the APIVersion in the response
so the rest of the code looks like :
arResponse := v1beta1.AdmissionReview{
Response: &v1beta1.AdmissionResponse{
Result: &metav1.Status{},
Allowed: false,
},
}
podnamingReg := regexp.MustCompile(`kuku`)
if podnamingReg.MatchString(string(pod.Name)) {
fmt.Printf("the pod %s is up to the name standard", pod.Name)
arResponse.Response.Allowed = true
}
arResponse.APIVersion = "admission.k8s.io/v1"
arResponse.Kind = arRequest.Kind
arResponse.Response.UID = arRequest.Request.UID
so I needed to add the 2 parts and make sure that in case the pod name is not up to standard then I need to return the right response
I made a model serving server with Python Tornado library and its sole purpose is to accept http request with payload and return result in json. The request can be made with either application/json or multipart/form-data.
To authenticate and authorise users, I made another server with Golang echo library. So all user requests should reach here before reaching my resource server.
Here I have a problem, because my program requires images as input, so users will dispatch their request with FormData. When it first hit my Golang server, I need to do the following steps
Read the form file.
Save it in local disk.
Load the file and save it in a byte buffer.
Initialise a multipart writer
Make a request to my resource server
Got result, return to user
I feel like this is redundant as I imagine there is a way to propagate those request directly to my resource server (after auth is done), without having to go through the I/O parts.
My code currently looks like this, at this point authentication is done through middleware. Is there a way to optimise this flow?
func (h Handler) ProcessFormData(c echo.Context) error {
// some validation
file, err := c.FormFile("file")
if err != nil {
return c.JSON(http.StatusBadRequest, response.Exception{
Code: errcode.InvalidRequest,
Detail: "Invalid file uploaded",
Error: err,
})
}
filePath, err := fileUtil.SaveNetworkFile(file)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
f, err := os.Open(filePath)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
defer f.Close()
fi, err := f.Stat()
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
var body bytes.Buffer
writer := multipart.NewWriter(&body)
part, err := writer.CreateFormFile("file", fi.Name())
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
if _, err := io.Copy(part, f); err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
writer.Close()
req, err := http.NewRequest("POST", fmt.Sprintf("%s", env.ResourceServer), &body)
req.Header.Set("Content-Type", writer.FormDataContentType())
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.APIRequestError,
Error: err,
})
}
client := &http.Client{}
res, err := client.Do(req)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.APIRequestError,
Detail: "Error when posting request to resource server",
Error: err,
})
}
defer res.Body.Close()
data, _ := ioutil.ReadAll(res.Body)
if res.StatusCode != 200 {
errorData := &model.PanicResponse{}
err := json.Unmarshal(data, errorData)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.UnmarshalError,
Error: err,
})
}
return c.JSON(res.StatusCode, errorData)
}
result := &model.SuccessResponse{}
err = json.Unmarshal(data, result)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.UnmarshalError,
Error: err,
})
}
if fileUtil.IsFileExists(filePath) {
fileUtil.DeleteFile(filePath)
}
// track and update usage
userData := c.Get("USER")
user := userData.(model.User)
db.UpdateUsage(h.Db, &user.ID)
return c.JSON(200, result)
}
Found a solution thanks to the comment from #cerise-limón
Essentially, I need just 2 lines
f, err := file.Open()
if _, err := io.Copy(part, f); err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
})
}