UDP server / client in Go not reading data - udp

I have written a UDP server and client in Go. I am not seeing any error messages when the server is running on port 1200, I also do not see any errors when client is trying to connect to port 1200 on the same machine (OS X 10.9.1)
The server is not printing "SOS...", that is the message the client is writing, in a infinite loop.
The client is able to send a message to the server though, however what the server is reading is 0 bytes.
Server Code
package main
import ( "net" "fmt" "time" )
func main() {
port := "127.0.0.1:1200"
udpAddress, err := net.ResolveUDPAddr("udp4",port)
if err != nil {
fmt.Println("error resolving UDP address on ", port)
fmt.Println(err)
return
}
conn ,err := net.ListenUDP("udp",udpAddress)
if err != nil {
fmt.Println("error listening on UDP port ", port)
fmt.Println(err)
return
}
defer conn.Close()
var buf []byte
for {
time.Sleep(100 * time.Millisecond)
n,address, err := conn.ReadFromUDP(buf)
if err != nil {
fmt.Println("error reading data from connection")
fmt.Println(err)
return
}
if address != nil {
fmt.Println("got message from ", address, " with n = ", n)
if n > 0 {
fmt.Println("from address", address, "got message:", string(buf[0:n]), n)
}
}
}
}
client code, running on same server, with command go run udp-client.go :1200 or go run udp-client.go 127.0.0.1:1200
package main
import (
"fmt"
"net"
"os"
"time"
)
func main() {
if len(os.Args) != 2{
fmt.Fprintf(os.Stderr, "Usage:%s host:port", os.Args[0])
os.Exit(1)
}
service := os.Args[1]
fmt.Println("Connecting to server at ", service)
conn, err := net.Dial("udp",service)
if err != nil {
fmt.Println("Could not resolve udp address or connect to it on " , service)
fmt.Println(err)
return
}
fmt.Println("Connected to server at ", service)
defer conn.Close()
fmt.Println("About to write to connection")
for {
time.Sleep(1000*time.Millisecond)
n, err := conn.Write([]byte("SOS ... \n"))
if err != nil {
fmt.Println("error writing data to server", service)
fmt.Println(err)
return
}
if n > 0 {
fmt.Println("Wrote ",n, " bytes to server at ", service)
}
}
}

The UDPConn.ReadFromUDP method reads data and puts it into the slice you provided.
In your case this slice is nil. Therefore your buffer does not provide enough space for data.
You can fix this by changing one line in the server code:
var buf []byte = make([]byte, 1500)
The buffer size should probably be chosen to fit your network protocol. Or you create a 64k buffer so that you can receive maximum sized udp packets. This however seems a bit wastefull :)

Related

How to trigger an Api call when it's the scheduled time

I'm building a simple platform to allow Twitter users to post to their Twitter accounts on a time schedule of their choosing. So for example someone might want to make a post to his/her Twitter account at exactly 12:00 am midnight, but the platform allows them to create the post by 4:00 pm and automatically post to their timeline when it's the scheduled time. I'm confused as to how to implement this since the Api endpoint responsible for this action has to be called manually for this event to take place, I thought of using cron jobs to run every minute and check for all the scheduled posts, call the endpoint if it's time to create the post, but I feel this is cumbersome and error-prone. Are there any more programmatic ways of implementing this, please?
Ps: I'm trying to do this in golang...
You can use a Pub/Consumer model with a message queue. It will make your solution more robust and distributed. I always try not to use sleep in any kind of distributed environment.
Your system will have 3 components:
Producer: This will take care of scheduling and writing messages to a message queue.
Message Queue: You can use RabbitMQ(with delayMessageExchange) or ActiveMQ(it has in-built scheduling). Your message queue will take care of the scheduling problem. Because the message will be delivered only after a specific time delay to the consumer.
Consumer: The consumer will read data from the message queue and perform operations as the functionality
HLD will look like this:
You can use https://pkg.go.dev/github.com/streadway/amqp this package for RabbitMQ
and https://pkg.go.dev/github.com/go-stomp/stomp this package for ActiveMQ
Here is an simple example, I have used AWS ActiveMQ.
package main
import (
"crypto/tls"
"flag"
"fmt"
"github.com/go-stomp/stomp/v3"
"os"
"time"
)
const defaultPort = ":61613"
var serverAddr = flag.String("server", "b-50ad5529-0347-4308-af59-f6265d68d290-1.mq.us-east-1.amazonaws.com:61614", "STOMP server endpoint")
var messageCount = flag.Int("count", 2, "Number of messages to send/receive")
var queueName = flag.String("queue", "/queue/client_test", "Destination queue")
var helpFlag = flag.Bool("help", false, "Print help text")
var stop = make(chan bool)
// these are the default options that work with RabbitMQ
var options []func(*stomp.Conn) error = []func(*stomp.Conn) error{
stomp.ConnOpt.Login("activemquser", "activemqpassword"),
}
func main() {
flag.Parse()
if *helpFlag {
fmt.Fprintf(os.Stderr, "Usage of %s\n", os.Args[0])
flag.PrintDefaults()
os.Exit(1)
}
subscribed := make(chan bool)
go recvMessages(subscribed)
// wait until we know the receiver has subscribed
<-subscribed
go sendMessages()
<-stop
<-stop
}
func sendMessages() {
defer func() {
stop <- true
}()
netConn, err := tls.Dial("tcp", *serverAddr, &tls.Config{})
if err != nil {
println("cannot connect to server", err.Error())
}
conn, err := stomp.Connect(netConn, options...)
if err != nil {
println("cannot connect to server", err.Error())
}
for i := 1; i <= *messageCount; i++ {
text := fmt.Sprintf("Message #%d", i)
fmt.Println("sending message ", text, " ", time.Now())
// scheduling a message with 15 seconds delay
err = conn.Send(*queueName, "text/plain",
[]byte(text), stomp.SendOpt.Header("AMQ_SCHEDULED_DELAY", "15000"))
if err != nil {
println("failed to send to server", err)
return
}
// schedule each message after 3 secs
time.Sleep(3 * time.Second)
}
println("sender finished")
}
func recvMessages(subscribed chan bool) {
defer func() {
stop <- true
}()
netConn, err := tls.Dial("tcp", *serverAddr, &tls.Config{})
if err != nil {
println("cannot connect to server", err.Error())
}
conn, err := stomp.Connect(netConn, options...)
if err != nil {
println("cannot connect to server", err.Error())
}
sub, err := conn.Subscribe(*queueName, stomp.AckAuto)
if err != nil {
println("cannot subscribe to", *queueName, err.Error())
return
}
close(subscribed)
for i := 1; i <= *messageCount; i++ {
msg := <-sub.C
expectedText := fmt.Sprintf("Message #%d", i)
actualText := string(msg.Body)
fmt.Println("got message", actualText, " ", time.Now())
if expectedText != actualText {
println("Expected:", expectedText)
println("Actual:", actualText)
}
}
println("receiver finished")
}

How to optimize database connections

In my Go application I use crontab package to run Tracker function every minute. As you can notice from the code I call PostgreSQL function. To interact with the PostgreSQL database, I use the gorm package. Application worked several days without any problem but now I notice an error in logs: pq: sorry, too many clients already. I know that same questions was asked several times in StackOverflow before. For example in this post people advice to use Exec or Scan methods. In my case as you can see I use Exec method but anyway I have error. As far as I understand, each database request makes a separate connection and does not close it. I can't figure out what I'm doing wrong.
main.go:
package main
import (
"github.com/mileusna/crontab"
)
func main() {
database.ConnectPostgreSQL()
defer database.DisconnectPostgreSQL()
err = crontab.New().AddJob("* * * * *", controllers.Tracker); if err != nil {
utils.Logger().Fatal(err)
return
}
}
tracker.go:
package controllers
import (
"questionnaire/database"
"time"
)
var Tracker = func() {
err := database.DBGORM.Exec("CALL tracker($1)", time.Now().Format("2006-01-02 15:04:05")).Error; if err != nil {
utils.Logger().Println(err) // ERROR: pq: sorry, too many clients already
return
}
}
PostgreSQL.go:
package database
import (
"fmt"
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/postgres"
"github.com/joho/godotenv"
"questionnaire/utils"
)
var DBGORM *gorm.DB
func ConnectPostgreSQL() {
err := godotenv.Load(".env")
if err != nil {
utils.Logger().Println(err)
panic(err)
}
databaseUser := utils.CheckEnvironmentVariable("PostgreSQL_USER")
databasePassword := utils.CheckEnvironmentVariable("PostgreSQL_PASSWORD")
databaseHost := utils.CheckEnvironmentVariable("PostgreSQL_HOST")
databaseName := utils.CheckEnvironmentVariable("PostgreSQL_DATABASE_NAME")
databaseURL:= fmt.Sprintf("host=%s user=%s dbname=%s password=%s sslmode=disable", databaseHost, databaseUser, databaseName, databasePassword)
DBGORM, err = gorm.Open("postgres", databaseURL)
if err != nil {
utils.Logger().Println(err)
panic(err)
}
err = DBGORM.DB().Ping()
if err != nil {
utils.Logger().Println(err)
panic(err)
}
DBGORM.LogMode(true)
}
func DisconnectPostgreSQL() error {
return DBGORM.Close()
}

OpenVAS Go TLS Certificate Error

I'm currently developing a custom scanner in go for OpenVAS. Problem is that the handshake with my custom server fails.
I traced the problem down to error -73: GNUTLS_E_ASN1_TAG_ERROR from gnutls_handshake, but I can't find any resources on that problem. I read something about the certificates being incorrect then but I can't do anything other than regenerating the OpenVAS certificates. The tls functionality in the go server just uses a simple ListenAndServeTLS and gets the server cert and key.
edit:
So this is the relevant network part on the custom scanner:
var (
port = ":1234"
cert = "/usr/local/var/lib/openvas/CA/servercert.pem"
ca = "/usr/local/var/lib/openvas/CA/cacert.pem"
key = "/usr/local/var/lib/openvas/private/CA/serverkey.pem"
)
func start_server() {
ca_file, err := ioutil.ReadFile(ca)
if err != nil {
panic(err)
}
blocks, _ := pem.Decode( ca_file )
ca, err := x509.ParseCertificate(blocks.Bytes)
if err != nil {
panic(err)
}
priv_file, _ := ioutil.ReadFile(key)
blocks2, _ := pem.Decode( priv_file )
priv, err := x509.ParsePKCS1PrivateKey(blocks2.Bytes)
if err != nil {
panic(err)
}
pool := x509.NewCertPool()
pool.AddCert(ca)
cert := tls.Certificate{
Certificate: [][]byte{ca_file},
PrivateKey: priv,
}
config := tls.Config{
ClientAuth: tls.RequireAndVerifyClientCert,
Certificates: []tls.Certificate{cert},
ClientCAs: pool,
}
config.Rand = rand.Reader
service := "0.0.0.0" + port
listener, err := tls.Listen("tcp", service, &config)
if err != nil {
log.Fatalf("server: listen: %s", err)
}
log.Print("server: listening")
for {
conn, err := listener.Accept()
if err != nil {
log.Printf("server: accept: %s", err)
break
}
defer conn.Close()
log.Printf("server: accepted from %s", conn.RemoteAddr())
go handle(conn)
}
}
func handle(conn net.Conn) {
str := "Hello"
defer conn.Close()
buf := make([]byte, 512)
for {
log.Print("server: conn: waiting")
conn.Write( ([]byte)(str) )
n, err := conn.Read(buf)
if err != nil {
if err != nil {
fmt.Println (err)
}
break
}
tlscon, ok := conn.(*tls.Conn)
if ok {
state := tlscon.ConnectionState()
sub := state.PeerCertificates[0].Subject
log.Println(sub)
}
log.Printf("server: conn: echo %q\n", string(buf[:n]))
n, err = conn.Write(buf[:n])
n, err = conn.Write(buf[:n])
log.Printf("server: conn: wrote %d bytes", n)
if err != nil {
log.Printf("server: write: %s", err)
break
}
}
log.Println("server: conn: closed")
}
func main() {
start_server()
}
It's taken from an example but it didn't work properly at first ( there was no decode before parsecertificates). Maybe the certificate is mal-formatted now because of that? Before adding the two decodes I had a similar error about the asn1 tags not matching. So also an asn1 error. I thought of generating my own certificate but I don't know if this will not break OpenVAS for the other scanners. I had the same results when just using listenandservetls from go. The error is definitely produced in gnutls_handshake. It's frustrating that I only get an error code from that.

Fetch an HTTPS Asset from Specific IP and Skipping DNS lookup

I am working on some code for a device that will not be able to query a DNS. I will only have ports 80 and 443 available.
The following works, but of course hits the DNS. The domain used is my personal domain, not the real domain the problem is for - it’s work related and redacted. This is simply used to illustrate the issue.
package main
import (
“log”
“net/http”
)
func main() {
client := &http.Client{}
req, err := http.NewRequest(“GET”, “https://donatstudios.com/images/Spacecat/spacecat.svg”, nil)
if err != nil {
log.Fatal(err)
}
_, err = client.Do(req)
if err != nil {
log.Fatal(err)
}
log.Fatal(“no errors”)
}
I change the code to hit the specific IP address ala:
package main
import (
“log”
“net/http”
)
func main() {
client := &http.Client{}
req, err := http.NewRequest(“GET”, “https://162.243.23.224/images/Spacecat/spacecat.svg”, nil)
if err != nil {
log.Fatal(err)
}
req.Host = “donatstudios.com”
_, err = client.Do(req)
if err != nil {
log.Fatal(err)
}
log.Fatal(“no errors”)
}
And now receive “cannot validate certificate for 162.243.23.224 because it doesn't contain any IP SANs”
When not using an https domain the above code works.
Presumably this is something to do with SSL. #go-nuts told me they believe this to happen before it ever hit the HTTP layer? I’ve been poking this for hours and cannot figure out how to make it work.
I'm assuming your server is using SNI like the one used in the example. Give this a try and see if it works for you.
package main
import (
"crypto/tls"
"log"
"net/http"
)
func main() {
tlsConfig := &tls.Config{
ServerName: "moupon.co",
}
tlsConfig.BuildNameToCertificate()
transport := &http.Transport{TLSClientConfig: tlsConfig}
client := &http.Client{Transport: transport}
req, err := http.NewRequest("GET", "https://216.239.32.21/s/img/logo.png", nil)
if err != nil {
log.Fatal(err)
}
req.Host = "moupon.co"
_, err = client.Do(req)
if err != nil {
log.Fatal(err)
}
log.Fatal("no errors")
}

Golang write input and get output from terminal process

I have a question regarding how to send input and receive output from a terminal subprocess such as ssh. An example in python would be something like this:
how to give subprocess a password and get stdout at the same time
I cannot find a simple example in Golang that is similar how the above work.
In Golang I would want to do something like this but it does not seem to work:
cmd := exec.Command("ssh", "user#x.x.x.x")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
stdin, _ := cmd.StdinPipe()
stdin.Write([]byte("password\n"))
cmd.Run()
However; I'm not sure how to do this in go because every time i exec this ssh command I am only able to get the output. I am unable to input my password automatically from code.
Does anyone have examples of writing to terminal processes such as ssh? If so, please share.
Thanks to the comments above, I was able to get ssh access working with a password. I used golang's ssh api library. It was fairly simple as I followed the examples from:
https://code.google.com/p/go/source/browse/ssh/example_test.go?repo=crypto
Specifically:
func ExampleDial() {
// An SSH client is represented with a ClientConn. Currently only
// the "password" authentication method is supported.
//
// To authenticate with the remote server you must pass at least one
// implementation of AuthMethod via the Auth field in ClientConfig.
config := &ClientConfig{
User: "username",
Auth: []AuthMethod{
Password("yourpassword"),
},
}
client, err := Dial("tcp", "yourserver.com:22", config)
if err != nil {
panic("Failed to dial: " + err.Error())
}
// Each ClientConn can support multiple interactive sessions,
// represented by a Session.
session, err := client.NewSession()
if err != nil {
panic("Failed to create session: " + err.Error())
}
defer session.Close()
// Once a Session is created, you can execute a single command on
// the remote side using the Run method.
var b bytes.Buffer
session.Stdout = &b
if err := session.Run("/usr/bin/whoami"); err != nil {
panic("Failed to run: " + err.Error())
}
fmt.Println(b.String())
}
This is a modified/complete version of above example https://godoc.org/golang.org/x/crypto/ssh#example-Dial
First get terminal package by go get golang.org/x/crypto/ssh
package main
import (
"bufio"
"bytes"
"fmt"
"os"
"strings"
"golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/terminal"
)
func main() {
if len(os.Args) < 3 {
usage := "\n./remote-ssh {host} {port}"
fmt.Println(usage)
} else {
host := os.Args[1]
port := os.Args[2]
username, password := credentials()
config := &ssh.ClientConfig{
User: username,
Auth: []ssh.AuthMethod{
ssh.Password(password),
},
}
connectingMsg := fmt.Sprintf("\nConnecting to %s:%v remote server...", host, port)
fmt.Println(connectingMsg)
hostAddress := strings.Join([]string{host, port}, ":")
// fmt.Println("Host add %s ", hostAddress)
client, err := ssh.Dial("tcp", hostAddress, config)
if err != nil {
panic("Failed to dial: " + err.Error())
}
for {
session, err := client.NewSession()
if err != nil {
panic("Failed to create session: " + err.Error())
}
defer session.Close()
// Once a Session is created, can execute a single command on remote side
var cmd string
str := "\nEnter command (e.g. /usr/bin/whoami OR enter 'exit' to return) : "
fmt.Print(str)
fmt.Scanf("%s", &cmd)
if cmd == "exit" || cmd == "EXIT" {
break
}
s := fmt.Sprintf("Wait for command '%s' run and response...", cmd)
fmt.Println(s)
var b bytes.Buffer
session.Stdout = &b
if err := session.Run(cmd); err != nil {
panic("Failed to run: " + err.Error())
}
fmt.Println(b.String())
}
}
}
func credentials() (string, string) {
reader := bufio.NewReader(os.Stdin)
fmt.Print("Enter Username: ")
username, _ := reader.ReadString('\n')
fmt.Print("Enter Password: ")
bytePassword, err := terminal.ReadPassword(0)
if err != nil {
panic(err)
}
password := string(bytePassword)
return strings.TrimSpace(username), strings.TrimSpace(password)
}
https://play.golang.org/p/4Ad1vKNXmI