Golang - API Server and Socket at the same time - api

Question

I try to make sockets to communicate with my clients.
A socket would be created after some requests to my API. It means, a client connects itself (only by request), but then, he joins a chat, so a socket is created and linked to the good channel.
I already used sockets so I understand how it works (C, C++, C#, Java), but what I want to make, with what I saw on the web, I think it's possible, but I don't understand how to handle it with golang.
I create a first server:
func main() {
r := mux.NewRouter()
r.HandleFunc("/", HomeHandler)
r.HandleFunc("/products", ProductsHandler)
r.HandleFunc("/articles", ArticlesHandler)
http.Handle("/", r)
}
But for socket, I need another one?
package main
import "net"
import "fmt"
import "bufio"
import "strings" // only needed below for sample processing
func main() {
fmt.Println("Launching server...")
// listen on all interfaces
ln, _ := net.Listen("tcp", ":8081")
// accept connection on port
conn, _ := ln.Accept()
// run loop forever (or until ctrl-c)
for {
// will listen for message to process ending in newline (\n)
message, _ := bufio.NewReader(conn).ReadString('\n')
// output message received
fmt.Print("Message Received:", string(message))
// sample process for string received
newmessage := strings.ToUpper(message)
// send new string back to client
conn.Write([]byte(newmessage + "\n"))
}
}
Thank for help !

Answer 1

Based on our chat discussion.
OVERsimplified example with lots of pseudocode
import (
"net"
"encoding/json"
"errors"
)
type User struct {
name string
}
type Message {
Action string
Params map[string]string
}
type Server struct {
connected_users map[*User]net.Conn
users_connected_with_each_other map[*User][]*User
good_users map[string]*User
}
func (srv *Server) ListenAndServe(addr string) error {
ln, err := net.Listen("tcp", addr)
if err != nil {
return err
}
return srv.Serve(tcpKeepAliveListener{ln.(*net.TCPListener)})
}
func (srv *Server) Serve(l net.Listener) error {
defer l.Close()
for {
rw, e := l.Accept()
if e != nil {
return e
}
// you want to create server_conn here with buffers, channels and stuff
// to use async thread safe read/write from it
go srv.serve_conn(rw)
}
}
func (srv *Server) serve_conn(rw net.Conn) error {
dec := json.NewDecoder(rw)
var message Message
//read 1st message he sent, should be token to connect
dec.Decode(&message)
token := get_token(Message)
user, ok := srv.good_users[token]
if !ok {
return errors.New("BAD USER!")
}
// store connected user
srv.connected_users[user] = rw
for {
// async reader will be nice
dec.Decode(&message)
switch message.Action {
case "Message":
// find users to send message to
if chats_with, err := users_connected_with_each_other[user]; err == nil {
for user_to_send_message_to := range chats_with {
// find connections to send message to
if conn, err := srv.connected_users[user_to_send_message_to]; err == nil {
// send json encoded message
err := json.NewEncoder(conn).Encode(message)
//if write failed store message for later
}
}
}
//other cases
default:
// log?
}
}
}
func main() {
known_users_with_tokens := make(map[string]*User)
srv := &Server{
connected_users: make(map[*User]net.Conn),
users_connected_with_each_other: make(map[*User][]*User),
good_users: known_users_with_tokens, // map is reference type, so treat it like pointer
}
// start our server
go srv.ListenAndServe(":54321")
ConnRequestHandler := function(w http.ResponseWriter, r *http.Request) {
user := create_user_based_on_request(r)
token := create_token(user)
// now user will be able to connect to server with token
known_users_with_tokens[token] = user
}
ConnectUsersHandler := function(user1,user2) {
// you should guard your srv.* members to avoid concurrent read/writes to map
srv.users_connected_with_each_other[user1] = append(srv.users_connected_with_each_other[user1], user2)
srv.users_connected_with_each_other[user2] = append(srv.users_connected_with_each_other[user2], user1)
}
//initialize your API http.Server
r := mux.NewRouter()
r.HandleFunc("/", HomeHandler)
r.HandleFunc("/products", ProductsHandler)
r.HandleFunc("/articles", ArticlesHandler)
r.HandleFunc("/connection_request", ConnRequestHandler) // added
http.Handle("/", r)
}
Call ConnectUsersHandler(user1, user2) to allow them communicate with each other.
known_users_with_tokens[token] = user to allow user to connect to server
You need to implement async reader/writer for connections to your server. Usefull structs to keep good Users.
Guard Server struct members and provide thread safe access to update it.
UDP
Looks like json.NewEncoder(connection).Encode(&message) and json.NewDecoder(connection).Decode(&message) is async and thread safe. So you can write simultaneously from different goroutines. No need to manual sync, YAY!

Answer 2

default http server accepts connection on one "host:port" only
Answer depends on what protocol you are going to use to communicate via your sockets.
I suggest to do it this way: (much simplified)
Leave http.Server alone to serve your API (it implements protocols HTTP 1.*/2 so you dont need to worry about it)
Implement your own "MultiSocketServer", do to so:
2.1 Implement GracefulListener (must implement net.Listener) (you need to shutdown your sockets when you dont need them anymore, right?)
2.2 Implement MultiSocketServer.Serve(l GracefulListener) (hello http.Server.Serve() ) to serve individual connection (your protocol to communicate with client via sockets goes here. something like net.textproto will be easy to implement since you GracefulListener.Accept() will return net.Conn)
2.3 Add methods MultiSocketServer.ListenAndServe(addr), MultiSocketServer.StopServe(l GracefulListener) to your MultiSocketServer
type MultiSocketServer struct {
listeners GracefulListener[] //or map?
// lots of other stuff
}
// looks familiar? (http.Server.ListenAndServe)
func (s *MultiSocketServer) ListenAndServe(addr string) {
ln, err := net.Listen("tcp", addr)
graceful_listner = &GracefulListener(ln)
s.listeners = append(s.listeners, graceful_listner)
go s.Serve(graceful_listner)
return graceful_listner
}
func (s *MultiSocketServer) StopServe(graceful_listner GracefulListener) {
graceful_listner.Stop()
//pseudocode
remove_listener_from_slice(s.listeners, graceful_listner)
}
Ofcourse, you need to add error checking and mutex (propably) to guard MultiSocketServer.listeners to make it thread safe.
In your main() start your API http.Server, and initialize your MultiSocketServer. Now from your http.Handler/http.HandlerFunc of http.Server you should be able to call MultiSocketServer.ListenAndServe(addr) to listen and serve your sockets connections.
UPDATE based on question
however, I'm not sure to understand the part "In your main()". If I understand it good, you said I have my API, and after starting it, I initialize MultiSocketServer. But where? after the starting of my API? Or you mean it would be better that I use the logic of your code as an API? Every request trough a socket
BTW: updated MultiSocketServer.ListenAndServe to start Listen and return graceful_listner
func main() {
//init MultiSocketServer
multi_socket_server = &MultiSocketServer{} //nil for GracefulListener[] is fine for now, complex initialization will be added later
// no listners yet, serves nothing
// create new Handeler for your "socket requests"
SocketRequestHandler := function(w http.ResponseWriter, r *http.Request) {
// identify client, assign him an address to connect
addr_to_listen := parse_request(r) //pseudocode
listener := multi_socket_server.ListenAndServe(addr_to_listen)
// TODO: handle errors
// now your multi_socket_server listen to addr_to_listen and serves it with multi_socket_server.Serve method in its own goroutine
// as i said MultiSocketServer.Serve method must implement your protocol (plaintext Reader/Writer on listener for now?)
save_listener_in_context_or_whatever_you_like_to_track_it(listener) //pseudo
}
SocketDisconnectHandler := function(w http.ResponseWriter, r *http.Request) {
// identify client
some_client := parse_request(r) //pseudocode
// get listener based on info
listener := get_listener_from_context_or_whatever(some_client) //pseudo
multi_socket_server.StopServe(listener)
// TODO: handle errors
}
//initialize your API http.Server
r := mux.NewRouter()
r.HandleFunc("/", HomeHandler)
r.HandleFunc("/products", ProductsHandler)
r.HandleFunc("/articles", ArticlesHandler)
r.HandleFunc("/socket_request", SocketRequestHandler) // added
r.HandleFunc("/socket_disconnect", SocketDisconnectHandler) //added
http.Handle("/", r)
// it creates new http.Server with DefaultServeMux as Handler (which is configured with your http.Handle("/", r) call)
http.ListenAndServe(":8080") // start serving API via HTTP proto
}
Actually, you may call multi_socket_server.ListenAndServe(addr_to_listen) and multi_socket_server.StopServe(listener) from any handler in your API server.
Every time you call multi_socket_server.ListenAndServe(addr_to_listen) it will create new listener and serve on it, you have to control it (dont listen on the same address more then once (i think it will error out anyway))
Your MultiSocketServer.Serve may looks like:
func (s *MultiSocketServer) Serve(l net.Listener) {
defer l.Close()
for {
// will listen for message to process ending in newline (\n)
message, _ := bufio.NewReader(conn).ReadString('\n')
// output message received
fmt.Print("Message Received:", string(message))
// sample process for string received
newmessage := strings.ToUpper(message)
// send new string back to client
conn.Write([]byte(newmessage + "\n"))
}
}
Possible GracefulListener implementation github
Or are you trying to achieve something completely different? =)