Using Vue and Webpack to create an MPA - vue.js

Question

I am trying to learn how to use Vue and Webpack 4 to create a multi-page application. The main reason for this is that is that the server is not stateless and I need the server to handle routing because of the complexity of permissions and protected routes.
However, I want to use Vue.js to leverage the power of single file components to create reusable and maintainable code. I understand how to modify my wepack.config.js to set up multiple entry points and I assume that I would just be serving a different bundle that would be injected into a single index.html but there is where the things start becoming unclear to me.
How would I handle this with Vue.js? Just to be clear, I am not using the Vue CLI and I want to use single file components to design the front end. A simple project skeleton/boilerplate code would be much appreciated with an emphasis on configuration.
My main entry point (main.js)
import Vue from 'vue';
import App from './App.vue';
import router from './router';
import $ from 'jquery';
import 'bootstrap/dist/js/bootstrap.js';
import 'bootstrap/dist/css/bootstrap.min.css';
import '#fortawesome/fontawesome-free/js/all.js'
Vue.config.productionTip = false;
new Vue({
el: '#app',
router,
render: h => h(App),
});
webpack.config.js
const HtmlWebpackPlugin = require('html-webpack-plugin');
const VueLoaderPlugin = require('vue-loader/lib/plugin');
const CopyPlugin = require('copy-webpack-plugin');
const webpack = require('webpack');
const path = require('path');
module.exports = {
entry: './src/main.js',
module: {
rules: [
{ test: /\.js$/, use: 'babel-loader', exclude: '/node_modules'},
{ test: /\.vue$/, use: 'vue-loader' },
{ test: /\.css$/, use: ['vue-style-loader', 'css-loader']},
//{ test: /\.(png|svg|jpg|gif)$/, use: 'file-loader'},
]
},
devServer: {
open: true,
hot: true,
},
resolve: {
alias: {
'#': path.resolve(__dirname, 'src')
}
},
plugins: [
new HtmlWebpackPlugin({template: './src/index.html'}),
new VueLoaderPlugin(),
//new webpack.HotModuleReplacementPlugin(),
new CopyPlugin([{from: 'src/images', to: 'images'}])
]
};