Why are my Nuxt/vue pages getting blocked by robot.txt? - vue.js

Question

This is not a question about best practice within SEO but a question about how to setup config.js and script sections correctly in VUE
I have build my site with Vue/Nuxt and what have earlier been a walk in the park for me with angular, are now causing errors.
My overall problem is that I am not sure if I have build my script section right as my pages are not getting indexed by Google. In my nuxt.config.js file I have build my sitemap, robot.txt and some general meta-tags. For every page I have build dynamic meta tags in their script section.
Google Search Console is giving 3 type of errors.
Some pages are blocked by my robot.txt
Some pages is said to be dublicated rel-canonical
When inspecting my site it cant fint pages without typing in '/' at the end of the URL. This is also seen when using Screaming Frog SEO tool.
My assumption is that I am missing some form of a redirect that makes the crawler index pages ending with '/' as those are getting indexed fine in Search Console?
Nuxt.config.js file (Sections, not all content is showed)
head: {
title: 'NorthArc',
meta: [
{ charset: 'utf-8' },
{ name: 'viewport', content: 'width=device-width, initial-scale=1' },
{ name: 'language', content: 'da_DK' },
{ name: 'robots', content: 'index, follow' },
{ name: 'og:type', content: 'website' },
],
link: [
{ rel: 'icon', type: 'image/x-icon', href: '/favicon.ico' },
]
},
sitemap: {
path: '/sitemap.xml',
hostname: 'https://northarc.dk/',
routes: [
{
url: '/',
changefreq: 'monthly',
priority: 1,
},
{
url: '/team/',
changefreq: 'monthly',
priority: 1,
},
{
url: '/groen-planlaegning/',
changefreq: 'monthly',
priority: 1,
},
{
url: '/strategisk-samarbejde/',
changefreq: 'monthly',
priority: 1,
},
{
url: '/blog/',
changefreq: 'monthly',
priority: 1,
},
{
url: '/blog/er-ruteplanlaegning-svaert/',
changefreq: 'monthly',
priority: 1,
},
{
url: '/blog/automatisk-ruteplanlaegning/',
changefreq: 'monthly',
priority: 1,
},
{
url: '/faq/',
changefreq: 'monthly',
priority: 1,
},
{
url: '/contact/',
changefreq: 'monthly',
priority: 1,
},
{
url: '/policies/',
changefreq: 'monthly',
priority: 1,
}
]
},
robots: {
UserAgent: 'Googlebot',
Disallow: ['/roi', '/pricing'],
Sitemap: 'https://northarc.dk/sitemap.xml',
},
Script section from a page that is said to be blocked bt robot.txt and has dublicated rel-canonical.
<script>
export default {
name: 'home',
head() {
return {
title: 'test',
meta: [
{
hid: 'description',
name: 'description',
content: 'test',
},
{ hid: 'robots', name: 'robots', content: 'index, follow' },
{hid: 'og-title', property: 'og:title', content: 'Fjern spildtid på vejen og minimere antal kørte kilometer'},
{hid: 'og-url', property: 'og:url', content: 'https://northarc.dk/groen-planlaegning'},
{hid: 'og-description', property: 'og:description', content: 'test'},
{hid: 'og-image', property: 'og:image', content: '/Applications/Northarc_landing/assets/Preview_sløret.jpg'},
],
link: [
{
rel: 'canonical',
href: 'https://northarc.dk/groen-planlaegning/'
}
]
}
}
};
</script>
Notes: (change-log)
I have tried to add a '/' to all sites URL in my sitemap and at the rel-canonical for the pages example showed above.
I have tried to change the user of robot.txt to googlebot to disallow two pages. Before the user was set to '*' where it still blocked some pages.

Answer 1

By default Nuxt allows each route without or with a trailing slash, example:
https://northarc.dk/blog
https://northarc.dk/blog/
It can be detect as duplicate content by crawlers.
So you can define which is the main URL with the "canonical" header.
But if you want keep URLs only with a trailing slash at end, you have to allow only route with a trailing slash by the router configuration:
// nuxt.config.js
router: {
trailingSlash: true
}
See docs https://nuxtjs.org/docs/2.x/configuration-glossary/configuration-router#trailingslash
in addition,
you don't need to hardcoded all your routes in the sitemap-module config, it's automatic for all static routes, eg.:
// nuxt.config.js
sitemap: {
hostname: 'https://northarc.dk',
defaults: {
changefreq: 'monthly',
priority: 1,
trailingSlash: true
},
exclude: ['roi', 'pricing'],
trailingSlash: true // if necessary
},