How to fetch big data in vue - vue.js

Csv Link:https://raw.githubusercontent.com/owid/covid-19-data/master/public/data/owid-covid-data.csv
Hello, I need to import the above csv file, but I need to do it from this link to be up to date. When I take this data and use pinia for storage data, about 700mb is added to my memory usage every time I refresh the page. Since I don't know the backend, I need to fetch and use the data like this.
What should I do so that the memory usage does not increase every time I refresh the page?
Pinia Code:
import {defineStore} from "pinia";
import {computed, ref} from "vue";
export const useCovidDataStore = defineStore('covidData', () => {
const data = ref(null)
const loaded = ref(false)
const selectedCountry = ref('Germany')
function setData(veri) {
data.value = veri
}
function setCountry(country) {
selectedCountry.value = country
}
function setLoaded() {
loaded.value = !loaded.value
}
const getData = computed(() => data.value)
const getLoaded = computed(() => loaded)
const getSelectedCountry = computed(()=>selectedCountry)
return {data, setData, getData, setLoaded, getLoaded, loaded,selectedCountry,setCountry,getSelectedCountry}
})
Data Code:
Papa.parse("https://raw.githubusercontent.com/owid/covid-19-data/master/public/data/owid-covid-data.csv", {
download: true,
header: true,
complete: function(results) {
const groupedByLocation = results.data.reduce((acc, obj) => {
if (!acc[obj.location]) {
acc[obj.location] = [];
}
acc[obj.location].push(obj);
return acc;
}, {});
store.setData(groupedByLocation)
}
})

When you have a huge amount of data you can either:
use some pagination to get only some chunks and not the whole thing
use a middleware backend/serverless function to do that work on a more appropriate place
No other magic sauce. .slice'ing on the frontend will not make the loading of the page more performant, just reduce the DOM overhead of injecting all of those nodes.
Memoization can also help, but not on the initial load.

Related

Updates are slow with Redux

I'm building a React Native app, using Redux and AsyncStorage. When a user deletes an item, updates happen a bit slow (2-3 seconds).
I have an array of objects (rather small).
I delete an item in this array with such function:
let idFordeleteFav = categoryArrFav.map(function(el) {return el['name']}).indexOf(itemName)
let cleanedCategory = [...categoryArrFav.slice(0, idFordeleteFav), ...categoryArrFav.slice(idFordeleteFav+1)]
let completeNewList = {...allAffirmation, [category]: cleanedCategory}
props.dispatch(setAffArr(completeNewList))
My mapStateToProps looks like this:
const mapStateToProps = (state) => {
const { appR } = state
return {
allAffirmation: appR.allAffirmation,
affirmations: appR.affirmations}
}
I wonder, what I can do to update faster.
First of all, check how much middleware you have and the logic inside them. Also, I recommend you replace async storage with react native MMKV (https://github.com/mrousavy/react-native-mmkv) which is much faster:
You need to do 2 things. First, create a wrapper for MMKV because it is sync:
import { Storage } from 'redux-persist'
import { MMKV } from "react-native-mmkv"
const storage = new MMKV()
export const reduxStorage: Storage = {
setItem: (key, value) => {
storage.set(key, value)
return Promise.resolve(true)
},
getItem: (key) => {
const value = storage.getString(key)
return Promise.resolve(value)
},
removeItem: (key) => {
storage.delete(key)
return Promise.resolve()
},
}
The second step is migration from the async store for existing users if you have https://github.com/mrousavy/react-native-mmkv/blob/master/docs/MIGRATE_FROM_ASYNC_STORAGE.md

How to pass reactive data to Vue components using pinia store elements?

GamePage.vue
Destructured the pinia state elements and action method
const $store = useGameStore();
const {game, teamOne, teamTwo} = storeToRefs($store);
const { getGame } = $store;
Passed the destructed variables to components
<player-stat-table
:title="teamTwo.name"
:players="teamTwo.players"
:teamColor="teamTwo.team_color"
/>
Table Display
store/game_store.js
I am trying to edit data from the above table using updatePlayer action, after successfully completing the action I am updating the entire store data by recalling the get action method. But the data in the table is not updating reactively, it's updating after page reload. How to update it reactively?
import { api } from 'boot/axios'
import { defineStore } from 'pinia'
import { splitPlayers } from 'src/helpers'
export const useGameStore = defineStore('game', {
state: () => ({
game: null,
teamOne: null,
teamTwo: null,
}),
getters: {
getTeamOne: state => state.teamOne,
getTeamTwo: state => state.teamTwo,
getGameData: state => state.game,
},
actions: {
getGame(payload) {
return new Promise((resolve, reject) => {
api.get(`/games/${payload.gameID}/`)
.then(resp => {
const data = resp.data;
const teams = splitPlayers(data)
this.game = data
this.teamOne = teams[0]
this.teamTwo = teams[1]
resolve(data)
})
})
},
updatePlayer(payload) {
return new Promise((resolve, reject) => {
api.put(`/playerstat/${payload.id}/`, data)
.then(resp => {
const data = resp.data;
this.getGame({gameID: data.game})
resolve(data)
})
})
},
}
})
First, you can get rid of you getters, cause due to pinia documentation,
as getters you can think of as the computed properties
and you're not computing anything. So you can simply access the state properties, what you are already doing in your GamePage.vue file.
Secondly, you should also consider async/await pattern instead of Promiste.then(). Like mentioned in the comments, there's a problem with promise constructor antipattern in the OP.
I also prefer writing my pinia stores with the setup() approach, because I think it fits the vue3/composition-api approach a bit better.
import { api } from 'boot/axios'
import { defineStore } from 'pinia'
import { splitPlayers } from 'src/helpers'
export const useGameStore = defineStore('game', () => {
const game = ref(null);
const teamOne = ref(null);
const teamTwo = ref(null);
const getGame = async (gameId) => {
const resp = await api.get(`/games/${gameId}/`);
const teams = splitPlayers(resp.data)
game.value = resp.data
teamOne.value = teams[0]
teamTwo.value = teams[1]
};
const updatePlayer = async (data) => {
const resp = await api.put(`/playerstat/${data.id}/`, data)
const gameId = resp.data.game;
await getGame(gameId)
};
return {
game,
teamOne,
teamTwo,
getGame,
updatePlayer
}
});

How to get state in Nuxt js with composition api?

setup(){
const columns = computed(()=>store.state['subCategory'].subCategoryColumnsData[subCategoryName.value]);
const { fetch } = useFetch(async () => {
await store.dispatch('subCategory/getColumnsQuery', {
categories: subCategoryId.value,
page: 1,
subCategoryName: subCategoryName.value,
})
});
fetch();
}
I want to switch between pages in my project. Whenever I switched another page, I send request to get data with latest updates. This code works well for the first time when page was loaded, but it doesn't work when I switched from one page to another page. But if I check store state, I can see it in store. If I visit same page second time , I can see data this time.
But if I change my code like this, it works well. I did not get why it does not work true in the first sample
setup(){
const columns = ref([])
const { fetch } = useFetch(async () => {
await store.dispatch('subCategory/getColumnsQuery', {
categories: subCategoryId.value,
page: 1,
subCategoryName: subCategoryName.value,
})
}).then(() => (columns.value = store.state['subCategory'].subCategoryColumnsData[subCategoryName.value]));
fetch();
}
Can you test it? sample:
const state = reactive({ columns: computed(() => yourstore })
// do not need to call fetch because this hook is a function
useFetch(async () => { await store.dispatch(url) })
return {
...toRefs(state),
}

NuxtJS dispatch is not loading data

I've been struggling for 5 hours with the following issue.
I have a service file where I have API calls using Axios. In the store, I have an action that uses the service to pull a list of schools, then I commit the data to the mutations. If I console log the data on the mutation object, it works correctly and shows the data. However, when I call dispatch from the component inside the onMounted hook, I get an empty object. Any help is greatly appreciated. (see the code below)
store/schools.js
export const state = () => ({
mySchools: []
});
export const mutations = {
getSchools(state, data) {
state.schools = data;
console.log(state.schools); // works;
}
};
export const actions = {
async getMySchools({ commit }) {
await this.$getSchools().then(response => {
commit("getSchools", response.data);
});
}
};
portal/dashboard.vue
import {onMounted, ref, useStore} from "#nuxtjs/composition-api";
export default {
layout: 'portal',
setup() {
const store = useStore();
const schools = ref([]);
onMounted(async() => {
await store.dispatch('schools/getMySchools'); // is not pulling data
schools.value = store.state.schools.mySchools;
console.log(schools); // empty
});
return {
schools
}
}
};
Thank you
You shouldn't use await with then
try this
async getMySchools({ commit }) {
const response = await this.$getSchools();
commit("getSchools", response.data);
}
I'm assuming that your this.$getSchools() actually works since I'm not sure what that is and it's not part of the code

NuxtJS - Prevent fetch if data already exists in state?

I have a portfolio site built using NuxtJS and a headless Wordpress CMS. On several pages, I'm importing a mixin that looks like this:
import { mapActions, mapState } from 'vuex';
export default {
computed: {
...mapState({
galleries: state => state.portfolio.galleries[0],
})
},
methods: {
...mapActions('portfolio', ['fetchGalleries']),
},
async fetch() {
await this.fetchGalleries();
}
}
The Vuex module looks like this:
export const state = () => ({
galleries: [],
});
export const actions = {
async fetchGalleries({ commit }) {
let res = await this.$axios.$get(`${process.env.WP_API_URL}/wp/v2/media`);
const data = res.reduce((acc, item) => {
const { slug } = item.acf.category;
(acc[slug] || (acc[slug] = [])).push(item);
return acc;
}, {});
commit('setGalleries', data);
}
};
export const mutations = {
setGalleries(state, data) {
state.galleries.push(data);
}
};
fetch is being used in the mixin to return data from the api before page load. I noticed however that each time I navigate to a new page, it's running that same fetch and continually adding duplicate data to Vuex state.
How do I prevent fetch from running and continually adding duplicate data to my state if it already exists?
I'm not sure why this was tripping me up so much, but I figured out a very simple solution.
async fetch() {
if (this.galleries.length) return;
await this.fetchGalleries();
}
Just added a conditional return statement as the first line within the fetch function.