Can I make firebase load my data lazily? Let's say my app consists of a tree with a million nodes that the user can collapse, expand and modify. The vast majority of those nodes will stay collapsed the vast majority of the time so it doesn't make sense to keep everything in memory. As far as I can tell, firebase transfers everything in the database to the client on launch and is not meant to be used in any other way. Is that correct?
That's incorrect. Firebase synchronizes data only as you request it. To accomplish something like this, it's all about how you store the data.
For instance, a simplistic example would be this structure, which achieves the desired result:
/records/root/record1
/records/root/record2
/records/record1/record1-1
/records/record2/record2-1
Now you do ref.child('root_level').on('child_added'...) for your starting point. When a node is expanded, run a child_added on the child path.
You could also use priorities, storing all the records in the same path and loading only those you need based on the priority:
/records/record1 (priority null)
/records/record2 (priority null)
/records/record1-1 (priority 'record1')
/records/record2-1 (priority 'record2')
Now, to retrieve your root records, you use:
ref.child('records').startAt(null).endAt(null)
When a node is expanded, you use the following:
ref.child('records').startAt(parentId).endAt(parentId)
Related
I'm new to React, I think the basics have sunk in but I'm stuck on something. We're going to re-build one of our old systems and I'd like to do it in React.
Our system is an internal CRM, each set of client data is about a Mb in size, so efficiency is one of our priorities. The logic is done on a separate API, used by lots of different systems, so 99% of this front end is CRUD only.
(I hope I'm explaining this Ok!)
So onto my question. If I make a small change to a part of the client data, say I add an 'Audit' to the client... there is a chance that LOTS of other data changes. Complex enough that I don't want to replicate the logic both front end & API side.
Would I need to have the API return the full Mb of data, to have the root level app re-render all it's components? Or is there a more efficient way of doing it? Should I be setting up each component to periodically ping the API to check for changes individually?
I'm just a little bit lost where to start tackling the idea of it. Any help is much appreciated!
First things first - React Components rerender when any props or state field was changed.
If you change smth on the client-side and changes should affect server-side which important for user, then you do should updated your app view. To make it more smooth you can use shouldComponentUpdate method of Component's lifecycle to prevent unnecessary re-renders.
If server-side updates are not important for user (some meta data...), then you may not update the state of you application, by that you prevent re-renders.
I am new to WinRT and was playing around with session state. I am navigating to a page to collect data and then want to return to the main page. Just before navigation I am using:
SuspensionManager.SessionState["CurrentState"] = someObject;
The object contains lists of other mildly complex objects, etc... All seems to be working but is this the correct way to use the Suspension Manager?
I have looked at other posts on the topic and some people report that it is necessary to use [DataContract] and [DataMember] attributes to all the classes that are serialized. I omitted them and it still works, (getting the data across pages). So what is the recommended approach?
I may be reading too much into one aspect your question, but the role of SuspensionManager and SessionState is to store just enough information to bring your application back to the place the user left it if the application is actually terminated while it's suspended.
In the Windows 8 application lifecycle, your app gets 'suspended' if another app comes to the foreground. While your app is suspended all of its state is retained in memory, and if reactivated (you flip back to it) everything* is restored "for free".
A suspended app could, however, also be terminated by the OS (b/c of memory pressure, for instance) and there is no opportunity to react to that scenario in your app, so what you are really doing with SessionState is storing what's necessary to 'recreate' the last place the user was at IF the application had actually terminated. It's essentially an insurance policy: if the application is merely suspended, SessionState isn't really needed.
The 'what's necessary' is the grey area, I could store all of the information about say a user profile that was in progress OR I could save just the userid that indexes into my persistent storage of all the user profile data. I generally have more of a minimalist view and will retain as little as possible in SessionState - I make the analogy that I don't need to remember everything, I only need to remember how/where to get/find everything.
There's an implication as well in your question that you're using SessionState to pass information between pages in your app, and that's not really the intent. Each page of your app is typically connected with a view model, and when you interact with a page of that app, you'd update the view model and drive additional screens and experiences from the changes already in the view model. Leaving one screen of your app and returning the main one would also imply to me that you've persisted what ever information you collected - certainly to the view model, but also to something persistent like a data base or local storage. When you revisit that page, you'd then pull the data back out of your view model (or that persistent storage); the main page doesn't need that information so why hold on to it?
Lastly, since you mentioned being new to WinRT, you may want to check out App Builder, which pulls together a number of resources in consumable chunks to lead you through building an app over a period of 30-days (though all material is available, so you can consume at any pace you want :)) The discussion of lifecycle management that's germane to your question comes in on Day 17 of that sequence.
*"everything is restored for free" doesn't necessarily mean you don't have any work to do when an app comes out of the suspended state. There may be stale data that requires refreshing, and connections or other transient or short-lived entities may need to be refreshed/recreated.
I'm in the process of building a data store for keeping track of all the remote images being stored in my app. I've decided to use a singleton that will keep track of all the images being referenced.
As I remember, iOS automatically begins purging objects from memory based on recency of usage, whether or not it's being referenced by the current view controller, etc. However, if I store these images in a data store, those objects are always being referenced by the store itself. My solution for memory management was to keep track of when images were last called and keep some form of limit on the # and size of images being stored in the data store and purge the oldest based on age.
Is this solution a good one? Why or why not? Should I be depending on Apple's automatic memory management, or is having my own manager fine?
Further explanation:
Here's how requesting an image from one of my view controllers will end up looking with my solution:
[[HollerImages store]getImageWithUrl:#"https://www.google.com/logos/classicplus.png"
completionBlock:^(BOOL succeeded, UIImage *image){
if( succeeded ){
//Update the UIImageView with the returned image
}
}];
The store will then manage how many images are currently being referenced in the app and automatically de-reference old images as we hit some pre-defined limit. Thoughts?
Renaud Boisjoly (#rboisjoly) just sent me a link to this library which appears to provide the solution I was describing: https://github.com/rs/SDWebImage/
The easiest way to handle memory concerns is to just implement the -(void)didReceiveMemoryWarning function and clear out all your cached data there.
What you're talking about is implementing an expiring cache. You could just count the elements in your data structure at each insertion and remove elements from the head when you've hit the limit (provided it is an ordered data structure). The former solution is easier and works in most cases.
Using Ncqrs, is there a way to replay every single event ever happened (all aggregate types) and feed these through my denormalizers in order to recreate the whole read model from scratch?
Edit:
I though it's be good to provide a more specific use case. I'm building this inside a ASP.NET MVC application and using Entity Framework (Code first) for working with the read models. In order to speed up development (and because I'm lazy), I want to use a database initializer that recreates the database schemas once any read model changes. Then using the initializer's seed method to repopulate them.
There is unfortunately nothing built in to do this for you (though I haven't updated the version of ncqrs I use in quite a while so perhaps that's changed). It is also somewhat non-trivial to do it since it depends on exactly what you want to do.
The way I would do it (up to this point I have not had a need) would be to:
Call to the event store to get all relevant events
Depending on what you are doing this could be all events or just the events for one aggregate root, or a subset of events for one or more aggregate roots.
Re-create the read-model in memory from scratch (to save slow and unnecessary writing)
Store the re-created read-model in place of the existing one
Call to the event store one more time to get any events that may have been missed
Repeat until there are no new events being returned
One thing to note, if you are recreating the entire read-model database from scratch I would off-line the service temporarily or queue up new events until you finish.
Again there are different ways you could approach this problem, your architecture and scenarios will probably dictate how best to do it.
We use a MsSqlServerEventStore, to replay all the events I implemented the following code:
var myEventBus = NcqrsEnvironment.Get<IEventBus>();
if (myEventBus == null) throw new Exception("EventBus is not found in NcqesEnvironment");
var myEventStore = NcqrsEnvironment.Get<IEventStore>() as MsSqlServerEventStore;
if (myEventStore == null) throw new Exception("MsSqlServerEventStore is not found in NcqesEnvironment");
var myEvents = myEventStore.GetEventsAfter(GetFirstEventIdFromEventStore(), int.MaxValue);
myEventBus.Publish(myEvents);
This will push all the events on the eventbus and the denormalizers will process all the events. The function GetFirstEventIdFromEventStore just queries the eventstore and returns the first Id from the eventstore (where SequentialId = 1)
What I ended up doing is the following. At the service startup, before any commands are being processed, if the read model has changed, I throw it away and recreate it from scratch by processing all past events in my denormalizers. This is done in the database initializer's seed method.
This was a trivial task using the MS SQL event storage as there was a method for retrieving all events. However, I'm not sure about other event storages.
Let me begin with an illustrative example (assume the implementation is in a statically typed language such as Java or C#).
Assume that you are building a content management system (CMS) or something similar. The data is hierarchically organised into Folders. Each folder has a collection of children; a child may be a Page or a Folder. All items are stored within a root folder. No cycles are allowed. We have an acyclic graph.
The system will have a remote API and instances of Folder and Page must be serialized / de-serialized across the network. With a typical implementation of folder, in which a folder's children are a List, serialization of the root node would send the entire graph. This is unacceptable for obvious reasons.
I am interested to hear people have solved this problem in the past.
I have two potential suggestions:
Navigation by query: Change the domain model so that the folder class contains only a list of IDs for each child. To access a child we must query for it. Serialisation is now trivial since the graph ends at a well defined point. The major downside is that we lose type safety - the ID could be for something other than a folder/child.
Stop and re-attach: During serialization stop whenever we detect a reference to a folder or page, send the ID instead. When de-serializing we must then look up the corresponding object for each ID and re-attach it at the relevant position in the nascent object.
I don't know what kind of API you are trying to build, but your suggestion #1 sounds like it is close to what is recommended for REST style services and APIs. Basically, a Folder object would contain a list of URLs to its children.
The Navigation by query solution was used for NFS. By reading through your question, it looks to me, as if you're trying to implements kind of a file system yourself.
If you're looking specifically into sending objects over the network there is always CORBA. Aside from that there is DCOM and the newer WCF. But wait there is more like RMI. Furthermore there are Web Services. I'll stop here now.
Suppose You model the whole tree with every element being a Node, specialisations of Node being Folder and, umm, Leaf. You have a "root" Node. Nodes have a methods
canHaveChildren()
getChildren()
Leaf nodes have the obvious behaviours (never even need to hit the network)
Folders getChildren() get the next set of nodes.
I did devise a system with Restful services along these lines. Seemed to be reasonably easy to program to.
I would not do it by the Navigation by query method. Simply because I would like to stick with the domain model where folders contains folders or pages.
Customizing the serialization might also be tricky, bug prone and difficult to change\understand.
I would suggest that you introduce and object like FolderBowser in your model which takes an id and gives you a list of contents of the folder. That will make your service operations simpler.
Cheers,
Unmesh
The classical solution is probably to use a proxy pattern, where some of the graph is sent over the network and some of the folders are replaced by proxies that will not have their lists of children populated until they are queried. A round trip to the server takes a significant amount of time and it will probably result in too many requests if all folders are proxies (this would yield a new request each time the contents of a folder is inspected), so you want to go for some trade off between the size of each chunk of data and the number of server requests needed in a typical scenario. This is of course application specific, but sending the contents of all child folders in for instance depth 2 might be a useful strategy...
Long story short: What will probably work best is your solution #1 with the exception that you want to send more than one folder at a time because of the overhead of a round trip to the server...