Avoid replicating whole tables in Matlab `join`'s - sql

Question

I have a table T of records and fields. I want to create a new field and populate it with the result of a lookup of another table L. This means that I will use one or more fields in T as a foreign key. In SQL, I can UPDATE the newly created field in table T using a JOIN with table L. Conversely, Matlab has no updating of an existing table when doing a join; that creates a whole new table, which is then used to replace the original table T. It seems like a lot of data replication to populate one field. Is this avoided under-the-hood? Is there a code design pattern or idiom that avoids this, but is still reasonably readable and doesn't compromise on code compactness too much?
While I asked this question in the context of join, I'd be interested in general in strategies for avoiding table replication in all variations of Matlab joins.
I'll describe an example of how, for each record in Table1, the ForeignKey is used to look up Data in Table2.
Table1
-----------------------------
SomeField NewField ForeignKey
--------- -------- ----------
someData1 dummy a
someData2 dummy b
someData3 dummy a
someData4 dummy b
someData5 dummy a
Table2
--------
Key Data
--- ----
a apple
b banana
The following SQL code performs the lookup. The entry in the Data field is then concatenated with the content of field SomeField in Table1 and stored into field NewField.
UPDATE Table1 INNER JOIN Table2
ON Table1.ForeignKey = Table2.Key
SET Table1.NewField = Table1.SomeField & Table2.Data
The updated Table1 is:
Table1
------------------------------------
SomeField NewField ForeignKey
--------- --------------- ----------
someData1 someData1apple a
someData2 someData2banana b
someData3 someData3apple a
someData4 someData4banana b
someData5 someData5apple a
Interesting to note that Table INNER JOIN Table2 isn't actually created. It is only "virtually" created to enable the calculation with which to update Table1. In contrast, Matlab's JOIN creates the actual joined table, and a separate operation is needed to do the calculation.

Answer 1

The operation that you're doing must look something like this:
table1 = table({'someData1';'someData2';'someData3';'someData4';'someData5'},...
{'a';'b';'a';'b';'a'},'VariableNames',{'SomeField','ForeignKey'});
table2 = table({'a';'b'},{'apple';'banana'},'VariableNames',{'Key','Data'});
table3 = join(table1,table2,'LeftKeys','ForeignKey','RightKeys','Key')
This produces the following table:
SomeField ForeignKey Data
___________ __________ ________
'someData1' 'a' 'apple'
'someData2' 'b' 'banana'
'someData3' 'a' 'apple'
'someData4' 'b' 'banana'
'someData5' 'a' 'apple'
And then you are applying some sort of operation on the columns SomeField and Data.
I think this join function has been added to make use easy for those familiar with SQL but less familiar with MATLAB syntax.
If you are still worried about copying large amounts of data (as I mentioned, this is not the case because of lazy copying), you can obtain the column Data above using the following set-based operations:
[~,index] = ismember(table1.ForeignKey,table2.Key);
data = table2.Data(index);
Here, data is a cell array, identical to table3.Data. In any case, the index values created here are what SQL would create internally for this JOIN operation. If a table1.ForeignKey is not in table2.Key, the corresponding index value is 0 (MATLAB indexes starting at 1). In that case, you cannot use index directly to index, you would need to use an additional level of indexing to get only the valid rows:
[valid,index] = ismember(table1.ForeignKey,table2.Key);
data1 = table1.SomeField(valid)
data2 = table2.Data(index(valid));
Note that table/join uses ismember in this exact same way, then copies the left table (which causes the copy to reference the data in the input table due to lazy copying), and adds columns to it for the right table.

Answer 2

I’m no expert on tables, but I can give insight into how MATLAB operates with this type of data.
A MATLAB table object contains a matrix for each column.
Copying a matrix in MATLAB does not copy the data. MATLAB uses lazy copying. This means that the copy references the same data as the original (until the copy or original is changed, at which point a copy is made). This behavior is well documented (1), (2).
Thus, creating a new table using whole columns from other tables would cause matrices to be copied, but these copies don’t incur any actual copying of matrix contents, the new table references data in the original tables.
But if any of the values in a column is changed, the whole column will need to be copied to avoid the other table to see the same change. The reference is internal and temporary, and invisible to the user. For all intends and purposes, it looks like the new table contains a copy of the original data.
However, if the join operation causes rows to be swapped or removed, all of this is a moot point. The data will be copied.