This question already has answers here:
Computed / calculated / virtual / derived columns in PostgreSQL
(8 answers)
Closed 2 years ago.
Forewords
I've given some examples from PHP just to point my development environment. The question is not about PHP, it's purely about PostgreSQL.
PostgreSQL documentation about generated column states that:
There are two kinds of generated columns: stored and virtual. A stored generated column is computed when it is written (inserted or updated) and occupies storage as if it were a normal column. A virtual generated column occupies no storage and is computed when it is read.
However, it only shows an example of stored column in the example of that page and not a virtual one. Also it says:
The keyword STORED must be specified to choose the stored kind of generated column. See CREATE TABLE for more details.
...where it links to CREATE TABLE page. In that page, the documentation clearly points the pattern is GENERATED ALWAYS AS (expression) STORED.
GENERATED ALWAYS AS ( generation_expr ) STORED
This clause creates the column as a generated column. The column cannot be written to, and when read the result of the specified
expression will be returned.
The keyword STORED is required to signify that the column will be computed on write and will be stored on disk.
The generation expression can refer to other columns in the table, but not other generated columns. Any functions and operators used must be immutable. References to other tables are not allowed.
I actually try to implement a virtual field in Laravel (PHP) and this is what I've came up with so far in my migrations:
DB::statement('ALTER TABLE entries ADD COLUMN do_hint BOOLEAN GENERATED ALWAYS AS (hint_hash OR hint_tags OR hint_due_date OR hint_created_at OR hint_updated_at) VIRTUAL');
As you can see in this statement, there are hint_hash, hint_tags and other several boolean columns (named in hint_* glob pattern) in entries table. I'd like to compute do_hint on the fly so that:
I can actually query it from the database. Sure, I could also generate it on PHP but I'd like to query, that's the point I'd like to use a virtual column.
I'd like to rely on the performance of PostgreSQL rather than PHP.
And considering stored columns only generate once (when the row is inserted), this is not the behavior I desire. I'd like to have a virtual column so that I can read it on the fly.
I get statement errors from my code.
SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near "VIRTUAL"
LINE 1: ...hint_due_date OR hint_created_at OR hint_updated_at) VIRTUAL
^ (SQL: ALTER TABLE entries ADD COLUMN do_hint BOOLEAN GENERATED ALWAYS AS (hint_hash OR hint_tags OR hint_due_date OR hint_created_at OR hint_updated_at) VIRTUAL)
It points at VIRTUAL part is the problem. So I thought "Maybe, virtual generated column is the default behavior." so I removed it and tried again, yet it failed again, complaining the type of the generated column must be defined:
SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at end of input
LINE 1: ...tags OR hint_due_date OR hint_created_at OR hint_updated_at)
^ (SQL: ALTER TABLE entries ADD COLUMN do_hint BOOLEAN GENERATED ALWAYS AS (hint_hash OR hint_tags OR hint_due_date OR hint_created_at OR hint_updated_at))
So, my question is in the title: Does PostgreSQL, at this point in time, only support stored virtual columns? Is there no way to get virtual columns at this point?
I mean, the documentation is confusing me when it says "There are two kinds of generated columns: stored and virtual.". Is it (i) only trying to introduce the concept of virtual and stored generated column here or (ii) advertising a new feature it has on PostgreSQL 12? I mean, if there is no way to create virtual generated columns, they could add a warning saying "Virtual generated columns are not possible at this point. We only support stored ones yet." but there is not such a warning anywhere in the documentation. I'm confused.
Thanks in advance.
Environment
PHP 7.4.5
Laravel 7
PostgreSQL 12.2
The documentation clearly states that virtual generated columns are not supported.
PostgreSQL currently implements only stored generated columns.
This appears just one sentence after the part of the documnetation that you quoted.
Yes, currently PostgreSQL only supports stored generated columns.
You can use a view to get the functionality of virtual generated columns.
Related
The main problem I want to discuss is the schema synchronization conflicts that occur when tables already have data and some new required attribute is added or I rename an required attribute. This question already has some possible solutions but these are not acceptable in a production environment where you already have user data since they simply suggest to delete the data. I also want to enforce required fields so setting the column to {nullable: true} is also not an option for me.
As an example suppose I had a column named "time" that I renamed to "minutes". When I synchronize the schemas TypeORM produces the following error:
QueryFailedError: column "minutes" contains null values
Is there a more elegant/automated way to deal with these errors other than just setting the column to {nullable: true}? I can imagine that you could write some custom SQL with the migration script to also modify the row values. Seems like a little to much manual effort for me though.
In the bigquery console, when creating a table, there used to be type JSON as an option for the column types but weirdly enought it was never present in their docs We used this column type in our production tables, and discovered later on that you can't select it in queries otherwise bigquery throws an error, and the json functions also didn't work with it. So we simply stopped using this column in the queries but they still exist in our tables.
However, in the past couple of days, all queries against this table are failing with this error 400 Json is not enabled for current project. and this column type is not present in the bigquery console anymore. It seems it was removed or deprecated? I checked the release notes, but the latest release was way before the error occured. This broke our production environment, and we couldnt even export the data because exporting gave the same error. Instead we had to use a new table without this column which meant we lost all our history.
Did anyone face the same problem with any other column types before, is it normal that a type is deprecated without users being notified beforehand. This is making me question the reliability of bigquery.
Please reach out to Google Cloud support and we will help you fix your issue with that problematic table. You may also want to try fixing it yourself using the ALTER TABLE DROP COLUMN statement that is currently in public preview [1]. This will drop the erroneous column (the data in that column only will be lost). The rest of the data will remain usable.
[1] https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#alter_table_drop_column_statement
I ran into the same error message few days ago and was surprised to read about this policy change that's not backed up by a mitigation process. My attempt to use Vlad Grachev suggestion to drop this column did not prevail, as the console does not allow to query this table (same "Json is not enabled for current project." error).
My only remediation at this point is:
build a new table where the json column is switched to type string
create a pipeline that transforms the objects to strings
migrate the data through the pipeline to the new table
In BigQuery Json data can be stored in a column type "Record.Are you referring the same by JSON column type?
BigQuery uses the RECORD (or STRUCT) type to represent nested structure. A column of RECORD type is in fact a large column containing multiple child columns. For more information Refer the link below,
Json Data in BigQuery
if you are not refering to the Record Data type, The Json Column type might be a test feature that might not dependent on deprecation scheme
I'm trying to add data from a internal table to a custom one.
DATA: BEGIN OF TMP_CTRYGRP_T OCCURS 1000,
CTYGR TYPE /SAPSLL/CTYGR,
TEXT1 TYPE /SAPSLL/TEXT60,
END OF TMP_CTRYGRP_T.
SELECT ctygr, text1 FROM /SAPSLL/CTYGPT INTO TABLE #DATA(lt_countryGroupsTable)
LOOP AT lt_countryGroupsTable ASSIGNING FIELD-SYMBOL(<ls_countryGroups>).
APPEND <ls_countryGroups> TO TMP_CTRYGRP_T.
ENDLOOP.
Then I want to add the line in a custom Table Type ZZ_T_TAB
So I've tried to create a field-symbol of this table, creating an internal table from it, but none of the solutions I've tried was permitting me to add lines in that Custom table (even if the one in the program had the lines).
The problem I mainly encountered was:
are not mutually convertible in a Unicode program.
So my questions are:
Why does that error happen? Googling it didn't provide me an understandable answer
For the moment I'm using an internal table limited to 1000 rows. But I don't really know by advance the number of lines the search could provide. Is there any way to improve that?
How to add lines from any solution to my ZZ_T_TAB then? And afterwards how could I add other fields in the same table, for the rows already existing?
As some of you maybe understood, I'm quite a rookie in ABAP.
So if there's any useful link to understand all of that I would be happy if you can share it with me.
Why don't you directly select into the table?
Don't use OCCURS as it is declared obsolete and already forbidden in classes.
Declare your own structure as type and mark your custom internal table as TYPE STANDARD TABLE OF struct_type. This way, there will be no upper bounds
TYPES:
BEGIN OF struct_type,
CTYGR TYPE /SAPSLL/CTYGR,
TEXT1 TYPE /SAPSLL/TEXT60,
END OF struct_type.
DATA tmp_ctrygrp_t TYPE STANDARD TABLE OF struct_type WITH EMPTY KEY.
Why does that error happen? Googling it didn't provide me an
understandable answer
You cannot use APPEND with non-identical structures. You have to "convert" it before. Look up for the command MOVE-CORRESPODING in ABAP help (F1 on command in editor).
For the moment I'm using an internal table limited to 1000 rows. But I
don't really know by advance the number of lines the search could
provide. Is there any way to improve that?
Do not use OCCURS extension it is deprecated (as lausek wrote), old syntax.
How then to add lines from any solution to my ZZ_T_TAB ? And
afterwards how could I add other fields in the same table, for the
rows already existing?
You can modify a DB table various ways.:
1, Use UPDATE statement to directly update a field value.
2, Use MODIFY statement to modify field values from a (for example) pre-selected
structure.
Look up the UPDATE and MODIFY command in ABAP help, there are really helpful code examples.
Here is the problem,
I recently installed postgresql with postGIS extension (with functions)
I built a small database model with some geography data.
I'm actually trying to insert some geography data into it, and use functions on it... like points / polygons / ...
Problem is that when I try to use postGis functions like ST_Area, ST_Perimeter, ... PostgreSQL always returns error(s). Most of them are 42P01 meaning "unknown table" if I'm right. But tables exist...
Here is a screenshot of my actual test db :
http://s30.postimg.org/prnyyw7gh/image.png
You can see on the screenshot that the postGIS extension is active on my current model (the 1050 functions of this extension are available in this model)
I inserted data this way :
INSERT INTO "Base".points (point,lat,lng) VALUES ("Base".ST_GeographyFromText('POINT(45.5555 32.2222)'),'45.5555','32.2222');
and
INSERT INTO "Base".polygons (polygon) VALUES ("Base".ST_GeographyFromText('POLYGON((x y,x1 y1,x2 y2,x y))'));
For table Points, I have a serial field (id), a geography field (point) and 2 text fields (lat, lng).
For table Polygons, I have a serial field (id) and a geography field (polygon).
Here are the two queries I'm tryin to make :
SELECT "Base".ST_Area(polygon) FROM "Base".polygons WHERE id=1
or
SELECT "Base".ST_Perimeter(polygon) FROM "Base".polygons WHERE id=1
These 2 tests do not work. They return error 42P01.
When tryin to test another function on my table "points", this also fails but returns a strange message. What I'm tryin is this :
SELECT "Base".ST_Distance((SELECT point FROM "Base".points WHERE id=1),(SELECT point FROM "Base".points WHERE id=2))
This function exists, but returns error message SQL state: 42883 with message :
ERROR : function _st_distance("Base".geography, "Base".geography, numeric, boolean) does not exist
I don't send any numeric or boolean... I can't explain where these errors are coming from...
I have to say that I'm new to postgresql... Problem may come from this...
Thanks for reading/Help
[Tip - post your create table statement, so we can see what the column types actually are].
I think your problems are because you are using geography data type rather than geometry; not all functions work (either at all or with the same arguments) for geography type. Here's an explanation of why - in short, according to the article.
If you do a lot of measurements and e.g. have to compare sizes of
large polygons, it would make sense to use geography rather than
geometry.
To find out what arguments are expected by which functions, check the postgis documentation. It will show you e.g.
float ST_Area(geometry g1);
float ST_Area(geography geog, boolean use_spheroid=true);
So you can see there are two versions of st_area. One accepts a geometry as an argument, the other accepts a geography, but you also have to add another argument.
If "polygon" is of type geography, you need
SELECT "Base".ST_Area(polygon, TRUE) FROM "Base".polygons WHERE id=1
-- true will measure around the geography spheroid and is more accurate
-- than false - read the documentation for what you need!
st_perimeter is similar.
Regarding your error on st_distance, did you see that underscore before "st_distance" in the error message? I suspect that you are running into problems because you have created the postgis extension in a schema. In PGAdmin, have a look for the function "ST_distance". You'd see that it calls in turn the function "_st_distance" - it can't find that because in your case the function is in a different schema. Try just doing this:
CREATE EXTENSION postgis
I think that will save you a world of pain.
Lastly, I think you have the lat and lon arguments in your st_geographyfromtext the wrong way round (but I may be wrong).
The usual way to create a geometry column is AddGeometryColumn, however I have to work with pre-existing columns, so I can't use that function (as far as I know).
Thanks to the PostGIS docs, I can already register the column in the "geometry_columns" table, however AddGeometryColumn seems to do more than create a column and add a row in geometry_columns, for example it adds checks on the column.
So my question is what: what do I need to do to register the column manually, besides adding a row in geometry_columns ?
(for example, is there a modified version AddGeometryColumn that works with an existing column ?)
The easiest way of doing it on existing columns is using the function Populate_Geometry_Columns:
https://postgis.net/docs/Populate_Geometry_Columns.html
In other words: The function you are asking for is already there :-)
HTH
Nicklas
As you said, AddGeometryColumn is only a handy shortcut for creating not only the column, but adding type checks and indexes. Of course, you can add these by hand to an existing column: you simply need to do the same things that the AddGeometryColumn does for you in a single command.
If you need to transfer one "regular" column to a "gis" column, why not use SELECT INTO for transfering the data?