For some time now I have the impression that a field and a column are the same thing in SQL server but today I stumbled across the statement that they are not actually the same thing. It was stated that a field is the intersection of a row and a column, so if a table has 10 rows and 10 columns it will have 100 fields in total.
I looked at Microsoft Docs and this is what it says:
A column is collection of cells aligned vertically in a table. A field is an element in which one piece of information is stored, such as the eceived field. Usually, a column in a table contains the values of a single field.
In this case then it looks like they are not the same thing.
I would highly appreciate it if someone can clarify this! Thank you in advance!
A field is part of a row, not a table, a "column of a row", if you will.
However, a lot of people use these terms interchangeably, and you can (read: have to) often deduce what they actually meant by the context of the sentence.
In some contexts, a field may refer to an element within a data value, where a column contains the entire data value. A common example is a date data type, where the month, day, and year are fields within the column. Spatial data types (e.g., the PostGIS data type in Postgres) is another example, where the x coordinate, y coordinate, and spatial reference ID are all fields within the column. Some kinds of identifiers have application-specific fields within them, where, for example, the first two characters carry some information, the next four characters convey some other information, etc. In cases like these, the distinction between column and field is important.
Related
I tried using a name that more accurately describes my question but msg said I am limited to 150 chars.
Looking for assistance from someone who has advanced SQL skills. Ideally I want to do it in SQL to let the computer do the work. Too much manual manipulation is ripe with the possibility of mistakes.
I've already searched for users groups within Google. All emails are being returned saying the email does not exist anymore.
What I am using appears to be a proprietary version of Dremel SQL / Google SQL, however, someone experienced in Dremel SQL will probably be able to guide me in the right direction.
BACKGROUND INFO:
Pulling a column that is an array column which holds another array (a notes column). I think maybe an array of arrays?
I have not figured a way to do what I am trying to do with Google or Dremel SQL yet.
So for now, I am doing it the hard way.
As originally pulled, the data looks like this [{Array of arrays}, {Array of arrays}, {Array of arrays}, etc., repeat... :
More specifically: [{4 or more text fields which could also hold numbers and separated by commas}, {another set of fields}, {another set of fields}...]
I.E. (this is all in just one column of data and hundreds of rows)
[
{"created":"1540236216969","notes": blah... blah... blah", "original_text_length":534, "User_email":"someone#emailaddress.com","user_shortname":"someone"},
{"created":"1540236216969","notes": blah... blah... blah", "original_text_length":1224, "User_email":"someone#emailaddress.com","user_shortname":"someone"},
{"created":"1540236216969","notes": blah... blah... blah", "original_text_length":1664, "User_email":"someone#emailaddress.com","user_shortname":"someone"}
...
]
The number of these is different for each row pulled and each has a specific ID #
A typical row of data is:
ID #, start_date, end_date, some other fields, notes_(the array field)
WHAT I AM DOING NOW is:
SQL data pull,
exporting to google sheets,
make separate tabs for the different array columns.
copying the notes column (the array column holding arrays) to a separate tab on Google Sheets, then
Split Text To Columns using the first curly brace "{" as the separater.
Here is where my dillema is.
Once pulled, I need to split all of those columns again to separate each of the individual elements in each array. Unable to Split text to Columns again with all of them highlighted. I can Split Text to Columns again one at a time but will really be a pain if I have to do that individually for each column and every row (hundreds of rows). Need to find a way to automate this.
I will also need to change each of unix dates to calendar dates within each array PLUS add rows to the spreadsheet depending on the number of columns from the first split. The columns are different for each row depending on how many notes have been added.
OR... do it with SQL (which appears to be a proprietary type of SQL similar to NoSQL but not the same). I have tried using syntax's for IBM SQL, Oracle SQL, SQL Server, and others found online but none work.
OR... do it with a looping function within Google Sheets.
Possibly re-add it to the database as a new table once both sets of arrays are completely split up.
END RESULT
ID#, date1, date 2, first created date (right now a unix date), first note, first other field, etc...
Then add a new row with
Same ID# from above, date1 from row above, date 2 from row above, next (2nd) created date (right now a unix date), 2nd note, 2nd other field, etc...
Add a new row...
3rd set of notes etc.
Explanation- In column "Raw content" i am having data like -MSH|^~\&|PYXIS|SJO|PHA|SJF1|201707171217||ACK|587044650|P|2.2|0| MSA|AA|587044650|....... Message in each row in b/w MSH data is different and i need to search for all the rows having MSH|* |* |CACPA| in place of SJO as shown in above example.. *(astric) means any value
What SQL query should be any suggestion?
Fix your data structure! You should not be storing multiple values in a single column. Instead, you should have a separate table with one row per whatever and per color. That is the right way to use a relational database.
That said, sometimes we are stuck with other people's really bad design decisions. There is a way around this:
where concat('|', colors, '|') not like '%|Red|%'
So, I'm constantly being given data in new and different formats. I'm on a crusade to get my work to standardize data for easy use, and if I managed to convince the powers that be to standardize data, this problem becomes entirely moot. Until then, I have the following problem:
I get data in a variety of ways. Sometimes my gross sales are called total sales. Sometimes gross sales before discounts, total sales before discounts, Gross_Sales, etc. Discounts, deductions, exempt amounts, etc. form another column. So on and so forth. I'd like to be able to do the following:
1) Figure out what columns I want,
2) Turn those columns into a pivot table.
For part 1, I have two options, and I'm wondering if there's anymore: The 1st is to use Microsoft's fuzzy-matching add-in to help me match. I'd have a separate tab dedicated to fuzzy matching each column I need. The second is to just generate a long list of all the variants, and to test each one until I find a hit, assign it, and move onto testing the next one.
The second part is turning all of this into a pivot table - the resouces I have so far are https://www.thespreadsheetguru.com/blog/2014/9/27/vba-guide-excel-pivot-tables and How to Create a Pivot Table in VBA
Is there a better method? Is there another way?
Edit: Slightly better method - Grab the data columns, place them into a table, and pivot everything off of that table - it removes the need to re-create pivot tables, just need to move the data over.
Having the same problem, I use a mix of your two methods.
My data consists of a bunch of logs for rejected x-ray images, and the reject reason is a free text field. My solution was to create a table where the first column contains my desired output categories, and then each subsequent column contains a different variation of it.
For example, a row might have (column one/ouput first entry):
Positioning, POS, Positioning Error, Patient Positioning
Note that these are all fairly different from each other. Where the fuzzy matching comes in - it is used to capture all the smaller differences and mispellings around those other columns. When the fuzzy matching section decides a given reason matches a column's entry, it is then replaced with the appropriate desired output reason from column 1 of the table. In my example, a reason of 'Possitioning Err' [sic] would match to column 3 (Positioning Error) and then get converted to Positioning.
Then wash rinse repeat over the rest of your data as needed. This approach was super useful and fairly flexible in helping standardize my data. It was also computationally more expensive, but you'd only need to run the matching portion once I guess.
As for the actual mechanics of going about doing this - I use 2010, so no inbuilt functionality. I run the fuzzy matching code on a temporary worksheet until best percentage matches are found, and then overwrite the actual source data afterwards.
I am trying Sense/Net Community edition features.
I defined and installed content type called "Vacation Request" successfully.
I want to know what is the purpose of "mapping" field in table: SchemaPropertyTypes
Many thanks,
I really, really hope you are asking this only out of curiosity, and not because you want to change something manually in the db - because it is not recommended :). Please always access the content repository through the API, do not query or modify the db directly.
Property types and values
Simple property values (like int or short text values) are stored in the FlatProperties table. This is a fixed-with table, containing a predefined number of columns dedicated to different types (e.g. x pieces of string columns, y pieces of int columns - see column names in the table).
Property definitions are stored in the SchemaPropertyTypes table, as you have found out.
The zero-based Mapping field in the SchemaPropertyTypes table defines the column index in the FlatProperties table for a particular property. E.g. a value of a string property with mapping 6 will be stored in the FlatProperties table's 'nvarchar_7' column (note the index is shifted by one, because the column name index is one-based).
If you take a look at the PropertyInfoView view (not table), it may help clarifying this: the last column of the view is a computed column that displays the column name that you can look up in the flat properties table.
(there are other useful SQL views there that display data in a more readable way)
Property 'overflow'
It is possible to register more properties of the same type (e.g. int) than can fit in one row in the FlatProperties table. Solution: Sense/Net stores these nodes in multiple rows - this is why there is a Page column there.
Although MS SQL Server supports a huge number of columns for some time, this design has been kept for compatibility reasons.
This is why you see mapping values in PropertyInfoView like 249 with column name nvarchar_10: the value is stored on page 3, which means that content occupies 3 records in the FlatProperties table.
'Other' property types
You may have noticed that in case of reference or long text properties there is no mapping. This is because we do not store them in the FlatProperties table, they have their own tables like ReferenceProperties or TextPropertiesNText.
I have problems with my records within my database, so I have a template with about 260,000 records and for each record they have 3 identification columns to determine what time period the record is from and location: one for year, one for month, and one for region. Then the information for identifying the specific item is TagName, and Description. The Problem I am having is when someone entered data into this database they entered different description for the same device, I know this because the tag name is the same. Can I write code that will go through the data base find the items with the same tag name and use one of the descriptions to replace the ones that are different to have a more uniform database. Also some devices do not have tag names so we would want to avoid the "" Case.
Also moving forward into the future I have added more columns to the database to allow for more information to be retrieved, is there a way that I can back fill the data to older records once I know that they have the same tag name and Description once the database is cleaned up? Thanks in advance for the information it is much appreciated.
I assume that this will have to be done with VBA of some sort to modify records by looking for the first record with that description and using a variable to assign that description to all the other items with the same tag name? I just am not sure of the correct VBA syntax to go about this. I assume a similar method would be used for the backfilling process?
Your question is rather broad and multifaceted, so I'll answer key parts in steps:
The Problem I am having is when someone entered data into this
database they entered different description for the same device, I
know this because the tag name is the same.
While you could fix up those inconsistencies easily enough with a bit of SQL code, it would be better to avoid those inconsistencies being possible in the first place:
Create a new table, let's call it 'Tags', with TagName and TagDescription fields, and with TagName set as the primary key. Ensure both fields have their Required setting to True and Allow Zero Length to False.
Populate this new table with all possible tags - you can do this with a one-off 'append query' in Access jargon (INSERT INTO statement in SQL).
Delete the tag description column from the main table.
Go into the Relationships view and add a one-to-many relation between the two tables, linking the TagName field in the main table to the TagName field in the Tags table.
As required, create a query that aggregates data from the two tables.
Also some devices do not have tag names so we would want to avoid the
"" Case.
In Access, the concept of an empty string ("") is different from the concept of a true blank or 'null'. As such, it would be a good idea to replace all empty strings (if there are any) with nulls -
UPDATE MyTable SET TagName = Null WHERE TagName = '';
You can then set the TagName field's Allow Zero Length property to False in the table designer.
Also moving forward into the future I have added more columns to the
database to allow for more information to be retrieved
Think less in terms of more columns than more tables.
I assume that this will have to be done with VBA of some sort to modify records
Either VBA, SQL, or the Access query designers (which create SQL code behind the scenes). In terms of being able to crunch through data the quickest, SQL is best, though pure VBA (and in particular, using the DAO object library) can be easier to understand and follow.