I have two tables that I'd like to join via the MAC Address field, but each table stores MAC addresses slightly different:
Table 1 data: 0:1e:8:c5:9e:fe
Table 2 data: 00:1e:08:c5:9e:fe
The first one removes starting 0's for ANY of the 6 groups of colon-separated fields.
Is there a way I can join on these in SQL without having to modify the data?
I'm guessing I would have to convert both values to XX:XX:XX:XX:XX:XX, then compare them...I'm just not quite sure how to do that.
You can remove leading zeroes with the REPLACE and STUFF functions:
SELECT *
FROM Table1
INNER JOIN Table2
ON Table1.ShortMac = STUFF(REPLACE(':' + Table2.LongMac, ':0', ':'), 1, 1, '')
Unfortunately, no, they cannot be joined without modifying the data in some way.
I would avoid using the MAC addresses as primary keys since they can change.
If you must use them, is it possible to ensure they're all being inserted in a consistent format? Once that's taken care of, you could write a script (whether it's in SQL, or using another language in an application that can interface with the db) to update all of your existing records to a consistent format. You can just split them apart at a delimiter like the ':' and code to pad a 0 if needed. This would allow you to do your joins without having to create any sort of conversion methods.
Simple way to convert the values would be to split or tokenize the string on the :. Then loop over the items and if size != 2, prepend a 0, and then concatenate them back together. Kinda brute force, but it'd do the job.
Similar to a brute force approach like #plattitude describes but somewhat simpler you could use a regular expresion and REPLACE where each ':' character matches either ':' or ':0' and the shorter string or '0'+ the shorter string matches the longer.
Sorry in hotel room with no access to build and test expression but someone may like to add it to the answer.
Related
I have a table with a state column. Inside the state column I have a value like TX` I want to replace that ending character to make the State read TX. How would I do that please give examples
You already have answers for replacing the quote, but I wanted to provide methods for avoiding this problem in the first place.
As noted in #SeanLange's answer, you can use define your State field as a CHAR(2) , so you know that you'll never have a dummy character following a valid state code. You could also handle this in your client code, sanitizing the input before even sending to the database.
One could argue that it would even be a good idea to define a lookup table with a foreign key constraint, so users could only input valid values. You could also use this lookup table client-side (e.g. to provide a list of states).
Of course, you also have to consider internationalization: What about when/if you need to store locations outside of the United States, that may have > 2 characters?
You can escape a single quote by doubling it and including it in quotes. So:
select replace(state, '''', ''')
Of course, if the problem is just a bad third character, then LEFT(state, 2) might do the trick as well.
As a Sean Lang's comment said, you can do this in many different ways. For simplicity, you can just use LEFT(string, #) function for the whole typecasting as long as your raw values are all in the TX` format (**two letter abbrev and one ` , so three characters total for every value in that field.
If that is the case then just do:
SELECT CAST(LEFT(t.state_column, 2) As Varchar(2)) As state,
t.column_2,
t.column_3
/* and so on for all the columns you want */
FROM table t;
--
Further Reference:
- https://w3resource.com/PostgreSQL/left-function.php
I have a textbox to make a search in my table.My table name is ADDRESSBOOK and this table holds the personel records like name,surname,phone numbers and etc.The phone numbers holding like "0 123 456789".If I write "0 123 456789" in my textbox in the background this code is working
SELECT * FROM ADDRESSBOOK WHERE phonenumber LIKE "0 123 456789"
My problem is how can I select the same row with writing "0123456789" in the textbox.Sorry for my english
You can use replace():
WHERE REPLACE(phonenumber, ' ', '') LIKE REPLACE('0 123 456789', ' ', '')
If performance is an issue, you can do the following in SQL Server:
alter table t add column phonenumber_nospace as (replace(phonenumber, ' ', '');
create index idx_t_phonenumber_nospace on t(phonenumber_nospace);
Then, remove the spaces in the parameter value before constructing the query, and use:
WHERE phonenumber_nospace = #phonenumber_nospace
This assumes an equality comparison, as in your example.
If there is a specific format in which the Phone number is stored than you can insert space at the specific locations and than pass that to the database query.
For Example as you have mentioned in the question for number 0 123 456789.
If there is a space after first number and space after fourth number then you could take the text from the textbox and insert space at second position and sixth position(as after adding space at second position + next three positions are number so sixth position) and pass that text to the database query.
An important part of Db design is ensuring data consistency. The more consistently it's stored, the easier it is to query. That's why you should make a point of ensuring your columns use the correct data types:
Dates/time columns should use an appropriate date/time type.
Number columns should use a numeric type of the appropriate size. (None of this numeric varchar rubbish.)
String columns should be of the appropriate length (whether char or varchar).
Columns with referential relationships should never store invalid references to the referenced table.
And similarly, you need to determine the exact format you wish to use when storing telephone numbers; and ensure that any time you store a number it's done so consistently.
Some queries will be complex enough as is. As soon as you're unable to rely on a consistent format, your queries to find data need to cater for all the possible variations. They'll be less likely to leverage indexes effectively.
I have seen argument in favour of storing telephone numbers as numeric data. (It is after all a "number".) Though I'm not really convinced because this approach would be unable to represent leading zeroes (which might be desirable).
Conclusion
Whenever you insert/update a telephone number, ensure it's stored in a consistent format. (NOTE: You can be flexible about how the number appears to your users. It's only the stored value that needs to be consistent.)
Whenever you search for a telephone number, convert the search value into the compatible format before searching.
It's up to you exactly where/how you do these conversions. But you might wish to consider CHECK constraints to ensure that if you failed to convert a number appropriately at some point, that it isn't accidentally stored in the incorrect format. E.g.
CONSTRAINT CK_NoSpacesInTelno CHECK (Telephone NOT LIKE '% %')
I have a PostgreSQL column of type text that contains data like shown below
(32.85563, -117.25624)(32.855470000000004, -117.25648000000001)(32.85567, -117.25710000000001)(32.85544, -117.2556)
(37.75363, -121.44142000000001)(37.75292, -121.4414)
I want to convert this into another column of type text like shown below
(-117.25624, 32.85563)(-117.25648000000001,32.855470000000004 )(-117.25710000000001,32.85567 )(-117.2556,32.85544 )
(-121.44142000000001,37.75363 )(-121.4414,37.75292 )
As you can see, the values inside the parentheses have switched around. Also note that I have shown two records here to indicate that not all fields have same number of parenthesized figures.
What I've tried
I tried extracting the column to Java and performing my operations there. But due to sheer amount of records I have, I will run out of memory. I also cannot do this method in batched due to time constraints.
What I want
A SQL query or a sequence of SQL queries that will achieve the result that I have mentioned above.
I am using PostgreSQL9.4 with PGAdmin III as the client
this is a type of problem that should not be solved by sql, but you are lucky to use Postgres.
I suggest the following steps in defining your algorithm.
First part will be turning your strings into a structured data, second will transform structured data back to string in a format that you require.
From string to data
First, you need to turn your bracketed values into an array, which can be done with string_to_array function.
Now you can turn this array into rows with unnest function, which will return a row per bracketed value.
Finally you need to slit values in each row into two fields.
From data to string
You need to group results of the first query with results wrapped in string_agg function that will combine all numbers in rows into string.
You will need to experiment with brackets to achieve exactly what you want.
PS. I am not providing query here. Once you have some code that you tried, let me know.
Assuming you also have a PK or some unique column, and possibly other columns, you can do as follows:
SELECT id, (...), string_agg(point(pt[1], pt[0])::text, '') AS col_reversed
FROM (
SELECT id, (...), unnest(string_to_array(replace(col, ')(', ');('), ';'))::point AS pt
FROM my_table) sub
GROUP BY id; -- assuming id is PK or no other columns
PostgreSQL has the point type which you can use here. First you need to make sure you can properly divide the long string into individual points (insert ';' between the parentheses), then turn that into an array of individual points in text format, unnest the array into individual rows, and finally cast those rows to the point data type:
unnest(string_to_array(replace(col, ')(', ');('), ';'))::point AS pt
You can then create a new point from the point you just created, but with the coordinates reversed, turn that into a string and aggregate into your desired output:
string_agg(point(pt[1], pt[0])::text, '') AS col_reversed
But you might also move away from the text format and make an array of point values as that will be easier and faster to work with:
array_agg(point(pt[1], pt[0])) AS pt_reversed
As I put in the question, I tried extracting the column to Java and performing my operations there. But due to sheer amount of records I have, I will run out of memory. I also cannot do this method in batched due to time constraints.
I ran out of memory here as I was putting everything in a Hashmap of
< my_primary_key,the_newly_formatted_text >. As the text was very long sometimes and due to the sheer number of records that I had, it wasnt surprising that I got an OOM.
Solution that I used:
As suggested my many folks here, this solution was better solved with a code. I wrote a small script that formatted the text as per my liking and wrote the primary key and the newly formatted text to a file in tsv format. Then I imported the tsv in a new table and updated the original table from the new one.
I have a column of type bigint (ProductSerial) in my table. I need to filter the table by the Product serial using like operator. But I found that, like operator can't be used for integer type.
Is there any other method for this (I don't want to use the = operator).
If you must use LIKE, you can cast your number to char/varchar, and perform the LIKE on the result. This is quite inefficient, but since LIKE has a high potential of killing indexes anyway, it may work in your scenario:
... AND CAST(phone AS VARCHAR(9)) LIKE '%0203'
If you are looking to use LIKE to match the beginning or the end of the number, you could use integer division and modulus operators to extract the digits. For example, if you want all nine-digit numbers starting in 407, search for
phone / 1000000 = 407
Although I'm a bit late to the party, I'd like to add the method I'm using to match the first N given numbers (in the example, 123) in any numeric-type column:
SELECT * FROM MyTable WHERE MyColumn / POWER(10, LEN(MyColumn) - LEN(123)) = 123
The technique is similar to #dasblinkenlight's one, but it works regardless of the number of digits of the target column values. This is a viable workaround if your column contain numbers with different length and you don't want to use the CAST+LIKE method (or a calculated column).
For additional details on that (and other LIKE workarounds) check out this blog post that I wrote on this topic.
If you have control over the database you could add a calculated column to copy the integer value to a string:
ALTER TABLE MyTable
ADD CalcCol AS (CAST(ProductSerial AS VARCHAR)) PERSISTED
And query like:
SELECT *
FROM MyTable
WHERE ProductSerial LIKE '%2548%'
This will move the calculation to the insert/update and only on rows inserted/updated rather then converting every row for each query.
This may be a problem if there are a lot of updated to columns as it will add a very small overhead to these.
There may be a way to do it mathematically using modulus but this would take a lot of working out and testing.
You can change your Field PhoneNumbers and store as String and then use the Like You can alter your table so that you can use the LIKE statement, if you still want to use BIGint for your phone numbers, you cannot get the exact Phone Number without using = the method you can use is Between method that looks for the Numbers that are inside the range.
For the edited question: I think you should use = sign for their ID, or convert the Int to String and then Use Like.
The original question related to a phone number. OP has since edited it to refer to serial numbers. This answer refers to the original question only.
My suggestion is to avoid storing your phone numbers as integers in the first place, and thus the problem does not occur. My phone number is in the form, internationally, of:
+44 7844 51515
Storing it as an integer makes no sense here, as you will never need to do any mathematical operation on it, and you would lose the leading plus. Within the UK, it is:
07844 51515
and thus storing it as an integer would lose its leading zero. Unless you have a very very specific requirement to store it as an integer, you would fare significantly better storing it as a string instead.
[Note: Not actually my phone number]
I have two SQLite tables, that I would love to join them on a name column. This column contains accented characters, so I am wondering how can I compare them for join. I would like the accents dropped for the comparison to work.
You can influence the comparison of characters (such as ignoring case, ignoring accents) by using a Collation. SQLLite has only a few built in collations, although you can add your own.
SqlLite, Data types, Collating Sequences
SqlLite, Define new Collating Sequence
EDIT:
Given that it seems doubtful if Android supports UDFs and computed columns, here's another approach:
Add another column to your table, normalizedName
When your app writes out rows to your table, it normalizes name itself, removing accents and performing other changes. It saves the result in normalizedName.
You use normalizedName in your join.
As the normalization function is now in java, you should have few restrictions in coding it. Several examples for removing accents in java are given here.
There is an easy solution, but not very elegant.
Use the REPLACE function, to remove your accents. Exemple:
SELECT YOUR_COLUMN FROM YOUR_TABLE WHERE replace(replace(replace(replace(replace(replace(replace(replace(
replace(replace(replace( lower(YOUR_COLUMN), 'á','a'), 'ã','a'), 'â','a'), 'é','e'), 'ê','e'), 'í','i'),
'ó','o') ,'õ','o') ,'ô','o'),'ú','u'), 'ç','c') LIKE 'SEARCH_KEY%'
Where SEARCH_KEY is the key word that you wanna find on the column.
As mdma says, a possible solution would be a User-Defined-Function (UDF). There is a document here describing how to create such a function for SQLite in PHP. You could write a function called DROPACCENTS() which drops all the accents in the string. Then, you could join your column with the following code:
SELECT * FROM table1
LEFT JOIN table2
ON DROPACCENTS(table1.column1) = DROPACCENTS(table2.column1)
Much similar to how you would use the UCASE() function to perform a case-insensitive join.
Since you cannot use PHP on Android, you would have to find another way to create the UDF. Although it has been said that creating a UDF is not possible on Android, there is another Stack Overflow article claiming that a content provider could do the trick. The latter sounds slightly complicated, but promising.
Store a special "neutral" column without accented characters and compare / search only this column.