I have a table that has a TEXT column that holds IP, IPs or range (for example 1.1.1.1/24).
In case of multiple IPs, the IPs will be separated by a ####
for example 1.1.1.1####2.2.2.2
The table with 4 rows:
ip
------------------
1.1.1.1
1.1.1.1####2.2.2.2
1.1.1.1/24
3.3.3.3
2.2.2.2
I want to get all the rows that contain the ip 1.1.1.1 or 3.3.3.3, meaning I want to get the first 4 rows.
(1.1.1.1,1.1.1.1####2.2.2.2,1.1.1.1/24,3.3.3.3)
I found this solution in another stack-overflow question:
select inet '192.168.1.5' << any (array['192.168.1/24', '10/8']::inet[]);
but I cannot understand how can I make it work for my specific table and to get me all the first 4 rows.
Please help
Thanks in advance
I think this does what you want:
select t.*
from t
where '1.1.1.1'::inet <<= any(regexp_split_to_array(t.ips, '####')::inet[])
Here is a db<>fiddle.
Related
I have a table with text column that holds ip with subnet
| ip
-------------
| 1.1.1.1/30
when you convert 1.1.1.1/30 to list of ip you get:
1.1.1.0
1.1.1.1
1.1.1.2
1.1.1.3
I want to run a sql on this table and give a list of ips somehow as part of "where" or anything else, and get this row only if the list of the ips that I give contain the ips of the range in the row.
meaning,
where ('1.1.1.0','1.1.1.1)
--> I will not get the row
but:
where ('1.1.1.0','1.1.1.1,1.1.1.2,1.1.1.3)
--> I will get the row
but:
where ('1.1.1.0','1.1.1.1,1.1.1.2,1.1.1.3,1.1.1.4,1.1.1.5)
--> I will get the row
Is there anyway to do that ?
You have to expand out the inet into all its host values and then use containment to accomplish this:
with blowout as (
select t.ip, array_agg(host(network(t.ip::inet) + gs.n)) as all_ips
from t
cross join lateral
generate_series(0, broadcast(t.ip::inet) - network(t.ip::inet)) as gs(n)
group by t.ip;
)
select *
from blowout
where all_ips <# array['1.1.1.0', '1.1.1.1', '1.1.1.2',
'1.1.1.3', '1.1.1.4', '1.1.1.5']::text[]
;
Since you are not using any special inet functions in the comparison, it is best to do the comparisons using text.
I have a table that holds 2 text columns: type and IP.
Both of them can be a single text or a multiple text separated with '####'.
in addition, the ip can also be a range such as 1.1.1.1/24
the table below
row |type | ip
-------------------------------------------
1 |red | 1.1.1.1
2. |red####blue | 1.1.1.1####2.2.2.2
3. |blue | 1.1.1.1/32####2.2.2.2/32
4. |yellow | 1.1.1.1
5. |red | 3.3.3.3
6. |yellow####red | 1.1.1.1
7. |blue | 1.1.1.1####3.3.3.3
I want to get all the rows that have
type red or blue or both (exactly red and blue or less, meaning a single red or a single blue)
AND
IP 1.1.1.1 or 2.2.2.2 or both including ranges (exactly 1.1.1.1 and 2.2.2.2 or less meaning a single 1.1.1.1 or a single 2.2.2.2 or if we have multiple ips, they need to match the range ecactly or less)
meaning I want to get rows 1,2,3
I started to write the next query but I can't get it right:
SELECT * FROM t where
regexp_split_to_array(t.type, '####')::text[] in ('red','blue')
and
regexp_split_to_array(t.ip, '####')::inet[] in ('1.1.1.1','2.2.2.2')
Thanks in advance!
You want the overlaps operator:
SELECT *
FROM t
WHERE regexp_split_to_array(t.type, '####')::text[] && array['red', 'blue'] and
regexp_split_to_array(t.ip, '####')::inet[] && array['1.1.1.1', '2.2.2.2']
Matching type so that one, the other, or both match can be accomplished with the containment operator since the underlying comparison is equality.
Matching inet types to subnets is a different story. That has to use the inet && operator (contains or is contained by), so the ip array has to be turned to rows by unnest. The requirement to again match one, the other, or both means we need a count of ip values and return rows only where the count of matches equals the count of ip values.
This query appears to do the job. Fiddle here.
with asarrays as (
SELECT row, regexp_split_to_array(t.type, '####')::text[] as types,
unnest(regexp_split_to_array(t.ip, '####')::inet[]) as ip
FROM t
), typematch as (
select *, count(*) over (partition by row) as totcount
from asarrays
where types <# array['red', 'blue']
), ipmatch as (
select *, count(*) over (partition by row) as matchcount
from typematch
where ip && any(array['1.1.1.1'::inet, '2.2.2.2'::inet])
)
select row, types, array_agg(ip) as ip
from ipmatch
where matchcount = totcount
group by row, types;
I've got a postgres database that contains a table with IP, User, and time fields. I need a query to give me the complete set of all IPs that have only a single user active on them over a defined time period (i.e. I need to filter out IPs with multiple or no users, and should only have one row per IP). The user field contains some null values, that I can filter out. I'm using Pandas' read_sql() method to get a dataframe directly.
I can get the full dataframe of data from the defined time period easily with:
SELECT ip, user FROM table WHERE user IS NOT NULL AND time >= start AND time <= end
I can then take this data and wrangle the information I need out of it easily using pandas with groupby and filter operations. However, I would like to be able to get what I need using a single SQL query. Unfortunately, my SQL chops ain't too hot. My first attempt below isn't great; the dataframe I end up with isn't the same as when I create the dataframe manually using the original query above and some pandas wrangling.
SELECT DISTINCT ip, user FROM table WHERE user IS NOT NULL AND ip IN (SELECT ip FROM table WHERE user IS NOT NULL AND time >= start AND time <= end GROUP BY ip HAVING COUNT(DISTINCT user) = 1)
Can anyone point me in the right direction here? Thanks.
edit: I neglected to mention that there are multiple entries for each user/ip combination. The source is network authentication traffic, and users authenticate on IPs very frequently.
Sample table head:
---------------------------------
ip | user | time
---------------------------------
172.18.0.0 | jbloggs | 1531987000
172.18.0.0 | jbloggs | 1531987100
172.18.0.1 | jsmith | 1531987200
172.18.0.1 | jbloggs | 1531987300
172.18.0.2 | odin | 1531987400
If I were to query this example table for the time range 1531987000 to 1531987400 I would like the following output:
---------------------
ip | user
--------------------
172.18.0.0 | jbloggs
172.18.0.2 | odin
This should work
SELECT ip
FROM table
WHERE user IS NOT NULL AND time >= start AND time <= end
GROUP BY ip
HAVING COUNT(ip) = 1
Explanation:
SELECT ip FROM table WHERE user IS NOT NULL AND time >= start AND time <= end - filtering out the nulls and time periods
...GROUP BY ip HAVING COUNT(ip) = 1 - If an ip has multiple users, the count(no. of rows with that ip) would be greater > 1.
If by "single user" you mean that there could be multiple rows with only one user, then:
SELECT ip
FROM table
WHERE user IS NOT NULL AND time >= start AND time <= end
GROUP BY ip
HAVING MIN(user) = MAX(user) AND COUNT(user) = COUNT(*);
I have figured out a query that gets me what I want:
SELECT DISTINCT ip, user
FROM table
WHERE user IS NOT NULL AND time >= start AND time <= end AND ip IN
(SELECT ip FROM table
WHERE user IS NOT NULL AND time >= start AND time <= end
GROUP BY ip HAVING COUNT(DISTINCT user) = 1)
Explanation:
The inner select gets me all IPs that have only one user across the specified time range. I then need to select the distinct ip/user pairs from the main table where the IPs are in the nested select.
It seems messy that I have to do the same filtering (of time range and non-null user fields) twice though, is there a better way to do this?
This question already has answers here:
Select first row in each GROUP BY group?
(20 answers)
Closed 5 years ago.
I have a table with three columns, hostname, address, and virtual. The address column is unique, but a host can have up to two address entries, one virtual and one non-virtual. In other words, the hostname and virtual column pair are also unique. I want to produce a result set that contains one address entry for a host giving priority to the virtual address. For example, I have:
hostname | address | virtual
---------+---------+--------
first | 1.1.1.1 | TRUE
first | 1.1.1.2 | FALSE
second | 1.1.2.1 | FALSE
third | 1.1.3.1 | TRUE
fourth | 1.1.4.2 | FALSE
fourth | 1.1.4.1 | TRUE
The query should return the results:
hostname | address
---------+--------
first | 1.1.1.1
second | 1.1.2.1
third | 1.1.3.1
fourth | 1.1.4.1
Which is the virtual address for every host, and the non-virtual address for hosts lacking a virtual address. The closest I've come is asking for one specific host:
SELECT hostname, address
FROM system
WHERE hostname = 'first'
ORDER BY virtual DESC NULLS LAST
LIMIT 1;
Which gives this:
hostname | address
---------+--------
first | 1.1.1.1
I would like to get this for every host in the table with a single query if possible.
What you're looking for is a RANK function. It would look something like this:
SELECT * FROM (
SELECT hostname, address
, RANK() OVER (PARTITION BY hostname ORDER BY virtual DESC NULLS LAST) AS rk
FROM system
)
WHERE rk = 1
This is a portable solution that also works in Oracle and SQL Server.
In Postgres, the simplest way is distinct on:
SELECT DISTINCT ON (hostname) hostname, address
FROM system
ORDER BY hostname, virtual DESC NULLS LAST
I'm try to select all ports from another table that has port_start and port_end columns.
Let's assume I have ports table is called Ports and the table with range is called Whitelist.
So, It looks something like
Ports
-----
80
120
700
Whitelist
---------
1 - 100
381 - 500
Let's focus on port 80, I thought of selecting what is between the whitelist and then ignore it by using not in/except/whatsoever with between clause.
With not between it will no select the opposite since I need to assure all ranges doesnt fit with the port.
Is there any nice solution to solve this? Thanks
Try:
select p.*
from ports p
left join whitelist w
on p.id between w.port_start and w.port_end
where w.port_start is null
Fiddle:
http://sqlfiddle.com/#!2/b37a3/1/0