Split column with Json values into multiple columns in Postgres - sql

Calling DBeaver table columns:
Column Name # Data type Encoding
created_at 1 timestamp az64
column_with_json_values 22 varchar(2048) lzo
.
select column_with_json_values from table_name
column_with_json_values
{"messaging": true, "newsletters": true, "ranking_coach": true, "sms_reminders": true, "birthday_reminders": true, "merchant_shiftplan": true}
{"messaging": false, "yext_sync": false, "marketplace": false, "newsletters": false, "ranking_coach": false, "sms_reminders": false, "sms_newsletters": false, "merchant_payment": true, "customer_feedback": true, "birthday_reminders": false, "merchant_dashboard": true, "merchant_shiftplan": false, "reserve_with_google": false, "sms_customer_confirmation": false, "individual_email_from_name": true, "appointment_location_customer": false}
{"messaging": false, "yext_sync": false, "marketplace": false, "newsletters": false, "ranking_coach": false, "sms_reminders": false, "sms_newsletters": false, "merchant_payment": true, "customer_feedback": true, "birthday_reminders": false, "merchant_dashboard": true, "merchant_shiftplan": false, "reserve_with_google": false, "sms_customer_confirmation": false, "individual_email_from_name": true, "appointment_location_customer": false}
{"messaging": false, "newsletters": false, "merchant_payment": false, "birthday_reminders": false, "merchant_shiftplan": false, "double_opt_in_required": false}
{"messaging": false, "yext_sync": false, "marketplace": false, "newsletters": false, "ranking_coach": false, "sms_reminders": false, "sms_newsletters": false, "merchant_payment": true, "customer_feedback": true, "birthday_reminders": false, "merchant_dashboard": true, "merchant_shiftplan": false, "reserve_with_google": false, "sms_customer_confirmation": false, "individual_email_from_name": true, "appointment_location_customer": false}
{"square": false, "insights": true, "api_token": false, "messaging": true, "wait_list": false, "yext_sync": false, "marketplace": false, "newsletters": true, "pdf_prefill": false, "free_product": false, "website_duda": false, "ranking_coach": false, "sms_reminders": true, "contact_widget": false, "online_booking": true, "sms_newsletters": false, "merchant_payment": false, "shared_customers": false, "customer_feedback": true, "external_services": false, "birthday_reminders": true, "merchant_dashboard": true, "merchant_shiftplan": false, "net_promoter_score": false, "new_booking_widget": true, "reserve_with_google": false, "file_download_widget": false, "automated_newsletters": true, "double_opt_in_required": false, "sms_customer_confirmation": false, "individual_email_from_name": false, "new_closingtime_background": false, "gdpr_marketing_opt_in_modal": false, "appointment_location_customer": false, "facebook_instagram_integration": false, "new_full_screen_booking_widget": true, "merchant_logo_on_customer_email": true, "show_all_branch_option_on_insights": false, "participating_account_notifications": false, "show_newsletter_non_subscriber_selection": false}
{"reporting": true, "newsletters": false, "merchant_payment": false, "customer_feedback": true, "reserve_with_google": true, "new_closingtime_background": true, "merchant_logo_on_customer_email": true}
{"messaging": false, "yext_sync": false, "marketplace": false, "newsletters": false, "ranking_coach": false, "sms_reminders": false, "sms_newsletters": false, "merchant_payment": true, "customer_feedback": true, "birthday_reminders": false, "merchant_dashboard": true, "merchant_shiftplan": false, "reserve_with_google": false, "sms_customer_confirmation": false, "individual_email_from_name": true, "appointment_location_customer": false}
{"messaging": false, "newsletters": false, "merchant_payment": false, "birthday_reminders": false, "merchant_shiftplan": false, "double_opt_in_required": false, "new_closingtime_background": true, "facebook_instagram_integration": true, "show_all_branch_option_on_insights": true, "show_newsletter_non_subscriber_selection": false}
{"sms_reminders": true, "customer_feedback": true, "merchant_dashboard": true, "reserve_with_google": false, "show_all_branch_option_on_insights": true}
An interesting point to note is that not all of the rows have the same features. For instance, row#1 to row#5 start with messaging, row #6 start with square, row #7 start with reporting.
I would like to split this one column into multiple columns, where each column has only 1 feature.
Expected Output:
No. messaging newsletter ranking_coach sms_reminders
1 true true true true
2 false null null null
3 false null null null
..
6 null null null null

This is an example query on how to access Json properties, you can cast to JSONB and then access by using ->.
Some info on -> and ->>:
PostgreSQL provides two native operators -> and ->> to help you query JSON data.
The operator -> returns JSON object field as JSON. The operator ->> returns JSON object field as text.
with jsonvalue as (
select '{"messaging": true, "newsletters": true, "ranking_coach": true, "sms_reminders": true, "birthday_reminders": true, "merchant_shiftplan": true}'::jsonb as jsonvalues
)
select
jsonvalues -> 'messaging',
jsonvalues -> 'newsletters'
from jsonvalue;
this will result in your expected output if you adjust my example query on your case. Hope this helped.

Related

How can I get a DIFFERENT random float between 0 and 1 tested on each element of an array using a mask?

**This is the question for my homework: Write a function which simulates the DMS mutation process on one read using np.random.random to generate randomness according to a uniform distribution. Recall, with probability DMS_mutation_prob and eligible base acquires a DMS mutation.
The DMS_mutation_prob = 0.1
My function looks like this, where eligible_to_mutate is an np.array containing Boolean values - True at mutation eligible positions, False elsewhere, and DMS_mutation_prob is equal to 0.1**
def single_read_random_DMS_mutation_mask(eligible_to_mutate, DMS_mutation_prob):
mutation_profile = (eligible_to_mutate==True) & (np.random.random() <DMS_mutation_prob)
return mutation_profile
**
The only problem is that it choses one random number between 0 and 1 to test on all boolean values instead of a different one for each value.
How can I make it so that there's a different random float tested for each value?
When I tried this function where
eligible_to_mutate=
array([False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, True, True, True, True, True, False, True,
False, True, True, True, True, True, False, True, False,
True, False, False, False, False, True, True, True, True,
False, True, True, True, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False])
I either get the same eligible_to_mutate array:
array([False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, True, True, True, True, True, False, True,
False, True, True, True, True, True, False, True, False,
True, False, False, False, False, True, True, True, True,
False, True, True, True, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False])
**or
All false values**
array([False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False])

DevOps migration tool: ArgumentNullException: Value cannot be null

I'm trying to migrate tickets from one org/project to an other org/project, but not with success at the moment.
I have tried to config it the correct way, but I guees I have filled it in correctly now (see below) and still I get the following error.
note: it looks like the 'Source Node Path StartsWith' in the log is not correct. We do not use (sub) Area's. Area should be: vdp-bam-gen.
I tried to leave out the LanguageMaps and also empty the value (AreaPath": ""). But I still get (other) errors.
NodeStructureEnricher.ProcessCommonStructure(Area, Area)
-Source Node Path StartsWith [\vdp-bam-gen\area]
Error while running WorkItemMigration
System.ArgumentNullException: Value cannot be null.
Parameter name: source
at System.Linq.Enumerable.Where[TSource](IEnumerable1 source, Func2 predicate)
I cannot copy my whole config, but here are some parts:
Version": "12.0",
"TelemetryEnableTrace": false,
"workaroundForQuerySOAPBugEnabled": false,
"Source": {
"$type": "TfsTeamProjectConfig",
"Collection": "https://a_url_of my project/",
"Project": "VDP-BAM-GEN",
"ReflectedWorkItemIDFieldName": "Custom.ReflectedWorkItemId",
"AllowCrossProjectLinking": false,
"AuthenticationMode": "Prompt",
"LanguageMaps": {
"AreaPath": "Area",
"IterationPath": "Iteration"
}
"Target": {
"$type": "TfsTeamProjectConfig",
"Collection": "https://a_url_of my project/",
"Project": "CIT-BAM-SOURCING-CUSTOMERS",
"ReflectedWorkItemIDFieldName": "Custom.ReflectedWorkItemId",
"AllowCrossProjectLinking": false,
"AuthenticationMode": "Prompt",
"LanguageMaps": {
"AreaPath": "Area",
"IterationPath": "Iteration"
"Processors": [
"$type": "WorkItemDeleteConfig",
"Enabled": true,
"QueryBit": "AND [System.WorkItemType] NOT IN ('Test Suite', 'Test Plan', 'Epic', 'Feature', 'User Story','Bug','Inbox Item')",
"OrderBit": "[System.ChangedDate] desc"
,
"$type": "WorkItemMigrationConfig",
"ReplayRevisions": true,
"PrefixProjectToNodes": false,
"UpdateCreatedDate": false,
"UpdateCreatedBy": false,
"BuildFieldTable": false,
"AppendMigrationToolSignatureFooter": false,
"QueryBit": "AND [System.WorkItemType] NOT IN ('Test Suite', 'Test Plan', 'Epic', 'Feature', 'User Story','Bug','Inbox Item')",
"OrderBit": "[System.ChangedDate] desc",
"Enabled": true,
"LinkMigration": true,
"AttachmentMigration": true,
"AttachmentWorkingPath": "c:\tmp\WorkItemAttachmentWorkingFolder",
"FixHtmlAttachmentLinks": false,
"SkipToFinalRevisedWorkItemType": false,
"WorkItemCreateRetryLimit": 5,
"FilterWorkItemsThatAlreadyExistInTarget": true,
"PauseAfterEachWorkItem": false,
"AttachmentMaxSize": 480000000,
"CollapseRevisions": false,
"LinkMigrationSaveEachAsAdded": false

Split dictionary into multiple columns in SQL

The data fetched from API returns a dictionary format stored in a column, as below:
{"messaging": true, "newsletters": true, "ranking_coach": true, "sms_reminders": true, "birthday_reminders": true, "merchant_shiftplan": true}
{"messaging": false, "yext_sync": false, "marketplace": false, "newsletters": false, "ranking_coach": false, "sms_reminders": false, "sms_newsletters": false, "merchant_payment": true, "customer_feedback": true, "birthday_reminders": false, "merchant_dashboard": true, "merchant_shiftplan": false, "reserve_with_google": false, "sms_customer_confirmation": false, "individual_email_from_name": true, "appointment_location_customer": false}
{"messaging": false, "yext_sync": false, "marketplace": false, "newsletters": false, "ranking_coach": false, "sms_reminders": false, "sms_newsletters": false, "merchant_payment": true, "customer_feedback": true, "birthday_reminders": false, "merchant_dashboard": true, "merchant_shiftplan": false, "reserve_with_google": false, "sms_customer_confirmation": false, "individual_email_from_name": true, "appointment_location_customer": false}
{"messaging": false, "newsletters": false, "merchant_payment": false, "birthday_reminders": false, "merchant_shiftplan": false, "double_opt_in_required": false}
{"messaging": false, "yext_sync": false, "marketplace": false, "newsletters": false, "ranking_coach": false, "sms_reminders": false, "sms_newsletters": false, "merchant_payment": true, "customer_feedback": true, "birthday_reminders": false, "merchant_dashboard": true, "merchant_shiftplan": false, "reserve_with_google": false, "sms_customer_confirmation": false, "individual_email_from_name": true, "appointment_location_customer": false}
{"square": false, "insights": true, "api_token": false, "messaging": true, "wait_list": false, "yext_sync": false, "marketplace": false, "newsletters": true, "pdf_prefill": false, "free_product": false, "website_duda": false, "ranking_coach": false, "sms_reminders": true, "contact_widget": false, "online_booking": true, "sms_newsletters": false, "merchant_payment": false, "shared_customers": false, "customer_feedback": true, "external_services": false, "birthday_reminders": true, "merchant_dashboard": true, "merchant_shiftplan": false, "net_promoter_score": false, "new_booking_widget": true, "reserve_with_google": false, "file_download_widget": false, "automated_newsletters": true, "double_opt_in_required": false, "sms_customer_confirmation": false, "individual_email_from_name": false, "new_closingtime_background": false, "gdpr_marketing_opt_in_modal": false, "appointment_location_customer": false, "facebook_instagram_integration": false, "new_full_screen_booking_widget": true, "merchant_logo_on_customer_email": true, "show_all_branch_option_on_insights": false, "participating_account_notifications": false, "show_newsletter_non_subscriber_selection": false}
{"reporting": true, "newsletters": false, "merchant_payment": false, "customer_feedback": true, "reserve_with_google": true, "new_closingtime_background": true, "merchant_logo_on_customer_email": true}
{"messaging": false, "yext_sync": false, "marketplace": false, "newsletters": false, "ranking_coach": false, "sms_reminders": false, "sms_newsletters": false, "merchant_payment": true, "customer_feedback": true, "birthday_reminders": false, "merchant_dashboard": true, "merchant_shiftplan": false, "reserve_with_google": false, "sms_customer_confirmation": false, "individual_email_from_name": true, "appointment_location_customer": false}
{"messaging": false, "newsletters": false, "merchant_payment": false, "birthday_reminders": false, "merchant_shiftplan": false, "double_opt_in_required": false, "new_closingtime_background": true, "facebook_instagram_integration": true, "show_all_branch_option_on_insights": true, "show_newsletter_non_subscriber_selection": false}
{"sms_reminders": true, "customer_feedback": true, "merchant_dashboard": true, "reserve_with_google": false, "show_all_branch_option_on_insights": true}
An interesting point to note is that not all of the rows have the same features.
For instance, row#1 to row#5 start with messaging, row #6 start with square, row #7 start with reporting.
I would like to split this one column into multiple columns, where each column has only 1 feature.
Expected Output:
No. messaging newsletter ranking_coach sms_reminders
1 true true true true
2 false null null null
3 false null null null
..
6 null null null null
you can use json functionality in postgresql:
select
Jsoncol ->> 'messaging' messaging,
Jsoncol ->> 'newsletters' newsletters,
Jsoncol ->> 'ranking_coach' ranking_coach,
Jsoncol ->> 'sms_reminders' sms_reminders
from tablename
db<>fiddle here

Vuetify Data Table Slots with nested item properties

I'm having a hard time figuring out how to access item properties from within a v-data-table slot with a specific JSON format.
The items bound to the v-data-table have a nested property called "technicals" which also contains several nested properties for each interval like "5m", "15m" etc...
I generated the v-data-table headers with a computed property and everything is displaying fine but now I want to access the data with a slot inside the table to manipulate the boolean values of true/false with some additional styling etc. I just can't find out how to access the properties in the slot tag.
The values assigned to the headers are of this structure
Object.technicals.side.interval.value
Where "side" and "interval" are component props stored in the data object.
{
"market": {
"id": "xyx"
},
"technicals": {
"buy": {
"1m": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": true,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
},
"5m": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": true,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
},
"15m": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": false,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
},
"30m": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": false,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
},
"1h": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": false,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
},
"4h": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": true,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
}
}
}
<v-data-table
:headers="headers"
:items="tickerStats"
:sort-by.sync="sortBy"
:sort-desc.sync="isDescending"
>
<template v-slot:item.technicals.side.interval.RSI_OVERSOLD="{ item }">
<v-chip>doSomething</v-chip>
</template>
</v-data-table>
I need to get access from this line but I can't access the props from the component (side and interval) and can't access the other things either by trying.
<template v-slot:item.technicals.side.interval.RSI_OVERSOLD="{ item }">
It only works accessing the market ID like so, but I don't need to modify the ID column.
<template v-slot:item.market.id="{ item }">
UPDATE
Here is how I generate the headers
headers() {
let fnc = this.functions.filter(f => {
return f.type == this.side;
});
let headers = [
{
text: "MARKET",
align: "start",
sortable: true,
value: "market.id",
type: String
}
];
let side = this.side;
for (let i = 0; i < fnc.length; i++) {
headers.push({
text: fnc[i].name,
align: "start",
sortable: true,
type: Boolean,
value:
"technicals[" +
this.side +
"]." +
this.interval +
"[" +
fnc[i].name +
"]"
});
}
return headers;
}
Function is an array of objects of this format
{
name: "RSI_OVERSOLD",
formatted_name: "RSI OVERSOD"
}
This is the structure of one item inside the items array used for: items from which I only want to display the market.id and the technicals for the given side and interval passed as a prop to the component.
{
"id": "ETHUSDT",
"ticker": {
"exchange": "Binance",
"base": "ETH",
"quote": "USDT",
"timestamp": 1613375545712,
"last": "1753.52000000",
"open": "1671.49000000",
"high": "1847.54000000",
"low": "1655.67000000",
"volume": "935312.43155000",
"quoteVolume": "1656379605.10542940",
"change": "-82.03000000",
"changePercent": "-4.469",
"bid": "1753.37000000",
"bidVolume": "5.70749000",
"ask": "1753.52000000",
"askVolume": "0.21402000"
},
"market": {
"limits": {
"amount": {
"min": 0.001,
"max": 10000
},
"price": {
"min": 0.01,
"max": 100000
},
"cost": {
"min": 1
},
"market": {
"min": 0.001,
"max": 10000
}
},
"precision": {
"base": 8,
"quote": 8,
"amount": 3,
"price": 2
},
"tierBased": false,
"percentage": true,
"taker": 0.001,
"maker": 0.001,
"id": "ETHUSDT",
"lowercaseId": "ethusdt",
"symbol": "ETH/USDT",
"base": "ETH",
"quote": "USDT",
"baseId": "ETH",
"quoteId": "USDT",
"info": {
"symbol": "ETHUSDT",
"pair": "ETHUSDT",
"contractType": "PERPETUAL",
"deliveryDate": 4133404800000,
"onboardDate": 1569398400000,
"status": "TRADING",
"maintMarginPercent": "2.5000",
"requiredMarginPercent": "5.0000",
"baseAsset": "ETH",
"quoteAsset": "USDT",
"marginAsset": "USDT",
"pricePrecision": 2,
"quantityPrecision": 3,
"baseAssetPrecision": 8,
"quotePrecision": 8,
"underlyingType": "COIN",
"underlyingSubType": [],
"settlePlan": 0,
"triggerProtect": "0.0500",
"filters": [
{
"minPrice": "0.01",
"maxPrice": "100000",
"filterType": "PRICE_FILTER",
"tickSize": "0.01"
},
{
"stepSize": "0.001",
"filterType": "LOT_SIZE",
"maxQty": "10000",
"minQty": "0.001"
},
{
"stepSize": "0.001",
"filterType": "MARKET_LOT_SIZE",
"maxQty": "10000",
"minQty": "0.001"
},
{
"limit": 200,
"filterType": "MAX_NUM_ORDERS"
},
{
"limit": 100,
"filterType": "MAX_NUM_ALGO_ORDERS"
},
{
"notional": "1",
"filterType": "MIN_NOTIONAL"
},
{
"multiplierDown": "0.8500",
"multiplierUp": "1.1500",
"multiplierDecimal": "4",
"filterType": "PERCENT_PRICE"
}
],
"orderTypes": [
"LIMIT",
"MARKET",
"STOP",
"STOP_MARKET",
"TAKE_PROFIT",
"TAKE_PROFIT_MARKET",
"TRAILING_STOP_MARKET"
],
"timeInForce": [
"GTC",
"IOC",
"FOK",
"GTX"
]
},
"type": "future",
"spot": false,
"margin": true,
"future": true,
"delivery": false,
"active": true
},
"technicals": {
"buy": {
"1m": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": true,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
},
"5m": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": true,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
},
"15m": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": false,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
},
"30m": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": false,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
},
"1h": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": false,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
},
"4h": {
"RSI_OVERSOLD": false,
"LOWER_BBAND": false,
"UPTRENDING": true,
"MFI_OVERSOLD": false,
"BULLISH_MACD_CROSS": false
}
},
"sell": {
"1m": {
"RSI_OVERBOUGHT": false,
"UPPER_BBAND": false,
"DOWNTRENDING": false,
"MFI_OVERBOUGHT": false,
"BEARISH_MACD_CROSS": false
},
"5m": {
"RSI_OVERBOUGHT": false,
"UPPER_BBAND": false,
"DOWNTRENDING": false,
"MFI_OVERBOUGHT": false,
"BEARISH_MACD_CROSS": false
},
"15m": {
"RSI_OVERBOUGHT": false,
"UPPER_BBAND": false,
"DOWNTRENDING": true,
"MFI_OVERBOUGHT": false,
"BEARISH_MACD_CROSS": false
},
"30m": {
"RSI_OVERBOUGHT": false,
"UPPER_BBAND": false,
"DOWNTRENDING": true,
"MFI_OVERBOUGHT": false,
"BEARISH_MACD_CROSS": false
},
"1h": {
"RSI_OVERBOUGHT": false,
"UPPER_BBAND": false,
"DOWNTRENDING": true,
"MFI_OVERBOUGHT": false,
"BEARISH_MACD_CROSS": false
},
"4h": {
"RSI_OVERBOUGHT": false,
"UPPER_BBAND": false,
"DOWNTRENDING": false,
"MFI_OVERBOUGHT": false,
"BEARISH_MACD_CROSS": false
}
}
}
},

numpy mask covers another mask

I have different masks from different shapes in an image. Since some of the shapes contain other shapes, I would like to check if a given mask covers another given mask. For instance consider the followings:
A=[True, True, True, True, True,
True, False, False, False, True,
True, False, False, False, True,
True, False, False, False, True,
True, True, True, True, True]
B=[False, False, False, False, False,
False, True, True, False, False,
False, True, True, False, False,
False, False, False, False, False,
False, False, False, False, False]
In a 5x5 image, A covers B. How can I check if A covers B?
EDIT:
A and B could also share some points and A still covers B:
A=[True, True, True, True, True,
True, False, False, False, True,
True, False, False, False, True,
True, False, False, False, True,
True, True, True, True, True]
B=[False, False, False, False, False,
True, True, True, False, False,
True, True, True, False, False,
False, False, False, False, False,
False, False, False, False, False]
We could fill holes for A to have a blob and then perform OR-ing with B. If the OR-ing results in the same array as the holes-filled A, it concludes A as the "cover". To fill holes, we can use SciPy's binary_fill_holes -
from scipy.ndimage.morphology import binary_fill_holes
Af = binary_fill_holes(A)
out = (Af == Af | binary_fill_holes(B)).all()
# or np.array_equal(Af, Af | binary_fill_holes(B))