I'll try and explain how the table is laid out so that what I need might be a bit more clear.
###############################################################
# cid # iid # child cid # child iid # target cid # target iid #
###############################################################
# 112 # 1 # null # null # 116 # 1 #
# 112 # 2 # 112 # 1 # null # null #
# 112 # 3 # 112 # 1 # 116 # 2 #
# 112 # 4 # 112 # 1 # 100 # 3 #
# 112 # 101 # null # null # 116 # 101 #
# 112 # 102 # 112 # 101 # null # null #
# 112 # 103 # 112 # 101 # 116 # 102 #
# 112 # 201 # null # null # 116 # 201 #
# 112 # 202 # 112 # 201 # null # null #
# 112 # 203 # 112 # 201 # 116 # 202 #
# 112 # 301 # null # null # 116 # 301 #
# 112 # 302 # 112 # 301 # null # null #
# 112 # 302 # 112 # 301 # 116 # 302 #
Above there is a cut down representation of the table I'm trying to get data from. Sorry if the layout is a bit crap. Each row here is an object. Each of these objects can have child objects so for example, the first row has no child objects but is linked to the target object. Row two has a child object and isn't linked to a target object, however, it is linked back to row 1 via the child cid and iid which does have a target object. Row three is also linked to row one but it also has a target object so I don't actually want to go back to row one.
Other table
#########################################
# cid # iid # col1 # col2 # col3 # col4 #
#########################################
# 116 # 1 # a # null # 16 # 1 #
# 116 # 2 # b # 1 # 6 # null #
# 116 # 3 # n # 1 # 11 # 2 #
# 116 # 101 # n # 2 # 61 # 3 #
# 116 # 102 # b # null # 161 # 101 #
# 116 # 201 # a # 33 # 312 # 116 #
# 116 # 202 # a # 33 # 312 # 116 #
# 116 # 301 # s # 56 # 1321 # 33 #
# 116 # 302 # r # 6 # 22 # 12 #
Resulting table
###########################################################################################
# cid # iid # child cid # child iid # target cid # target iid # col1 # col2 # col3 # col4 #
###########################################################################################
# 112 # 1 # null # null # 116 # 1 # a # null # 16 # 1 #
# 112 # 2 # 112 # 1 # null # null # a # null # 16 # 1 #
# 112 # 3 # 112 # 1 # 116 # 2 # b # 1 # 6 # null #
# 112 # 4 # 112 # 1 # 100 # 3 # n # 1 # 11 # 2 #
# 112 # 101 # null # null # 116 # 101 # n # 2 # 61 # 3 #
# 112 # 102 # 112 # 101 # null # null # n # 2 # 61 # 3 #
# 112 # 103 # 112 # 101 # 116 # 102 # b # null # 161 # 101 #
# 112 # 201 # null # null # 116 # 201 # a # 33 # 312 # 116 #
# 112 # 202 # 112 # 201 # null # null # a # 33 # 312 # 116 #
# 112 # 203 # 112 # 201 # 116 # 202 # a # 33 # 312 # 116 #
# 112 # 301 # null # null # 116 # 301 # s # 56 # 1321 # 33 #
# 112 # 302 # 112 # 301 # null # null # s # 56 # 1321 # 33 #
# 112 # 302 # 112 # 301 # 116 # 302 # r # 6 # 22 # 12 #
[Just to clarify, in the first table, target cid and iid relate to cid and iid in the other table im linking to it.]
Essentially what I need is to recursively go back through the table until a row has a target object reference.
If a row has both a child c/i id and a target c/i id i just want the target c/i id.
Can anybody point me in the right direction?
I'm slowly reading through
http://docs.oracle.com/cd/B19306_01/server.102/b14200/queries003.htm but I'm finding it a bit confusing. I wouldn't exactly be an expert in the easier SQL queries so recursion is a bit over my head right now.
Thanks
EDIT: Added example of other table and outcome
I dont know what exactly do you need, but
you could start with tihs statment
select cid, iid, level, connect_by_root(target_cid), connect_by_root(target_iid)
from tab
connect by prior cid = child_cid
AND prior iid = child_iid
AND target_cid is null
;
and then filtern the entries you need
select *
from
(
select cid, iid, level, connect_by_root(target_cid) as target_cid, connect_by_root(target_iid) as target_iid
from tab
connect by prior cid = child_cid
AND prior iid = child_iid
AND target_cid is null
)
where target_cid is not null
;
CID IID TARGET_CID TARGET_IID
++++++++++++++++++++++++++++++
112 1 116 1
112 2 116 1
112 3 116 2
112 4 100 3
112 101 116 101
112 102 116 101
112 103 116 102
112 201 116 201
112 202 116 201
112 203 116 202
112 301 116 301
112 302 116 301
112 302 116 302
Related
Thanks for looking at my problem. I want to add target column to new_thing, what should I do. Thanks.
import pandas as pd
# reading data from csv
df = pd.read_csv('data.csv')
df.head()
# The csv format
# ID cot1 num1 target num2 cat3
# 0 123 Santa Elena 100 1 52.00 a
# 1 124 India 77 1 25.00 d
# 2 125 Ruanda 60 0 32.10 b
# 3 126 Lesoto 11 0 -11.00 h
# 4 127 Singapur 79 0 0.07 j
df.dtypes
# diffrent columns category(int,string)
# out
# ID int64
# cot1 object
# num1 int64
# target int64
# num2 float64
# cat3 object
# dtype: object
new_thing = df.select_dtypes(include = ['object']).columns
new_thing
# out
# Index(['cot1', 'cat3'], dtype='object')
type(new_thing)
# out
#pandas.core.indexes.base.Index
# I want to add target column to the new_thing
# I have tried the below but no success
new_thing.append(df.target)
#
# TypeError: all inputs must be Index
As you see I could not add the target to new_thing
This error is because new_thing store values of type pandas.DataFrame.index (columns) if you want to add 'target' to new_thing you must do this
new_thing = df.select_dtypes(include = ['object']).columns
new_thing.append(df.columns[df.columns.get_loc("target")])
to add column and rows selecting type object you should not use instance .columns
new_thing = df.select_dtypes(include = ['object'])
print(pd.concat([new_thing, df[['target']]]))
I tried using meshroom to make a 3d model using google colab. Everything worked fine but there was no output file. I tried mounting mega instead of google drive, but no luck. Instead, I got the following text(some of output cut due to word limit). This is the colab notebook that I used: https://colab.research.google.com/drive/10T2pDZGRUd5r1UiAvQUwJZTqE_tLydcu
[12:26:49.520867][info] Bundle Adjustment Statistics:
- local strategy enabled: no
- adjustment duration: 0.00275099 s
- poses:
- # refined: 2
- # constant: 0
- # ignored: 0
- landmarks:
- # refined: 88
- # constant: 0
- # ignored: 0
- intrinsics:
- # refined: 0
- # constant: 1
- # ignored: 0
- # residual blocks: 176
- # successful iterations: 12
- # unsuccessful iterations: 0
- initial RMSE: 0.386763
- final RMSE: 0.36501
[12:26:49.520935][info] Remove outliers:
- # outliers residual error: 0
- # outliers angular error: 0
[12:26:49.520954][info] Bundle adjustment iteration: 0 took 3 msec.
[12:26:49.520966][info] Bundle adjustment with 1 iterations took 3 msec.
[12:26:49.521135][info] Initial pair is: 738871193, 833403405
[12:26:49.521201][info] Begin Incremental Reconstruction:
- mode: SfM augmentation
- # images in input: 236
- # images in resection: 234
- # landmarks in input: 44
- # cameras already calibrated: 2
[12:26:49.521225][info] Incremental Reconstruction start iteration 0:
- # number of resection groups: 0
- # number of poses: 2
- # number of landmarks: 44
- # remaining images: 234
[12:26:49.522265][info] Update Reconstruction:
- resection id: 0
- # images in the resection group: 1
- # images remaining: 234
[12:26:49.522355][info] [3/236] Robust Resection of view: 18451152
[12:26:49.529856][info] Robust Resection information:
- resection status: true
- threshold (error max): 2.89881
- # points used for resection: 32
- # points validated by robust resection: 31
[12:26:49.532694][info] Bundle adjustment start.
[12:26:49.532746][info] Start bundle adjustment iteration: 0
block_sparse_matrix.cc:81 Allocating values array with 15600 bytes.
detect_structure.cc:95 Dynamic f block size because the block size changed from 6 to 4
detect_structure.cc:113 Schur complement static structure <2,3,-1>.
detect_structure.cc:95 Dynamic f block size because the block size changed from 6 to 4
detect_structure.cc:113 Schur complement static structure <2,3,-1>.
[12:26:49.552254][info] Bundle Adjustment Statistics:
- local strategy enabled: no
- adjustment duration: 0.0190014 s
- poses:
- # refined: 3
- # constant: 0
- # ignored: 0
- landmarks:
- # refined: 75
- # constant: 0
- # ignored: 0
- intrinsics:
- # refined: 1
- # constant: 0
- # ignored: 0
- # residual blocks: 150
- # successful iterations: 51
- # unsuccessful iterations: 0
- initial RMSE: 0.476511
- final RMSE: 0.313505
[12:26:49.552361][info] Remove outliers:
- # outliers residual error: 0
- # outliers angular error: 0
[12:26:49.552432][info] Bundle adjustment iteration: 0 took 19 msec.
[12:26:49.552454][info] Bundle adjustment with 1 iterations took 19 msec.
[12:26:49.625419][info] Incremental Reconstruction start iteration 1:
- # number of resection groups: 1
- # number of poses: 0
- # number of landmarks: 0
- # remaining images: 233
[12:26:49.625465][info] Incremental Reconstruction completed with 2 iterations:
- # number of resection groups: 1
- # number of poses: 0
- # number of landmarks: 0
- # remaining images: 233
[12:26:49.625540][info] Structure from Motion statistics:
- # input images: 236
- # cameras calibrated: 0
- # poses: 0
- # landmarks: 0
- elapsed time: 0.104
- residual RMSE: -nan
[12:26:49.625566][info] Histogram of residuals:
0 | 0
0.1 | 0
0.2 | 0
0.3 | 0
0.4 | 0
0.5 | 0
0.6 | 0
0.7 | 0
0.8 | 0
0.9 | 0
1
[12:26:49.625587][info] Histogram of observations length:
0 | 0
0.1 | 0
0.2 | 0
0.3 | 0
0.4 | 0
0.5 | 0
0.6 | 0
0.7 | 0
0.8 | 0
0.9 | 0
1
[12:26:49.625605][info] Histogram of nb landmarks per view:
0 | 0
0 | 0
0 | 0
0 | 0
0 | 0
0 | 0
0 | 0
0 | 0
0 | 0
0 | 0
1
Have you tried this part of the code?
# Choose format (tar.gz or zip)
!tar -czvf out.tar.gz ./out
from google.colab import files
files.download('out.tar.gz')
!zip -r out.zip ./out
files.download('out.zip')
For more information you can check this source:
https://colab.research.google.com/gist/natowi/3044484ad0c98877692c399297e3ab7e/meshroomcolab.ipynb#scrollTo=VQ8F_rxPw4dK
This is the data with which I'm working
And I'm trying to get something like the following:
The documentation speaks of transposition, but I'm lost with how this would be applied to multi-indexed data.
Assuming that the first six columns are the multi-index, you need not a transpose, but an unstack:
df1 = df.unstack().fillna(0).astype(int)
# Count
#Parliamentarian: Vote 1 2 3
#Parliament Session Sitting Vote Party
#42 1 164 255 BQ 7 0 0
# C 0 81 0
# Lib 162 0 1
# NDP 0 34 1
# 256 BQ 10 0 0
# C 81 0 0
# Lib 0 164 1
# NDP 34 0 1
You can add the fourth column, if you wish:
df1['Count',4] = 0
I have a set of files in a folder like this:
hsa-miR-106a-5p.filtered.txt hsa-miR-182-5p.filtered.txt hsa-miR-2467-5p.filtered.txt hsa-miR-421.filtered.txt hsa-miR-592.filtered.txt
hsa-miR-106b-3p.filtered.txt hsa-miR-183-3p.filtered.txt hsa-miR-25-3p.filtered.txt hsa-miR-424-3p.filtered.txt hsa-miR-615-3p.filtered.txt
hsa-miR-106b-5p.filtered.txt hsa-miR-183-5p.filtered.txt hsa-miR-25-5p.filtered.txt hsa-miR-424-5p.filtered.txt hsa-miR-625-3p.filtered.txt
hsa-miR-1180-3p.filtered.txt hsa-miR-188-5p.filtered.txt hsa-miR-27a-3p.filtered.txt hsa-miR-431-5p.filtered.txt hsa-miR-625-5p.filtered.txt
hsa-miR-1246.filtered.txt hsa-miR-18a-3p.filtered.txt hsa-miR-27a-5p.filtered.txt
file:
ENSG00000224531.4 SMIM13 ENST00000416247.2 9606 hsa-miR-135b-5p 3 132 139 -0.701 99 -0.701 99
ENSG00000112357.8 PEX7 ENST00000541292.1 9606 hsa-miR-135b-5p 3 428 435 -0.683 99 -0.640 99
ENSG00000138279.11 ANXA7 ENST00000372921.5 9606 hsa-miR-135b-5p 3 205 212 -0.631 99 -0.631 99
ENSG00000135248.11 FAM71F1 ENST00000315184.5 9606 hsa-miR-135b-5p 3 488 495 -0.581 99 -0.581 99
ENSG00000087302.4 C14orf166 ENST00000556760.1 9606 hsa-miR-135b-5p 3 34 41 -0.566 99 -0.566 99
ENSG00000104722.9 NEFM ENST00000433454.2 9606 hsa-miR-135b-5p 3 25 32 -0.565 99 -0.565 99
ENSG00000132485.8 ZRANB2 ENST00000254821.6 9606 hsa-miR-135b-5p 3 284 291 -0.566 99 -0.565 99
ENSG00000185127.5 C6orf120 ENST00000332290.2 9606 hsa-miR-135b-5p 3 125 132 -0.564 99 -0.553 99
I would like to combine the rows for all files so that I get one output files that contain all the rows.
I have been playing around with R, but decided I wanted to try awk:
for f in *.filtered.txt
do
....
done
Maybe this can help you (R solution):
files <- dir(pattern="\\.txt$") # Get the field list and put it in a vector
df <- NA # Initialize the variable that will hold the data frame
for(f in files){
if(is.na(df)) { # If the dataframe is empty ...
df <- read.table(f) # ... read the first file in the list
} else { # ... otherwise ...
df <- rbind(df, read.table(f)) # ... bind the dataframe with the rows
# from the next file
}
}
if there is a name column
name
prashant
ram
then the column values should become like this
name
##############################
# Name | Replaced_value #
##############################
# prashant | XXXXXXXX #
# | #
# ram | XXX #
##############################
It has to replaced by same number of Xs.
You can combine lpad/rpad and length
LPAD('X',LENGTH(InputString),'X')
This will work !!
select name,substr(lpad(name,length(name)+length(name),'X'),1,length(name)) as replaced_name from table