I am new in Octave and I would like to create the following things:
A .txt file with 10 elements = 10 values
Load this file and read the data (values)
A function to add all these values
Return the total
Any idea?
Create a text file called "in.txt" that looks something like this:
1 2 3 4 5 6 7 8 9 10
Then read it into Octave and sum the elements using something like this (assuming Octave's working directory is the same as where you saved the file, use "pwd" to confirm):
load("in.txt"); sum(x)
Related
I have a very wide Excel sheet, from Column A - DIE (about 2500 columns wide), of survey data. Each column is a question, and each row is a response. I'm trying to upload the data to SQL and convert it to a more SQL-friendly format using the UNPIVOT function, but I can't even get it loaded into SQL because it exceeds the 1024-column limit.
Basically, I have an Excel sheet that looks like this:
But I want to convert it to look like this:
What options do I have to make this change, either in Excel (prior to upload) or SQL (while circumventing the 1024 column limit)?
I have had to do this quite a bit. My solution was to write a Python script that would un-crosstab a CSV file (typically exported from Excel), creating another CSV file. The Python code is here: https://pypi.python.org/pypi/un-xtab/ and the documentation is here: http://pythonhosted.org/un-xtab/. I've never run it on a file with 2500 columns, but don't know why it wouldn't work.
R has a very specific function call in one of it's libraries. You can also connect, read, and write data with R into a database. Would suggest downloading R and Rstudio.
Here is a working script to get you started that does what you need:
Sample data:
df <- data.frame(id = c(1,2,3), question_1 = c(1,0,1), question_2 = c(2,0,2))
df
Input table:
id question_1 question_2
1 1 1 2
2 2 0 0
3 3 1 2
Code to transpose the data:
df2 <- gather(df, key = id, value = values)
df2
Output:
id id values
1 1 question_1 1
2 2 question_1 0
3 3 question_1 1
4 1 question_2 2
5 2 question_2 0
6 3 question_2 2
Some helper functions for you to import and export the csv data:
# Install and load the necessary libraries
install.packages(c('tidyr','readr'))
library(tidyr)
library(readr)
# to read a csv file
df <- read_csv('[some directory][some filename].csv')
# To output the csv file
write.csv(df2, '[some directory]data.csv', row.names = FALSE)
Thanks for all the help. I ended up using Python due to limitations in both SQL (over 1024 columns wide) and Excel (well over 1 million rows in the output). I borrowed the concepts from rd_nielson's code, but that was a bit more complicated than I needed. In case it's helpful to anyone else, this is the code I used. It outputs a csv file with 3 columns and 14 million rows that I can upload to SQL.
import csv
with open('Responses.csv') as f:
reader = csv.reader(f)
headers = next(reader) # capture current field headers
newHeaders = ['ResponseID','Question','Response'] # establish new header names
with open('PythonOut.csv','w') as outputfile:
writer=csv.writer(outputfile, dialect='excel', lineterminator='\n')
writer.writerow(newHeaders) # write new headers to output
QuestionHeaders = headers[1:len(headers)] # Slice the question headers from original header list
for row in reader:
questionCount = 0 # start counter to loop through each question (column) for every response (row)
while questionCount <= len(QuestionHeaders) - 1:
newRow = [row[0], QuestionHeaders[questionCount], row[questionCount + 1]]
writer.writerow(newRow)
questionCount += 1
[SPSS] How can I copy values from a variable (column) and paste it below the other one by syntax?
I need to merge 10 columns and I cant do this only by copy paste.
I have this: [1]: https://i.imgur.com/I5DFV.jpg "tooltip"
var1 var2
1 3 6
2 4 7
3 5 8
4
5
.
.
.
and I want this:
newvar
1 3
2 4
3 5
4 6
5 7
6 8
If you want to create new lines (so you get two lines with one variable instead of one line with two variables), You can use varstocases like this:
varstocases /make NewVar from Var1 Var2/index=originVar(NewVar).
this will get both the old variables into the new one, and create an additional variable called originVar which will contain the name of the original variable that each number in NewVar came from.
ADDITION:
if your file was originally sorted by a specific variable(s) you can now just sort again by your original variable and by originVar. If you don't have a variable that conserves the original order, just create one before rustructure:
compute OrigOrder=$casenum.
restructure....
sort cases by OrigOrder originVar./* or by originVar OrigOrder.
Your example may imply that you already have empty lined to which you want to copy values from previous lines. This is a different situation, you can do it this way:
compute NewVar=Var1.
if missing(NewVar) NewVar=lag(Var2).
I'm trying to use gnuplot 4.6 patchlevel 6 to visualize some data from a file test.dat which looks like this:
#Pkg 1
type min max avg
small 1 10 5
medium 5 15 7
large 10 20 15
#Pkg 2
small 3 9 5
medium 5 13 6
large 11 17 13
(Note that the values are actually separated by tabs even though it shows as spaces here.)
My gnuplot commands are
reset
set datafile separator "\t"
plot 'test.dat' index 0 using 2:xticlabels(1) title col, '' using 3 title col, '' using 4 title col
This works fine as long as there is only a single data block in test.dat. When I add the second block spurious data points appear. Why is that and how can it be fixed?
YFTR: Using stat on the file yields only expected results. It reports two data blocks for the full file and correct values (for min, max and sum) when I specify one of the two using index
as mentioned in the comment to the question, one has to explicitly repeat the index 0 specification within all parts of the plot command as
plot 'test.dat' index 0 using 2, '' index 0 using 3, ...
otherwise '' refers to all blocks in the data file
I have a text file "celldata.txt" containing a very simple table of data.
1 2 3 4
5 6 7 8
9 10 11 12
1 2 3 4
2 3 4 5
The problem is when it comes to accessing the data at a certain column and row.
My approach has been to load using loadTable.
Table table;
int numCols;
int numRows;
void setup() {
size(200,200);
table = loadTable("celldata.txt","tsv");
numRows=table.getRowCount();
numCols=table.getColumnCount();
}
void draw() {
background(255);
fill(0);
text(numRows +" "+ numCols,100,100); // Check num of cols and rows
println(table.getFloat(0,0));
}
Question 1: When I do this, it says the number of rows are 5 and the number of columns is just 1. Why is it not 5 x 4?
Question 2: Why is table.getFloat(0,0) "NaN" instead of the first element of the data?
I want to use a much bigger matrix later and access certain elements (of type double) with something like getFloat(i,j) and be able to loop through all elements.
Using the same example data as I, can someone please help me understand what is wrong with my code and how to access the textfile's data? Should I be using another method than loadTable?
You've told Processing that the file contains tab separated values (by using the "tsv" option), but your file contains space separated values.
Since your file does not contain any tabs, it reads the entire row as a single value. So the 0,0 position of your table is 1 2 3 4, which isn't a number- hence the NaN. This is also why it thinks your table only has one column.
You should modify your celldata.txt file to actually be separated by tabs instead of spaces:
1 2 3 4
5 6 7 8
9 10 11 12
1 2 3 4
2 3 4 5
You could also separate them by commas and then use the "csv" option.
If you're still having trouble, you can see what Processing is reading in by adding saveTable(table, "data/new.csv"); to the end of your setup() function and then looking at that file. It will be a list of values separated by commas, so you can see exactly where Processing thinks the cells of the table are.
After run Macro on my Excel file (.xlsx) I have output like this:
With 3 first empty columns for each row.
Then when I try to save this as Text with Tab delimited I got output (.txt) but without 3 first empty rows:
Others empty rows was displayed properly as tabulation, but these 3 first rows was somehow deleted. But in my case I need this.
Any solution how to avoid that situation? Adding it manually don't be a soltuion, because I have huge amounts of data.
Thanks.
In the First Row of First 3 Columns enter any dummy special character like "#".
Example:
# # # 1 999 999 2 10 3
Just enter these # symbols in first ROW. and now save the excel as Tab delimited text file. I get output as below.
Output:
# # # 1 999 999 2 10 3
1 999 999 2 10 3
1 999 999 2 10 3
1 999 999 2 10 3
Hope this solves the problem in this case. If the empty rows or columns are not consistent, then the code present in Alex page can be used.
Put a formula in the last columns of rows that are empty that evaluate to empty (e.g. =""). And then export.