First of all thanks for your time on my issue.
Second sorry if this is a duplicate, I legit searched for 3 days on and off and didn't find answer to my particular situation.
Visual on the error obtained
This first line transforms an array's values into a string of index(es)
$path = self::extract_path($path);
After extraction $path becomes a string with a format similar to['key1']['key2']
Here I test if one valuable element (ondemand) is found within that Keys string, so far so good.
if( $path !== FALSE && strpos($path, $element) !== FALSE){
$var_dim_str = 'coupon' . $path . '[recurrence]';
As a self-proofing, I hard-coded one of my specific scenario element and it does exist so my issue is not really non-initialization of my variable as most other topic were suggesting.
var_dump($coupon['item']['ondemand']['recurrence']);
My issue lies here, I get "ErrorException [ Notice ]: Undefined variable: coupon[item][ondemand][recurrence]"
$recurrence = $$var_dim_str;
[...]
Here are my var_dump output:
These are my indexes as an array they get extracted to reconstruct the variable's string', I have some variable-dimension arrays that gets called so the dimension depth might not always be of fixed value, hence why I have to dynamically test this.
1- $path, before being extracted
[...]\modules\payment\classes\Helper\Payment.php:290:
array (size=2)
0 => string 'item' (length=4)
1 => string 'ondemand' (length=8)
2- We see here that (['item']['ondemand']['recurrence'] => 3) exists
[...]\modules\payment\classes\Payment\Cart.php:266:
array (size=1)
'item' =>
array (size=3)
'ondemand' =>
array (size=14)
[...]
'recurrence' => string '3' (length=1)
[...]
3- My var_dump above is outputting it properly.
[...]\modules\payment\classes\Payment\Cart.php:270:string '3' (length=1)
**So, my conclusion is that PHP doesn't really appreciate my 'built' variable but there has to be some way to make this logic work... any pointers on this would be really appreciated. **
The issue does lie with $$var_dim_str. What the $$ does is resolve $$variableName to be the variable name contained in $variableName. It basically holds a reference to another variable based on the variable's name. From the PHP site:
A variable variable takes the value of a variable and treats that as the name of a variable. In the above example, hello, can be used as the name of a variable by using two dollar signs. i.e.
So ...
// some variables
$name = 'Ellan' ;
$site = 'Stack Exchange' ;
$tags = 'PHP-7, Variables' ;
// we want $site
$variableName = 'site' ;
echo $$variableName ;
// we want $tags
$variableName = 'tags' ;
echo $$variableName ;
The code above will produce:
Stack Exchange
PHP-7, Variables
When you assign a value to $var_dim_str:
$var_dim_str = 'coupon' . $path . '[recurrence]';
$var_dim_str does not hold the name of a variable. It contains the a string.
Are you sure you want to use $$ in this instance? Maybe you really want to use:
$recurrence = $var_dim_str;
v1- old var, v2 - new var.. Currently if anyone sets v1 it will error out so I am making it backward compataible. So if old var v1 is defined anywhere I want to make those 2 variables the same i.e v2 = v1 (Also value of v1 should be in v2)
set v1 test
For this I have a hash with old and new names
set oldvars(v1) v2
foreach ele [array names oldvars] {
if {([info exists ele] || $ele == "") && $oldvars($ele)!= ""} {
o2n $ele $oldvars($ele) //o2n is a proc defined below
}
}
proc o2n {gvar gval}
global v1
puts "gvar is $gvar, gval is $gval" //prints gvar - v1 and gval - v2
set $gval [set $$gvar] **//Error - Cant read $v1 no such variable. Above line does print gvar = v1 so $$gvar should be $v1 =test**
puts "$gval [set $gvar]" // Has the value of v1 i.e test
}
puts "v2 is $v2" **: Error : Can't read v2 no such variable**
If we assume
set a1 1
set b1 2
set c1 3
set oldvars(a1) a2
set oldvars(b1) b2
set oldvars(c1) c2
All you have to do to copy values from x1 variables to corresponding x2 variables is
foreach {x1 x2} [array get oldvars] {
set $x2 [set $x1]
}
The assignment looks a little off, but what it means is ”assign to the variable whose name is in x2 the value of the variable whose name is in x1.
If you want to do this via a procedure, you need to take into account that the procedure’s code executes in another scope. The uplevel command helps with that, and as a sideeffect the assignment code becomes simpler:
proc o2n {x1 x2} {
upvar 1 $x1 v1 $x2 v2
set v2 $v1
}
foreach {x1 x2} [array get oldvars] {
o2n $x1 $x2
}
Checking existence
If you keep references to variables in the form of variables that store those variables’ names, testing for existence also looks different from the usual. If you have
foreach ele [array names oldvars] {
the variable ele is guaranteed to exist (it is created by foreach as long as the list isn’t empty – but then the body of foreach won’t be executed anyway), but it isn’t the variable you wanted to verify: it only holds the name of the variable you wanted to verify. So you’ll need to write the test like this:
info exists $ele
(you also need to be in the same scope where the variable referenced by ele exists).
This looks very strange since we usually are very careful to verify the name of the variable, not the value. But in this case the value of the variable is the name we want to verify, so.
Inside the procedure it’s easier to write the test. The upvar command creates names in the current scopes that are linked to variables in another scope. If those variables don’t exist, the local names will in effect be names of unset variables.
upvar 1 $x1 v1 $x2 v2
if {[info exists v1]} {
Documentation:
array,
foreach,
if,
info,
proc,
set,
upvar
I have 133 variables on income (each variable represents a group). I want the Gini coefficients of all these groups, so I use ineqdeco in Stata. I can't compute all these coefficients by hand so I created a for loop:
gen sgini = .
foreach var of varlist C07-V14 {
forvalue i=1/133 {
ineqdeco `var'
replace sgini[i] = $S_gini
}
}
Also tried changing the order:
foreach var of varlist C07-V14 {
ineqdeco `var'
forvalue i=1/133 {
replace sgini[i] = $S_gini
}
}
And specifying i beforehand:
gen i = 1
foreach var of varlist C07-V14 {
ineqdeco `var'
replace sgini[i] = $S_gini
replace i = i+1
}
}
I don't know if this last method works anyway.
In all cases I get the error: weight not allowed r(101). I don't know what this means, or what to do. Basically, I want to compute the Gini coefficient of all 133 variables, and store these values in a vector of length 133, so a single variable with all the coefficients stored in it.
Edit: I found that the error has to do with the replace command. I replaced this line with:
replace sgini = $S_gini in `i'
But now it does not "loop", so I get the first value in all entries of sgini.
There is no obvious reason for your inner loop. If you have no more variables than observations, then this might work:
gen sgini = .
gen varname = ""
local i = 1
foreach var of varlist C07-V14 {
ineqdeco `var'
replace sgini = $S_gini in `i'
replace varname = "`var'" in `i'
local i = `i' + 1
}
The problems evident in your code (seem to) include:
Confusion between variables and local macros. If you have much experience with other languages, it is hard to break old mental habits. (Mata is more like other languages here.)
Not being aware that a loop over observations is automatic. Or perhaps not seeing that there is just a single loop needed here, the twist being that the loop over variables is easy but your concomitant loop over observations needs to be arranged with your own code.
Putting a subscript on the LHS of a replace. The [] notation is reserved for weights but is illegal there in any case. To find out about weights, search weights or help weight.
Note that with this way of recording results, the Gini coefficients are not aligned with anything else. A token fix for that is to record the associated variable names alongside, as done above.
A more advanced version of this solution would be to use postfile to save to a new dataset.
I'm trying to convert a csv file containing 3 columns (ATTRIBUTE_NAME,ATTRIBUTE_VALUE,ID) into a flat table whose each row is (ID,Attribute1,Attribute2,Attribute3,....). The samples of such tables are provided at the end.
Either Python, Perl or SQL is fine. Thank you very much and I really appreciate your time and efforts!
In fact, my question is very similar to this post, except that in my case the number of attributes is pretty big (~300) and not consistent across each ID, so hard coding each attribute might not be a practical solution.
For me, the challenging/difficult parts are:
There are approximately 270 millions lines of input, the total size of the input table is about 60 GB.
Some single values (string) contain comma (,) within, and the whole string will be enclosed with double-quote (") to make the reader aware of that. For example "JPMORGAN CHASE BANK, NA, TX" in ID=53.
The set of attributes is not the same across ID's. For example, the number of overall attributes is 8, but ID=53, 17 and 23 has only 7, 6 and 5 respectively. ID=17 does not have attributes string_country and string_address, so output blank/nothing after the comma.
The input attribute-value table looks like this. In this sample input and output, we have 3 ID's, whose number of attributes can be different depending on we can obtain such attributes from the server or not.
ATTRIBUTE_NAME,ATTRIBUTE_VALUE,ID
num_integer,100,53
string_country,US (United States),53
string_address,FORT WORTH,53
num_double2,546.0,53
string_acc,My BankAcc,53
string_award,SILVER,53
string_bankname,"JPMORGAN CHASE BANK, NA, TX",53
num_integer,61,17
num_double,34.32,17
num_double2,200.541,17
string_acc,Your BankAcc,17
string_award,GOLD,17
string_bankname,CHASE BANK,17
num_integer,36,23
num_double,78.0,23
string_country,CA (Canada),23
string_address,VAN COUVER,23
string_acc,Her BankAcc,23
The output table should look like this. (The order of attributes in the columns is not fixed. It can be sorted alphabetically or by order-of-appearance.)
ID,num_integer,num_double,string_country,string_address,num_double2,string_acc,string_award,string_bankname
53,100,,US (United States),FORT WORTH,546.0,My BankAcc,SILVER,"JPMORGAN CHASE BANK, NA, TX"
17,61,34.32,,,200.541,Your BankAcc,GOLD,CHASE BANK
23,36,78.0,CA (Canada),VAN COUVER,,Her BankAcc,,
This program will do as you ask. It expects the name of the input file as a parameter on the command line.
Update Looking more carefully at the data I see that not all of the data fields are available for every ID. That makes things more complex if the fields are to be kept in the same order as they appear in the file.
This program works by scanning the file and accumulating all the data for output into hash %data. At the same time it builds a hash %headers, that keeps the position each header appears in the data for each ID value.
Once the file has been scanned, the collected headers are sorted by finding the first ID for each pair that includes information for both headers. The sort order for that pair within the complete set must be the same as the order they appeared in the data for that ID, so it's just a matter of comparing the two position values using <=>.
Once a sorted set of headers has been created, the %data hash is dumped, accessing the complete list of values for each ID using a hash slice.
Update 2 Now that I realise the sheer size of your data I can see that my second attempt was also flawed, as it tried to read all of the information into memory before outputting it. That isn't going to work unless you have a monster machine with about 1TB of memory!
You may get some mileage from this version. It scans twice through the file, the first time to read the data so that the full set of header names can be created and ordered, then again to read the data for each ID and output it.
Let me know if it's not working for you, as there's still things I can do to make it more memory-efficient.
use strict;
use warnings;
use 5.010;
use Text::CSV;
use Fcntl 'SEEK_SET';
my $csv = Text::CSV->new;
open my $fh, '<', $ARGV[0] or die qq{Unable to open "$ARGV[0]" for input: $!};
my %headers = ();
my $last_id;
my $header_num;
my $num_ids;
while (my $row = $csv->getline($fh)) {
next if $. == 1;
my ($key, $val, $id) = #$row;
unless (defined $last_id and $id eq $last_id) {
++$num_ids;
$header_num = 0;
$last_id = $id;
print STDERR "Processing ID $id\n";
}
$headers{$key}[$num_ids-1] = ++$header_num;
}
sub by_position {
for my $id (0 .. $num_ids-1) {
my ($posa, $posb) = map $headers{$_}[$id], our $a, our $b;
return $posa <=> $posb if $posa and $posb;
}
0;
}
my #headers = sort by_position keys %headers;
%headers = ();
print STDERR "List of headers complete\n";
seek $fh, 0, SEEK_SET;
$. = 0;
$csv->combine('ID', #headers);
print $csv->string, "\n";
my %data = ();
$last_id = undef;
while () {
my $row = $csv->getline($fh);
next if $. == 1;
if (not defined $row or defined $last_id and $last_id ne $row->[2]) {
$csv->combine($last_id, #data{#headers});
print $csv->string, "\n";
%data = ();
}
last unless defined $row;
my ($key, $val, $id) = #$row;
$data{$key} = $val;
$last_id = $id;
}
output
ID,num_integer,num_double,string_country,string_address,num_double2,string_acc,string_award,string_bankname
53,100,,"US (United States)","FORT WORTH",546.0,"My BankAcc",SILVER,"JPMORGAN CHASE BANK, NA, TX"
17,61,34.32,,,200.541,"Your BankAcc",GOLD,"CHASE BANK"
23,36,78.0,"CA (Canada)","VAN COUVER",,"Her BankAcc",,
Use Text::CSV from CPAN:
#!/usr/bin/env perl
use strict;
use warnings;
# --------------------------------------
use charnames qw( :full :short );
use English qw( -no_match_vars ); # Avoids regex performance penalty
use Text::CSV;
my $col_csv = Text::CSV->new();
my $id_attr_csv = Text::CSV->new({ eol=>"\n", });
$col_csv->column_names( $col_csv->getline( *DATA ));
while( my $row = $col_csv->getline_hr( *DATA )){
# do all the keys but skip if ID
for my $attribute ( keys %$row ){
next if $attribute eq 'ID';
$id_attr_csv->print( *STDOUT, [ $attribute, $row->{$attribute}, $row->{ID}, ]);
}
}
__DATA__
ID,num_integer,num_double,string_country,string_address,num_double2,string_acc,string_award,string_bankname
53,100,,US (United States),FORT WORTH,546.0,My BankAcc,SILVER,"JPMORGAN CHASE BANK, NA, TX"
17,61,34.32,,,200.541,Your BankAcc,GOLD,CHASE BANK
23,36,78.0,CA (Canada),VAN COUVER,,Her BankAcc,,
I am counting servers in three different forests and I'd like to email the results.
I have an integer ($i) that increases for each server, but I reset this number for every forest.
Is there a way that I can create a variable from the forest name ($forest) and a $i count?
I would also love if my mail would foreach the servers in forests count so this would be dynamic.
I guess I am asking if I can name a variable after a variable.. For example $forest.$i or similar...
Please help!
And happy new year!!
Edit:
# Get password (this is secure!)
$password = Read-Host "Enter password" -AsSecureString
# Create a container for your objects.
$forestContainer = #()
$forests = "corp.foresta.com","corp.forestb.com","corp.forestc.com"
foreach ($forest in $forests) {
# Create credentials for forest
$credentials = new-object -typename System.Management.Automation.PSCredential -argumentlist "$forest\administrator",$password
# Connect to current forest
$forestconnection = Connect-QADService -Service $forest -Credential $credentials
$servers = Get-QADComputer -WarningAction SilentlyContinue -OSName *server*,*hyper*
echo $($servers.Length)
$currentForest = New-Object PsObject -Property #{
Name = $forest
Count = $($servers.Length)
}
$forestContainer += $currentForest
}
Yes, you can do this. What you're looking to do is create a map/table between the forest and the count for the forest.
Something like
Forest count
====== ======
F1 120
F2 120
Three approaches are available
Use numerical indexing
Use a HashTable for named indexing
Use dynamic properties for named indexing
Let's look at each
1. Use numerical indexing
$fList = #() #initialize an empty array
$fList += $count # add a new entry
$fList += $count # add another entry
$fList # show all values
$fList[0] # get the 1st value
$fList[1] = 30 # change the 2nd value
numerical indexing is great if you know the order of values in the table.
2. Use a HashTable for named indexing
$fHT = #{} # hold counts of each forest
$forest = "example"
$count = 100
$fHT.$forest = $count
$fHT # show all
$fHT.$forest # show the count for the current value of $forest
3. Use dynamic properties for named indexing
$fDP = new-object PSObject
$fDP | add-member -membertype noteproperty -name $forest -value $count
and as Andy suggests, these can even be combined
$fHT = #{} # hold counts of each forest
$forest = "example forest 1"
$count = 100
$fHT.$forest = $count
$fDP = New-Object PSObject -Property $fHT
$fDP # show all
$fDP.$forest # show the count for the current value of $forest
How about creating a custom object. Creating your own object is a great way to associate pieces of data together. In your case you'd like to associate a number to either a name of a forest or a forest object created by something else. Regardless if you are working with a list of forest name strings or objects you can use this approach.
Create a container (an array) for your custom objects.
$forestContainer = #()
As you loop through your forests, create a custom object for each and add it to the array. In this example $myForests is either a collection of objects or a string array of names.
$i = 1
foreach ($forest in $myForests) {
# Create a custom object associating all the data you want.
$currentForest = New-Object PsObject -Property #{
Forest = $forest
Count = $i
# Note: You can add as many properties to the object here as you like.
}
$forestContainer += $currentForest
$i += 1
}
The benefit of this approach is you will be able to use the array filled with your new objects with other Powershell cmdlets easily. For example:
$forestContainer | where {$_.Count -lt 2}
Or
$forestContainer | format-table
Or
$forestContainer | Out-GridView
To access the Count property which contains the current integer when that forest was processed you can use this syntax:
$forestContainer[0].Count
Hopefully you can apply this to your code. If you post some of your code I can help you more.