I want to adjust the lower and upper bound of three pyo.Var() depending on the outcome of another variable.
the variable model.inverter_power should be in the i-th iteration bigger than model.inverter_power[i].lb + model.fcr_power[i] but also smaller than model.inverter_power[i].ub - model.fcr_power[i])
How can i implement this? Unfortuntately my idea is not working....
def fcr_inverter_reduction(model, i):
return (model.inverter_power[i] >= model.inverter_power[i].lb + model.fcr_power[i],
model.inverter_power[i] <= model.inverter_power[i].ub - model.fcr_power[i])
model.fcr_inverter_rule = pyo.Constraint(model.i, rule = fcr_inverter_reduction)
I tried various versions of this code, not only is this code not linear anymore, so I used ipopt as a solver but no solution can be found, i got this error message:
File "D:\.conda\envs\PythonEnviromentV2\lib\site-packages\pyomo\opt\base\solvers.py", line 596, in solve
raise ApplicationError(
pyomo.common.errors.ApplicationError: Solver (ipopt) did not exit normally
This is very doable if you reformulate just a bit. Also, I don't think it is possible to return a tuple of 2 constraints like you are doing with that function, so you should break it up... it is clearer as well.
You probably could access the upper/lower bound and use that within the constraint because they are fixed/constant with respect to the solver, but I think it is probably clearer to break out your min/max values as parameters. Some variation of this works.
(also, in the future, you are more likely to get better help/results if you post a fully minimal-reproducible example instead of just 1-line.)
Code:
import pyomo.environ as pyo
model = pyo.ConcreteModel()
model.I = pyo.Set(initialize=[1,2,3])
# power parameters...
model.min_inverter = pyo.Param(model.I, initialize={1:10, 2:15, 3:22})
model.max_inverter = pyo.Param(model.I, initialize={1:55, 2:45, 3:80})
# vars...
model.inverter_power = pyo.Var(model.I)
model.fcr_power = pyo.Var(model.I)
def fcr_inverter_min(model, i):
return model.inverter_power[i] >= model.min_inverter[i] + model.fcr_power[i]
model.fcr_inverter_rule_min = pyo.Constraint(model.I, rule=fcr_inverter_min)
def fcr_inverter_max(model, i):
return model.inverter_power[i] <= model.max_inverter[i] - model.fcr_power[i]
model.fcr_inverter_rule_max = pyo.Constraint(model.I, rule=fcr_inverter_max)
model.pprint()
Output:
1 Set Declarations
I : Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 3 : {1, 2, 3}
2 Param Declarations
max_inverter : Size=3, Index=I, Domain=Any, Default=None, Mutable=False
Key : Value
1 : 55
2 : 45
3 : 80
min_inverter : Size=3, Index=I, Domain=Any, Default=None, Mutable=False
Key : Value
1 : 10
2 : 15
3 : 22
2 Var Declarations
fcr_power : Size=3, Index=I
Key : Lower : Value : Upper : Fixed : Stale : Domain
1 : None : None : None : False : True : Reals
2 : None : None : None : False : True : Reals
3 : None : None : None : False : True : Reals
inverter_power : Size=3, Index=I
Key : Lower : Value : Upper : Fixed : Stale : Domain
1 : None : None : None : False : True : Reals
2 : None : None : None : False : True : Reals
3 : None : None : None : False : True : Reals
2 Constraint Declarations
fcr_inverter_rule_max : Size=3, Index=I, Active=True
Key : Lower : Body : Upper : Active
1 : -Inf : inverter_power[1] - (55 - fcr_power[1]) : 0.0 : True
2 : -Inf : inverter_power[2] - (45 - fcr_power[2]) : 0.0 : True
3 : -Inf : inverter_power[3] - (80 - fcr_power[3]) : 0.0 : True
fcr_inverter_rule_min : Size=3, Index=I, Active=True
Key : Lower : Body : Upper : Active
1 : -Inf : 10 + fcr_power[1] - inverter_power[1] : 0.0 : True
2 : -Inf : 15 + fcr_power[2] - inverter_power[2] : 0.0 : True
3 : -Inf : 22 + fcr_power[3] - inverter_power[3] : 0.0 : True
I am new to optimization programming using python and I have a problem with defining a variable in both pyomo and PuLP for a MILP problem. I am using gene expression data and I am confused about how to define a variable as a MXN matrix. I am trying to use either pyomo or PuLP. I know how to use MATLAB and define it so I am putting it make it more helpful to anyone to understand.
y = optimvar('y',M,N,'Type','integer','LowerBound',0,'UpperBound',1)
Also If you think there are better python package please give me your suggestion.
Thank you very much in advance !!
I'd pick pyomo. I think it is more expressive and easier to read/work with. You'll need to install a solver separately. If that is daunting, pulp comes with CBC built in.
This should get you started. There are a decent number of examples in the pyomo documentation to get you started.
import pyomo.environ as pe
m = pe.ConcreteModel('example')
m.M = pe.Set(initialize=[1, 2, 3], name='the M set')
m.N = pe.Set(initialize=['A', 'B', 'C'], name='the N set')
m.X = pe.Var(m.M, m.N, domain=pe.NonNegativeReals)
m.pprint()
Yields:
3 Set Declarations
M : Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 3 : {1, 2, 3}
N : Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 3 : {'A', 'B', 'C'}
X_index : Size=1, Index=None, Ordered=True
Key : Dimen : Domain : Size : Members
None : 2 : M*N : 9 : {(1, 'A'), (1, 'B'), (1, 'C'), (2, 'A'), (2, 'B'), (2, 'C'), (3, 'A'), (3, 'B'), (3, 'C')}
1 Var Declarations
X : Size=9, Index=X_index
Key : Lower : Value : Upper : Fixed : Stale : Domain
(1, 'A') : 0 : None : None : False : True : NonNegativeReals
(1, 'B') : 0 : None : None : False : True : NonNegativeReals
(1, 'C') : 0 : None : None : False : True : NonNegativeReals
(2, 'A') : 0 : None : None : False : True : NonNegativeReals
(2, 'B') : 0 : None : None : False : True : NonNegativeReals
(2, 'C') : 0 : None : None : False : True : NonNegativeReals
(3, 'A') : 0 : None : None : False : True : NonNegativeReals
(3, 'B') : 0 : None : None : False : True : NonNegativeReals
(3, 'C') : 0 : None : None : False : True : NonNegativeReals
4 Declarations: M N X_index X
[Finished in 1.5s]
================
Edit: Making crossed sets
Some variation of below should work fine. You can either make a full x-set by initializing to the cross of two sets, or if your model is sparse, and it makes sense, I'd encourage you to use the within keyword to control the domain and only populated necessary pairs with initialize as shown.
import pyomo.environ as pe
m = pe.ConcreteModel('example')
m.M = pe.Set(initialize=[1, 2, 3], name='the M set')
m.N = pe.Set(initialize=['A', 'B', 'C'], name='the N set')
m.cross = pe.Set(within=m.M*m.N, initialize = [(1,'A'), (3,'B')])
Is there a way to set multiple indexes to a set value within one constraint without having to type out the same variable for each indexed time. I have provided an example, imagine that you want to optimize when to charge your electric vehicle but you don't want it to charge at certain hours in a day. The example below works to avoid charging at the 4th and 5th hour. However, what if I want it to not charge for 15 hours of the day but don't feel like writing m.EVcharge[0]+m.EVcharge[1]+... putting m.EVcharge[:15] == 0 will not work because the constraints don't handle slices that well in pyomo.
def time_rule(m):
return m.EVcharge[4]+m.EVcharge[5] == 0
m.time_rule = Constraint(time, rule=time_rule)
Yes. There are a variety of ways to do this. You could make a subset of m.time and only pass that subset in to the constraint rule, which would constrain them to zero, or sum across the subset and make constrain it to zero (both assume negative charging is not possible.)
Or, you could do it more cleanly with data or a parameter that holds the limit for any arbitrary time block and use that, which keeps the data separate from the model, which is generally a good idea...
import pyomo.environ as pyo
# some background data on limits...
use_limit = { 0:3, # limited juice avial
1:3,
2:0, # no juice avail. :)
3:0}
m = pyo.ConcreteModel('EV Charge')
m.T = pyo.Set(initialize=range(6))
m.EV_charge = pyo.Var(m.T, domain=pyo.NonNegativeReals)
# Constraints
def charge_limit(m, time):
return m.EV_charge[time] <= use_limit[time]
m.C1 = pyo.Constraint(use_limit.keys(), rule=charge_limit)
m.pprint()
Yields:
2 Set Declarations
C1_index : Size=1, Index=None, Ordered=False
Key : Dimen : Domain : Size : Members
None : 1 : Any : 4 : {0, 1, 2, 3}
T : Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 6 : {0, 1, 2, 3, 4, 5}
1 Var Declarations
EV_charge : Size=6, Index=T
Key : Lower : Value : Upper : Fixed : Stale : Domain
0 : 0 : None : None : False : True : NonNegativeReals
1 : 0 : None : None : False : True : NonNegativeReals
2 : 0 : None : None : False : True : NonNegativeReals
3 : 0 : None : None : False : True : NonNegativeReals
4 : 0 : None : None : False : True : NonNegativeReals
5 : 0 : None : None : False : True : NonNegativeReals
1 Constraint Declarations
C1 : Size=4, Index=C1_index, Active=True
Key : Lower : Body : Upper : Active
0 : -Inf : EV_charge[0] : 3.0 : True
1 : -Inf : EV_charge[1] : 3.0 : True
2 : -Inf : EV_charge[2] : 0.0 : True
3 : -Inf : EV_charge[3] : 0.0 : True
I am currently attempting to solve a network problem that is not fully connected.Thus,I have attempted to do some preprocessing of data so as to form a set of tuples, e.g. {(a,b , (c,e)}..., i.e. from a to b, from c to e.
I am able to declare binary decision variables with keys such as (a,b), (c,e), via using the set of tuples for indexing.
However, when I tried to use rules to declare constraints, with decision variables such as x[i][j], errors are thrown stating that (a,b) is an invalid index.
Hence, I would like to ask if tuples can be used as indices for decision variables.
If not, is there a way to only declare the only decision variables that are needed, rather than declaring all, and then setting those unneeded to 0.
Thank you!
Welcome to the site.
You certainly CAN index variables with tuples of arbitrary dimensionality. The code I have below shows an example of indexing bad_X with a tuple of nodes. This makes sense when you have a variable that has a logical representation for all combinations of your indices or you control the indices somehow with smart set notation or you risk a whole bunch of nonsense variables as you can see in my example below with bad_X in the printout.
For your network, I would suggest just making a set of ARCS that are the valid combination of the NODES. This set would just contain the tuples of the valid connections. Note in my example below, the set of NODES could be blown away as it is not needed. You could just create ARCS directly. Sometimes it is handy to have both if you have some node-based data like supply/demand.
import pyomo.environ as pyo
mdl = pyo.ConcreteModel()
arc_data = { ('a', 'b'): 10,
('c', 'd'): 5,
('e', 'j'): 44,
('a', 'j'): 2,
('j', 'e'): 12}
# collapse the nodes into a set from the keys of the arc data
node_set = set()
for k in arc_data.keys():
node_set.update(*k)
# Sets
mdl.NODES = pyo.Set(initialize=node_set)
mdl.ARCS = pyo.Set(within=mdl.NODES * mdl.NODES, initialize=arc_data.keys())
# Params
mdl.cost = pyo.Param(mdl.NODES, mdl.NODES, initialize=arc_data)
# Selection Variable: poor choice...
mdl.bad_X = pyo.Var(mdl.NODES, mdl.NODES, domain=pyo.Binary)
# some examples of using this "bad X"
print('printing just to show how to access...this will produce a None')
print(mdl.bad_X['a', 'e'].value)
print(mdl.bad_X[('c', 'd')].value) # also valid
# Better selection variable
mdl.X = pyo.Var(mdl.ARCS, domain=pyo.Binary)
# toy constraint...ensure at least 3 are selected
mdl.c1 = pyo.Constraint(expr=sum(mdl.X[arc] for arc in mdl.ARCS) >= 3)
# Objective: Minimize cost of 3 selected arcs
mdl.OBJ = pyo.Objective(expr=sum(mdl.X[arc] * mdl.cost[arc] for arc in mdl.ARCS))
# to show the difference between the choices of variables
mdl.pprint()
# solve and show results
solver = pyo.SolverFactory('cbc')
results = solver.solve(mdl)
for arc in mdl.ARCS:
print(arc, mdl.X[arc].value)
Output:
printing just to show how to access...this will produce a None
None
None
5 Set Declarations
ARCS : Dim=0, Dimen=2, Size=5, Domain=ARCS_domain, Ordered=False, Bounds=None
[('a', 'b'), ('a', 'j'), ('c', 'd'), ('e', 'j'), ('j', 'e')]
ARCS_domain : Dim=0, Dimen=2, Size=36, Domain=None, Ordered=False, Bounds=None
Virtual
NODES : Dim=0, Dimen=1, Size=6, Domain=None, Ordered=False, Bounds=None
['a', 'b', 'c', 'd', 'e', 'j']
bad_X_index : Dim=0, Dimen=2, Size=36, Domain=None, Ordered=False, Bounds=None
Virtual
cost_index : Dim=0, Dimen=2, Size=36, Domain=None, Ordered=False, Bounds=None
Virtual
1 Param Declarations
cost : Size=5, Index=cost_index, Domain=Any, Default=None, Mutable=False
Key : Value
('a', 'b') : 10
('a', 'j') : 2
('c', 'd') : 5
('e', 'j') : 44
('j', 'e') : 12
2 Var Declarations
X : Size=5, Index=ARCS
Key : Lower : Value : Upper : Fixed : Stale : Domain
('a', 'b') : 0 : None : 1 : False : True : Binary
('a', 'j') : 0 : None : 1 : False : True : Binary
('c', 'd') : 0 : None : 1 : False : True : Binary
('e', 'j') : 0 : None : 1 : False : True : Binary
('j', 'e') : 0 : None : 1 : False : True : Binary
bad_X : Size=36, Index=bad_X_index
Key : Lower : Value : Upper : Fixed : Stale : Domain
('a', 'a') : 0 : None : 1 : False : True : Binary
('a', 'b') : 0 : None : 1 : False : True : Binary
('a', 'c') : 0 : None : 1 : False : True : Binary
('a', 'd') : 0 : None : 1 : False : True : Binary
('a', 'e') : 0 : None : 1 : False : True : Binary
('a', 'j') : 0 : None : 1 : False : True : Binary
('b', 'a') : 0 : None : 1 : False : True : Binary
('b', 'b') : 0 : None : 1 : False : True : Binary
('b', 'c') : 0 : None : 1 : False : True : Binary
('b', 'd') : 0 : None : 1 : False : True : Binary
('b', 'e') : 0 : None : 1 : False : True : Binary
('b', 'j') : 0 : None : 1 : False : True : Binary
('c', 'a') : 0 : None : 1 : False : True : Binary
('c', 'b') : 0 : None : 1 : False : True : Binary
('c', 'c') : 0 : None : 1 : False : True : Binary
('c', 'd') : 0 : None : 1 : False : True : Binary
('c', 'e') : 0 : None : 1 : False : True : Binary
('c', 'j') : 0 : None : 1 : False : True : Binary
('d', 'a') : 0 : None : 1 : False : True : Binary
('d', 'b') : 0 : None : 1 : False : True : Binary
('d', 'c') : 0 : None : 1 : False : True : Binary
('d', 'd') : 0 : None : 1 : False : True : Binary
('d', 'e') : 0 : None : 1 : False : True : Binary
('d', 'j') : 0 : None : 1 : False : True : Binary
('e', 'a') : 0 : None : 1 : False : True : Binary
('e', 'b') : 0 : None : 1 : False : True : Binary
('e', 'c') : 0 : None : 1 : False : True : Binary
('e', 'd') : 0 : None : 1 : False : True : Binary
('e', 'e') : 0 : None : 1 : False : True : Binary
('e', 'j') : 0 : None : 1 : False : True : Binary
('j', 'a') : 0 : None : 1 : False : True : Binary
('j', 'b') : 0 : None : 1 : False : True : Binary
('j', 'c') : 0 : None : 1 : False : True : Binary
('j', 'd') : 0 : None : 1 : False : True : Binary
('j', 'e') : 0 : None : 1 : False : True : Binary
('j', 'j') : 0 : None : 1 : False : True : Binary
1 Objective Declarations
OBJ : Size=1, Index=None, Active=True
Key : Active : Sense : Expression
None : True : minimize : 10*X[a,b] + 5*X[c,d] + 44*X[e,j] + 2*X[a,j] + 12*X[j,e]
1 Constraint Declarations
c1 : Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : 3.0 : X[a,b] + X[c,d] + X[e,j] + X[a,j] + X[j,e] : +Inf : True
10 Declarations: NODES ARCS_domain ARCS cost_index cost bad_X_index bad_X X c1 OBJ
('a', 'b') 1.0
('c', 'd') 1.0
('e', 'j') 0.0
('a', 'j') 1.0
('j', 'e') 0.0
[Finished in 4.3s]
Second answer to get at your second point about indexing in constraints...
This shows some variations of the concept you asked about. Note that because I built params for every possible combination of city-year, that I needed to supply a default value for the ones that were not in my sparse data.
import pyomo.environ as pyo
mdl = pyo.ConcreteModel()
demand_data = { ('LA', 1999): 500,
('LA', 2001): 700,
('NY', 2001): 600,
('Pheonix', 2000): 300,
('Pheonix', 2001): 400 }
my_favorites = {'LA'} # a subset for example use
# Sets
mdl.CITIES = pyo.Set(initialize= {k[0] for k in demand_data.keys()})
mdl.YEARS = pyo.Set(initialize= {k[1] for k in demand_data.keys()})
# Params
mdl.demand = pyo.Param(mdl.CITIES, mdl.YEARS, initialize=demand_data, default=0)
mdl.cost = pyo.Param(mdl.YEARS, initialize={1999: 5, 2000: 10, 2001: 15}, default=0)
# Variable for supply
mdl.supply = pyo.Var(mdl.CITIES, mdl.YEARS, domain=pyo.NonNegativeReals)
#### Constraints ####
#ensure 1/2 of demand in each city-year pair
def half_demand(self, city, year):
return mdl.supply[city, year] >= 0.5 * mdl.demand[city, year]
mdl.c1 = pyo.Constraint(mdl.CITIES, mdl.YEARS, rule=half_demand)
# ensure favorite cities get all demand every year
def all_demand(self, city, year):
return mdl.supply[city, year] >= mdl.demand[city, year]
mdl.c2 = pyo.Constraint(my_favorites, mdl.YEARS, rule=all_demand)
# ensure all demand is eventually met over all years
def eventually_met(self, city):
return sum(mdl.supply[city, year] for year in mdl.YEARS) >= \
sum(mdl.demand[city, year] for year in mdl.YEARS)
mdl.c3 = pyo.Constraint(mdl.CITIES, rule=eventually_met)
#### OBJECTIVE ####
def objective(self):
return sum(mdl.supply[city, year] * mdl.cost[year]
for year in mdl.YEARS
for city in mdl.CITIES)
mdl.OBJ = pyo.Objective(rule=objective)
# solve and show results
solver = pyo.SolverFactory('cbc')
results = solver.solve(mdl)
mdl.supply.display()
Output:
supply : Size=9, Index=supply_index
Key : Lower : Value : Upper : Fixed : Stale : Domain
('LA', 1999) : 0 : 500.0 : None : False : False : NonNegativeReals
('LA', 2000) : 0 : 0.0 : None : False : False : NonNegativeReals
('LA', 2001) : 0 : 700.0 : None : False : False : NonNegativeReals
('NY', 1999) : 0 : 300.0 : None : False : False : NonNegativeReals
('NY', 2000) : 0 : 0.0 : None : False : False : NonNegativeReals
('NY', 2001) : 0 : 300.0 : None : False : False : NonNegativeReals
('Pheonix', 1999) : 0 : 350.0 : None : False : False : NonNegativeReals
('Pheonix', 2000) : 0 : 150.0 : None : False : False : NonNegativeReals
('Pheonix', 2001) : 0 : 200.0 : None : False : False : NonNegativeReals
[Finished in 3.1s]
I think I manage to solve it, please refer to the example below
##First, create the set of tuples needed for filtering
#Op_Machine: set of (operation, machine) tuples created to avoid redundancy in decision variable declaration
Op_Machine=list()
for machine_id, op_proctime in Machine_Op_Time.items():
for op in op_proctime.keys():
print(Op_Machine)
print((op,machine_id))
Op_Machine.append((op,machine_id))
print(Op_Machine)
##Next, invoke the rule using the if statement to filter across all possible indices accepting those combinations that are aligned with the tuples within the set
##Use Constraint.Skip to Skip creating constraints that do not belong to the set of tuples
def F1_rule(model,i,k):
if (i,k) in Op_Machine:
##print(i,k)
return model.Cmax>=model.completion_time[i,k]
else:
return Constraint.Skip
#model.makespan= Constraint(model.op_set, model.mach_set, rule=Cmax_rule)
model.F1= Constraint(Operation_Set, Machine_Set, rule=F1_rule)
Note that the sets Operation_Set, Machine_Set function as the universal set as it comprises all combinations of operations and machines. Hence the statement model.F1= Constraint(Operation_Set, Machine_Set, rule=F1_rule) can be thought a for loop that iterates over all combinations while the if statement within the def function acts a filter to generate the needed constraints.
In the below, the purpose is to extract information from specific lines, I output the desired file with my code, but i have small problem in the selection of.
/Live Seis/{s=s","substr($0,12,1500)}
I selected the range of 12-1500 fields, because in my code I select always the last field, but in this line this option does not work.
Is is possible to improve or change the code to other more efficient.
Input file
SCI TB Timestamp Local : 1/2/10 9:54:37.780 AM
File # : 5000
Is Raw : false
Shot # : 754
Swath Name : 1000
Tape # : 2240
Tape Label : X5
Tape Time : 5/4/18 9:55:01.619 AM
Comment : N/A
Line Name : 40559.0
Point Number : 38741.0
Point Index : 1
Easting (m) : 0
Northing (m) : 0
Elevation (m) : 0.0
Source Number : 6
Source ID : 1
SFL : 0
SFN : 0
Spread : 754
Process Type : 0
Nb of Aux : 4
Exploitation Mode : 0
Acq : 1
Correl Pilot : CorrelWith
Type of Test : N/A
Record Type : NORMAL
Acq Length (ms) : 0
Sweep Length (ms) : 0
Record Length (ms) : 0
SR : SR_2_MS
Filter Type : 0
SF : 1
Dump SF : 1
Live Seis : 40560:38466-39172/2(1-354) 40572:38466-39172/2(355-708) 40584:38466-39172/2(709-1062) 40596:38466-39172/2(1063-1416) 40608:38466-39172/2(1417-1770) 40620:38466-39172/2(1771-2124) 40632:38466-39172/2(2125-2478) 40644:38466-39172/2(2479-2832) 40656:38466-39172/2(2833-3186) 40668:38466-39172/2(3187-3540) 40680:38466-39172/2(3541-3894) 40692:38466-39172/2(3895-4248) 40704:38466-39172/2(4249-4602) 40716:38466-39172/2(4603-4956) 40728:38466-39172/2(4957-5310) 40740:38466-39172/2(5311-5664) 40752:38466-39172/2(5665-6018) 40764:38466-39172/2(6019-6372) 40776:38466-39172/2(6373-6726) 40788:38466-39172/2(6727-7080) 40800:38466-39172/2(7081-7434) 40812:38466-39172/2(7435-7788) 40824:38466-39172/2(7789-8142) 40836:38466-39172/2(8143-8496) 40848:38466-39172/2(8497-8850) 40860:38466-39172/2(8851-9204) 40872:38466-39172/2(9205-9558) 40884:38466-39172/2(9559-9912) 40896:38466-39172/2(9913-10266) 40908:38466-39172/2(10267-10620) 40920:38466-39172/2(10621-10974) 40932:38466-39172/2(10975-11328) 40944:38466-39172/2(11329-11682) 40956:38466-39172/2(11683-12036) 40968:38466-39172/2(12037-12390) 40980:38466-39172/2(12391-12744)
Live Aux : 1-2
Dead Seis : N/A
Missing Seis : N/A
Acq Error : N/A
ITB : N/A
CTB (?s) : N/A
Uphole Time (?s) : N/A
User Header : N/A
Jday : 124
Nb Of Seis : 12746
SCI TB Timestamp GPS Time : 1209452095780002
COG State : COG_ACTUAL
Deviation : 0.5
Cog Easting (m) : 0
Cog Northing (m) : 0
Cog Elevation (m) : 0
Traces Total : 00000
Nb Of Live Seis : 00000
Nb Of Live Aux : 2
Nb Of Dead Seis : 0
Nb Of Missing Seis : 0
Max Of Max Seis : 7.25919534678016E15
Hist Zeroing Length : N/A
________________________________________________________________________
SCI TB Timestamp Local : 2/1/10 9:15:51.580 PM
File # : 5001
Is Raw : false
Shot # : 50
Swath Name : 1000
Tape # : 2240
Tape Label : X5
Tape Time : 5/4/18 9:16:15.011 PM
Comment : N/A
Line Name : 40549.0
Point Number : 39053.0
Point Index : 1
Easting (m) : 0
Northing (m) : 0
Elevation (m) : 0.0
Source Number : 4
Source ID : 1
SFL : 0
SFN : 0
Spread : 50
Process Type : 0
Nb of Aux : 4
Exploitation Mode : 0
Acq : 1
Correl Pilot : 0
Type of Test : N/A
Record Type : NORMAL
Acq Length (ms) : 0
Sweep Length (ms) : 0
Record Length (ms) : 0
SR : SR_2_MS
Filter Type : 0
SF : 1
Dump SF : 1
Live Seis : 40560:38622-39484/2(1-432) 40572:38622-39484/2(433-864) 40584:38622-39484/2(865-1296) 40596:38622-39484/2(1297-1728) 40608:38622-39484/2(1729-2160) 40620:38622-39484/2(2161-2592) 40632:38622-39484/2(2593-3024) 40644:38622-39484/2(3025-3456) 40656:38622-39484/2(3457-3888) 40668:38622-39484/2(3889-4320) 40680:38622-39484/2(4321-4752) 40692:38622-39484/2(4753-5184) 40704:38622-39484/2(5185-5616) 40716:38622-39484/2(5617-6048) 40728:38622-39484/2(6049-6480) 40740:38622-39484/2(6481-6912) 40752:38622-39484/2(6913-7344) 40764:38622-39484/2(7345-7776) 40776:38622-39484/2(7777-8208) 40788:38622-39484/2(8209-8640) 40800:38622-39484/2(8641-9072) 40812:38622-39484/2(9073-9504) 40824:38622-39484/2(9505-9936) 40836:38622-39484/2(9937-10368) 40848:38622-39484/2(10369-10800) 40860:38622-39484/2(10801-11232) 40872:38622-39484/2(11233-11664) 40884:38622-39484/2(11665-12096) 40896:38622-39484/2(12097-12528) 40908:38622-39484/2(12529-12960) 40920:38622-39484/2(12961-13392) 40932:38622-39484/2(13393-13824) 40944:38622-39484/2(13825-14256) 40956:38622-39484/2(14257-14688) 40968:38622-39484/2(14689-15120) 40980:38622-39484/2(15121-15552)
Live Aux : 1-2
Dead Seis : N/A
Missing Seis : N/A
Acq Error : N/A
ITB : N/A
CTB (?s) : N/A
Uphole Time (?s) : N/A
User Header : N/A
Jday : 124
Nb Of Seis : 15554
SCI TB Timestamp GPS Time : 1209492969580002
COG State : COG_ACTUAL
Deviation : 3.0
Cog Easting (m) : 0
Cog Northing (m) : 0
Cog Elevation (m) : 178.4
Traces Total : 0000
Nb Of Live Seis : 0000
Nb Of Live Aux : 2
Nb Of Dead Seis : 0
Nb Of Missing Seis : 0
Max Of Max Seis : -1.13516329566208E14
Hist Zeroing Length : N/A
________________________________________________________________________
Output file
21,1000,2240,X5,40559.0,38741.0, 40560:38466-39172/2(1-354) 40572:38466-39172/2(355-708) 40584:38466-39172/2(709-1062) 40596:38466-39172/2(1063-1416) 40608:38466-39172/2(1417-1770) 40620:38466-39172/2(1771-2124) 40632:38466-39172/2(2125-2478) 40644:38466-39172/2(2479-2832) 40656:38466-39172/2(2833-3186) 40668:38466-39172/2(3187-3540) 40680:38466-39172/2(3541-3894) 40692:38466-39172/2(3895-4248) 40704:38466-39172/2(4249-4602) 40716:38466-39172/2(4603-4956) 40728:38466-39172/2(4957-5310) 40740:38466-39172/2(5311-5664) 40752:38466-39172/2(5665-6018) 40764:38466-39172/2(6019-6372) 40776:38466-39172/2(6373-6726) 40788:38466-39172/2(6727-7080) 40800:38466-39172/2(7081-7434) 40812:38466-39172/2(7435-7788) 40824:38466-39172/2(7789-8142) 40836:38466-39172/2(8143-8496) 40848:38466-39172/2(8497-8850) 40860:38466-39172/2(8851-9204) 40872:38466-39172/2(9205-9558) 40884:38466-39172/2(9559-9912) 40896:38466-39172/2(9913-10266) 40908:38466-39172/2(10267-10620) 40920:38466-39172/2(10621-10974) 40932:38466-39172/2(10975-11328) 40944:38466-39172/2(11329-11682) 40956:38466-39172/2(11683-12036) 40968:38466-39172/2(12037-12390) 40980:38466-39172/2(12391-12744) ,1209452095780002
1853,1000,2240,X5,40549.0,39053.0, 40560:38622-39484/2(1-432) 40572:38622-39484/2(433-864) 40584:38622-39484/2(865-1296) 40596:38622-39484/2(1297-1728) 40608:38622-39484/2(1729-2160) 40620:38622-39484/2(2161-2592) 40632:38622-39484/2(2593-3024) 40644:38622-39484/2(3025-3456) 40656:38622-39484/2(3457-3888) 40668:38622-39484/2(3889-4320) 40680:38622-39484/2(4321-4752) 40692:38622-39484/2(4753-5184) 40704:38622-39484/2(5185-5616) 40716:38622-39484/2(5617-6048) 40728:38622-39484/2(6049-6480) 40740:38622-39484/2(6481-6912) 40752:38622-39484/2(6913-7344) 40764:38622-39484/2(7345-7776) 40776:38622-39484/2(7777-8208) 40788:38622-39484/2(8209-8640) 40800:38622-39484/2(8641-9072) 40812:38622-39484/2(9073-9504) 40824:38622-39484/2(9505-9936) 40836:38622-39484/2(9937-10368) 40848:38622-39484/2(10369-10800) 40860:38622-39484/2(10801-11232) 40872:38622-39484/2(11233-11664) 40884:38622-39484/2(11665-12096) 40896:38622-39484/2(12097-12528) 40908:38622-39484/2(12529-12960) 40920:38622-39484/2(12961-13392) 40932:38622-39484/2(13393-13824) 40944:38622-39484/2(13825-14256) 40956:38622-39484/2(14257-14688) 40968:38622-39484/2(14689-15120) 40980:38622-39484/2(15121-15552) ,1209492969580002
My code
awk '/File/{if(s){print s;s=$NF+0}else{s=$NF+0}}
/Swath Name/{s=s","$NF}
/Tape #/{s=s","$NF}
/Tape Label /{s=s","$NF}
/Line Name/{s=s","$NF}
/Point Number/{s=s","$NF}
/Live Seis/{s=s","substr($0,12,1500)}
/SCI TB Timestamp GPS Time/{s=s","$NF}
END{print s}' file
Thanks in advance