I have a .rrd db which is collecting data from a temperature gauge. Now I have a second gauge so I'd like to add this new gauge to the existing .rrd database. I tried many times with the "rrdtool tune" command, but after that I run a "rrdtool info" on my database, and I see that there's not the last data source (another gauge) that I tried to insert.
How can I do this?
The command you need is, as you say, rrdtool tune. The documentation is available online at https://oss.oetiker.ch/rrdtool/doc/rrdtune.en.html
The ability to extend an RRA and to add or remove a DS was only added late in RRDTool 1.4. Check that you are not using an older version of RRDTool, as if you are, you will not be able to use this feature until you upgrade.
I just checked, and I see I'm using RRDTOOL 1.4 so I would not have problems. Anyway, the fact is that I used this command:
/usr/bin/rrdtool tune TEMPCucina.rrd DS:METEOTEMPEXT:GAUGE:1200:U:U RRA:AVERAGE:0.5:1:180000
I got this back from the computer:
DS[TEMPCucina] typ: GAUGE hbt: 1200 min: nan max: nan
But it seems that I'm not able to write into TEMPCucina.rrd
And if I try to perform the following command:
rrdtool info TEMPCucina.rrd
I just get the following, and it seems that no new gauge has been created
filename = "TEMPCucina.rrd"
rrd_version = "0003"
step = 60
last_update = 1510780261
header_size = 556
ds[TEMPCucina].index = 0
ds[TEMPCucina].type = "GAUGE"
ds[TEMPCucina].minimal_heartbeat = 1200
ds[TEMPCucina].min = NaN
ds[TEMPCucina].max = NaN
ds[TEMPCucina].last_ds = "18"
ds[TEMPCucina].value = 1,8000000000e+01
ds[TEMPCucina].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 30000
rra[0].cur_row = 1304
rra[0].pdp_per_row = 1
rra[0].xff = 0,0000000000e+00
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
(when I try to write I get this, but I don't know how to proceed at this point)
ERROR: TEMPCucina.rrd: illegal attempt to update using time 1510780527 when last update time is 1510780527 (minimum one second step)
I finally did it, but I wasn't able to use the rrdtool tune function.
I finally found here how to perform a dump of the database, how to modify it, and finally restore it to its original location (so I could also correct some data).
This is not what I was searching for, but it solved my problem so I want to share it.
Related
I recently got into camelot and but for some reason need to use my own backend to remove text during the imaging process.
Here is what I tried based on the documentation:
class ConversionBackend(object):
def convert(pdf_path, png_path):
arg2 = '-sOutputFile=' + png_path
p = subprocess.Popen(['/usr/bin/gs', '-sDEVICE=png16m', '-dNOPAUSE', '-dBATCH', '-dQUIET', "-dFILTERIMAGE", "-dFILTERTEXT", "-r300",str(arg2), pdf_path], stdout = subprocess.PIPE)
pass`
Aside from the fact that my ghostscript use is not optimal, all of this runs does work when I run it step by step in the python console. However when calling camelot using
tables = camelot.read_pdf(path, line_scale=100, split_text=True, flag_size=True, layout_kwargs={'detect_vertical': False, 'char_margin': 2.0}, pages='all', backend=ConversionBackend())
Camelot still executes my command but with complete disregard for the backend=ConversionBackend()
Any ideas on how to fix this?
Sh4yce
Do you know how to set up the Spreading Factor to 12 in a Mbed-OS LoRaWAN protocol APIs to connect to a LoRaWAN network using OTAA?
I'm trying to make LoRA node to use Spreading Factor SF12, because the default one is SF7. I know that in the PHY layer we can change Radio configurations. There are several examples to change between the different sub-GHz frequency bands, however, I can't find one on how to change the LoRa modulation SF between 7 and 12 and with a bandwidth of 125 kHz.
I'm using an SX1276 radio at EU 868 MHz config.
In the source code you can find the SF7-12 different configurations, but there is not a clear way to set it up. These configs are the definitions (#define) DR_0, DR_, etc ).
In the configuration file in the Phy part you find some example like this:
"phy": {
"help": "LoRa PHY region. 0 = EU868 (default), 1 = AS923,
2 = AU915, 3 = CN470, 4 = CN779, 5 = EU433,
6 = IN865, 7 = KR920, 8 = US915, 9 = US915_HYBRID",
"value": "0"
},
But there is no examples or description for the Spreading Factor.
I would like to change it via source code, rather than the configuration file.
EDIT 1:
after Jon's answer, I add the following lines, but still not forcing the SF12 Joins.
retcode = lorawan.disable_adaptive_datarate ();
retcode = lorawan.set_datarate (0); // DR_0
Call:
lorawan.set_datarate(0); // SF12 125 KHz
Make sure to:
Disable ADR.
Either use ABP, or call the function above in the JOIN_SUCCESS event handler. This is because join procedure always starts at SF7, and then keeps the data rate on which the join succeeded.
I coding in julia using JuMP of Juliopt.
I would like to know how to print the solve progress using CbcSolver?
I already used the log level parameter:
m = Model(solver = CbcSolver(log=1))
it works and you can choose between 0 to 3, higher more details.
but my problem is that the log level only prints after the solver finish, i already put the the time limit parameter "sec" but when it is set for higher values never stops and i don't know why!!
m = Model(solver = CbcSolver(sec=90,log=1))
I was going to try callbacks but Cbc does not support callback.
I am positive that there is a way to print because when i use the opensolver in excel a set Cbc to solve it shows the progress.I just dont know how to do this!!
I'm trying to merge around 100 dataframes with a for loop and am getting a memory error. I'm using ipython jupyter notebook
Here is a sample of the data:
timestamp Namecoin_cap
0 2013-04-28 5969081
1 2013-04-29 7006114
2 2013-04-30 7049003
Each frame is around 1000 lines long
Here's the error in detail, I've also include my merge function.
My system is currently using up 64% of it memory
I have searched for similar issues but it seems most are for very large arrays >1GB, my data is relatively small in comparison.
EDIT: Something is suspicious. I wrote a beta program before, this was to test with 4 dataframes, i just exported that through pickle and it is 500kb. Now when i try to export the 100 frames one I get a memory error. It does however export a file that is 2GB. So i suspect somewhere down the line my code has created some kind of loop, creating a very large file. NB the 100 frames are stored in a dictionary
EDIT2: I have exported the scrypt to .py
http://pastebin.com/GqaHr7xc
This is a .xlsx that cointains asset names the script needs
The script fetches data regarding various assets, then cleans it up and saves each asset to a data frame in a dictionary
I'd be really appreciative if someone could have a look and see if there's anything immediately wrong. Other wise please advise on what tests I can run.
EDIT3: I'm finding it really hard to understand why this is happening, the code worked fine in the beta, all i have done now is add more assets.
EDIT4: I ran I size check on the object (dict of dfs) and it is 1,066,793 bytes
EDIT5: The problem is in the merge function for coin 37
for coin in coins[:37]:
data2['merged'] = pd.merge(left=data2['merged'],right=data2[coin], left_on='timestamp', right_on='timestamp', how='left')
This is when the error occurs. for coin in coins[:36]:' doesn't produce an error howeverfor coin in coins[:37]:' produces the error, any ideas ?
EDIT6: the 36th element is 'Syscoin', i did coins.remove('Syscoin') however the memory problem still occurs. So it seems to be a problem with the 36th element in coins no matter what the coin is
EDIT7: goCards suggestions seemed to work however the next part of the code:
merged = data2['merged']
merged['Total_MC'] = merged.drop('timestamp',axis=1).sum(axis=1)
Produces a memory error. I'm stumped
In regard to storage, I would recommend using a simple csv over pickle. Csv is a more generic format. It is human readable,and you can check your data quality easier especially as your data grows.
file_template_string='%s.csv'
for eachKey in dfDict:
filename = file_template_string%(eachKey)
dfDict[eachKey].to_csv(filename)
If you need to date the files you can also put a timestamp in the filename.
import time
from datetime import datetime
cur = time.time()
cur = datetime.fromtimestamp(cur)
file_template_string = "%s_{0}.csv".format(cur.strftime("%m_%d_%Y_%H_%M_%S"))
There are some obvious errors in your code.
for coin in coins: #line 61,89
for coin in data: #should be
df = data2['Namecoin'] #line 87
keys = data2.keys()
keys.remove('Namecoin')
for coin in keys:
df = pd.merge(left=df,right=data2[coin], left_on='timestamp', right_on='timestamp', how='left')
Same issue happened to me!
"MemoryError:" by notebook on execution of pandas. I have also screen printed quite lot of observations before issued happened.
Reinstalling Anaconda didn't help. Later realized that i was working with IPython notebook instead Jupyter notebook. Switched to Jupyter notebook. Everything worked fine!
I am using the function getgroup() to read all of the groups of a user in the active directory.
I'm not sure if I'm doing something wrong but it is very very slow. Each time it arrives at this point, it takes several seconds. I'm also accessing the rest of Active directory using the integrated function of "Accountmanagement" and it executes instantly.
Here's the code:
For y As Integer = 0 To AccountCount - 1
Dim UserGroupArray As PrincipalSearchResult(Of Principal) = UserResult(y).GetGroups()
UserInfoGroup(y) = New String(UserGroupArray.Count - 1) {}
For i As Integer = 0 To UserGroupArray.Count - 1
UserInfoGroup(y)(i) = UserGroupArray(i).ToString()
Next
Next
Later on...:
AccountChecker_Listview.Groups.Add(New ListViewGroup(Items(y, 0), HorizontalAlignment.Left))
For i As Integer = 0 To UserInfoGroup(y).Count - 1
AccountChecker_Listview.Items.Add(UserInfoGroup(y)(i)).Group = AccountChecker_Listview.Groups(y)
Next
Item(,) contains my normal Active directory data that I display Item(y, 0) contain the username.
y is the number of user accounts in AD. I also have some other code for the other information in this loop but it's not the issue here.
Anyone know how to make this goes faster or if there is another solution?
I'd recommend trying to find out where the time is spent. One option is to use a profiler, either the one built into Visual Studio or a third-party profiler like Redgate's Ants Profiler or the Yourkit .Net Profiler.
Another is to trace the time taken using the System.Diagnostics.Stopwatch class and use the results to guide your optimization efforts. For example time the function that retrieves data from Active Directory and separately time the function that populates the view to narrow down where the bottleneck is.
If the bottleneck is in the Active Directory lookup you may want to consider running the operation asynchronously so that the window is not blocked and populates as new data is retrieved. If it's in the listview you may want to consider for example inserting the data in a batch operation.