After implementing the answer to this question on how to set up a script for time visualization in this project (which uses a small extension to the published Log::Timeline that allows me to set the logging file from the program itself), I still get the same error
12:18 Timeline connection error: Could not get timeline data: java.net.ConnectException: Conexión rehusada
(which means refused connection). I've also checked the created files, and they are empty, they don't receive anything. I'm using this to log:
class Events does Log::Timeline::Event['ConcurrentEA', 'App', 'Log'] { }
(as per the README.md file). It's probably the case that there's no such thing as a default implementation, as shown in the tests, but, in that case, what would be the correct way of making it print to the file and also connect to the timeline visualizer?
If you want to use the timeline visualization, leave the defaults for logging, commenting out any modification of the standard logging output. In my case:
#BEGIN {
# PROCESS::<$LOG-TIMELINE-OUTPUT>
# = Log::Timeline::Output::JSONLines.new(
# path => log-file
# )
#}
Not really sure if this would have happened if an output file is defined using an environment variable, but in any case, better to be on the safe side. You can use the output file when you eventually drop the script into production.
Related
I have a celery app which has to be pinged by another app. This other app uses json to serialize celery task parameters, but my app has a custom serialization protocol. When the other app tries to ping my app (app.control.ping), it throws the following error:
"Celery ping failed: Refusing to deserialize untrusted content of type application/x-stjson (application/x-stjson)"
My whole codebase relies on this custom encoding, so I was wondering if there is a way to configure a json serialization but only for this ping, and to continue using the custom encoding for the other tasks.
These are the relevant celery settings:
accept_content = [CUSTOM_CELERY_SERIALIZATION, "json"]
result_accept_content = [CUSTOM_CELERY_SERIALIZATION, "json"]
result_serializer = CUSTOM_CELERY_SERIALIZATION
task_serializer = CUSTOM_CELERY_SERIALIZATION
event_serializer = CUSTOM_CELERY_SERIALIZATION
Changing any of the last 3 to [CUSTOM_CELERY_SERIALIZATION, "json"] causes the app to crash, so that's not an option.
Specs: celery=5.1.2
python: 3.8
OS: Linux docker container
Any help would be much appreciated.
Changing any of the last 3 to [CUSTOM_CELERY_SERIALIZATION, "json"] causes the app to crash, so that's not an option.
Because result_serializer, task_serializer, and event_serializer doesn't accept list but just a single str value, unlike e.g. accept_content
The list for e.g. accept_content is possible because if there are 2 items, we can check if the type of an incoming request is one of the 2 items. It isn't possible for e.g. result_serializer because if there were 2 items, then what should be chosen for the result of task-A? (thus the need for a single value)
This means that if you set result_serializer = 'json', this will have a global effect where all the results of all tasks (the returned value of the tasks which can be retrieved by calling e.g. response.get()) would be serialized/deserialized using the json-serializer. Thus, it might work for the ping but it might not for the tasks that can't be directly serialized/deserialized to/from JSON which really needs the custom stjson-serializer.
Currently with Celery==5.1.2, it seems that task-specific setting of result_serializer isn't possible, thus we can't set a single task to be encoded in 'json' and not 'stjson' without setting it globally for all, I assume the same case applies to ping.
Open request to add result_serializer option for tasks
A short discussion in another question
Not the best solution but a workaround is that instead of fixing it in your app's side, you may opt to just add support to serialize/deserialize the contents of type 'application/x-stjson' in the other app.
other_app/celery.py
import ast
from celery import Celery
from kombu.serialization import register
# This is just a possible implementation. Replace with the actual serializer/deserializer for stjson in your app.
def stjson_encoder(obj):
return str(obj)
def stjson_decoder(obj):
obj = ast.literal_eval(obj)
return obj
register(
'stjson',
stjson_encoder,
stjson_decoder,
content_type='application/x-stjson',
content_encoding='utf-8',
)
app = Celery('other_app')
app.conf.update(
accept_content=['json', 'stjson'],
)
You app remains to accept and respond stjson format, but now the other app is configured to be able to parse such format.
I get the following error:
Keen.delete(:iron_worker_analytics, filters: [{:property_name => 'start_time', :operator => 'eq', :property_value => '0001-01-01T00:00:00Z'}])
Keen::ConfigurationError: Keen IO Exception: Project ID must be set
However, when I set the value, I get the following:
warning: already initialized constant KEEN_PROJECT_ID
iron.io/env.rb:36: warning: previous definition of KEEN_PROJECT_ID was here
Keen works fine when I run the app and load the values from a env.rb file but from the console I cannot get past this.
I am using the ruby gem.
I figured it out. The documentation is confusing. Per the documentation:
https://github.com/keenlabs/keen-gem
The recommended way to set keys is via the environment. The keys you
can set are KEEN_PROJECT_ID, KEEN_WRITE_KEY, KEEN_READ_KEY and
KEEN_MASTER_KEY. You only need to specify the keys that correspond to
the API calls you'll be performing. If you're using foreman, add this
to your .env file:
KEEN_PROJECT_ID=aaaaaaaaaaaaaaa
KEEN_MASTER_KEY=xxxxxxxxxxxxxxx
KEEN_WRITE_KEY=yyyyyyyyyyyyyyy KEEN_READ_KEY=zzzzzzzzzzzzzzz If not,
make a script to export the variables into your shell or put it before
the command you use to start your server.
But I had to set it explicitly as Keen.project_id after doing a Keen.methods.
It's sort of confusing since from the docs, I assumed I just need to set the variables. Maybe I am misunderstanding the docs but it was confusing at least to me.
I have written a code in twisted .I need to write the log information in when we have call
d.addErrback(on_failure).
from twisted.python import log
log.startLogging(open('/home/crytek.etl/foo.log', 'w'))
def on_failure(failure):
log.msg(failure)
d.addErrback(on_failure)
Is this the correct way of implementing this.I don't get any values written to the file.Can someone suggest on how this can be implemented
You probably want to consider opening your log file in append mode. Otherwise, every time your application starts you'll wipe out all your old logs. This could make it appear as though the log messages you're expecting to see aren't being logged.
from twisted.python import log
log.startLogging(open('/home/crytek.etl/foo.log', 'a'))
You should also log failures using log.err instead of log.msg
def on_failure(failure):
log.err(failure)
And you can do this more easily since on_failure has exactly the same signature as log.err. Just write:
d.addErrback(log.err)
Also, I liked, log.err doesn't have exactly the same signature as on_failure. It is better, it accepts a 2nd argument which is used to present a header for the failure in the log file. You can use it like this:
d.addErrback(log.err, "Frobbing the widget failed")
This will present "Frobbing the widget failed" together with the failure in the log file.
So I have a little trouble on the opening of a JFrame. I searched extensively on the net, but I really can not find a solution ...
I explained the situation:
I need to develop an application that needs to retrieve information tracking application while meeting new safety standards. For that I use JMX that allows monitoring and VisualVM to see these information.
I therefore I connect without problems (recently ^ ^) to JMX since VisualVM.
There is thus in a VisualVM plugin for recovering information on MBean, including those on Methods (Operations tab in the plugin).
This allows among others to stop a service or create an event.
My problem then comes when I try to display a result of statistics.
In fact, I must show, at the click of a button from the list of methods in the "Operations", a window with a table in HTML (titles, colors and everything else).
For that I use a JFrame:
public JFrame displayHTMLJFrame(String HTML, String title){
JFrame fen = new JFrame();
fen.setSize(1000, 800);
fen.setTitle(title);
JEditorPane pan = new JEditorPane();
pan.setEditorKit(new HTMLEditorKit());
pan.setEditable(false);
pan.setText(HTML);
fen.add(pan);
return fen;
}
I call it in my method:
public JFrame displayHtmlSqlStatOK_VM(){
return displayHTMLJFrame(displaySQLStat(sqlStatOK, firstMessageDate), "SqlStatOK");
}
The method must therefore giving me back my JFrame, but she generates an error:
Problem invoking displayHtmlSqlStatOK_VM : java.rmi.UnmarshalException: error unmarshalling return; nested
exception is:
java.io.InvalidClassException: javax.swing.JFrame; local class incompatible: stream classdesc serialVersionUID =
-5208364155946320552, local class serialVersionUID = -2386951414768123374
I saw on the internet that this was a version problem (Serialization), and I believe strongly that it comes from the fact that I have this:
Server - JDK5 <----> Client (VisualVM) - JDK6
Knowing that I can not to change the server version (costs too important ...) as advocated by some sites and forums.
My question is as follows:
Can I display this damn window keeping my current architecture (JDK5 server side and client side JDK6)?
I could maybe force the issue? Tell him that there's nothing bad that can run my code? Finally I'm asking him but he does not answer me maybe to you he will tell you ... (Yes I crack ^^).
Thank you very much to those who read me and help me!
If you need more info do not hesitate.
EDIT
The solution to my problem might be elsewhere, because in fact I just want a table with minimal formatting (this is just for viewing application for an for an officer to have his little table him possibly putting critical data in red...).
But I have nowhere found a list of types that I can return with VisualVM ... This does not however seem to me too much to ask.
After I had thought of a backup solution, which would be to create a temporary HTML file and open it automatically in the browser, but right after that is perhaps not very clean ... But if it can work ^^
I am open to any area of research!
It looks like you are sending instance javax.swing.JFrame over the JMX connection - this is a bad idea.
Well good I found myself, as a great :)
Thank you bye!
..........
Just kidding of course I will give the solution that I found ^ ^
So here's what I did:
My display to be done on the client (normal...) my code to display a JFrame that I had set up on the server was displayed obviously ... On the server xD
I didn't want to change the customer (VisualVM) to allow users maximum flexibility. However I realized that to display my HTML table to be rendered usable (with colors and everything) I had to change the client (as JMX does not support the type JFrame as type back an operation).
My operation running from the MBeans plugin for VisualVM, it was necessary that I find the source code for it to say "Be careful if you see that I give you the HTML you display it in a JFrame".
Here is my approach:
- Get the sources
The link SVN to get sources VisualVM is as follows:
https: //svn.java.net/svn/visualvm~svn/branches/release134
If like me you have trouble with the SVN client includes in NetBeans because you are behind a proxy, you can do it by command line:
svn --config-option servers:global:http-proxy-host=MY_PROXY_HOST --config-option servers:global:http-proxy-port=MY_PROXY_PORT checkout https: //svn.java.net/svn/visualvm~svn/branches/release134 sources-visualvm
Putting you on your destination folder of course (cd C:\Users\me\Documents\SourcesVisualVM example).
- Adding the platform VisualVM
NetBeans needs the platform VisualVM to create modules (plugins) for it. For this, go to "Tools" -> "NetBeans Platforms".
Then click "Add Platform ..." at the bottom left of the window and select the folder to the bin downloaded at this address: http:// visualvm.java.net/download.html
You should have this:
http://img15.hostingpics.net/pics/543268screen1.png
- Adding sources in the workspace (NetBeansProjects)
Copy/paste downloaded sources (SVN from the link above) to your NetBeans workspace (by default in C:\Users\XXX\Documents\NetBeansProjects).
- Ouverture du projet du plugin MBeans
In NetBeans, right click in the Project Explorer (or go to the menu "Files") and click "Open Project ...".
You will then have a list of projects in your workspace.
Open the project "mbeans" found in "release134" -> "Plugins", as below:
http://img15.hostingpics.net/pics/310487screen2.png
- Changing the file "platform.properties"
To build plugin you must define some variables for your platform.
To do this, open the file platform.properties in the directory release134\plugins\nbproject of your workspace.
Replace the content (by changing the paths compared to yours):
cluster.path=\
C:\\Program Files\\java6\\visualvm_134\\platform:\
C:\\Program Files\\java6\\visualvm_134\\profiler
# Deprecated since 5.0u1; for compatibility with 5.0:
disabled.clusters=
nbjdk.active=default
nbplatform.active=VisualVM_1.3.4
suite.dir=${basedir}
harness.dir= C:\\Program Files\\NetBeans 7.1.2\\harness
- Changing the class XMBeanOperations
To add our feature (displaying an HTML table), you must change the class that processes operations, namely the class XMBeanOperations in package com.sun.tools.visualvm . modules.mbeans.
At line 173, replace:
if (entryIf.getReturnType() != null &&
!entryIf.getReturnType().equals(Void.TYPE.getName()) &&
!entryIf.getReturnType().equals(Void.class.getName()))
fireChangedNotification(OPERATION_INVOCATION_EVENT, button, result);
By :
if (entryIf.getReturnType() != null &&
!entryIf.getReturnType().equals(Void.TYPE.getName()) &&
!entryIf.getReturnType().equals(Void.class.getName())) {
if (entryIf.getReturnType() instanceof String) {
String res = result + "";
if (res.indexOf("<html>") != -1) {
JFrame frame = displayHTMLJFrame(res, button.getText());
frame.setVisible(true);
}
else
fireChangedNotification(OPERATION_INVOCATION_EVENT, button, result);
} else
fireChangedNotification(OPERATION_INVOCATION_EVENT, button, result);
}
With the method of creating the JFrame that you place above "void performInvokeRequest (final JButton button)" for example:
// Display a frame with HTML code
public JFrame displayHTMLJFrame(String HTML, String title){
JFrame fen = new JFrame();
fen.setSize(1000, 800);
fen.setTitle(title);
JEditorPane pan = new JEditorPane();
pan.setEditorKit(new HTMLEditorKit());
pan.setEditable(false);
pan.setText(HTML);
fen.add(pan);
return fen;
}
We can see that we already did a test on the return type, if it is a String which is returned, if the case, if we see in this string the balise , then we replace the result of the click by opening a JFrame with the string you put in, what makes us display our HTML code!
- Creating a .nbm
The file .nbm is the deployment file of your plugin. Simply right-click your project (in the Project Explorer) and click on "Create NBM".
Your file .nbm will be created in the folder "build" the root of your project.
- Installing the plugin in VisualVM
To install your plugin, you must just go in VisualVM, go into "Tools" -> "Plugins" tab and then "Downloaded", click "Add Plugins ...". Select your plugin .nbm then click "Install". Then follow the instructions.
Useful Sources
http: //docs.oracle.com/javase/6/docs/technotes/guides/visualvm/
http: //visualvm.java.net/"]http://visualvm.java.net/
http: //visualvm.java.net/api-quickstart.html (Créer un plugin VisualVM avec NetBeans)
Thank you very much for your help Tomas Hurka ;)
for some test I need to run a data driven test with a configuration that is generated (via reflection) in the ClassInitialize method (by using reflection). I tried out everything, but I just can not get the data source properly set up.
The test takes a list of classes in a csv file (one line per class) and then will test that the mappings to the database work out well (i.e. try to get one item from the database for every entity, which will throw an exception when the table structure does not match).
The testmethod is:
[DataSource(
"Microsoft.VisualStudio.TestTools.DataSource.CSV",
"|DataDirectory|\\EntityMappingsTests.Types.csv",
"EntityMappingsTests.Types#csv",
DataAccessMethod.Sequential)
]
[TestMethod()]
public void TestMappings () {
Obviously the file is EntityMappingsTests.Types.csv. It should be in the DataDirectory.
Now, in the Initialize method (marked with ClassInitialize) I put that together and then try to write it.
WHERE should I write it to? WHERE IS THE DataDirectory?
I tried:
File.WriteAllText(context.TestDeploymentDir + "\\EntityMappingsTests.Types.csv", types.ToString());
File.WriteAllText("EntityMappingsTests.Types.csv", types.ToString());
Both result in "the unit test adapter failed to connect to the data source or read the data". More exact:
Error details: The Microsoft Jet database engine could not find the
object 'EntityMappingsTests.Types.csv'. Make sure the object exists
and that you spell its name and the path name correctly.
So where should I put that file?
I also tried just writing it to the current directory and taking out the DataDirectory part - same result. Sadly, there is limited debugging support here.
Please use the ProcessMonitor tool from technet.microsoft.com/en-us/sysinternals/bb896645. Put a filter on MSTest.exe or the associate qtagent32.exe and find out what locations it is trying to load from and at what point in time in the test loading process. Then please provide an update on those details here .
After you add the CSV file to your VS project, you need to open the properties for it. Set the Property "Copy To Output Directory" to "Copy Always". The DataDirectory defaults to the location of the compiled executable, which runs from the output directory so it will find it there.