Saving VisualVM information as data - visualvm

Using VisaulVM, I'd like to obtain this as data, without image processing algorithms being applied... How can I do that? I think this won't come out of a snapshot.
I am not sure how VisualVM and jVisualVM vary, the naming is sure confusing, but I'm running the Oracle supplied one (Version 1.7.0_80 (Build 150109))
Thanks!

You can use Tracer plugin with various probes. Tracer can export data in CSV, HTML or XML.

All this information is available through JMX. That's how VisualVM gets the information and you can use the same technology to get it too. First install the VisualVM-MBeans plugin from the Tools menu. This will add another tab titled MBeans where you can see all the available data for your application. You will find the graphed data under java.lang.Memory and java.lang.OperationSystem.
If you're trying to check information for your own process, it's as simple as calling ManagementFactory.getOperatingSystemMXBean().getSystemLoadAverage() and ManagementFactory.getMemoryMXBean().getHeapMemoryUsage(). There are more, but these should get you started.
To get precise CPU usage see: Using OperatingSystemMXBean to get CPU usage
If you want to get information on another process, you'd need some more code. There is a complete answer on Accessing a remote MBean server but basically:
// replace host and port
// not tested, might not work
JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://<addr>:<port>");
JMXConnector jmxConnector = JMXConnectorFactory.connect(url);
MBeanServerConnection connection = jmxConnector.getMBeanServerConnection();
OperatingSystemMXBean bean = ManagementFactory.getPlatformMXBean(connection, OperatingSystemMXBean.class);
bean.getSystemLoadAverage();
You will also have to start your Java process with exposed JMX as explained on How to activate JMX on my JVM for access with jconsole? but basically:
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9010
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
There is also a way to enumerate Java processes running on the local machine and even connect to processes that don't have JMX enabled (though you get less data). If that's what you're looking for, VisualVM Source Code will be a good place to start.

To answer your other question about naming:
VisualVM is opensource project hosted at visualvm.java.net and Java VisualVM is stable version of VisualVM with Oracle branding and other small changes. Java VisualVM is distributed in JDK. There is a table where you can find which VisualVM release is the basis for Java VisualVM in respective JDK update.

Related

How to deploy an atg project in weblogic?

I created a simple project using ATG 10.2 .I want to know how to deploy it in weblogic. Please provide detailed procedure with screenshots,if possible.
To provide a 'detailed' procedure is beyond the scope of what StackOverflow is trying to provide. That said, if you have an understanding of the Weblogic Management Console you should be able to follow these steps to setup your initial deployment:
Create a Server
1.1 Specify a server name (eg. commerce) and the port number this server will run on (eg. 8180). Select it as a 'Stand-alone server'.
1.2 Once created go to Configuration > Server Start for the newly created server and modify the 'Arguments' block and include the following setings (assuming you are running windows, for Unix update your own paths)
-Datg.dynamo.data-dir=c:\ATG-Data -Datg.dynamo.server.name=commerce -d64 -XX:ParallelGCThreads=8 -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Xms1152m -Xmx2048m -XX:NewSize=128m -XX:MaxNewSize=256m -XX:PermSize=128m -XX:MaxPermSize=256m
1.3 Save your Server
Create DataSources
2.1 In the Console click on 'Services > Data Sources'
2.2 Create 'New' datasources for each of your connections. As a minimum you will need connections for ATGSwitchingDS_A, ATGSwitchingDS_B (Assuming you are doing switching datasources) and ATGProductionDS. These names should match your JNDI names in your property files. Remember to specify the 'commerce' server as the target for each of the datasources.
Create Deployment
3.1 Assuming you've already built your EAR (eg. ATGProduction.ear) and it is available in c:\deployments you need to create a deployment in Weblogic. You need to create the deployment in the console and specify the target as 'commerce'. Once done you need to also 'start serving requests' on the deployment.
Start Server
You should now be able to see your server running on port 8180 with the log files being written to c:\ATG-Data\servers\commerce\logs.
If after this things aren't running, post specific questions about your issues and someone here might be able to help you.

How to submit code to a remote Spark cluster from IntelliJ IDEA

I have two clusters, one in local virtual machine another in remote cloud. Both clusters in Standalone mode.
My Environment:
Scala: 2.10.4
Spark: 1.5.1
JDK: 1.8.40
OS: CentOS Linux release 7.1.1503 (Core)
The local cluster:
Spark Master: spark://local1:7077
The remote cluster:
Spark Master: spark://remote1:7077
I want to finish this:
Write codes(just simple word-count) in IntelliJ IDEA locally(on my laptp), and set the Spark Master URL to spark://local1:7077 and spark://remote1:7077, then run my codes in IntelliJ IDEA. That is, I don't want to use spark-submit to submit a job.
But I got some problem:
When I use the local cluster, everything goes well. Run codes in IntelliJ IDEA or use spark-submit can submit job to cluster and can finish the job.
But When I use the remote cluster, I got a warning log:
TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
It is sufficient resources not sufficient memory!
And this log keep printing, no further actions. Both spark-submit and run codes in IntelliJ IDEA result the same.
I want to know:
Is it possible to submit codes from IntelliJ IDEA to remote cluster?
If it's OK, does it need configuration?
What are the possible reasons that can cause my problem?
How can I handle this problem?
Thanks a lot!
Update
There is a similar question here, but I think my scene is different. When I run my codes in IntelliJ IDEA, and set Spark Master to local virtual machine cluster, it works. But I got Initial job has not accepted any resources;... warning instead.
I want to know whether the security policy or fireworks can cause this?
Submitting code programatically (e.g. via SparkSubmit) is quite tricky. At the least there is a variety of environment settings and considerations -handled by the spark-submit script - that are quite difficult to replicate within a scala program. I am still uncertain of how to achieve it: and there have been a number of long running threads within the spark developer community on the topic.
My answer here is about a portion of your post: specifically the
TaskSchedulerImpl: Initial job has not accepted any resources; check
your cluster UI to ensure that workers are registered and have
sufficient resources
The reason is typically there were a mismatch on the requested memory and/or number of cores from your job versus what were available on the cluster. Possibly when submitting from IJ the
$SPARK_HOME/conf/spark-defaults.conf
were not properly matching the parameters required for your task on the existing cluster. You may need to update:
spark.driver.memory 4g
spark.executor.memory 8g
spark.executor.cores 8
You can check the spark ui on port 8080 to verify that the parameters you requested are actually available on the cluster.

Weblogic 12c setting PermSize when using NodeManager

We have a Windows Server 2012 64bit + Weblogic 12c setup. The AdminServer requires a higher PermSize when being used with a 64bit OS, thus we need to modify the "setDomainEnv.cmd" (as described in other questions here on stackoverflow).
When starting the AdminServer through the usual "startWeblogic.cmd" script, it uses the settings in "setDomainEnv.cmd" that sets the PermSize etc. successfully, but when using NodeManager "startServer()" command, it does not.
I read something in the documentation about the fact that one can control the parameters that are loaded on startup of a managed server (with NodeManager), but I did not find the right way to do it.
I would hope that we can achieve a consistent behaviour when starting a managed server (and the AdminServer) through NodeManager or manually.
Any ideas?
UPDATE:
I checked what's going on when starting managed server and(!) in comparison what's going on when starting the AdminServer. Result: the AdminServer process (it starts a 'javaw.exe' instance in contrast to a 'java.exe' instance for a managed server) never get's passed ANY parameters set in the setDomainEnv.cmd script.. it's basically full of Oracle internal parameters.
To me all this looks completely messed up and inconsistent. In addition to this I found an issue reported by Oracle that mystically talks about setting environment variables when running on a 64bit OS (see headline "Developer ZIP Distribution Fails on Windows 64-bit and Linux 64-bit"):
https://docs.oracle.com/cd/E24329_01/doc.1211/e26593/issues.htm#WLSRN238
I have idea if this applies to my version or not, since the version I downloaded does not say "developer" version, it basically was the primary weblogic download for the latest release.
The question that comes to my mind is this: what is the expected way of starting the AdminServer if not using "startServer"? Is there a bug that nobody cares about, since it is usually done differently? I am really disappointed to how confusing this rather simple topic evolves when starting to read Oracle documentation: it simply does not say anything about it at all.
Command line that is triggered when starting the AdminServer through "startServer()" command:
C:\PROGRA~1\Java\JDK17~1.0_6\jre\bin\javaw.exe -classpath "C:\PROGRA~1\Java\JDK17~1.0_6\jre\lib\rt.jar;C:\PROGRA~1\Java\JDK17~1.0_6\jre\lib\i18n.jar;C:\PROGRA~1\Java\JDK17~1.0_6\lib\tools.jar;D:\Oracle\Middleware\wlserver\server\lib\weblogic_sp.jar;D:\Oracle\Middleware\wlserver\server\lib\weblogic.jar;D:\Oracle\Middleware\oracle_common\modules\net.sf.antcontrib_1.1.0.0_1-0b3\lib\ant-contrib.jar;D:\Oracle\Middleware\wlserver\modules\features\oracle.wls.common.nodemanager_2.0.0.0.jar;D:\Oracle\Middleware\oracle_common\modules\com.oracle.cie.config-wls-online_8.1.0.0.jar;D:\Oracle\Middleware\wlserver\common\derby\lib\derbyclient.jar;D:\Oracle\Middleware\wlserver\common\derby\lib\derby.jar;D:\Oracle\Middleware\wlserver\server\lib\xqrl.jar" "-Djava.runtime.name=Java(TM) SE Runtime Environment" -Dpython.cachedir=C:\Users\ADMINI~1\AppData\Local\Temp\2\wlstTempAdministrator -Djava.protocol.handler.pkgs=weblogic.utils|weblogic.utils|weblogic.utils -Djava.vm.version=24.65-b04 "-Djava.vm.vendor=Oracle Corporation" -Djava.vendor.url=http://java.oracle.com/ -Dpath.separator=; "-Djava.vm.name=Java HotSpot(TM) 64-Bit Server VM" -Dweblogic.RootDirectory=D:\Oracle\Middleware\user_projects\domains\test1234\. "-Djava.vm.specification.name=Java Virtual Machine Specification" -Djava.runtime.version=1.7.0_67-b01 -Djavax.rmi.CORBA.UtilClass=weblogic.iiop.UtilDelegateImpl -Djava.awt.graphicsenv=sun.awt.Win32GraphicsEnvironment -Djava.endorsed.dirs=C:\PROGRA~1\Java\JDK17~1.0_6\jre\lib\endorsed -Dos.arch=amd64 -Djava.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\2\ -Dline.separator=
"-Djava.vm.specification.vendor=Oracle Corporation" -Djava.naming.factory.url.pkgs=weblogic.jndi.factories:weblogic.corba.j2ee.naming.url "-Dos.name=Windows Server 2012 R2" -Dprod.props.file=D:\Oracle\Middleware\wlserver\.product.properties -Dorg.omg.CORBA.ORBSingletonClass=weblogic.corba.orb.ORB -Djava.library.path=C:\PROGRA~1\Java\JDK17~1.0_6\jre\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;;D:\Oracle\Middleware\wlserver\server\native\win\x64;D:\Oracle\Middleware\wlserver\server\bin;D:\Oracle\Middleware\oracle_common\modules\org.apache.ant_1.9.2\bin;C:\PROGRA~1\Java\JDK17~1.0_6\jre\bin;C:\PROGRA~1\Java\JDK17~1.0_6\bin;D:\Oracle\product\12.1.0\dbhome_1\BIN;C:\Windows\System32;C:\Windows;C:\Windows\System32\wbem;C:\Windows\System32\WINDOW~1\v1.0\;C:\PROGRA~2\VISUAL~1\bin;C:\PROGRA~1\doxygen\bin;C:\PROGRA~1\TORTOI~1\bin;C:\PROGRA~2\WINDOW~4\8.0\WINDOW~1\;C:\PROGRA~1\MICROS~1\110\Tools\Binn\;D:\Oracle\Middleware\wlserver\server\native\win\x64\oci920_8;. "-Djava.specification.name=Java Platform API Specification" -Djava.class.version=51.0 -Dorg.omg.CORBA.ORBClass=weblogic.corba.orb.ORB -Dos.version=6.3 -Djavax.rmi.CORBA.PortableRemoteObjectClass=weblogic.iiop.PortableRemoteObjectDelegateImpl -Djava.awt.printerjob=sun.awt.windows.WPrinterJob -Djava.specification.version=1.7 -Djava.class.path=C:\PROGRA~1\Java\JDK17~1.0_6\lib\tools.jar;D:\Oracle\Middleware\wlserver\server\lib\weblogic_sp.jar;D:\Oracle\Middleware\wlserver\server\lib\weblogic.jar;D:\Oracle\Middleware\oracle_common\modules\net.sf.antcontrib_1.1.0.0_1-0b3\lib\ant-contrib.jar;D:\Oracle\Middleware\wlserver\modules\features\oracle.wls.common.nodemanager_2.0.0.0.jar;D:\Oracle\Middleware\oracle_common\modules\com.oracle.cie.config-wls-online_8.1.0.0.jar;D:\Oracle\Middleware\wlserver\common\derby\lib\derbyclient.jar;D:\Oracle\Middleware\wlserver\common\derby\lib\derby.jar;D:\Oracle\Middleware\wlserver\server\lib\xqrl.jar -Djava.vm.specification.version=1.7 -Dweblogic.management.GenerateDefaultConfig=false -Djava.home=C:\PROGRA~1\Java\JDK17~1.0_6\jre "-Djava.specification.vendor=Oracle Corporation" -Dawt.toolkit=sun.awt.windows.WToolkit "-Djava.vm.info=mixed mode" -Djava.version=1.7.0_67 -Djava.ext.dirs=C:\PROGRA~1\Java\JDK17~1.0_6\jre\lib\ext;C:\Windows\Sun\Java\lib\ext "-Djava.vendor=Oracle Corporation" -Djava.vendor.url.bug=http://bugreport.sun.com/bugreport/ -Dweblogic.store.DisableDiskScheduler=true -Dpython.verbose=warning weblogic.Server
UPDATE 2:
Start the AdminServer through node manager (nmStart('AdminServer')) creates a usual "java.exe" process and starts up the AdminServer with correct memory settings. But this is even more confusing: why is "startServer()" creating a separate process (javaw.exe) with entirely different settings? Why are my settings now totally different for AdminServer? What is the "correct" way of starting the AdminServer (development/production?). Two thumbs down on this environment.
UPDATE 3:
After repeating further tests the solution of getting "startServer()" to work is basically as follows: do not worry about the node manager settings at all, edit the "startWeblogic" script directly by adding additional java options inside of it (as usual by adding -D start parameters). The reason for all this is basically that the global settings (as used by node manager) are ignored completely, see my pasted command line output.
Check the nodemanager.properties file in your Oracle install ( e.g. /opt/ora/mw/wlserver_10.3/common/nodemanager/nodemanager.properties ) and verify that these options are set:
StartScriptName=startManagedWebLogic.sh
StartScriptEnabled=true
so the nodemanager is starting your servers with the appropriate scripts. You also have to option of setting server specific start attributes via the admin console - go to:
Servers -> Server Name -> Server Start tab -> Arguments
You can fill in server specific JVM args, like -XX:MaxPermSize=4096m in this field that will be used by the nodemanager. This may be a better/easier idea than hard coding it in the setDomainEnv script.
UPDATE
Attempt issuing an nmStart() command rather than a startServer() command for the AdminServer.
startServer allows you to start a server WITHOUT the nodemanager. It uses javaw.exe to effectively background the process
nmStart allows you to start the server WITH the nodemanager - which is why you get the correct memory settings. Because the process is started via a service, it is more or less automatically backgrounded, which is why you see the normal java.exe

tomcat7 clickstack not finding Config params

I am testing the tomcat7 clickstack for our application which has some config parameters set using the built in Config features of Cloudbees. The tomcat7 clickstack does not find them, but the standard tomcat6 container does. I have double checked them and reset them through the cloudbees sdk and they are there and correct, but are coming back as null for tomcat7.
The switch to clickstacks requires us to refactor how the servlet container gets configured so that the injection points such as cloudbees-web.xml and jvm system properties behave consistently across all the servlet container clickstacks.
Some of that refactoring has been committed but some of the work is still in my backlog... Assuming none of the other bees steal that task from my backlog before I get to it ;-)
IF I recall correctly, the parameters should be available as environment variables (sub optimal I know, but all containers should be giving this as a consistent UX for all clickstacks, eg both non-java based and java-based) and may be already available as system properties (again sub optimal, but the java container refactoring should be giving this as a consistent UX for all java based clickstacks). The consistent java servlet UX has not been committed yet but should be available soon.

How to use java.util.logging in Weblogic?

I have an application that was migrated from Glassfish to Weblogic, and it uses java.util.logging as logging framework.
The only way I have found to make the logs work is by editing the logging.properties file of the JVM and restart the server. This solution is awkward and gives problems because the log is written to a different file than the standard ones for weblogic, so we have to look at too many files for a log in a clustered environment. Besides, for some reason this does not work on some Windows systems.
Is there a way to keep using standard java logging to write messages to weblogic's standard log files? I tried the instructions on this page but it doesn't work either.
WebLogic Server ships with a JDK logging handler which will pick up log messages emitted from JDK logging framework and direct them into the WebLogic Server logging system.
Set the default logging level for new ServerLoggingHandler instances in logging.properties as well as adding the ServerLoggingHandler to the handlers.
handlers = weblogic.logging.ServerLoggingHandler
weblogic.logging.ServerLoggingHandler.level = ALL
http://docs.oracle.com/cd/E14571_01/web.1111/e13739/logging_services.htm#CHDBBEIJ
To direct the JDK logging framework to use the logging.properties file, the standard System property java.util.logging.config.file is used. With WebLogic Server, this can be easily accomplished by setting the JAVA_OPTIONS System property with the corresponding value.
$ export JAVA_OPTIONS="-Djava.util.logging.config.file=/Users/xxx/Projects/Domains/wls1035/logging.properties"
Some more hints here: http://buttso.blogspot.de/2011/06/using-slf4j-with-weblogic-server.html