jdbc and processing output using Jakarta Commons Math - apache

Using jdbc I am querying my database of ambulance response times. My goal is to take the output and process it into statistics using Jakarta Commons Math library. So far I am successful in querying my database and outputting the response times to the console. My next step is to process this output statistically, such as mean, medians, mode, etc. This is where I am stuck. Shown below is my code.
package javaDatabase;
import java.sql.*;
import org.apache.commons.math3.stat.StatUtils;
public class javaConnect3
{
public static void main(String[] args)
{
Connection conn = null;
Statement stmt = null;
try
{
conn = DriverManager
.getConnection("jdbc:sqlserver://myServerAddress;database=myDatabase;integratedsecurity=false;user=myUser;password=myPassword");
stmt = conn.createStatement();
String strSelect = "SELECT M_SecondsAtStatus FROM MManpower WHERE M_tTime > 'august 25, 2014' AND M_Code = 'USAR'";
ResultSet rset = stmt.executeQuery(strSelect);
while (rset.next())
{
int values = rset.getInt("M_SecondsAtStatus");
System.out.println(values);
}
// I am hoping to derive useful statistics from my database, such as the following.
// this uses Jakarta Commons Math
// System.out.println("min: " + StatUtils.min(values));
// System.out.println("max: " + StatUtils.max(values));
// System.out.println("mean: " + StatUtils.mean(values));
// System.out.println("product: " + StatUtils.product(values));
// System.out.println("sum: " + StatUtils.sum(values));
// System.out.println("variance: " + StatUtils.variance(values));
} catch (SQLException ex)
{
ex.printStackTrace();
} finally
{
try
{
if (stmt != null)
stmt.close();
if (conn != null)
conn.close();
} catch (SQLException ex)
{
ex.printStackTrace();
}
}
}
}
An error message pops up in Eclipse and the variable "values" is red underlined; "values cannot be resolved to a variable".
I am not sure how to get this to work. I don't understand how to output my ambulance response times from the database into something Apache Commons math will understand.
How can I get Apache Commons math to take the output from my database and generate a statistical result?

You need something like this:
List<Double> values = new ArrayList<Double>();
while (rset.next()) {
values.add(rset.getInt("M_SecondsAtStatus"));
}
double mean = StatsUtils.mean(values.toArray(new Double [](values.size()));
It's likely that you could query the database for statistics. The name of the method depends on your database:
SELECT avg(M_SecondsAtStatus)
FROM MManpower
WHERE M_tTime > 'august 25, 2014'
AND M_Code = 'USAR'
I'd say the second one is more efficient, because you don't have to transfer all those values to the Java JVM to do the same calculation that the database can do for you.

Another alternative is to use a DescriptiveStatistics object to collect all the data and then use it to compute the summary statistics you need. Using it alleviates the need for doing all explicit type casting and conversion:
DescriptiveStatistics ds = new DescriptiveStatistics();
while(rset.next()) {
int observation = rset.getInt("M_SecondsAtStatus");
ds.addValue(observation);
}
System.out.println("min: " + ds.getMin());
System.out.println("max: " + ds.getMax());
...

Related

Google Bigquery java client listTables return numBytes and numRows as null

When I try to find the size of all the tables using listTables API of BigQuery java client, it returns null. But if I use getTable individually, I get the proper data. Is this a known issue, or am I doing something wrong. Following is the code that returns null value for numBytes:
Page<Dataset> datasetPage = getAllDatasets("projectId");
if(datasetPage!=null) {
for (Dataset dataset : datasetPage.iterateAll()) {
for(Table table : dataset.list().iterateAll()) {
System.out.println(table.getNumBytes()); // Returns Null. **
}
}
}
In this Public Issue Tracker thread, it has been discussed that getting null value for numBytes and numRows using listTables is the expected behaviour. The BigQuery API considers retrieving numBytes and numRows to be an expensive operation and thus returns null. So, the listTables only returns partial information on a table.
As a workaround, use getTable() to retrieve the information of the table individually in a loop. I tested the below code snippet and was able to get the table size in bytes for all the tables.
public static void getAllTableSize(String projectId) {
try {
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
Page<Dataset> datasetPage = bigquery.listDatasets(projectId);
if(datasetPage!=null) {
for (Dataset datasetTemp : datasetPage.iterateAll()) {
for(Table table : datasetTemp.list().iterateAll()) {
Table tableTemp = bigquery.getTable(table.getTableId());
String tableName = tableTemp.getTableId().getTable();
Long tableSize = tableTemp.getNumBytes();
System.out.println("Table Name: " + tableName + " " + "Table Size: " + tableSize);
}
}
}
} catch (BigQueryException e) {
System.out.println("Error occurred: " + e.toString());
}
}
To answer my own question listTables api is designed to return only the partial information. This is mentioned in the code document
https://github.com/googleapis/java-bigquery/blob/dfa15e5ca08a3227f015a389c4c08732178a73e7/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/spi/v2/BigQueryRpc.java#L155

Extract Data from a Data table to a text file in JPA

I'm working on data manipulation and my order is to extract all the content of a table into a text file ! I have already implemented this but apprently, I have commited a mistake while creating the request :
public void extractDonneesFichierPlat(){
TypedQuery<NotificationCnavOP> query = entityManager.createQuery("SELECT * INTO OUTFILE 'C:\\Test.txt' FROM NotificationCnavOP", NotificationCnavOP.class);
}
You cannot use "select * into outfile..." syntax in JPQL which is a separate query language different than SQL. If you want to execute native SQL queries in JPA you must use entityManager.createNativeQuery() method.
But still it is not possible to execute such 'bulk manipulation' query using this method.
The only solution I can think of is to use JDBC instead of JPA and execute:
Object o = entityManager.getDelegate();
System.out.println(o.getClass()); //prints the object class
// in my case it is org.hibernate.internal.SessionImpl
SessionImpl s = (SessionImpl)o;
Connection c = s.connection();
try {
Statement stmt = c.createStatement();
stmt.executeQuery("SELECT * INTO OUTFILE 'C:/Test.txt' FROM NotificationCnavOP");
} catch (SQLException e) {
e.printStackTrace();
}
The actual implementation of Session depends on the provider. In the case of Hibernate it is org.hibernate.internal.SessionImpl.
For newer versions of Hibernate it is org.hibernate.Session and it doesn't have the connection() method. So, you should try:
Object o = entityManager.getDelegate();
Session s = (Session)o;
s.doWork(connection -> {
try {
Statement stmt = connection.createStatement();
stmt.executeQuery("SELECT * INTO OUTFILE 'C:/Test2.txt' FROM NotificationCnavOP");
} catch (SQLException e) {
e.printStackTrace();
}
});
Both versions work in Hibernate 4.

Inserting data through GUI into sql server

I'm able to execute sql statements by writing the sql codes (Insert etc) on Eclipse and it is being displayed into sql server correctly. Connection has been done. But what should I do when a user wants to add data through a GUI interface (text field) and the data need to get stored into the database automatically ??
my code in the ADD button, but i'm getting the Error: java.lang.NullPointerException ! Help please..
try {
String pid = ProductID.getText();
String sql = "insert into Products_tbl values (' " +pid + " ')";
// Running the sql query
rs = st.executeQuery(sql);
int count = 0;
while (rs.next()) {
count = count + 1;
}
if (count == 1) {
JOptionPane.showMessageDialog(null, "Welcome");
}
else if (count > 1) {
JOptionPane.showMessageDialog(null,"Duplicate User Access Denied");
}
else {
JOptionPane.showMessageDialog(null, " User Not Found ");
}
}
catch (Exception ex) {
System.out.println("Error: " + ex);
}
1- Using (' " +pid + " ')" is not safe because SQL injection may occur. Use SqlParameters instead. Please check:
https://www.w3schools.com/sql/sql_injection.asp
2- I am pretty sure something is wrong with the line: rs = st.executeQuery(sql);
Here, I bet the value of st is null. Make sure that your connection variable is defined and set correctly and you created the statement like below:
st = connection.createStatement();
You can also try executeupdate(query) instead of executequery(query) like:
int flag = st.executeUpdate(query);
Ref: https://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html#executeUpdate%28java.lang.String%29
3- Please use printStackTrace() method while printing the error in the catch blog, the error message would be more understandable.
System.out.println("Error: " + ex.printStackTrace());

How to get the ID of inserted row in JDBC with old driver?

I want to get auto increment id of inserted row. I know that there is a lot of examples how to do that:
link1
link2
But I use HSQL 1.8.0.10 and following code:
PreparedStatement ps = conn.prepareStatement("insert into dupa (v1) values(3)", Statement.RETURN_GENERATED_KEYS);
throws expection:
java.sql.SQLException: This function is not supported
How to get id if driver does not support the above solution. Is any other way to get auto increment key of inserted row? I want to handle as much as possible drivers. So want to use obove code in try section and use another way in catch section.
Second question: Is possible that database does not support this feature. So even if I use new driver and old database It will still not work? I tried to use hsql 2.3.2 driver but I can not to connect to 1.8.0.10 database.
The following code illustrates how to retrieve generated keys from HSQLDB 2.2.9 and later using the included JDBC 4 driver. This method returns a two element long[]. The first element contains the number of rows that were updated; the second contains the generated key if any:
static final long[] doUpdate( ... ) {
final long[] k = new long[] {0, KEY_UNDEFINED};
PreparedStatement ps = null;
try {
ps = CXN.get().prepareStatement(sql, Statement.RETURN_GENERATED_KEYS);
JdbcValue jv;
for (int i = 0; i < data.size(); i++) {
jv = data.get(i);
jv.type().setJdbcParameter(i + 1, ps, jv);
}
k[NUM_OF_ROWS] = (long) ps.executeUpdate();
if (k[NUM_OF_ROWS] > 0L) {
try (ResultSet rs = ps.getGeneratedKeys()) {
final String identColName = idCol.colName();
while (rs.next()) {
if (k[ROW_CREATED] != KEY_UNDEFINED) throw new AssertionError();
k[ROW_CREATED] = rs.getLong(identColName);
}
}
}
} catch (SQLException e) { ... }
finally {
try { if (ps != null) ps.close(); } catch (SQLException e) { }
}
return k;
}
I am unable to say whether this approach will work with old versions of HSQLDB.
You will have to use some vendor-specific solution, i.e. in mysql you would call LAST_INSERT_ID function.
I don't have valid installation of HSQL to test it, but you can give a try to the highest voted solution from this topic: how to return last inserted (auto incremented) row id in HSQL?

Example Program to insert a row using BAPI with JCO3

I am trying to "insert" (or) "add a row" to Purchase Requisition using standard BAPI (PurchaseRequisition.CreateFromData).
I am using JCo3. The example in JCo3 indicates that we should use table.appendRow() OR table.insertRow() methods. I am trying with table.appendRow() & table.appendRows(1). When i try to insert a row, i dont get any error and the row is not inserted.
Below is the program i am trying to execute.
/** Below are the inputs required for this program to run /
/ Step 1 **/
String BAPI_NAME = "BAPI_REQUISITION_CREATE";
/** Step 2 **/
String query_input_column1 = "DOCUMENTY_TYPE";
String query_input_column1_value = "NB";
String query_input_column2 = "PREQ_NAME";
String query_input_column2_value = "Name";
String query_input_column3 = "ACCTASSCAT";
String query_input_column3_value = "U";
String query_input_column4 = "DELIV_DATE";
String query_input_column4_value = "20131101";
String query_input_column5 = "MATERIAL";
String query_input_column5_value = "DELL-RQ2013";
String query_input_column6 = "QUANITY";
int query_input_column6_value = 10100;
/** Step 3 **/
String targetTableUnderBAPI = "REQUISITION_ITEMS";
/** Step 4 **/
/** For the confirmation read the value from export parameter after insertion execution **/
String result_column1 = "NUMBER";
JCoDestination destination = null;
try {
destination = JCoDestinationManager.getDestination(DestinationManager.DESTINATION_NAME1);
JCoRepository repository = destination.getRepository();
JCoContext.begin(destination);
JCoFunction function = repository.getFunction(BAPI_NAME);
if(function == null)
throw new RuntimeException(BAPI_NAME + " not found in SAP.");
System.out.println("BAPI Name from function object: " + function.getName());
//function.getImportParameterList().setValue(query_input_column1, query_input_column1_value);
JCoTable table = function.getTableParameterList().getTable(targetTableUnderBAPI); //it is taken from the response value of metadata
//System.out.println("No of Columns: "+ table.getNumColumns());
System.out.println("Trying to execute append row");
table.appendRow();
table.setValue(query_input_column1,query_input_column1_value);
table.setValue(query_input_column2,query_input_column2_value);
table.setValue(query_input_column3,query_input_column3_value);
//table.setValue(query_input_column4,new java.util.Date(query_input_column4_value));
//skipped Other columns related code
try{
function.execute(destination);
}
catch(AbapException e){
System.out.println(e.toString());
return;
}
System.out.println("Let us check the result from export parameter");
String exportParamStructure = (String)function.getExportParameterList().getValue(result_column1); //getStructure(result_column1); // getValue(result_column1);
System.out.println("Resulting PR#: "+exportParamStructure);
} catch (JCoException e) {
e.printStackTrace();
}
finally
{
try {
JCoContext.end(destination);
} catch (JCoException e) {
e.printStackTrace();
}
}
I did not understand how to read the response and am trying to fetch it from exportParameters!!
Can anybody share a piece of code to insert and
getting confirmation response (do we get the PREQ_NO in response?)
I am adding date field value as "20131101", but not sure if the format and approach is right?
when i try to add Quantity column value, i get an error message complaining this column is not part of BAPIEBANC. But the column is visible in BAPIEBANC type.
any configuration on SAP side to be checked?
should i activate any fields in JCo side? if so, how
Please note that my knowledge on SAP is very limited.
Waiting for an expert's response.
Thanks.
First, you should take a look at SAP JCo documentation, e.g.
http://help.sap.com/saphelp_nw04/helpdata/en/6f/1bd5c6a85b11d6b28500508b5d5211/content.htm
Regarding your code:
Adding (one) row to the table looks right on first sight.
Your code says QUANITY instead of QUANTITY.
You should add date values as java.util.Date; if creating a Date from a String format, you should use java.text.DateFormat.parse(). See http://docs.oracle.com/javase/6/docs/api/java/util/Date.html (this is however Java specific and has nothing to do with JCo).
If changing anything in SAP, never forget to call BAPI_TRANSACTION_COMMIT in the end to finish the logical unit of work (aka transaction) or nothing will actually be changed.
If you don't like to fiddle with the more or less complicated and verbose JCo API, try using Hibersap which gives you a much nicer programming model when calling functions in SAP ERP: http://hibersap.org.
However, you will still need a basic understanding on how SAP function modules work technically (such as parameter types or data types) as well as on the domain specific model which lies behind them (in your case, creating a requisition). I.e. you may need to communicate with your SAP experts.
Here I added 2 types of insertion :
insertval() function for user defined module resides in sap with the help of abap programmer
Its an standard module for insert a ticket using jco to SOLMAN system. First you have to analyse import, export, table & structure parameters, and according to that you have to pass values and retrieve response. In second function it will return ticket n° after successfull insertion of ticket in solman.
I hope this sample code will help you, it worked for me.
public class jco
{
static String DESTINATION_NAME1 = "ABAP_AS_WITHOUT_POOL";
static String DESTINATION_NAME2 = "ABAP_AS_WITH_POOL";
static
{
Properties connectProperties = new Properties();
connectProperties.setProperty(DestinationDataProvider.JCO_ASHOST, "192.1.1.1");
connectProperties.setProperty(DestinationDataProvider.JCO_SYSNR, "01");
connectProperties.setProperty(DestinationDataProvider.JCO_CLIENT, "500");
connectProperties.setProperty(DestinationDataProvider.JCO_USER, "uname");
connectProperties.setProperty(DestinationDataProvider.JCO_PASSWD, "pwd");
connectProperties.setProperty(DestinationDataProvider.JCO_LANG, "en");
createDestinationDataFile(DESTINATION_NAME1, connectProperties);
connectProperties.setProperty(DestinationDataProvider.JCO_POOL_CAPACITY, "3");
connectProperties.setProperty(DestinationDataProvider.JCO_PEAK_LIMIT, "10");
createDestinationDataFile(DESTINATION_NAME2, connectProperties);
System.err.println("hai");
}
static void createDestinationDataFile(String destinationName, Properties connectProperties)
{
File destCfg = new File(destinationName+".jcoDestination");
try
{
try (FileOutputStream fos = new FileOutputStream(destCfg, false)) {
connectProperties.store(fos, "for tests only !");
}
}
catch (IOException e)
{
throw new RuntimeException("Unable to create the destination files", e);
}
}
public void insertval() throws JCoException
{
JCoDestination destination = JCoDestinationManager.getDestination(DESTINATION_NAME1);
JCoFunction jf=destination.getRepository().getFunction("ZUSER_DET");
jf.getImportParameterList().setValue("FIRST_NAME","member");
jf.getImportParameterList().setValue("LAST_NAME","c");
jf.getImportParameterList().setValue("USER_NO","1000");
jf.execute(destination);
System.out.println(jf);
}
public void insertticket() throws JCoException
{
JCoDestination destination = JCoDestinationManager.getDestination(DESTINATION_NAME2);
System.out.println("test"+"\n");
JCoFunction jf=destination.getRepository().getFunction("BAPI_NOTIFICATION_CREATE");
JCoTable jt1=jf.getTableParameterList().getTable("APPX_HEADERS");
JCoTable jt2=jf.getTableParameterList().getTable("APPX_LINES");
JCoTable jt3=jf.getTableParameterList().getTable("APPX_LINES_BIN");
JCoTable jt4=jf.getTableParameterList().getTable("NOTIF_NOTES");
JCoTable jt5=jf.getTableParameterList().getTable("NOTIF_PARTNERS");
JCoTable jt6=jf.getTableParameterList().getTable("NOTIF_SAP_DATA");
JCoTable jt7=jf.getTableParameterList().getTable("NOTIF_TEXT_HEADERS");
JCoTable jt8=jf.getTableParameterList().getTable("NOTIF_TEXT_LINES");
JCoStructure jfn1=jf.getImportParameterList().getStructure("NOTIF_EXT");
JCoStructure jfn2=jf.getImportParameterList().getStructure("NOTIF_CRM");
JCoStructure jfn3=jf.getImportParameterList().getStructure("IBASE_DATA");
jfn1.setValue("NUMB","1234");
jfn1.setValue("REFNUM","123");
jfn1.setValue("TYPE_NOTIF","SLFN");
jfn1.setValue("SUBJECT","tl");
jfn1.setValue("PRIORITY","2");
jfn1.setValue("LANGUAGE","EN");
jfn1.setValue("CATEGORY","Z01");
jfn2.setValue("CODE","0011");
jfn2.setValue("CODEGROUP","0011");
jfn2.setValue("CATEGORY","Z01");
jfn3.setValue("INSTANCE","489");
jfn3.setValue("IBASE","500");
jt1.appendRow();
jt1.setValue("DESCR","practise");
jt2.appendRow();
jt2.setValue("LINE","CVXCVXCV");
jt3.appendRow();
jt3.setValue("LINE","second text line");
jt4.appendRow();
jt4.setValue("TYPE_NOTE","my");
jt4.setValue("IDENT","hoe twwrtgw");
jt4.setValue("DESCRIPTION","its description ");
jt5.appendRow();
jt5.setValue("PARNR","new ");
jt5.setValue("TYPE_PAR","FN");
jt5.setValue("FUNC_PAR","EN");
jt5.setValue("PAR_ACTIVE","1");
jt6.appendRow();
jt6.setValue("INSTN","0020214076");
jt6.setValue("COMP","FI-AA");
jt6.setValue("SYSTYPE","P");
jt6.setValue("SYSID","PRD");
jt6.setValue("MANDT","900");
jt8.appendRow();
jt8.setValue("TXT_NUM","1");
jt8.setValue("TDFORMAT",">X");
jt8.setValue("TDLINE","/(performing all test)");
jf.execute(destination);
String jfex=jf.getExportParameterList().getString("REFNUM");
System.out.println("hi "+jfex);
}