I'm looking at the reference manual and can't find any documentation of bitwise operations/functions.
Is there any way to use, for example, a bitwise AND operation (equivalent to "A & B" in Hive) in a Pig script?
You can provide custom UDF for this. eg. see https://pig.apache.org/docs/r0.7.0/udf.html
In pig script you would do
REGISTER myudfs.jar;
And example for BinaryAND UDF:
package myudfs;
import java.io.IOException;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.util.WrappedIOException;
public class BitwiseAND extends EvalFunc (Integer)
{
public String exec(Tuple input) throws IOException {
// check input tuple:
if (input == null || input.size() < 2)
return null;
try{
return (Integer)input.get(0) & (Integer)input.get(1);
}catch(Exception e){
throw WrappedIOException.wrap("Caught exception processing input row ", e);
}
}
}
NOTE: This is not tested, it's just copied from the pig udf page.
Related
I am using CSVPrinter class from apache.commons.csv, and I am trying to print some lines in a csv file. As I know, we need to call close() method on a FileWriter, after the writing is done. And based on that assumption, I tried to call CSVPrinter.close().However, IntelliJ IDEA warns me that this method is redundant. Besides, examples in https://www.callicoder.com/java-read-write-csv-file-apache-commons-csv/ does not include this method, either. I want to know why is that method redundant and if I just use .flush() everything will be alright?
Here is an example copied from website mentioned above.
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;
import java.io.*;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Arrays;
public class CSVWriter {
private static final String SAMPLE_CSV_FILE = "./sample.csv";
public static void main(String[] args) throws IOException {
try (
BufferedWriter writer = Files.newBufferedWriter(Paths.get(SAMPLE_CSV_FILE));
CSVPrinter csvPrinter = new CSVPrinter(writer, CSVFormat.DEFAULT
.withHeader("ID", "Name", "Designation", "Company"));
) {
csvPrinter.printRecord("1", "Sundar Pichai ♥", "CEO", "Google");
csvPrinter.printRecord("2", "Satya Nadella", "CEO", "Microsoft");
csvPrinter.printRecord("3", "Tim cook", "CEO", "Apple");
csvPrinter.printRecord(Arrays.asList("4", "Mark Zuckerberg", "CEO", "Facebook"));
csvPrinter.flush();
// I added the following line
csvPrinter.close();
}
}
}
As #user207421 explained in the comments.
First: The try-with-resources statement provides an automatic close at the end of its scope.
Second: Flush is redundant before close.
I am working on porting a JUnit4 based test framework that uses JUnitCore to launch unit tests with JUnit5 launcher, junit-vintage-engine and a LauncherDiscoveryRequest with ClassSelector.
It is working for all cases except #ParameterizedTest. Launcher is not honoring parameterized tests.
How do I get Launcher to honor JUnit4 parameterized tests?
I have tried adding testCompile("org.junit.jupiter:junit-jupiter-params:${junit_jupiter_version}")
package ...;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import ...TestCategory;
import ...TestType;
import ...DataScramblingRegexTest;
#TestType(...TestCategory.UNIT)
#RunWith(Parameterized.class)
public class RegexGenTest {
private static final int LOOKAROUND_ATTEMPTS = 100;
#Parameterized.Parameter(0)
public String regex;
#Parameterized.Parameters(name = "{index}: RegexGenTest: {0}")
public static Iterable<String[]> data() {
return DataScramblingRegexTest.data();
}
#Test
public void testTransformAndGenerate() {
//even though Data Scramble Random Text doesn't transform every regex,
//even if we do, and run it through Generex, it should still work.
final String transformedRegex = RegexGen.transform(this.regex);
String randomStr = RegexGen.generate(transformedRegex);
if (RegexGen.containsLookArounds(this.regex)) {
for (int i = 0; i < LOOKAROUND_ATTEMPTS; i++) {
randomStr = RegexGen.generate(transformedRegex);
if (randomStr.matches(this.regex)) {
return;
}
}
Assert.fail("Lookaround failed");
}
Assert.assertTrue("generated string does not match regex: " + randomStr, randomStr.matches(this.regex));
}
}
build.gradle
compile("junit:junit:${junit_version}")
testCompile("org.junit.vintage:junit-vintage-engine:${junit_jupiter_version}")
testCompile("org.junit.jupiter:junit-jupiter-api:${junit_jupiter_version}")
testCompile("org.junit.jupiter:junit-jupiter-params:${junit_jupiter_version}")
testCompile("org.junit.jupiter:junit-jupiter-engine:${junit_jupiter_version}")
where
junit_version=4.12
junit_jupiter_version=5.4.2
Note the proprietary TestCategory.UNIT annotation, we use this in our test discovery code and create a Launcher and a LauncherdiscoveryRequest with ClassSelector.
I expect JUnit 4 parameters to be supported by JUnit5 launcher with junit-jupiter-vintage engine and it is not supported.
I am trying to write a simple fixture that opens the browser and navigates to www.google.com. When I run the wiki page, it passes with all green, but the browser never opens up (I don't think the method even gets called by the wiki). Can someone take a look at my fixture and wiki to see what I am doing wrong? Many thanks in advance,
Here is the Wiki -
!|SeleniumFitness|
|URL |navigateToSite?|
|http://www.google.com| |
After Running -
!|SeleniumFitnesse| java.lang.NoSuchMethodError: org.openqa.selenium.remote.service.DriverCommandExecutor.<init>(Lorg/openqa/selenium/remote/service/DriverService;Ljava/util/Map;)V
|URL |The instance decisionTable_4.setURL. does not exist|navigateToSite?
|http://www.google.com|!The instance decisionTable_4.navigateToSite. does not exist |
Here is the Fixture -
package FitNesseConcept.fitNesse;
import java.util.Properties;
import org.junit.BeforeClass;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.BeforeMethod;
//import com.google.common.base.Preconditions.*;
//import com.google.common.collect.Lists;
import fit.ColumnFixture;
public class SeleniumFitnesse extends ColumnFixture {
public static ChromeDriver driver = null;
private String navigateToSite = "";
public String URL = "";
public SeleniumFitnesse() {
Properties props = System.getProperties();
props.setProperty("webdriver.chrome.driver", "/home/ninad/eclipse-workspace/chromedriver");
driver = new ChromeDriver();
}
// SET-GET Methods
public String getURL() {
return URL;
}
public void setURL(String uRL) {
URL = uRL;
}
public String getNavigateToSite() {
return navigateToSite;
}
public void setNavigateToSite(String navigateToSite) {
this.navigateToSite = navigateToSite;
}
// Navigate to URL
public void navigateToSite() throws Throwable {
System.out.println("Navigating to Website");
try {
driver.navigate().to(URL);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
You are getting some good recommendations as comments - but to answer your question directly, for an old-style ColumnFixture, which is what you have written, the method "navigateToSite" is indeed not going to be called.
These styles of fixtures are not often used anymore, Slim is preferred, and your fitnesse instance in its documentation will show you how to use Slim style. However, for a column fixture as you have written, if you want a method to be called it needs to be a "?" following name of the method in the header row.
See basic docs for column fixture:
http://fitnesse.org/FitNesse.UserGuide.FixtureGallery.BasicFitFixtures.ColumnFixture
You are mis-using column fixture, even granted the old style though. Column fixture's pattern is "here is a series of columns that represent inputs, now here is a method call I want to make to get the output and check result". Navigating a website does not often fit that pattern. In old style fitnesse it would probably be approached by an ActionFixture:
http://fitnesse.org/FitNesse.UserGuide.FixtureGallery.BasicFitFixtures.ActionFixture
In the newer Slim style, a good fit for navigation and checking where you are would be a Scenario Table.
http://www.fitnesse.org/FitNesse.UserGuide.WritingAcceptanceTests.SliM.ScenarioTable
In general doing WebDriver / Selenium tests through a wiki is worth extra thought as to whether it's your best medium. Fitnesse is really designed to be a collaborative tool for documenting and verifying business requirements, directly against source code.
Here's an example of how to do with a ColumnFixture, although again ColumnFixture not exactly appropriate:
|url|navigateToUrl?|
|www.google.com| |
java class:
public String url;
public void navigateToUrl() {
}
You could return an "OK" if it navigates alright, or return the title of the page as opposed to void if you wanted.
I have a simple hive UDF:
package com.matthewrathbone.example;
import org.apache.hadoop.hive.ql.exec.Description;
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.Text;
#Description(
name="SimpleUDFExample",
value="returns 'hello x', where x is whatever you give it (STRING)",
extended="SELECT simpleudfexample('world') from foo limit 1;"
)
class SimpleUDFExample extends UDF {
public Text evaluate(Text input) {
if(input == null) return null;
return new Text("Hello " + input.toString());
}
}
When I am executing it using select query :
select helloUdf(method) from tests3atable limit 10;
method is the name of column in tests3atable table.
I am getting below exception :
FAILED: SemanticException [Error 10014]: Line 1:7 Wrong arguments 'method': Unable to instantiate UDF implementation class com.matthewrathbone.example.SimpleUDFExample: java.lang.IllegalAccessException: Class org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge can not access a member of class com.matthewrathbone.example.SimpleUDFExample with modifiers ""
Declare the class as public, it should work
I also had the same issue. It turned out, eclipse was not refreshing the program which i modified. So please make sure that the modifications you make in your code gets reflected in the jar.
Why I'm getting error "cannot find symbol: createStatement()"? My goal is to create a t1 Tab object in a TestClass and calls t1.Cr() with a String parameter.
I'm not forgetting the import java.sql.*;
public void Cr() throws Exception {
try {
ConexaoPlus con = new ConexaoPlus();
con.conect();
Statement st = con.createStatement();
st.execute("CREATE TABLE " + getName() + "();");
st.execute("DROP TABLE " + getName() + "();");
st.close();
} catch (Exception e) {
e.printStackTrace();
}
}
Thanks in advance!
A cannot find symbol error may occur for many reasons. You can read this post.
From what I see from the code you have provided, your class ConexaoPlus does not contain the method createStatement() or you might have not defined it.
java sql packages will have these methods already defined which you can use. For more help you will need to share the class ConexaoPlus.
This worked for me.
import java.sql.Statement;