I am working with Redis.
I got 2 Projects, one inserts data and the other is selects.
I put some key/field/value via org.springframework.data.redis.RedisTemplate. The project encoding is UTF-8.
key and field is String type and value is Object.
What I want to do is select data from remote database(Insert Project) and insert its own redis. When getting data, uses Jedis and in case of putting uses Template.
The problem.
I had serialize String and Key when I insert to Insert Project.
However, when I select in the other, it breaks.
using console(redis-cli) makes right answer.
Here is my source code. Is there anything should be considered? Thanks :
#Repository
public class RedisDuplicator implements RedisDecalcomanie {
#Inject
RedisManagerImpl serviceRedis;
#Override
public void duplicateHash(String key) throws Exception {
RedisHepler helper = RedisHepler.getInstance();
Jedis managerRedis = helper.getConnection();
Map<byte[], byte[]> hashAll = managerRedis.hgetAll(key.getBytes());
Iterator<byte[]> iter = hashAll.keySet().iterator();
while(iter.hasNext()){
byte[] field = iter.next();
byte[] value = hashAll.get(field);
System.out.println("REDIS DUPLICATOR - ORIGIN : " + field);
System.out.println("REDIS DUPLICATOR : " + new String(field, "ISO-8859-1"));
// System.out.println("\t[ val ] : " + new String(value));
System.out.println("utf-8 -> euc-kr : " + new String(field, "euc-kr"));
System.out.println("utf-8 -> ksc5601 : " + new String(field, "ksc5601"));
System.out.println("utf-8 -> x-windows-949 : " + new String(field, "x-windows-949"));
System.out.println("utf-8 -> iso-8859-1 : " + new String(field, "iso-8859-1"));
System.out.println("iso-8859-1 -> euc-kr : " + new String(field, "euc-kr"));
System.out.println("iso-8859-1 -> ksc5601 : " + new String(field, "ksc5601"));
System.out.println("iso-8859-1 -> x-windows-949 : " + new String(field, "x-windows-949"));
System.out.println("iso-8859-1 -> utf-8 : " + new String(field, "utf-8"));
System.out.println("euc-kr -> utf-8 : " + new String(field, "utf-8"));
System.out.println("euc-kr -> ksc5601 : " + new String(field, "ksc5601"));
System.out.println("euc-kr -> x-windows-949 : " + new String(field, "x-windows-949"));
System.out.println("euc-kr -> iso-8859-1 : " + new String(field, "iso-8859-1"));
System.out.println("ksc5601 -> euc-kr : " + new String(field, "euc-kr"));
System.out.println("ksc5601 -> utf-8 : " + new String(field, "utf-8"));
System.out.println("ksc5601 -> x-windows-949 : " + new String(field, "x-windows-949"));
System.out.println("ksc5601 -> iso-8859-1 : " + new String(field, "iso-8859-1"));
System.out.println("x-windows-949 -> euc-kr : " + new String(field, "euc-kr"));
System.out.println("x-windows-949 -> utf-8 : " + new String(field, "utf-8"));
System.out.println("x-windows-949 -> ksc5601 : " + new String(field, "ksc5601"));
System.out.println("x-windows-949 -> iso-8859-1 : " + new String(field, "iso-8859-1"));
}
helper.returnResource(managerRedis);
helper.destroyPool();
}
#Override
public void duplicateList(String key) throws Exception {
// TODO Auto-generated method stub
}
};
output :
REDIS DUPLICATOR - ORIGIN : [B#3d3c33b7
REDIS DUPLICATOR : ’t160
utf-8 -> euc-kr : ыt160
utf-8 -> ksc5601 : ыt160
utf-8 -> x-windows-949 : ыt160
utf-8 -> iso-8859-1 : ’t160
iso-8859-1 -> euc-kr : ыt160
iso-8859-1 -> ksc5601 : ыt160
iso-8859-1 -> x-windows-949 : ыt160
iso-8859-1 -> utf-8 : ��t160
euc-kr -> utf-8 : ��t160
euc-kr -> ksc5601 : ыt160
euc-kr -> x-windows-949 : ыt160
euc-kr -> iso-8859-1 : ’t160
ksc5601 -> euc-kr : ыt160
ksc5601 -> utf-8 : ��t160
ksc5601 -> x-windows-949 : ыt160
ksc5601 -> iso-8859-1 : ’t160
x-windows-949 -> euc-kr : ыt160
x-windows-949 -> utf-8 : ��t160
x-windows-949 -> ksc5601 : ыt160
x-windows-949 -> iso-8859-1 : ’t160
REDIS DUPLICATOR - ORIGIN : [B#4b0e18ba
REDIS DUPLICATOR : ’t130
utf-8 -> euc-kr : ыt130
utf-8 -> ksc5601 : ыt130
utf-8 -> x-windows-949 : ыt130
utf-8 -> iso-8859-1 : ’t130
iso-8859-1 -> euc-kr : ыt130
iso-8859-1 -> ksc5601 : ыt130
iso-8859-1 -> x-windows-949 : ыt130
iso-8859-1 -> utf-8 : ��t130
euc-kr -> utf-8 : ��t130
euc-kr -> ksc5601 : ыt130
euc-kr -> x-windows-949 : ыt130
euc-kr -> iso-8859-1 : ’t130
ksc5601 -> euc-kr : ыt130
ksc5601 -> utf-8 : ��t130
ksc5601 -> x-windows-949 : ыt130
ksc5601 -> iso-8859-1 : ’t130
x-windows-949 -> euc-kr : ыt130
x-windows-949 -> utf-8 : ��t130
x-windows-949 -> ksc5601 : ыt130
x-windows-949 -> iso-8859-1 : ’t130
I expected some of these should be '130'.
It comes out my console encoding problem. Since my Eclipse workspace setting was default, it could not print UTF-8 text:D (But it is still mystery why it could not print iso-8859-1 encoding...)
Related
When I download a file in Ubuntu to /tmp using selenium, the file doesn't appear. However, the count postfix of the filename keeps increasing in Chrome, e.g. file (1).txt - meaning the browser finds the previously downloaded files.
Example App.java:
public class App
{
public static void main( String[] args ) throws InterruptedException {
final ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", Map.of("download.default_directory", "/tmp/"));
final ChromeDriver driver = new ChromeDriver(options);
driver.get("file:///home/.../download.html");
Thread.sleep(5000);
driver.quit();
}
}
download.html
<div>Hello</div>
<script>
const element = document.createElement('a');
element.setAttribute('href', 'data:text/plain;charset=utf-8,' + encodeURIComponent("Hello World"));
element.setAttribute('download', "file.txt");
element.style.display = 'none';
document.body.appendChild(element);
element.click();
document.body.removeChild(element);
</script>
After running this java code, the file cannot be found in /tmp or anywhere else in the filesystem: find . 2>/dev/null -name file.txt yields nothing. However, while the code is running, selenium keeps increasing the postfix of the filename:
Where is the downloaded file?
I guess the files are downloaded into your /home/username/Downloads folder, not to /tmp folder as you expecting.
I recently upgraded my Chrome/Chromedriver from v69 to the latest v76, and my suite of Selenium tests - which use BrowserMob Proxy to inject HTTP headers - have stopped working. The ChromeDriver will open the browser OK, but the headers are no longer being injected.
Chrome, Selenium and BrowserMob Proxy are now all on the latest versions:
Chrome: 76.0.3809.132
BrowserMob: 2.1.5
Selenium: 3.141.59
Does anyone know whether there are compatibility issues with the latest Chrome and BrowserMob? Or, can anyone spot an issue in my code?
Any help is much appreciated!
Here is my code:
Proxy seleniumProxy = null;
try {
String hostIp = Inet4Address.getLocalHost().getHostAddress();
proxy.setTrustAllServers(true);
proxy.start(0, Inet4Address.getByName(hostIp));
proxy.addRequestFilter((request, contents, messageInfo) -> {
headerMap.forEach((key, value) -> {
if (key != null && value != null) {
request.headers().add((String) key, value);
}
});
return null; //continue regular processing...
});
seleniumProxy = ClientUtil.createSeleniumProxy(proxy, InetAddress.getByName(hostIp));
seleniumProxy.setHttpProxy(hostIp + ":" + proxy.getPort());
seleniumProxy.setSslProxy(hostIp + ":" + proxy.getPort());
System.out.println("Created proxy on port " + proxy.getPort());
} catch (UnknownHostException e) {
LOG.error("unable to create BrowerMob proxy", e);
}
return seleniumProxy;
I have the following problem/question:
I was with notebook automation. Now I bought an ultrabook MAC.
I did all the project import with automation, all right. However, when I run it, I get the following error:
java.lang.IllegalStateException: The driver executable does not exist: /Users/estevaomarcos/Documents/Projetos/DBServer/\Users\chromedriver
In my code is the following:
System.setProperty("webdriver.chrome.driver", "\\Users\\chromedriver");
WebDriver browser = new ChromeDriver(); .....
Set Driver Path: To avoid manual checking for the operating system we can get the file separator symbol from the system property using the file.separator key.
System.setProperty("webdriver.chrome.driver", "Users" + System.getProperty("file.separator") + "chromedriver");
WebDriver browser = new ChromeDriver();
browser.get("test.com");
Setting driver path based on which Operating System(OS) we are running on.
String os = System.getProperty("os.name").toLowerCase();
WebDriver driver = new ChromeDriver();
if(os.contains("mac")) {
System.setProperty("webdriver.chrome.driver", System.getProperty("usr.home") + System.getProperty("file.separator") + "chromedriver");
}else {
System.setProperty("webdriver.chrome.driver", System.getProperty("usr.home") + System.getProperty("file.separator") + "chromedriver.exe");
}
In a code based on operating system you should initialise new ChromeDriver after System.setProperty setup only.
user.home is correct option - NOT "usr"
Finally it works(java) great when looks like this:
String os = System.getProperty("os.name").toLowerCase();
if (os.contains("mac")) {
System.setProperty("webdriver.chrome.driver", System.getProperty("user.home") + System.getProperty("file.separator") + "chromedriver");
} else {
System.setProperty("webdriver.chrome.driver", System.getProperty("user.home") + System.getProperty("file.separator") + "chromedriver.exe");
}
WebDriver driver = new ChromeDriver();
I have this setup that is setting proxy ok in the local browser but when i try to use the grid the proxy will not be sent to the node:
var driver = new webdriver.Builder()
.withCapabilities(webdriver.Capabilities.firefox())
.setProxy(proxy.manual({ http : 'proxy:port',
https : 'proxy:port',
}))
.build();
Result : the browser proxy is - proxy:port ;
When i add :
var driver = new webdriver.Builder()
.withCapabilities(webdriver.Capabilities.firefox())
.setProxy(proxy.manual({ http : 'proxy:port',
https : 'proxy:port',
}))
.usingServer('http://hub:port/wd/hub')
.build();
The result : the browser proxy is - it showes me the ip of the hub.
Question : does anyone knows why the proxy set manualy is not sent to the hub and why the browser doesn't use it ? Or any other solution for this problem ?
This is the solution that worked :
var webdriver = require('selenium-webdriver'),
firefox = require('selenium-webdriver/firefox'),
proxy = require('selenium-webdriver/proxy')
driver = null,
profile = new firefox.Profile();
profile.setPreference("network.proxy.type", 1); // Manual proxy config
profile.setPreference("network.proxy.http", "proxy");
profile.setPreference("network.proxy.http_port", port);
profile.setPreference("network.proxy.ssl", "proxy");
profile.setPreference("network.proxy.ssl_port", port);
var opts = new firefox.Options();
opts.setProfile(profile);
var driver = new webdriver.Builder()
.withCapabilities(webdriver.Capabilities.chrome())
.setFirefoxOptions(opts);
.build();
I'm using Selenium Webdriver (Chromedriver) in Java, along with BrowserMob Proxy Server to capture HTTP traffic in a har file. I recently encountered a problem where sections of the website would not load, and I've narrowed it down to this error:
"Failed to load resource https://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js".
It seems that when using the proxy server, the Selenium driver can't access the SSL certificate for the https link. Here is a snippet of the code I am using:
ProxyServer server = new ProxyServer(4040);
server.start();
Proxy proxy = server.seleniumProxy();
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, proxy);
driver = new ChromeDriver(capabilities);
server.newHar("myHar");
Har har = server.getHar();
server.stop();
I have tried adding "capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);" and this solves the above problem, but only if I delete "capabilities.setCapability(CapabilityType.PROXY, proxy);" and then I am no longer able to capture the har file.
I tried switching to a firefox driver and setting up a firefox profile, but either I am not doing it properly or it won't work for my purposes either.
I have also tried setting up the cybervillainsCA certificate that comes with BrowserMob proxy in Trusted Root Certifaction Authorities, but that hasn't solved the issue either.
Does anyone know how to solve this in a way that I can collect HTTP traffic as well as successfully load the resource mentioned above?
What I ended up doing was scrapping the proxy entirely, and instead switched to using a Firefox profile with the Firebug and NetExport extensions in order to export the har file. This ended up significantly slowing down the tests, and exports a har for each page visited rather than for the entire session.
Code:
//Firefox profile
FirefoxProfile ffProfile = new FirefoxProfile();
ffProfile.addExtension(new File("firebug-1.11.4-fx.xpi"));
ffProfile.addExtension(new File("netExport-0.9b3.xpi"));
//Set default Firefox preferences
ffProfile.setPreference("app.update.enabled", false);
String domain = "extensions.firebug.";
//Set default Firebug preferences
ffProfile.setPreference(domain + "currentVersion", "1.11.4");
ffProfile.setPreference(domain + "allPagesActivation", "on");
ffProfile.setPreference(domain + "defaultPanelName", "net");
ffProfile.setPreference(domain + "net.enableSites", true);
//Set default NetExport preferences
ffProfile.setPreference(domain + "netexport.alwaysEnableAutoExport", true);
ffProfile.setPreference(domain + "netexport.autoExportToFile", true);
ffProfile.setPreference(domain + "netexport.showPreview", false);
ffProfile.setPreference(domain + "netexport.defaultLogDir", "string file path");
//WebDriver, instantiated outside the method
driver = new FirefoxDriver(ffProfile);
s = new WebDriverBackedSelenium(driver, "http://www.google.ca/");
I collected the har file after each page as follows:
HarFileReader r = new HarFileReader();
HarFileWriter w = new HarFileWriter();
int count = 1;
String allHars = "";
String harFolderPath = "file path for har";
File dir = new File(harFolderPath);
for (File child : dir.listFiles()) {
HarLog log = r.readHarFile(child);
File f = new File(harFolderPath + "\\test"+count+".txt");
w.writeHarFile(log, f);
allHars = allHars + readFileAsString(f.getPath());
count++;
}
FileUtils.cleanDirectory(dir);