I'm trying to update hive connector properties inside /etc/presto/conf/catalog/hive.properties. I added the following:
hive.s3.max-client-retries=50
hive.s3.max-error-retries=50
hive.s3.max-connections=500
hive.s3.connect-timeout=5m
hive.s3.socket-timeout=5m
But for some reason these are not accepted. The presto server restart fails with the following error:
2018-04-16T19:34:26.369Z WARN main Bootstrap UNUSED PROPERTIES
2018-04-16T19:34:26.369Z WARN main Bootstrap hive.s3.connect-timeout
2018-04-16T19:34:26.369Z WARN main Bootstrap hive.s3.max-client-retries
2018-04-16T19:34:26.369Z WARN main Bootstrap hive.s3.max-connections
2018-04-16T19:34:26.369Z WARN main Bootstrap hive.s3.max-error-retries
2018-04-16T19:34:26.369Z WARN main Bootstrap hive.s3.socket-timeout
2018-04-16T19:34:26.369Z WARN main Bootstrap
2018-04-16T19:34:26.587Z ERROR main com.facebook.presto.server.PrestoServer Unable to create injector, see the following errors:
1) Configuration property 'hive.s3.connect-timeout' was not used
at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)
2) Configuration property 'hive.s3.max-client-retries' was not used
at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)
3) Configuration property 'hive.s3.max-connections' was not used
at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)
4) Configuration property 'hive.s3.max-error-retries' was not used
at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)
5) Configuration property 'hive.s3.socket-timeout' was not used
at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)
5 errors
These properties are applicable when hive.s3-file-system-type is set to PRESTO. When it's eg. EMRFS, these configuration properties are not applicable and setting them causes the error you observe.
As Piotr mentioned, EMR changed the default FileSystem implementation starting with emr-5.12.0 from PrestoS3FileSystem to EmrFS. PrestoS3FileSystem properties will be unavailable, but you can configure EmrFS as you would for any other application. See 'EMRFS and PrestoS3FileSystem Configuration' on https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto-considerations.html for more information.
Related
After removing JndiLookupClass from the classpath under $HIVE_HOME/lib/log4j-core-2.6.2.jar we are getting multiple warnings when we open hive cli and also while running the queries.
PFA error lines.
2021-12-17 18:47:32,880 main WARN JNDI lookup class is not available because this JRE does not support JNDI. JNDI string lookups will not be available, continuing configuration. java.lang.ClassNotFoundException: org.apache.logging.log4j.core.lookup.JndiLookup
We have tried commenting the line #appenders = console, DRFA in log4j2.properties file present under $HIVE_HOME/conf but still the issue is persisting.
There are no updates in the official documentation of hive.
Can someone help with the fix.
Is there any way to suppress the warning messages as it is hampering the output.
I got this error message when using React Native on vscode. Is there a way to fix it?
[Error - 2:57:25 PM - MyRealmApp\.flowconfig] Error loading flow using option 'useNPMPackagedFlow'
Pkg flow-bin not found in c:\Users\ReactNativeProjects\MyRealmApp
[Error - 2:57:25 PM - MyRealmApp\.flowconfig] Error loading flow using option 'pathToFlow'
'flow' not found
[Info - 2:57:25 PM - MyRealmApp\.flowconfig] Falling back to bundled flow.
[Info - 2:57:27 PM - MyRealmApp\.flowconfig] Using flow 'c:\Users\.vscode\extensions\flowtype.flow-for-vscode-1.9.2\node_modules\flow-bin\flow-win64-v0.107.0\flow.exe' (v0.107.0)
.flowconfig:28 Unsupported option specified! (format.bracket_spacing)
[Error - 2:57:28 PM] Connection to server got closed. Server will not be restarted.
Basically this issue was caused by not having installed the same version of flow-bin as declared in the .flowconfig of the projects.
Now what you have to do to fix that, is to maintain coherence in the current .flowconfig of the project in your VS Code.
How to do that?
You can just do the below thing and it will fix your issue. If you don't have a yarn, then use npm in place of yarn.
yarn global add flow-bin#version
npm run ios or npm run android
The above will help you fix the issue, but you will always have to check whether they are getting pointed to the same version. In order to fix it stably, you can do the following:
In your VS Code settings you need to point flow.pathToFlow to your workspace node_module:
// settings.json
{
"flow.pathToFlow": "${workspaceFolder}/node_modules/.bin/flow",
}
${workspaceFolder} is a special variable provided by VS Code and it always points to the current project folder. You can find it in the variables reference page of the official docs.
You can prefer to use the local Flow module because this way you can ensure that the installed version and the configured one in .flowconfig always match even when you switch projects.
There is an extra setting for that: flow.useNPMPackagedFlow, just set it to true and you are done (no need to change flow.pathToFlow)
Testcafe 1.10 doesn’t seem to run via Yarn 2, while it just works running the globally installed one manually. I get this output:
ERROR Error: testcafe-legacy-api tried to access read-file-relative, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.
Required package: read-file-relative (via "read-file-relative")
Required by: testcafe-legacy-api#npm:4.0.0 (via /data/Fichiers/Code/testcafe/.yarn/cache/testcafe-legacy-api-npm-4.0.0-ff97b69c8c-e4a891ec41.zip/node_modules/testcafe-legacy-api/lib/)
I can see that read-file-relative is listed as a dependency of testcafe, but not as a dependency of testcafe-legacy-api (https://yarnpkg.com/package/testcafe, https://yarnpkg.com/package/testcafe-legacy-api), which itself is a dependency of testcafe.
EDIT : PRs on both sides have been merged so it should work now or soon without using the workaround below.
Apparently testcafe and its dependency testcafe-legacy-api have several actual dependencies that are not listed in their dependencies listing.
As the documentation for .yarnrc.yml explains:
Some packages may have been specified incorrectly with regard to their dependencies - for example with one dependency being missing, causing Yarn to refuse it the access. The packageExtensions fields offer a way to extend the existing package definitions with additional information.
Adding this to my .yarnrc.yml and then running yarn install (or yarn) fixed it:
packageExtensions:
"testcafe#*":
dependencies:
"#babel/runtime": "*"
"#babel/plugin-transform-for-of": "*"
"testcafe-legacy-api#*":
dependencies:
"read-file-relative": "*"
"testcafe-hammerhead": "*"
I'm trying to add views on Ambari 1.7.0.
For the Files view that is available here: https://github.com/apache/ambari/tree/trunk/contrib/views/files
I get an error after trying to launch an instance of the view:
500 HdfsApi connection failed. Check "webhdfs.url" property
With the following Stack Trace (just the first lines):
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.hdfs.web.WebHdfsFileSystem not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1720)
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2415)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2428)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2467)
I just add the following property during the configuration of the view:
webhdfs.url webhdfs://mycluster1:50070
I'm not sure but perhaps it's an issue during the building part like version conflict, i'm working on that.
Any ideas?
What is the value you have used for WebHDFS url in file view instance ?
Is your cluster Namenode HA enabled ? If yes , try to use url as webhdfs://:50070
If not HA enabled the use webhdfs://:50070
If none works use direct data node dfs address as webhdfs://:50070
I know that Mono supports the AjaxControlToolkit but I don't know how to integrate it into MonoDevelop. I added AjaxControlToolkit.dll, System.Web.Extensions.dll and System.Web.Extensions.Design.dll as References in the project but when I build the project I get the warning:
/Users/user1/Projects/FirstProject/Default.aspx(1,1): Warning: Parser failed with error The tag type 'ajaxToolkit:TabContainer' has not been registered.. CodeBehind members for this file will not be added. (FirstProject)
and I when I deploy it I get the error: Unknown server tag 'ajaxToolkit:TabContainer'.
How do I register the server tag in MonoDevelop?
You can use either the <%#Register directive, or add some lines to your Web.config. See here for instructions on either approach.
Using the Web.config mechanism has the advantage that you don't need to add a directive to every page where you want to use the controls.