What I am trying to do is mochijson2:decode(Ccode) generates any exception or error, program execution should not stop and case branch {error, Reason} should get executed.
But when I am trying to get it implemented, it generates error at first line while checking and code doesn't continue execution for lines below it.
SCustomid = case mochijson2:decode(Ccode) of
{struct, JsonDataa} ->
{struct, JsonData} = mochijson2:decode(Ccode),
Mvalll = proplists:get_value(<<"customid">>, JsonData),
Pcustomid = erlang:binary_to_list(Mvalll),
"'" ++ Pcustomid ++ "'";
{error, Reason} -> escape_str(LServer, Msg#archive_message.customid)
end,
You could suggest if I need to use Try Catch. I am a bit experienced with Ejabberd but newbie for Erlang. Any help is appreciated.
Seems the reason is an exception happens in mochijson2:decode/1. The function doesn't return a error as a tuple, instead the process crashes. There isn't enough information to tell what exactly the reason is. However I guess that the data format of Ccode might be wrong. You can handle exception using try ... catch statement:
SCustomid = try
case mochijson2:decode(Ccode) of
{struct, JsonDataa} ->
{struct, JsonData} = mochijson2:decode(Ccode),
Mvalll = proplists:get_value(<<"customid">>, JsonData),
Pcustomid = erlang:binary_to_list(Mvalll),
"'" ++ Pcustomid ++ "'";
{error, Reason} ->
escape_str(LServer, Msg#archive_message.customid)
end
catch
What:Reason ->
escape_str(LServer, Msg#archive_message.customid)
end,
Or just catch:
SCustomid = case catch(mochijson2:decode(Ccode)) of
{struct, JsonDataa} ->
{struct, JsonData} = mochijson2:decode(Ccode),
Mvalll = proplists:get_value(<<"customid">>, JsonData),
Pcustomid = erlang:binary_to_list(Mvalll),
"'" ++ Pcustomid ++ "'";
{error, Reason} ->
escape_str(LServer, Msg#archive_message.customid);
{What, Reason} ->
escape_str(LServer, Msg#archive_message.customid)
end,
You can use this:
SCustomid = try
{struct, JsonData} = mochijson2:decode(Ccode),
Mvalll = proplists:get_value(<<"customid">>, JsonData),
Pcustomid = erlang:binary_to_list(Mvalll),
"'" ++ Pcustomid ++ "'"
catch _:_ -> escape_str(LServer, Msg#archive_message.customid)
end
Related
I'm trying to parse a POST request using apache and mod_lua, i have something like this :
function authz_bulk_get(r, id_dash)
r:debug("BULK_DEBUG START")
if r.method == 'GET' then
for k, v in pairs(r:parseargs()) do
r:debug("BULK_DEBUG " .. k .. " " .. v)
end
elseif r.method == 'POST' then
r:debug("BULK_DEBUG TREAT POST")
local jv_content = r:requestbody()
r:debug("BULK_DEBUG REQUEST_BODY")
r:debug("BULK_DEBUG " .. jv_content)
jv_content = jv_content:gsub("%[", "")
jv_content = jv_content:gsub("%]", "")
local jv_jsonparse = json.decode(jv_content)
r:debug("BULK_DEBUG " .. jv_content)
r:debug("BULK_DEBUG " .. jv_jsonparse["type"])
r:debug("BULK_DEBUG " .. jv_jsonparse["id"])
end
r:debug("BULK_DEBUG END")
return apache2.AUTHZ_GRANTED
end
Everything seems OK in debug logs, i have all correct traces and I see requestbody and json parsing executing well but the response is {"statusCode":400,"error":"Bad Request","message":"[request body]: expected value of type [array] but got [null]"}
It looks like the request is emptied when i do r:requestbody()
Maybe I need to re-transmit the POST request after doing this processing ?
Any ideas ?
Thanks
I am querying an ignite cache like this:
try (QueryCursor<Cache.Entry<Long, IgniteAccountOrder>> qryCursor = cache.query(new ScanQuery<>())) {
qryCursor.forEach(
entry -> System.out.println("Key = " + entry.getKey() + ", Value = " + entry.getValue()));
}
This works fine and the value gets serialized fine.
As soon as any filter is added to the query an exception occurs. Here is the exact same code with a filter that always returns true which is technically equivalent to the above code without any filter:
IgniteBiPredicate<Long, IgniteAccountOrder> filter = (key, p) -> true;
try (QueryCursor<Cache.Entry<Long, IgniteAccountOrder>> qryCursor = cache.query(new ScanQuery<>(filter))) {
qryCursor.forEach(
entry -> System.out.println("Key = " + entry.getKey() + ", Value = " + entry.getValue()));
}
The following exception occurs with the second code:
Exception in thread "main" org.apache.ignite.client.ClientException: Ignite failed to process request [4]: Failed to deserialize object [typeName=java.lang.invoke.SerializedLambda] (server status code [1])
at org.apache.ignite.internal.client.thin.TcpClientChannel.convertException(TcpClientChannel.java:336)
at org.apache.ignite.internal.client.thin.TcpClientChannel.receive(TcpClientChannel.java:296)
at org.apache.ignite.internal.client.thin.TcpClientChannel.service(TcpClientChannel.java:218)
at org.apache.ignite.internal.client.thin.ReliableChannel.lambda$service$1(ReliableChannel.java:165)
at org.apache.ignite.internal.client.thin.ReliableChannel.applyOnDefaultChannel(ReliableChannel.java:763)
at org.apache.ignite.internal.client.thin.ReliableChannel.applyOnDefaultChannel(ReliableChannel.java:731)
at org.apache.ignite.internal.client.thin.ReliableChannel.service(ReliableChannel.java:164)
at org.apache.ignite.internal.client.thin.GenericQueryPager.next(GenericQueryPager.java:93)
at org.apache.ignite.internal.client.thin.ClientQueryCursor$1.nextPage(ClientQueryCursor.java:93)
at org.apache.ignite.internal.client.thin.ClientQueryCursor$1.hasNext(ClientQueryCursor.java:76)
at java.base/java.lang.Iterable.forEach(Iterable.java:74)
The issue was that IgniteAccountOrder class was not available to the remote server.
I was connecting to the remote server via a thin client and peer loading could not be enabled for the thin client.
I did the following to make this work:
Switched to a thick client
Enabled peer class loading via the thick client:
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setPeerClassLoadingEnabled(true);
I'm writing a test to check a function (called automatically by GenServer when a new file enters a folder) that calls other functions in the same module with pipes in order to read a file, process its content to insert it if needed and returns a list (:errors and :ok maps).
results looks like :
[
error: "Data not found",
ok: %MyModule{
field1: field1data,
field2: field2data
},
ok: %MyModule{
field1: field1data,
field2: field2data
},
error: "Data not found"
the code :
def processFile(file) do
insertResultsMap =
File.read!(file)
|> getLines()
|> extractMainData()
|> Enum.map(fn(x) -> insertLines(x) end)
|> Enum.group_by(fn x -> elem(x, 0) end)
handleErrors(Map.get(insertResultsMap, :error))
updateAnotherTableWithLines(Map.get(insertResultsMap, :ok))
end
defp getLines(docContent) do
String.split(docContent, "\n")
end
defp extractMainData(docLines) do
Enum.map(fn(x) -> String.split(x, ",") end)
end
defp insertLines([field1, field2, field3, field4]) do
Attrs = %{
field1: String.trim(field1),
field2: String.trim(field2),
field3: String.trim(field3),
field4: String.trim(field4)
}
mymodule.create_stuff(Attrs)
end
defp handleErrors(errors) do
{:ok, file} = File.open(#errorsFile, [:append])
saveErrors(file, errors)
File.close(file)
end
defp saveErrors(_, []), do: :ok
defp saveErrors(file, [{:error, changeset}|rest]) do
changes = for {key, value} <- changeset.changes do
"#{key} #{value}"
end
errors = for {key, {message, _}} <- changeset.errors do
"#{key} #{message}"
end
errorData = "data: #{Enum.join(changes, ", ")} \nErrors: #{Enum.join(errors, ", ")}\n\n"
IO.binwrite(file, errorData)
saveErrors(file, rest)
end
defp updateAnotherTableWithLines(insertedLines) do
Enum.map(insertedLines, fn {:ok, x} -> updateOtherTable(x) end)
end
defp updateOtherTable(dataForUpdate) do
"CLOSE" -> otherModule.doStuff(dataForUpdate.field1, dataForUpdate.field2)
end
I have several questions, and some will be pretty basic since I'm still learning :
What do you think of the code ? Any advices ? (take into account I voluntarily obfuscated names).
If I want to test this, is it the right way to test only processFile function ? Or should I make public more of them and test them individually ?
When I test the processFile function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thus error: "String" or ok: %{}" ?
What do you think of the code? Any advices? (take into account I voluntarily obfuscated names).
Opinion based.
If I want to test this, is it the right way to test only processFile function?
Yes.
Or should I make public more of them and test them individually?
No, this is an implementation detail and testing it is an anti-pattern.
When I test the processFile function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thus error: "String" or ok: %{}"?
You receive a Keyword. To check the explicit value, one might use:
foo = processFile(file)
assert not is_nil(foo[:ok])
OTOH, I’d better return a map from there and pattern match it:
assert %{ok: _} = processFile(file)
To assert that the result does not have anything save for :oks and :errors, one might use list subtraction:
assert Enum.uniq(Keyword.keys(result)) -- [:ok, :error] == []
I am trying to use the following code to unload data into S3 bucket. Which works but after unloading it throws some error.
Properties props = new Properties();
props.setProperty("user", MasterUsername);
props.setProperty("password", MasterUserPassword);
conn = DriverManager.getConnection(dbURL, props);
stmt = conn.createStatement();
String sql;
sql = "unload('select * from part where p_partkey in (select p_partkey from
part limit 10)') to"
+ " 's3://redshiftdump.****' "
+ " DELIMITER AS ','"
+ "ADDQUOTES "
+ "NULL AS ''"
+ "credentials 'aws_access_key_id=****;aws_secret_access_key=***' "
+ "parallel off" +
";";
boolean i = stmt.execute(sql);
stmt.close();
conn.close();
The unloading works. It is creating a file in the bucket. But it is giving me some error
java.sql.SQLException:
dataengine.impl.DSISimpleRowCountResult cannot be cast to
com.amazon.dsi.dataengine.interfaces.IResultSet
at
com.amazon.redshift.core.jdbc42.PGJDBC42Statement.createResultSet(Unknown
Source)
at com.amazon.jdbc.common.SStatement.executeQuery(Unknown Source)
what is this error and how to avoid it? Is there any way to dump the table in CSV format. Right now it is dumping the file in FILE format.
You say the UNLOAD works but you receive this error, that suggests to me that you are connecting successfully but there is an problem in the way your code interacts with the JDBC driver when the query completes.
We provide an example that may be helpful in our documentation on the page "Connect to Your Cluster Programmatically"
Regarding the output file format, you will get whatever is specified in your UNLOAD SQL but the filename will have a suffix (for example "000" or "6411_part_00") to indicate which part of the UNLOAD it is.
use executeUpdate .
def runQuery(sql: String) = {
Class.forName("com.amazon.redshift.jdbc.Driver")
val connection = DriverManager.getConnection(url, username, password)
var statement: Statement = null
try {
statement = connection.createStatement()
statement.setQueryTimeout(redshiftTimeoutInSeconds)
val result = statement.executeUpdate(sql)
logger.info(s"statement response code : ${result}")
} catch {
case e: Exception => {
logger.error(s"statement.isCloseOnCompletion :${e.getMessage} ::: ${e.printStackTrace()}")
throw new IngestionException(e.getMessage)
}
}
finally {
if(statement != null ) statement.close()
connection.close()
}
}
I'm playing with Elm and WebRTC, so I made a listen port which gets some messages from js:
type alias Message =
{ channel : String
, data : String
}
port listen : (Message -> msg) -> Sub msg
Now I would like to be able to divide the messages to different parts of my app. For instance, the chat uses the "chat" channel, while the game logic uses "game".
Is it possible to create a listenTo String subscription that filters out the messages with the correct channel (only returning the data)? Or perhaps a different way of doing it?
Update:
What I currently have, is something like this:
In my main.elm I have an update that looks like this. It can receive messages (from rtc) itself, and send messages for chat to it. (I would later add a "ForGame" then too)
type Msg = Received WebRTC.Message | ForChat Chat.Msg
update : Msg -> Model -> (Model, Cmd Msg)
update msg model =
case msg of
Received message ->
let
_ = Debug.log ("Received message on \"" ++ message.channel ++ "\": " ++ message.data)
in
( model
, Cmd.none
)
ForChat msg ->
let
(chatModel, chatCmd) = Chat.update msg model.chat
in
({ model | chat = chatModel}, Cmd.map ForChat chatCmd)
Then I have subscriptions that combines all my subscriptions:
subscriptions : Model -> Sub Msg
subscriptions model =
Sub.batch
[ WebRTC.listen Received
, Sub.map ForChat <| Chat.subscriptions model.chat
]
In Chat.elm I have a similar structure, with an update that handles it's messages. The subscription of the chat listens to all messages from WebRTC, but filters only the ones with channel chat:
subscriptions : Model -> Sub Msg
subscriptions model = WebRTC.listen forChatMessages
forChatMessages : WebRTC.Message -> Msg
forChatMessages webrtcMessage =
if webrtcMessage.channel == "chat"
then
let
message = decodeMessage webrtcMessage.data
in
case message of
Ok msg -> Receive msg
Err error -> Debug.log ("Received unreadable message on chat channel \"" ++ toString webrtcMessage.data ++ "\" with error \"" ++ error ++ "\"") Ignore
else
Ignore
(Ignore is a Msg for chat, which just does nothing case msg of Ignore -> (model, Cmd.none). decodeMessage uses a decoder to decode a message decodeMessage : String -> Result String Message.)
I'm quite happy with this, because this way all logic for chat is in Chat.elm. So main.elm doesn't need to know what channels chat is using. Chat just follows the standard structure (Msg, update, view, subscriptions) and main forwards everything.
The only thing that's still not great, is that in Chat.elm I have the forChatMessages function. Used like: subscriptions model = WebRTC.listen forChatMessages. I would like to make this more reuseable, so it would become something like:
subscriptions model = WebRTC.listen for "chat" decodeMessage Receive Ignore
It would then be reusable by the game:
subscriptions model = WebRTC.listen for "game" decodeGameInfo UpdateInfo Ignore
Update 2:
I managed to generalize the forChatMessages function into:
for : String -> (String -> Result String d) -> (d -> msg) -> msg -> Message -> msg
for channel decoder good bad webrtcMessage =
if
webrtcMessage.channel == channel
then
let
decoded = decoder webrtcMessage.data
in
case decoded of
Ok data -> good data
Err error -> Debug.log ("Failed decoding message on " ++ channel ++ "channel \"" ++ toString webrtcMessage.data ++ "\" with error \"" ++ error ++ "\"") bad
else
bad
So I think I found the solution myself. Unless someones has comments on this. Perhaps there is a cleaner/nicer/better way of doing the same?
Let's say you have the following Msg definition:
type Msg
= Listen Message
| GameChannel String
| ChatChannel String
Your update function could then act upon the channel value and call update again with the correct channel, ignoring all channel values except for "game" and "chat":
update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
case msg of
Listen message ->
case message.channel of
"game" ->
update (GameChannel message.data) model
"chat" ->
update (ChatChannel message.data) model
_ ->
model ! []
GameChannel data ->
...
ChatChannel data ->
...
Your subscription function would look something like this:
subscriptions : Model -> Sub Msg
subscriptions model =
listen Listen
I found a solution myself, and added it to the original question.
For clarity, this is the short version:
In my main.elm:
type Msg = Received WebRTC.Message | ForChat Chat.Msg
update : Msg -> Model -> (Model, Cmd Msg)
update msg model =
case msg of
Received message ->
let
_ = Debug.log ("Received message on \"" ++ message.channel ++ "\": " ++ message.data)
in
( model
, Cmd.none
)
ForChat msg ->
let
(chatModel, chatCmd) = Chat.update msg model.chat
in
({ model | chat = chatModel}, Cmd.map ForChat chatCmd)
subscriptions : Model -> Sub Msg
subscriptions model =
Sub.batch
[ WebRTC.listen Received
, Sub.map ForChat <| Chat.subscriptions model.chat
]
In Chat.elm:
subscriptions : Model -> Sub Msg
subscriptions model = WebRTC.listen <| for "game" decodeGameInfo UpdateInfo Ignore
In WebRTC.elm:
type alias Message =
{ channel : String
, data : String
}
port listen : (Message -> msg) -> Sub msg
for : String -> (String -> Result String d) -> (d -> msg) -> msg -> Message -> msg
for channel decoder good bad webrtcMessage =
if
webrtcMessage.channel == channel
then
let
decoded = decoder webrtcMessage.data
in
case decoded of
Ok data -> good data
Err error -> Debug.log ("Failed decoding message on " ++ channel ++ "channel \"" ++ toString webrtcMessage.data ++ "\" with error \"" ++ error ++ "\"") bad
else
bad