I'm trying to run some simple graph using noflo-filesystem and noflo-csv that reads CSV file, parses it and writes it down to file.
The issue is that the program waits for something that I don't know how to provide:
graph.fbp
Read(filesystem/ReadFile) OUT -> CSV ParseCSV(csv/ConvertCsvToMatrix)
ParseCSV OUT -> IN Write(filesystem/WriteFile)
'dump.csv' -> FILENAME Write(filesystem/WriteFile)
'data.csv' -> In Read
When I run
.\node_modules\.bin\noflo-nodejs --graph graphs\graph.fbp --batch --register=false --debug
I get:
DATA -> FILENAME Write() CONN
DATA -> FILENAME Write() DATA
DATA -> FILENAME Write() DISC
DATA -> IN Read() CONN
DATA -> IN Read() DATA
DATA -> IN Read() DISC
DATA -> ENCODING Read() CONN
DATA -> ENCODING Read() DATA
DATA -> ENCODING Read() DISC
Read() OUT -> CSV ParseCSV() CONN
Read() OUT -> CSV ParseCSV() < ..\python_code\web_app\TimeSeries.csv
Read() OUT -> CSV ParseCSV() DATA
Read() OUT -> CSV ParseCSV() > ..\python_code\web_app\TimeSeries.csv
Read() OUT -> CSV ParseCSV() DISC
ParseCSV() OUT -> IN Write() CONN
ParseCSV() OUT -> IN Write() DATA
And then the program is hanging.
I tried to feed Write's IN port with just a string like:
Read(filesystem/ReadFile) OUT -> CSV ParseCSV(csv/ConvertCsvToMatrix)
'some text' -> IN Write(filesystem/WriteFile)
'dump.csv' -> FILENAME Write(filesystem/WriteFile)
'data.csv' -> In Read
And this works fine:
DATA -> IN Write() CONN
DATA -> IN Write() DATA
DATA -> IN Write() DISC
DATA -> FILENAME Write() CONN
DATA -> FILENAME Write() DATA
DATA -> FILENAME Write() DISC
DATA -> IN Read() CONN
DATA -> IN Read() DATA
DATA -> IN Read() DISC
DATA -> ENCODING Read() CONN
DATA -> ENCODING Read() DATA
DATA -> ENCODING Read() DISC
Would anyone be so kind to advise how can I debug it?
The problem is that csv/ConvertCsvToMatrix is not sending disconnect on its output port. This causes NoFlo to believe that the network is not done running yet, so the process is not shutdown. Both of the .send() calls in the component must be followed by a .disconnect() call.
This can be seen from the --debug output in that it ends with:
ParseCSV() OUT -> IN Write() CONN
ParseCSV() OUT -> IN Write() DATA
Compare with the successful example which ends with
DATA -> ENCODING Read() DISC
With NoFlo 0.8 we have a new method for seeing what processes are keeping the network alive:
network.getActiveProcesses returns an array of process IDs for processes that are currently active.
This is useful for finding badly-behaving components.
Related
Akka - Discriminated Unions as messages in F#
I am unable to use discriminated unions as messages to akka actors. If anyone can point me at an example that does this, it would be much appreciated.
My own attempt at this is at git#github.com:Tweega/AkkaMessageIssue.git. (snippets below). It is a cutdown version of a sample found at https://github.com/rikace/AkkaActorModel.git (Chat project)
Problem
The DU message never finds its target on the server actor, but is sent to the deadletter box. If I send Objects, instead, they do arrive.
If I send a DU, but set my server actor to listen for generic Objects, the message does arrive, but its type is
seq [seq [seq []]
and I can't get at underlying DU.
The DU I am trying to send as message
type PrinterJob =
| PrintThis of string
| Teardown
The client code
let system = System.create "MyClient" config
let chatClientActor =
spawn system "ChatClient" <| fun mailbox ->
let server = mailbox.Context.ActorSelection("akka.tcp://MyServer#localhost:8081/user/ChatServer")
let rec loop nick = actor {
let! (msg:PrinterJob) = mailbox.Receive()
server.Tell(msg)
return! loop nick
}
loop ""
while true do
let input = Console.ReadLine()
chatClientActor.Tell(PrintThis(input))
Messages are forwarded to the client from console input
while true do
let input = Console.ReadLine()
chatClientActor.Tell(PrintThis(input))
The server code
let system = System.create "MyServer" config
let chatServerActor =
spawn system "ChatServer" <| fun (mailbox:Actor<_>) ->
let rec loop (clients:Akka.Actor.IActorRef list) = actor {
let! (msg:PrinterJob) = mailbox.Receive()
printfn "Received %A" msg //Received seq [seq [seq []]; seq [seq [seq []]]] ???
match msg with
| PrintThis str ->
Console.WriteLine("Printing: {0} Do we get this?", str)
return! loop clients
| Teardown ->
Console.WriteLine("Tearing down now")
return! loop clients
}
loop []
Dependencies
(I am not using paket here) - PM commands below:
Install-Package Akka -Version 1.4.23
Install-Package Akka.Remote -Version 1.4.23
Install-Package Akka.FSharp -Version 1.4.23
I am hosting the application in net5.0
Constructor argument names - oddity?
When passing in class instances as objects, akka seems to be sensitive to the name of constructor parameters. The message gets handled, but the data is not copied across from client to server. If you have a property called Username, the constructor parameter cannot be, for example, uName, otherwise its value is null when it reaches the server. Code for this is in branch params.
type DoesWork(montelimar: string) =
member x.Montelimar = montelimar
type DoesNotWork(montelimaro: string) =
member x.Montelimar = montelimaro
I opened an issue in the Akka.NET repository: https://github.com/akkadotnet/akka.net/issues/5194
And added a detailed reproduction for this: https://github.com/akkadotnet/akka.net/pull/5196
But it looks like Newtonsoft.Json really can't perform this deserialization without being given a type hint, which Akka.NET's network serialization does not do by default for JSON:
type TestUnion =
| A of string
| B of int * string
type TestUnion2 =
| C of string * TestUnion
| D of int
[<Fact(Skip="JSON.NET really does not support even basic DU serialization")>]
member _.``JSON.NET must serialize DUs`` () =
let du = C("a-11", B(11, "a-12"))
let settings = new JsonSerializerSettings()
settings.Converters.Add(new DiscriminatedUnionConverter())
let serialized = JsonConvert.SerializeObject(du, settings)
let deserialized = JsonConvert.DeserializeObject(serialized, settings)
Assert.Equal(du :> obj, deserialized)
That test will not pass and it doesn't use any of Akka.NET's infrastructure at all - so the default JSON serializer simply won't work for real-world F# use cases.
We can try changing the defaults of our serialization system to include a type hint, but that will take a lot of validation testing (for old Akka.Persistence data serialized without one).
A better solution, which my pull request validates, is to use Hyperion for polymorphic serialization instead - it will be similarly transparent to you but it has much more robust handling for complex types than Newtonsoft.Json and is actually faster: https://getakka.net/articles/networking/serialization.html#how-to-setup-hyperion-as-default-serializer
I'm using elm 0.18.
Let's say I have a function that strings together a bunch of stuff that I threw together in a hurry. It works, but I'm not sure what it's type signature is, and I'd like elm to tell me (or hint for me) that type signature.
For example, I use graphql and have a function that takes a graphql string, a decoder (which also doesn't have a type signature), and a Cmd Msg, and runs it through HttpBuilder.
graphQLPost graphiql decoder msg =
HttpBuilder.post (url ++ "api")
|> HttpBuilder.withStringBody "text/plain" graphiql
|> HttpBuilder.withExpect (Http.expectJson decoder)
|> HttpBuilder.send msg
This works, though I don't know why. I tried fitting it with the type signature graphQLPost : String -> Json.Decode.Decoder -> Cmd Msg, but I get an error.
Figuring out this type signature is not as important to me as finding a way to induce them through elm. Is there a command that I can enter into elm-repl or something that will tell me the signature?
Elm REPL will do this for you:
> import Http
> import HttpBuilder
> type Msg = Msg
> url = "..."
"..." : String
> graphQLPost graphiql decoder msg = \
| HttpBuilder.post (url ++ "api") \
| |> HttpBuilder.withStringBody "text/plain" graphiql \
| |> HttpBuilder.withExpect (Http.expectJson decoder) \
| |> HttpBuilder.send msg
<function>
: String
-> Json.Decode.Decoder a
-> (Result.Result Http.Error a -> msg)
-> Platform.Cmd.Cmd msg
When you write a function and hit <Enter>, it shows you the signature. In this case the signature is:
graphQLPost : String
-> Json.Decode.Decoder a
-> (Result.Result Http.Error a -> msg)
-> Platform.Cmd.Cmd msg
Running elm-make with the --warn option will cause the compiler to suggest that you include a type annotation on functions that don't have one, and it will provide one for you to copy and paste in.
Also, some editor integrations, such as the Visual Studio Code language extension for Elm, will display these kinds of warnings as a hint icon that you can click to add the missing type annotation automatically. You can set a keyboard shortcut for this to do it without your hands leaving the keyboard.
How do I implement and return a Cmd Msg?
For example, the following line generates a Cmd Msg:
Http.send msg request
It's used in the following function:
tryRegister : Form -> (Result Http.Error JsonProfile -> msg) -> Cmd msg
tryRegister form msg =
let
registerUrl =
"http://localhost:5000/register"
body =
encode form |> Http.jsonBody
request =
Http.post registerUrl body decoder
in
Http.send msg request
I'm trying to hand code a similar function within my TestAPI:
tryRegister : Form -> (Result Http.Error JsonProfile -> msg) -> Cmd msg
tryRegister form msg =
Cmd.none
The above code compiles. However, it's not clear to me how to implement a function that returns a Cmd Msg other than Cmd.none.
Appendix:
type Msg
=
...
| Submit
| Response (Result Http.Error JsonProfile)
update : Msg -> Form -> ( Form, Cmd Msg )
update msg model =
case msg of
...
Submit ->
( model, runtime.tryRegister model Response )
Source code on GitHub.
Edit
The original answer suggested mapping over Cmd.none, which compiles and may potentially be useful when mocking out functions for testing, but if you are actually trying to force another update cycle in The Elm Architecture, you will need to convert to a Task, as outlined in the send function described here.
send : msg -> Cmd msg
send msg =
Task.succeed msg
|> Task.perform identity
As #SwiftsNamesake mentioned above, in most cases this is not necessary, and the entire blog post on the subject is worth a read.
Original Answer
You can use Cmd.map over Cmd.none to change it to any Cmd:
Cmd.map (always Submit) Cmd.none
I'm trying to decode a http request to pokéapi in Elm, using StartApp as a base. Though I'm getting an error I don't really know how to fix:
The right argument of (|>) is causing a type mismatch.
76│ Http.getString testUrl
77│ |> Task.map parseMon
78│> |> Task.map OnPokemonLoaded
(|>) is expecting the right argument to be a:
Task Http.Error (Result String Pokemon) -> a
But the right argument is:
Task Http.Error (Result Http.Error Pokemon) -> Task Http.Error Action
The code it's talking about is:
-- Fetching test mon
testUrl : String
testUrl = "http://pokeapi.co/api/v2/pokemon/1/"
fetchTest : Effects.Effects Action
fetchTest =
Http.getString testUrl
|> Task.map parseMon
|> Task.map OnPokemonLoaded --line 78
|> Effects.task
parseMon : String -> Result String Pokemon.Pokemon
parseMon json = Json.Decode.decodeString Pokemon.decoder json
OnPokemonLoaded is one of my actions: OnPokemonLoaded (Result Http.Error Pokemon). Pokemon.decoder is a simple json decoder: decoder : Decoder Pokemon.
I'm still new to Elm, and only just trying out StartApp and Effects. The error seems to explain the problem pretty well, but I'm still a little lost as to how it should work.
So, how should I request and decode the json properly?
The use of Http.getString and parseMon is unnecessary. Instead, you can use Http.get and pass your Json decoder, then map it to a Result to get the functionality you're after:
fetchTest : Effects.Effects Action
fetchTest =
Http.get Pokemon.decoder testUrl
|> Task.toResult
|> Task.map OnPokemonLoaded
|> Effects.task
How to read HTTP response body for a non 200 HTTP status
getJson : String -> String -> Effects Action
getJson url credentials =
Http.send Http.defaultSettings
{ verb = "GET"
, headers = [("Authorization", "Basic " ++ credentials)]
, url = url
, body = Http.empty
}
|> Http.fromJson decodeAccessToken
|> Task.toResult
|> Task.map UpdateAccessTokenFromServer
|> Effects.task
The above promotes the error from
Task.toResult : Task Http.Error a -> Task x (Result Http.Error a)
The value of which becomes
(BadResponse 400 ("Bad Request"))
My server responds with what is wrong with the request as a JSON payload in the response body. Please help me retrieve that from the Task x a into ServerResult below.
type alias ServerResult = { status : Int, message : String }
The Http package (v3.0.0) does not expose an easy way to treat HTTP codes outside of the 200 to 300 range as non-error responses. Looking at the source code, the handleResponse function is looking between the hardcoded 200 to 300 range
However, with a bit of copy and pasting from that Http package source code, you can create a custom function to replace Http.fromJson in order to handle HTTP status codes outside the normal "success" range.
Here's an example of the bare minimum you'll need to copy and paste to create a custom myFromJson function that acts the same as the Http package except for the fact it also treats a 400 as a success:
myFromJson : Json.Decoder a -> Task Http.RawError Http.Response -> Task Http.Error a
myFromJson decoder response =
let decode str =
case Json.decodeString decoder str of
Ok v -> Task.succeed v
Err msg -> Task.fail (Http.UnexpectedPayload msg)
in
Task.mapError promoteError response
`Task.andThen` myHandleResponse decode
myHandleResponse : (String -> Task Http.Error a) -> Http.Response -> Task Http.Error a
myHandleResponse handle response =
if (200 <= response.status && response.status < 300) || response.status == 400 then
case response.value of
Http.Text str ->
handle str
_ ->
Task.fail (Http.UnexpectedPayload "Response body is a blob, expecting a string.")
else
Task.fail (Http.BadResponse response.status response.statusText)
-- copied verbatim from Http package because it was not exposed
promoteError : Http.RawError -> Http.Error
promoteError rawError =
case rawError of
Http.RawTimeout -> Http.Timeout
Http.RawNetworkError -> Http.NetworkError
Again, that code snippet is almost entirely copy and pasted except for that 400 status check. Copying and pasting like that is usually a last resort, but because of the library restrictions, it seems to be one of your only options at this point.