How do I implement a decoder that calls itself due to one of its fields having the same type? - elm

How do I implement a decoder that calls itself due to one of its fields having the same type?
providerDecoder : Decoder JsonProvider
providerDecoder =
Decode.map6 JsonProvider
(field "Profile" profileDecoder)
(field "Topics" <| Decode.list topicDecoder)
(field "Links" <| linksDecoder)
(field "RecentLinks" <| Decode.list linkDecoder)
(field "Subscriptions" <| Decode.list providerDecoder)
(field "Followers" <| Decode.list providerDecoder)
The following lines are causing issues:
(field "Subscriptions" <| Decode.list providerDecoder)
(field "Followers" <| Decode.list providerDecoder)
providerDecoder is defined directly in terms of itself, causing an
infinite
In conclusion, I am not sure how to resolve this error while still preserving the JsonProvider type.
Appendix:
type JsonProvider
= JsonProvider
{ profile : JsonProfile
, topics : List JsonTopic
, links : JsonLinks
, recentLinks : List JsonLink
, subscriptions : List JsonProvider
, followers : List JsonProvider
}

When you write recursive JSON decoders, you usually have to rely on Json.Decode.lazy. You can write those two lines as this:
(field "Subscriptions" <| Decode.list (Decode.lazy (\_ -> providerDecoder)))
(field "Followers" <| Decode.list (Decode.lazy (\_ -> providerDecoder)))
Once you change that you'll see another error message pop up about the types not matching up, and that's because you're using a single constructor union type that has a record as an argument (which is necessary when writing recursive record types). In this case I usually separate out the constructor and record type like this:
type JsonProvider
= JsonProvider JsonProviderFields
type alias JsonProviderFields =
{ profile : JsonProfile
, topics : List JsonTopic
, links : JsonLinks
, recentLinks : List JsonLink
, subscriptions : List JsonProvider
, followers : List JsonProvider
}
Now you can rewrite the provider decoder to first decode the JsonProviderFields record, then map it to a JsonProvider:
providerDecoder : Decoder JsonProvider
providerDecoder =
Decode.map6 JsonProviderFields
(field "Profile" profileDecoder)
(field "Topics" <| Decode.list topicDecoder)
(field "Links" <| linksDecoder)
(field "RecentLinks" <| Decode.list linkDecoder)
(field "Subscriptions" <| Decode.list (Decode.lazy (\_ -> providerDecoder)))
(field "Followers" <| Decode.list (Decode.lazy (\_ -> providerDecoder)))
|> Decode.map JsonProvider

Related

Ignore invalid item when decoding list

is it possible to ignore invalid items when decoding the list?
example: I have a Model
type Type
= A
| B
type alias Section =
{ sectionType : Type
, index : Int
}
getTypeFromString : String -> Maybe Type
getTypeFromString input =
case input of
“a” ->
Just A
“b” ->
Just B
_ ->
Nothing
decodeType : Decoder Type
decodeType =
Decode.string
|> Decode.andThen
(\str ->
case getTypeFromString str of
Just sectionType ->
Decode.succeed sectionType
Nothing ->
Decode.fail <| ("Unknown type" ++ str)
)
decodeSection : Decoder Section
decodeSection =
Decode.map2 Section
(Decode.field "type" decodeType)
(Decode.field "index" Decode.int)
if I decode the JSON
{
"sections": [{type: "A", index: 1}, {type: "invalid-type", index: 2}]
}
I expect my sections = [ {type = A, index= 1} ]
Generally the way you can deal with these is by decoding it to an Elm type that expresses the options, and then post processing with a map.
So for instance in your example, I would go for something like this:
decodeMaybeType : Decoder (Maybe Type)
decodeMaybeType =
Decode.string
|> Decode.map getTypeFromString
decodeMaybeSection : Decoder (Maybe Section)
decodeMaybeSection =
Decode.map2 (\maybeType index -> Maybe.map (\t -> Section t index) maybeType)
(Decode.field "type" decodeMaybeType)
(Decode.field "index" Decode.int)
decodeSections : Decoder (List Section)
decodeSections =
Decode.list decodeMaybeSection
|> Decode.map (List.filterMap identity)
NB: List.filterMap identity is a List (Maybe a) -> List a, it filters out the Nothing and gets rid of the Maybes in one go.
Given the comment by Quan Vo about one of many fields could be invalid, using Decode.oneOf might be a better fit.
You write the decoders for each field. If any field is illegal, the Section decoder fails and in oneOf, Nothing is returned.
(Here I am also using Json.Decode.Pipeline from NoRedInk).
import Json.Decode as Decode exposing (Decoder)
import Json.Decode.Pipeline exposing (required)
type Type
= A
| B
type alias Section =
{ sectionType : Type
, index : Int
}
getTypeFromString : String -> Maybe Type
getTypeFromString input =
case input |> String.toLower of
"a" ->
Just A
"b" ->
Just B
_ ->
Nothing
decodeType : Decoder Type
decodeType =
Decode.string
|> Decode.andThen
(\str ->
case getTypeFromString str of
Just sectionType ->
Decode.succeed sectionType
Nothing ->
Decode.fail <| ("Unknown type" ++ str)
)
decodeSection : Decoder Section
decodeSection =
Decode.succeed Section
|> required "type" decodeType
|> required "index" Decode.int
-- Either we succeed in decoding a Section or fail on some field.
decodeMaybeSection : Decoder (Maybe Section)
decodeMaybeSection =
Decode.oneOf
[ decodeSection |> Decode.map Just
, Decode.succeed Nothing
]
decodeSections : Decoder (List Section)
decodeSections =
Decode.list decodeMaybeSection
|> Decode.map (List.filterMap identity)

Debugging a JSON decoder

I'm trying to test a decoder - but I only ever get back the default values. Apologies for the wall of text, but when I've tried smaller examples, they always work, so I'm guessing there's a stupid error in here somewhere.
I've been trying to figure out why this won't work for quite a while now, with no luck. The JSON appears to be valid (I have tried parsing it in JS and in online validators).
I've tried different methods of decoding the JSON, again with no luck.
Any help at all is very much appreciated. If anything else should be added to the question, please let me know (I'm new to elm, if you can't tell).
I'm trying to decode JSON which looks like this:
{
"fade": 1,
"colour": "Campbells Red",
"stock": 1,
"site": "",
"url": "",
"plastic":"DX",
"name":"aviar",
"seenAt":1612884837886,
"weight":175,
"compositeIdentifier":"aviar||Innova||DX||Campbells Red||175",
"manufacturer":"Innova",
"expiresAt":1615476837886,
"glide":3,
"turn":0,
"speed":2,
"price":8.99
}
My type looks like this:
type alias DiscSighting =
{ fade: Int
, colour: String
, stock: Int
, site: String
, url: String
, plastic: String
, name: String
, weight: Int
, compositeIdentifier: String
, manufacturer: String
, glide: Int
, turn: Int
, speed: Int
, price: Float
}
And my decoder looks like this:
discDecoder: Decoder DiscSighting
discDecoder =
succeed DiscSighting
|> andMap (field "fade" (int) |> (withDefault) -1)
|> andMap (field "colour" (string) |> (withDefault) "")
|> andMap (field "stock" (int) |> (withDefault) -1)
|> andMap (field "site" (string) |> (withDefault) "")
|> andMap (field "url" (string) |> (withDefault) "")
|> andMap (field "plastic" (string) |> (withDefault) "")
|> andMap (field "name" (string) |> (withDefault) "")
|> andMap (field "weight" (int) |> (withDefault) -1)
|> andMap (field "compositeIdentifier" (string) |> (withDefault) "")
|> andMap (field "manufacturer" (string) |> (withDefault) "")
|> andMap (field "glide" (int) |> (withDefault) -1)
|> andMap (field "turn" (int) |> (withDefault) -1)
|> andMap (field "speed" (int) |> (withDefault) -1)
|> andMap (field "price" (float) |> (withDefault) -1)
The error I get is due to a test failing (it returns the error side of the result and thus fails the test):
Err (Failure "Expecting an OBJECT with a field named price
)
In discDecoder I'm not sure what the definitions are for andMap and withDefault, but the optional function in the package NoRedInk/elm-json-decode-pipeline will work instead of andMap and withDefault:
discDecoder : Decoder DiscSighting
discDecoder =
succeed DiscSighting
|> optional "fade" int -1
|> optional "colour" string ""
|> optional "stock" int -1
|> optional "site" string ""
|> optional "url" string ""
|> optional "plastic" string ""
|> optional "name" string ""
|> optional "weight" int -1
|> optional "compositeIdentifier" string ""
|> optional "manufacturer" string ""
|> optional "glide" int -1
|> optional "turn" int -1
|> optional "speed" int -1
|> optional "price" float -1
Full working example here: https://ellie-app.com/ckYm8HhMJfKa1

How to batch multiple http calls together?

I have a report which I'm using as a basis to perform a number of Http calls to get details for each row.
LoadReport ->
( model
, Http.toTask (loadReport model.token model.page)
|> Task.andThen
(\report ->
Task.map (addProductDetailsResultsToReport report) (Task.sequence (prepareRequests model.token report))
)
|> Task.map filterOnlyMissingBarcodes
|> Task.attempt ProductData
)
The calls are sequenced and perform one after another which is very slow as I need to perform 20 calls in a row. I would like to do something analogues to JavaScript
Promise.all(prepareRequests)
I used to have them being processed using Cmd.Batch but then I couldn't find a way to know when the whole batch is finished loading, I need to load another batch if there are not enough rows on the screen.
I believe the solution already posted by Murph is correct. The following code is an example that demonstrates that solution by first getting a collection of photos from flickr and then getting captions for all those photos by batching a bunch of http get tasks. Two lists are maintained in the model - untitled photos and titled photos. As the responses to the http gets come in, the appropriate photo is added to titled photos with the title assigned.
In this example the code can tell that all the gets have been responded to when the length of the titled list is the same as the length of the untitled list but it could just as easily have been done by removing photos from the untitled list until it is empty.
Here's a working demo
module Main exposing (..)
import Browser
import Html exposing (Html, div, text)
import Html.Attributes as HA
import Http
import Json.Decode as DC
import Svg
import Svg.Attributes as SA
import Task
type Msg
= SetPhotos (Result Http.Error (List Photo))
| SetDescription (Result Http.Error ( String, String ))
main : Program () Model Msg
main =
Browser.element
{ init = init
, view = view
, update = update
, subscriptions = \m -> Sub.none
}
type alias Model =
Result Http.Error
{ untitled : List Photo
, titled : List Photo
}
decodeUser : DC.Decoder String
decodeUser =
DC.at [ "user", "id" ] DC.string
type alias Photo =
{ id : String
, secret : String
, server : String
, farm : Int
, description : Maybe String
}
-- Create a Photo record from info retrieved from flickr api.
-- Get description later
initPhoto : String -> String -> String -> Int -> Photo
initPhoto id sec ser farm =
Photo id sec ser farm Nothing
decodePhotoList : DC.Decoder (List Photo)
decodePhotoList =
DC.list <|
DC.map4 initPhoto
(DC.at [ "id" ] DC.string)
(DC.at [ "secret" ] DC.string)
(DC.at [ "server" ] DC.string)
(DC.at [ "farm" ] DC.int)
-- Decode photos from "flickr.people.getPublicPhotos" request.
decodePhotos : DC.Decoder (List Photo)
decodePhotos =
DC.at [ "photos", "photo" ] decodePhotoList
-- Decode descripion of photo from "flickr.photos.getInfo" request.
decodePhotoDescription : DC.Decoder String
decodePhotoDescription =
DC.at [ "photo", "description", "_content" ] DC.string
-- api key from flickr. Anyone who clones this project should
-- get their own api key.
apiKey : String
apiKey =
"e9d3fdd5c2e26f9ebd13f4983cf727db"
flickrRestServices : String
flickrRestServices =
"https://api.flickr.com/services/rest/?"
noJsonCallback : String
noJsonCallback =
"&format=json&nojsoncallback=1"
userUrl : String -> String
userUrl name =
flickrRestServices
++ "&method=flickr.people.findByUserName"
++ "&api_key="
++ apiKey
++ "&username="
++ name
++ noJsonCallback
publicPhotosUrl : String -> String
publicPhotosUrl uid =
flickrRestServices
++ "&method=flickr.people.getPublicPhotos"
++ "&api_key="
++ apiKey
++ "&user_id="
++ uid
++ noJsonCallback
photoInfoUrl : String -> String
photoInfoUrl photo =
flickrRestServices
++ "&method=flickr.photos.getInfo"
++ "&api_key="
++ apiKey
++ "&photo_id="
++ photo
++ noJsonCallback
-- Cmd to get photo description from flickr.
-- Package results as SetDescription message.
-- Save the photo id with Task.map to apply the description to the right photo
setDescriptionCmd : Photo -> Cmd Msg
setDescriptionCmd dp =
case dp.description of
Nothing ->
Task.attempt SetDescription (Task.map (\s -> ( dp.id, s )) <| Http.toTask <| Http.get (photoInfoUrl dp.id) decodePhotoDescription)
Just des ->
Cmd.none
-- Cmd to get users public photos from flickr.
-- Package results as SetPhotos message.
getPhotosCmd : String -> Cmd Msg
getPhotosCmd name =
let
req =
Http.get (userUrl name) decodeUser
userTask =
Http.toTask req
publicPhotosTask uid =
Http.toTask (Http.get (publicPhotosUrl uid) decodePhotos)
userPhotosTask =
userTask |> Task.andThen publicPhotosTask
in
Task.attempt SetPhotos userPhotosTask
init : () -> ( Model, Cmd Msg )
init _ =
( Ok
{ untitled = []
, titled = []
}
, getPhotosCmd "elmDemo" -- flickr user name
)
-- UPDATE
update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
case msg of
SetPhotos (Ok photos) ->
( Ok
{ untitled = photos
, titled = []
}
, Cmd.batch <| List.map setDescriptionCmd photos
)
SetPhotos (Err e) ->
( Err e
, Cmd.none
)
-- Update description of the photo with matching id.
SetDescription (Ok ( photoId, desc )) ->
case model of
Ok photos ->
let
justTitled =
photos.untitled
|> List.filter (\ph -> ph.id == photoId)
|> List.map (\ph -> { ph | description = Just desc })
newTitled = photos.titled ++ justTitled
newPhotos = { photos | titled = newTitled }
in
( Ok newPhotos
, if
List.length newPhotos.titled
== List.length newPhotos.untitled
then
Cmd.none -- Could do something else here.
else
Cmd.none
)
Err e ->
( Err e
, Cmd.none
)
SetDescription (Err e) ->
( Err e
, Cmd.none
)
-- Compute a photo URL from a Photo record.
-- per: https://www.flickr.com/services/api/misc.urls.html
photoUrl : Photo -> String
photoUrl ps =
"https://farm"
++ String.fromInt ps.farm
++ ".staticflickr.com/"
++ ps.server
++ "/"
++ ps.id
++ "_"
++ ps.secret
++ "_b.jpg"
-- show an image and description if available.
viewPhoto : Photo -> Html Msg
viewPhoto ps =
div
[ HA.style "height" "20%"
, HA.style "width" "20%"
, HA.style "margin" "0"
]
[ div
[ HA.style "height" "90%"
, HA.style "width" "100%"
, HA.style "margin" "0"
]
[ Svg.svg
[ SA.version "1.1"
, SA.width "100%"
, SA.height "100%"
, SA.viewBox "-1 -0.6 2 1.2"
, SA.preserveAspectRatio "none"
]
[ Svg.image
[ SA.xlinkHref (photoUrl ps)
, SA.x "-1"
, SA.y "-0.6"
, SA.width "2"
, SA.height "1.2"
]
[]
]
]
, div
[ HA.style "height" "10%"
, HA.style "width" "100%"
, HA.style "margin" "0"
]
[ div
[ HA.style "text-align" "center" ]
[ text <| Maybe.withDefault "" ps.description ]
]
]
-- Draw an image or display the reason the image is not available.
view : Model -> Html Msg
view model =
case model of
Err s ->
text "Error: "
Ok photos ->
div []
[ div [] [ text "UNTITLED" ]
, div [] (List.map viewPhoto photos.untitled)
, div [] [ text "TITLED" ]
, div [] (List.map viewPhoto photos.titled)
]
Random thought:
Given that you will get a response back for each call you can keep track of the calls received by creating a collection of expected responses before calling batch and then removing the appropriate item from the collection each time a response is received.
At the point at which that collection is empty you've received all the responses and can fire off the next batch.
There are any number of variations on this pattern that should achieve the desired result. (And probably other patterns that would work just as well.)

How do I write a decoder to map a list of values for a custom data type?

I'm struggling to write a decoder for a list of links:
listOfLinksDecoder : Decoder (List JsonLink)
listOfLinksDecoder =
Decode.map (List JsonLink)
(field "Links" <| Decode.list linkDecoder)
Error:
Decode.map (List JsonLink)
Cannot find variable List
Please note that I have been successful in writing a decoder for a single link:
linkDecoder : Decoder JsonLink
linkDecoder =
Decode.map6 JsonLink
(field "Profile" profileDecoder)
(field "Title" Decode.string)
(field "Url" Decode.string)
(field "ContentType" Decode.string)
(field "Topics" <| Decode.list topicDecoder)
(field "IsFeatured" Decode.bool)
Please note that I attempted to search this documentation. However, I was still unable to find an example for my case.
Appendix:
topicLinks : Id -> Topic -> ContentType -> (Result Http.Error (List JsonLink) -> msg) -> Cmd msg
topicLinks providerId topic contentType msg =
let
url =
baseUrl ++ (getId providerId) ++ "/" ++ "topiclinks"
body =
encodeId providerId |> Http.jsonBody
request =
Http.post url body linksDecoder
in
Http.send msg request
You don't need to map, you can just do:
listOfLinksDecoder : Decoder (List JsonLink)
listOfLinksDecoder =
field "Links" <| Decode.list linkDecoder
Also note that Cannot find variable List error is because List is a type and not a constructor/function.

Currying (partial function) in custom event listener

Here's the short version:
I have a function:
onClick : String -> Bool -> (State -> msg) -> Options.Property c msg
onClick name isReversed toMsg =
Options.on "click" <| Json.succeed <|
toMsg (State name isReversed)
I want to change the function to this:
onClick : String -> Bool -> (State -> msg) -> (Maybe msg -> Options.Property c msg)
In other words, I'd like it to return a partial function which takes a Maybe msg. However, I'm not sure how to do that! Help would be appreciated.
Here's the long version:
I'm trying to build a 'fork' of the elm sortable table package which uses the elm mdl package and allows custom event listeners on the column headers. The sortable table package (obviously) applies event listeners on the column headers which sort the table by that column.
Unfortunately, you can't have two 'onClick' listeners on the same element, so what I've decided to do is to pass the custom message as an argument to the sort message, and then send it as a command from the sort update. Sorry if you're not familiar with the package and that all sounds like mumbo jumbo, I just thought I'd give the context for my question.
The developers of the sortable table package created a custom event listener which looks like this:
onClick : String -> Bool -> (State -> msg) -> Attribute msg
onClick name isReversed toMsg =
E.on "click" <| Json.map toMsg <|
Json.map2 State (Json.succeed name) (Json.succeed isReversed)
I've already changed it a bit to use the mdl custom event listener and to make it (in my opinion), slightly more readable:
onClick : String -> Bool -> (State -> msg) -> Options.Property c msg
onClick name isReversed toMsg =
Options.on "click" <| Json.succeed <|
toMsg (State name isReversed)
As I've said, I'd like it to be this:
onClick : String -> Bool -> (State -> msg) -> (Maybe msg -> Options.Property c msg)
However, if you're familiar with the package and have any other suggestions for using custom messages on clicking a column, please suggest them!! I really have no idea what I'm doing.
The longer version:
In the component's view code, there is a variable theadDetails which looks like this:
theadDetails =
customizations.thead (List.map (toHeaderInfo state toMsg columns)
state toMsg and columns all come from the config in the project's main view code. toMsg is a Msg which is handled in the main update (the component doesn't keep track of its own state).
toHeaderInfo looks like this:
toHeaderInfo : State -> (State -> msg) -> ColumnData data msg -> ( String, Status, Options.Property c msg )
toHeaderInfo (State sortName isReversed) toMsg { name, sorter } =
case sorter of
None ->
( name, Unsortable, onClick sortName isReversed toMsg )
Decreasing _ ->
( name, Sortable (name == sortName), onClick name False toMsg )
IncOrDec _ ->
if name == sortName then
( name, Reversible (Just isReversed), onClick name (not isReversed) toMsg )
else
( name, Reversible Nothing, onClick name False toMsg )
This is basically where the data that will be included in each element is rendered. All of this stuff about State and sorter has to do with how each column's sorting is configured and the current order. But you see, here onClick is being called and passed the required arguments.
As you see in theadDetails, this info is then passed to an function, customizations.tHead which looks like this:
defaultCustomizations : Customizations data msg c
defaultCustomizations =
{ tableAttrs = []
, caption = Nothing
, thead = simpleThead
, tfoot = Nothing
, tbodyAttrs = []
, rowAttrs = simpleRowAttrs
}
simpleThead : List ( String, Status, Options.Property { numeric : Bool, sorted : Maybe Table.Order } msg ) -> HtmlDetails {} msg
simpleThead headers =
HtmlDetails [] (List.map simpleTheadHelp headers)
simpleTheadHelp : ( String, Status, Options.Property { numeric : Bool, sorted : Maybe Table.Order } msg ) -> Html msg
simpleTheadHelp ( name, status, onClick ) =
let
check =
Debug.log "status" status
attrs =
case status of
Unsortable ->
[]
Sortable selected ->
if selected then
[ onClick
, Options.css "color" "rgb(0,0,0)"
]
else
[ onClick ]
Reversible Nothing ->
[ onClick
]
Reversible (Just isReversed) ->
[ onClick
, Options.css "color" "rgb(0,0,0)"
]
in
Table.th attrs [ Html.text name ]
It's precisely here where I'd like to pass the final argument. So simpleTheadHeald would become:
simpleTheadHelp : ( String, Status, Options.Property { numeric : Bool, sorted : Maybe Table.Order } msg ) -> Html msg
simpleTheadHelp ( name, status, onClick ) =
let
check =
Debug.log "status" status
attrs =
case status of
Unsortable ->
[]
Sortable selected ->
if selected then
[ onClick Nothing
, Options.css "color" "rgb(0,0,0)"
]
else
[ onClick Nothing ]
Reversible Nothing ->
[ onClick Nothing
]
Reversible (Just isReversed) ->
[ onClick Nothing
, Options.css "color" "rgb(0,0,0)"
]
in
Table.th attrs [ Html.text name ]
This, however, gives me an error saying onClick is not a function (because in the type definition it isn't expecting an argument).
Sorry for doing such a poor job explaining myself! I'm really trying to figure it out as I go, so I appreciate the patience.
I'm not familiar with the package, so if I'm missing something in your question or telling you something you already know, I apologize.
Functions in Elm are curried automatically. All you need to do is pass the function an incomplete set of arguments, and you'll get back a function that takes the remaining argument(s). So this would be your function signature:
onClick : String -> Bool -> (State -> msg) -> Maybe msg -> Options.Property c msg
onClick name isReversed toMsg maybeMsg =
You then write the function, using all the arguments and not worrying about partial application at all. Calling that function with only the first three arguments, like this:
onClick "myName" True MyMsg
will automatically return a function with this signature:
newFunction : Maybe msg -> Options.Property c msg
You don't need to do anything else.