How to receive tile data from ClusterBuster vector tile server in the sample example? - express

Summary
Seems the guys did a great work with the built-in filtering, clustering and caching on the ClusterBuster vector tile server.
I'm really excited about this project that looks very promising and i'm eager to try it!
I'm working on a project on plotting Deck.gl layers (MVTLayer) on top of Google Maps from a vector tile server through their integration.
I want to try to integrate and to serve from ClusterBuster server some clustered points saved in PostGIS.
Expected result:
I want to receive tile data from ClusterBuster vector tile server like in their provided sample example.
Actual result:
Provided sample example not working (points not showing on map, neither on Mapbox, nor on Google Maps with Deck.gl).
My attempts
I tried building and running the example provided but somehow, i'm not able to do this.
I ran the provided express server clusterbuster/example/express.ts and for frontend tried with the provided mapbox example from clusterbuster/example/mapbox.html (with my mapbox token) but nothing appears on mapbox map (nor on Google maps with my Deck.gl example).
As stated in the clusterbuster/example/readme.md i created an .env file with PostGIS connections settings.
After running the following commands:
yarn
yarn start
i get this output:
C:\zFVStuff\Google Maps - Deck.gl\_Servers\ClusterBuster\clusterbuster-master> yarn start
yarn run v1.22.4
$ yarn build && ts-node example/express.ts
$ rollup --config rollup.config.js
./lib/index.ts → dist/index.d.ts...
created dist/index.d.ts in 5.7s
./lib/index.ts → dist/index.js...
created dist/index.js in 5.1s
attempting to create supporting SQL functions
failure in creating First SQL function
failure in creating TileBBox SQL function
failure in creating TileDoubleBBox SQL function
Example app listening on port 3005!
send3663284b-0e58-4587-af82-b35604a99303: 4.964ms
3663284b-0e58-4587-af82-b35604a99303: 817.819ms
sendd0bdd671-1f47-4b40-84ba-2fed6adfd045: 3.451ms
d0bdd671-1f47-4b40-84ba-2fed6adfd045: 1276.588ms
sendcd36f672-4c4b-4a95-b772-7584e6eb8c55: 1.349ms
cd36f672-4c4b-4a95-b772-7584e6eb8c55: 1754.146ms
The requests seems to be ok, returning status 200 but tile data from ClusterBuster server is undefined nothing appears on the map (neither your Mapbox, or my Google Maps).
Also, it's curious that all tile responses seems to have the same size 216 B.
The data was imported ok in the PostGIS database.
Result is undefined after debugging the express.ts server file with VSCode Javascript Debug Terminal and starting it with the command ts-node .\example\express.ts.
In the Chrome network tab, in none of the sub-tabs Preview or Response i can't see any data.
I also tried logging the result tile in the express server in the success function,
server({
...//excluded for visibility
}).then(result => {
...//excluded for visibility
console.log(result)
res.status(200).send(result);
})
and below is the output, i get undefined:
Any help or advice is really appreciate! Thank you in advance!

After enabling debug in the TileServer configuration (with debug: true), i noticed 2 errors:
SELECT ST_AsMVT(tile, 'points', 4096, 'geom') AS mvt FROM tile
{
e: error: password authentication failed for user "userXYZ"
...
SELECT ST_AsMVT(tile, 'points', 4096, 'geom') AS mvt FROM tile
{
e: Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:205:27) {
...
Initially I ran from the root of clusterbuster with ts-node .\example\express.ts or with yarn start (that underneath uses the same and it tries to connect with my user instead of .env config file)
After running ts-node express.ts from the example folder it took into account the .env config file with postgres user instead of my user. Now the tile data from the server is returned ok and the example works at expected.
Many thanks to the contributor cosmin-petrescu that helped me.
Entire thread can be seen here on github.

Related

Vue.js error when importing amChart data from web API

I'm learning Vue.js with amCharts and trying to build a covid-19 dashboard into a test project using data from amChart's own web API.
Hopefully, I don't need to put all my code on here because I'm just using amChart's demo code and I have the chart working from a module import within my own project, e.g.:
import covid_us_timeline from "#/assets/data/us_timeline";
That's module code I have copied from amChart's web API # covid.amcharts.com/data/js/us_timeline.js
So, that works great, but I want to use the chart directly from the web API, so I'm using this in my Vue.js script section:
import covid_us_timeline from "https://covid.amCharts.com/data/js/us_timeline.js";
And getting this error:
These dependencies were not found:
https://covid.amCharts.com/data/js/us_timeline.js in ./node_modules/cache-loader/dist/cjs.js??ref--12-0!./node_modules/babel-loader/lib!./node_modules/vuetify-loader/lib/loader.js??ref--18-0!./node_modules/cache-loader/dist/cjs.js??ref--0-0!./node_modules/vue-loader/lib??vue-loader-options!./src/components/Covid19Chart.vue?vue&type=script&lang=js&
To install them, you can run: npm install --save https://covid.amCharts.com/data/js/us_timeline.js
Error from chokidar (C:): Error: EBUSY: resource busy or locked, lstat 'C:\hiberfil.sys'
It doesn't look like a cors issue and given this is all public domain data, I'd rather use in the front-end than build it in the back-end, so any ideas what I'm doing wrong?
Just as an update, I decided to go a different way anyway as accessing this volume of data from the front-end was a drain to say the least. So, switched to using axios in Node to access the data and load into Mongo and then manipulate it from there. It was only updated once a day anyway, so no need to access it every time the page was loaded.

Spartacus API calls return 504 (Gateway Timeout) when running using Server Side Rendering (SSR)

I'm trying to get Spartacus to work with SSR. When opening the default URL, http://localhost:4200, the storefront renders, as expected, but only after I clear the site data first. When I attempt to browse the storefront, API calls fail with a 504 (Gateway timeout). Chrome dev tools indicates the error is happening in the service worker. At this point, I'm wondering if I configured Spartacus incorrectly. When running Spartacus using yarn start rather than yarn serve:ssr, I can load the home page and browse the site normally.
OS: Ubuntu 16.04.6 LTS
Chrome Version: 73.0.3683.75
Node version: 11.15.0
Angular CLI version: 8.3.8
Yarn version: 1.19.1
ng new ssr-spartacus-app --style=scss
cd ssr-spartacus-app
ng add #spartacus/schematics --baseUrl https://localhost:9002 --baseSite cmssiteuid --pwa --ssr
rm src/app/app.component.html
echo "<cx-storefront>Loading...</cx-storefront>" > src/app/app.component.html
yarn build:ssr
yarn serve:ssr
Before running yarn build:ssr, I made following change to the app.module.ts file:
Before
context: {
baseSite: ['cmssiteuid'],
},
After
authentication: {
client_id: 'mobile_android',
client_secret: 'secret',
},
context: {
urlParameters: ['baseSite', 'language', 'currency'],
baseSite: ['cmssiteuid'],
},
I also set anonymousConsents to false. With this set to true, I was getting a lot of CORs errors.
If been scratching my head with this for a little while now and I'm hoping someone with more knowledge of Spartacus' inner workings can shed some light on why Spartacus is behaving this way with SSR.
I'm not sure that I can give you some certain recipe to fix the issue, obviously I need more details and logs relates to your problem, but still, based on my experience I can share with you some tips and tricks about how we should play with such issues (which relates to SSR).
Some set of theory which relates to SSR
https://angular.io/guide/universal (you can feel free to use Angular official documentation as a primary source, cuz Spartacus uses Angular OOTB features to make it works)
https://sap.github.io/spartacus-docs/server-side-rendering-in-spartacus/
https://enable.cx.sap.com/tag/tagid/spartacus (SSR related videos)
Practical approaches for debugging SSR
You should observe and analyze console output during starting your application in Node.js
You can use SSR configuration from example Storefront application (https://github.com/SAP/spartacus/tree/develop/projects/storefrontapp) like a starting point, cuz OOTB SSR works like a charm
Something from Spartacus team https://sap.github.io/spartacus-docs/how-to-debug-server-side-rendered-storefront/
Common set of theory to ensure that application has been configured correctly
SAP Commerce Cloud configuration for working with Spartacus https://sap.github.io/spartacus-docs/installing-sap-commerce-cloud/
Take a look on the guide https://sap.github.io/spartacus-docs/building-the-spartacus-storefront-from-libraries/ to ensure, that your frontend application has correct configuration
Double check your configuration which B2cStorefrontModule is using (here you can find an example project here https://github.com/SAP/spartacus/tree/develop/projects/storefrontapp)
Take a look on Network and Console browser tabs and try to resolve all errors
did you turn off PWA?
Turn PWA off.
As soon as Spartacus is installed in PWA mode, a service worker is installed, and it serves a cached version of index.html, along with the js files. This results in SSR being completely skipped. The following steps describe how to turn off PWA:
Check that there are no service workers registered in your app. If you do find any service workers, remove them.
Turn PWA off in your app module configuration, as follows:
StorefrontModule.withConfig({
backend: {
occ: {
baseUrl: 'https://[your_enpdoint],
},
},
pwa: {
enabled: false,
},
};
Rebuild your local Spartacus libraries by running the following command:
yarn build:core:lib
Build your local Spartacus shell app by running the following command:
yarn build --prod
Build the SSR version of your shell app by running the following command:
yarn build:ssr
Start Spartacus with the SSR server by running the following command:
yarn serve:ssr
If you are getting 504 after hitting the API service you need to check your API logs.
IF you have err log:
{"instant":{"epochSecond":1644915623,"nanoOfSecond":929833000},"thread":"hybrisHTTP1","level":"ERROR","loggerName":"org.springframework.web.servlet.DispatcherServlet","message":"Context initialization failed","thrown":{"commonElementCount":0,"localizedMessage":"Error creating bean with name 'cartEntriesController': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'defaultStockValidator' defined in ServletContext resource [/WEB-INF/config/v2/validators-v2-spring.xml]: Unsatisfied dependency expressed through constructor parameter 0: Could not convert argument value of type [de.hybris.platform.ycommercewebservices.stock.impl.DefaultCommerceStockFacade] to required type [de.hybris.platform.commercewebservices.core.stock.CommerceStockFacade]: Failed to convert value of type 'de.hybris.platform.ycommercewebservices.stock.impl.DefaultCommerceStockFacade' to required type 'de.hybris.platform.commercewebservices.core.stock.CommerceStockFacade'; nested exception is java.lang.IllegalStateException: Cannot convert value of type 'de.hybris.platform.ycommercewebservices.stock.impl.DefaultCommerceStockFacade' to required type 'de.hybris.platform.commercewebservices.core.stock.CommerceStockFacade': no matching editors or conversion strategy found","message":"Error creating bean with name 'cartEntriesController': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'defaultStockValidator'
You can try resolution:
Remove template extension ycommercewebservices extension from manifest.json, rebuild and redeploy with "Migrate Data" mode.

How can I test electron-builder auto-update flow?

I built an Electron app and I am now looking at how to distribute it.
I went with electron-builder to handle packaging etc.
For a bit of context, as a web developer, I am used to continuously deploy web apps on a web server but I have a hard time figuring out how to distribute a packaged one in Electron.
In electron-builder docs there is a brief mention about testing auto-update:
"Note that in order to develop/test UI/UX of updating without packaging the application you need to have a file named dev-app-update.yml in the root of your project, which matches your publish setting from electron-builder config (but in YAML format)"
But, it's rather vague...
So I actually have two questions:
1. How do I actually test the auto-update flow?
Do I need to actually publish a new version to trigger an update locally? Seems pretty unclear, it would be like developing against the production server.
2. Is it possible to have a fallback for unsigned code?
I don't have yet any certificate for code signing. So the OS/app will block the auto-update. But, I'd still want to tell the user that an update is available so they can go and download the app manually. Can I do that? (going back to point 1, I'd like to be able to test this flow)
I've just finished dealing with this. I also wanted to test against a non-production server and avoid having to package my app each time I iterated. To test downloads I had to sign my app, which slowed things down. But it sounds like you just need to check for updates. Which I think you can do as follows...
I created a dummy github repo, then created a a file dev-app-update.yml containing:
owner: <user or organization name>
repo: dev-auto-update-testing
provider: github
The path where this file is expected to be defaults to a place you can't access. Thankfully, you can override it like so:
if (isDev) {
// Useful for some dev/debugging tasks, but download can
// not be validated becuase dev app is not signed
autoUpdater.updateConfigPath = path.join(__dirname, 'dev-app-update.yml');
}
...that should be enough for your case -- since you don't need downloads.
If not, here are some other tips:
you can change the repo setting in your electron-builder config to point at your dummy repo then package your app. This will give you a packed, production build that points at your dummy repo -- this is how I did my download testing (though I have a cert, and signed my app)
you should be calling autoUpdate's checkForUpdates(), but if checkForUpdatesAndNotify() gives you a useful OS Notification then you should be able to set autoUpdater.autoDownload to false and end up with what you need.
Lastly, it sounds you could skip autoUpdater, since you won't be using the download feature anyway. Instead you could use github's releases api, assuming you use github to host your release. If not then your host should have something similar. Use that to check for updates then tell the user from within your App (could present them with a clickable URL too). If you want OS Notifications electron has a module for that.
We're using electron-updater with GitHub as a provider for auto-updates. Unfortunately, it breaks a lot and the electron-builder team doesn't support these issues well (1, 2, 3) (from my own experience, but you can find more examples on GitHub).
One way to test updates in dev mode:
Create a build of your app with an arbitrarily high version number
Create a public repo and publish the above build
Create a dev-app-update.yml next to your main entry point and configure it for the repo above (see)
In your main entry point:
import { autoUpdater } from "electron-updater";
...
if (process.env.NODE_ENV === "development") {
// Customize the test by toggling these lines
// autoUpdater.autoDownload = false
// autoUpdater.autoInstallOnAppQuit = false;
autoUpdater.checkForUpdates();
}
Then when running yarn dev you should see something like:
Checking for update
...
Found version 100.0.0 (url: <>.exe)
Downloading update from <>.exe
updaterCacheDirName is not specified in app-update.yml Was app build using at least electron-builder 20.34.0?
updater cache dir: C:\Users\<>\AppData\Local\Electron
New version 100.0.0 has been downloaded to C:\Users\<>\AppData\Local\Electron\pending\<>.exe
And it should install when you close the dev app.
This should give you some certainty but we still ran into issues in production. If you want to be sure, play through the full update flow with a test repo but packaged production apps just as you would do with the live one.

Angular 5 / Sentry error reporting shows error source reference in bundled js

I have applied Sentry.io error reporting in an Angular 5 project using Raven-js. Everything works fine and I get the error reports.
The problem is that the reported error location is provided using the bundle.js file references which is very hard to find in the actual project.
For e.g.
TypeError: Cannot read property 'id' of undefined
at e.logOffline (/mem/main.f858f9330b4c0ee7c38e.bundle.js:1:26440)
at e.phoneOffline (/mem/main.f858f9330b4c0ee7c38e.bundle.js:1:25883)
at t.handleEvent_12 (/mem/main.f858f9330b4c0ee7c38e.bundle.js:1:488537)
at HTMLAnchorElement.<anonymous> (/mem/vendor.f8004cbd231f87b295e2.bundle.js:1:311586)
at HTMLAnchorElement.r (/mem/vendor.f8004cbd231f87b295e2.bundle.js:1:666890)
Now this reference is not useful to locate the error.
Is there something that I missed in the sentry setup or this is how it actually shows the error?
Seems like Sentry can't find your source maps. If your site is public, Sentry will try to get the spurce maps automatically. Otherwise you can upload the source maps yourself.
https://docs.sentry.io/clients/javascript/sourcemaps/
The usage of source maps works out of the box for a deployed app. Sentry tries to fetch the source maps through the public internet and thus cannot fetch them on localhost.
If your source maps aren't public (e.g. out of security concerns), then you need to upload them manually like described here: https://docs.sentry.io/platforms/javascript/sourcemaps/availability/#uploading-source-maps-to-sentry .

Agile Central example cumulative flow diagram not rendering

After checking out the source from: https://github.com/RallyApps/app-catalog
I attempted to build the Portfolio Item cumulative flow diagram src/apps/charts/rpm/cfd locally using
rally-app-builder build which shows no errors
Launching App-debug.html allows me to input the relevant settings, but once I click 'save', nothing renders and there are no errors logged to the JS console either, making debugging challenging.
I attempted to copy the build output into a custom html app inside rally, which displayed the same behaviour.
Is this a bug with the example app? Or have I missed some crucial configuration step?
EDIT:
Extra info - I tried running via the rally-app-builder to no avail (think it's related to this issue: https://github.com/RallyApps/rally-app-builder/issues/45). I attempted to flatten the structure and the result of that was a cross-origin request error.
Running from file resulted in a 403 for the SDK
Sorry about that- that Rally App Builder issue is definitely the problem you're running into. Just as a test I downloaded the built html output for that app from the app catalog release: https://github.com/RallyApps/app-catalog/releases/download/2.1/2.1.zip
It worked when installed on a custom html app on my dashboard once I edited the app settings and configured a portfolio item.
If you're looking to tweak this app from source you'll probably have to copy all those files referenced in config.json that live in parent directories into the main app directory and fix the paths in config.json. Then you should be able to build and run the app normally using Rally App Builder.
It seems this app was really not designed to be run externally, so I added a little code to ease the issues I ran into:
In PortfolioChartAppBase.js, _loadSavedPortfolioItem function:
scope: this,
fetch: ['Name', 'ObjectID'] //this is new
And I added some default settings as a top level config object in CumulativeFlowChartApp.js (since the code to force into settings mode doesn't seem to work correctly when running externally):
config: {
defaultSettings: {
portfolioItemPicker: '/portfolioitem/feature/52725935318', //insert a valid oid here
startDate: 'actualstartdate',
endDate: 'actualenddate',
chartAggregationType: 'storycount'
}
},