How to migrate ArcGis Online data to external database? - dojo

I am using Arcgis Online. I want to migrate my feature layers (data which stored in esri's internal database) to external database.
Can anyone help me to figure out how I can do it? Also how can I use external database in my Application(PostgreSQL/Neo4j).What I need is, to host my own database server like neo4j and use as a replacement of feature-table provided by ArcGis.
Thanks,
Tarni

You have many options for downloading data from ArcGIS Online.
If you do not have many feature services, the easiest way would be to go to "My Content", then find your feature service. You should see an "Export Data" option towards the top-right.
If you have multiple feature services, you could repeat the directions above, but choose "Open in ArcGIS Desktop". This will download a file that will set up the connection in ArcMap for you. You can also hit the services directly from ArcGIS Desktop by going to "My Hosted Services" in ArcCatalog and logging in.
Another option is to use a Python script. This may be best if you have alot of data, and if the data is updated frequently. Check out https://github.com/tedrick/SyncSurvey for an example of getting data repeatedly from a feature service.

Related

Report Provider to Clients

I have a web solution with about 1000 clients. Nowadays they have access to reports that I provide as a PDF: an ASP.NET script executes SP's in the database and with PDFReactor I create a PDF. This is online and as you may already be guessing, it is unmanageable for large data sets.
I was thinking of using a reporting solution, as Power BI, Tableau or... Qlik. In my investigations I have found out that Qlik is the most complete solution for what I want. Nevertheless, I still have some questions, regarding the application architecture and how I can integrate Qlik with my application.
The first question is how I manage my users? I have a custom authentication provider and user management inside my application... What is the best way to integrate with Qlik? I manually configure all the clients in the Qlik Server also? There are endpoints to do it automatically?
The second question is how I manage what a client can see inside a Qlik document (QVW). I want to have several unique QVD, where several QVW get their data from. But I don't want to create a QVW for each client!! Can I filter a QVW based on a user authentication?
I would like that my clients could see the reports dynamically inside my web app. But my conclusion so far is that the web view of Qlik is static and that for more dynamism my users would have to use Qlik View desktop. Correct?
And this leads me to the final question: does my clients would have to pay licenses or is everything on my side and responsibility?
Thanks for your help!
Regards,
David
Out of the box QlikView Server (see 4.2) uses NTFS authentication (local windows users or Active Directory). There is something called DMS - ticket based authentication which allow non Windows users to be authorized to access apps. A bit more info here
QlikView have something called Section Access which is on file (qvw) level. Section Access is part of the QV script. Basically you can create link between user name and data field. The main file will contain all the data but when the user open the document he/she can see only the data that its relevant to this username. More info here and here
For example:
user1 -> can see only UK data
user2 -> can see UK and USA data
user3 -> can see all the data
... etc
The web app and the desktop app are the same. The web client provide same level of interaction as the desktop so you can slice and dice your data as you want
4.1 QV desktop (personal edition) is free but you can open up to 5 qvw files which are not "yours" (not created with the current instance of the personal edition) and the clients need to install it locally on their computers
4.2 QV server is not free. With QV server all the clients are accessing the QV apps via the browser. You (as administrator) are buying the licenses from Qlik Partner and it's up to you if you want to charge your customers or not (and how much)

How do I access netsuite data using SQL language

A company uses netsuites to make purchase orders and store sales information. The only way this company is able to access this information at the moment is through building netsuites reports
Is there a way SQL language could be written to access the netsuites data as some of the questions asked by the business need something a bit more complex than what netsuites reporting provides - and sound very easy using SQL language
I have experience in Oracle, mysql, and many other SQL languages so I would like to know how to set up a connection to the companies netsuites data so I can help write custom queries
Netsuite has SuiteAnalytics Connect, which is ODBC. It can be downloaded from a link at the very bottom the home screen of your Netsuite account.
After installation of the application bundle and connecting it to your Netsuite, you can write ORACLE SQL queries via Excel, Access, etc. My experience is that Netsuite uses Oracle SQL in ODBC queries and in Saved Search custom queries.
The way I understand it, the ODBC driver is a separately priced item from NetSuite. Once you have that, you could use Excel, Access, Crystal Reports or anything else that can use the ODBC driver to read the data and write queries against it.
The normal way to access NetSuite data is with SuiteScript -- javascript code that uses the nlapi* calls to get to the data.
You can access Netsuite backend using a number of different tools and it allows running Oracle SQL directly.
Netsuite provides JDBC, ADO.NET and ODBC if it was included in your licensing purchase. You can find the downloads from the main menu - lower left hand of the screen in Settings - Set Up SuiteAnalytics Connect.
ADO.net however is pretty worthless in SSIS as it doesn't allow parameter mapping or SQL from a variable value. I have yet to get the ODBC driver to work correctly - it can connect and show columns of a table but it won't validate saying "The ODBC Source.Outputs[ODBC Source Error Output].Columns[...] on the error output has properties that do not match the properties of its corresponding data source column."
The other options I'm looking into are Cozyroc and Kinsgwaysoft adapters for Netsuite. You could also do SOAP or REST connections if desired.
I've not used this, but there should as of 2014.2 there is an ODBC driver available for read access.
http://www.netsuite.com/portal/landing/2014-2/suiteanalytics.shtml
As previously mentioned, if you have it licensed you can download the ODBC driver from the Netsuite application. I tested some other adapters but found Netsuite's at least as good as the competitors and they provide timely security updates to the drivers as well as both 64 and 32 bit versions.
There are also metadata browsers that are on the net (example is URL below), the type of browser being based on which kind of access you are looking at (web services, verses ODBC, etc).
https://system.sandbox.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2017_2/odbc/record/account.html
Within the application GUI there is information you need to connect using their ODBC connection (you need the account id and the role id). The URL is just odbcserver.xxxx.netsuite.com where xxx is specific to your environment you are accessing. Note that other adapters such as Cozyroc, etc. require the admin to setup connections in Netsuite for those.
Here is a list of the main metadata tables:
OA_FKEYS
OA_TABLES
OA_COLUMNS
In addition, note that the URL above does NOT include customization. Plus its not easily consumed by automation tools. But Netsuite provides a set of metadata views within the repository which you can use ODBC to pull from which includes all keys - foreign and primary. AND it does include customizations! I built simple set of ETL jobs that use standard ADO.NET driver within SSIS and the ODBC DSN to pull all the metadata and then use BIML to automatically generate all the extract ETL and related staging tables. You can also easily then use the metadata to detect changes in the underlying database.
Let me know if I can be any help with doing any of this.
There are 3rd party tools, like CData ODBC Adapter for NetSuite: http://www.cdata.com/drivers/netsuite/odbc/ . This tool allows you to run standard SQL against the NetSuite API. Note that the NetSuite API presents a very different data model from that of the native NetSuite ODBC.
With this tool, you install in onto your machine (Win in my case) and then after you configure the driver, you can run SQL against it (the driver config is where you specify your NS credentials, etc.).
It revolutionizes access to NetSuite data IMHO.
Download the Netsuite ODBC drivers, set up your DSN locally with server/credentials, Use SSIS pkgs to automate data extract and load onto you local DB. Use ADO.NET drivers if preferred. Then utilize those tables to create/deliver your complex reports using SQL (stored procedures) and Reporting Services. This will require SQL Standard license at minimum though.
Only API call can access the NetSuite data
You can't access to Netsuite data using sql language.
The only way to gain access to Netsuite from third party systems is to use SOAP request or PHP toolkit.

Import a .atomsvc file from Reporting Services

I'm fairly sure I'm missing something really obvious here so hopefully someone can point me in the right direction.
A client of ours has spun up a standard public facing SQL Server 2008 Reporting Services webpage. It shows a few columns of data, based on a couple of drop down filters. It has the standard export to... options (Excel, PDF and so on). It also had a button that downloads an atomsvc file which I understand is for importing the data into Excel via PowerPivot.
Is it possible to import the data this file links to into SQL Server in a fairly straightforward way (T-SQL, linked data source, SSIS ect) or should I just import the Atom data the file links to as I would any other XML?
I can't find much online that doesn't talk about PowerPivot in relation to these atom feeds but it seems to me that, as a standard feature of SQL Serverreporting services, it should be easily importable into SQL server.
Alternatively am I completely missing a much easy way to import data from an HTTP reporting services page? Obviously I don't have console level access to the server as it's a public facing webpage.
So I've done a bit of digging and learned about URL Access (SSRS) for SQL Server Reporting Services.
The .atomsvc file contains a link to the raw data feed. This can be easily tweaked quite extensively. In particular there is a "Format" specifier available which can be used to export as normal XML / CSV / whatever. For example:
http://myrshost/ReportServer?/myreport&rs:Format=XML
http://myrshost/ReportServer?/myreport&rs:Format=CSV
I hope this helps someone. I was as I suspected missing something "obvious" but it was knowing what to Google that tripped me up.
I still can't help thinking that there should be an easy way to just download the .atomsvc file and open it with SQL Management Studio or SSIS rather than PowerPivot but XML / CSV is fairly straight forward o deal with.

Offline database solution for sql server

Here's task: We have an sql server database. which is hosted at our server. What we need to do is: we need to create a non-techy-users interface (basically insert/edit forms) and let these non-techy-users to install this database locally, since they are located in the areas without internet connection. Then when they're done using the database we get the data from them and inster it in our database.
The biggest concern is that it is not trivial for non-it people to install sql server. Can you please advise me what solution should I choose? Simple Access should work fine, but i really do not want to mess with it and have data conversion back and forth between engines.
Sync Framework for SQL Server: your application uses a lite weight, embedded SQL Server CE (no installation, just a couple of DLLs deployed along with your app) and the sync framework manages the synchronization with the 'mother ship' SQL Server.
Out of interest, why do they need their own installation? Can't you create a new database on your existing instance?
If you're looking for an easy way to create insert/edit forms on your database, have you considered looking at Microsoft's new LightSwitch product (currently in Beta) or Microsoft's Dynamic Data?

SSIS - SharePoint to SQL without Adapter Addin?

Im looking to Extract a SharePoint List (WSS 2.0) to a SQL(2005) Table using SQL Server Integrated Services.
First off I am aware of the "adapter" that does this from http://msdn.microsoft.com/en-us/library/dd365137.aspx however I'm just wondering for compatibility purposes if it can't just be done "out of the box".
There are only a limited number of "Data Flow Sources" to select as alternatives and I am unsure if any of these would be able to work in a similar way either directly to SharePoint or via SharePoints web services (e.g. http://server_name/_vti_bin/Lists.asmx) From the list of these sources it looks like the best option would be the OLE DB connector, but not sure how it would do this.
Any help you have would be great,
Mark
You can use Script Source task to talk to WSS Lists Data Retrieval Web Service. But I would strongly recommend using the custom Add-in.
Overview of WSS Webservices
Calling Sharepoint Webservices From SSIS