In my use case, ElasticSearch is already configured and has data that can be queried via REST API. I'm wondering if there is a way to write SQL statements that can query this data that ElasticSearch is already configured on.
Example, configure an adapter to ElasticSearch in MS SQLServer and use linkedserver to connect and run normal SQL statements.
I have read about the "river", but it seems to do the opposite of what I'm looking for. Any pointers will really help.
SQL Server is a relational database. It operates with tables in common. Posting requests to some URI is very unusual work for SQL Server. And there is no standard mechanism to do this.
What can you do:
Write a CLR-function to send post-requests
Map result json to some table (it can be difficult, because Elastic Search is document-oriented and SQL Server is not)
So, as for me, this way is very complicated. I advice to use some hand-written service to operate with DB and Elastic Search, and don't try to put all logic to SQL Server.
Something like Elasticsearch-SQL plugin might be of interest to you.
This won't allow you to use it as a linked server in MSSQL, but it will allow whatever application you have, to send SQL queries to the sql API on your ElasticSearch server and get results.
Example:
http://localhost:9200/_sql?sql=select * from indexName limit 10
Related
I am trying to migrate a database from a sql server into Azure. This database have 2 rather simple TSQL script that inserts data. Since the SQL Agent does not exist on Azure, I am trying to find an alternative.
I see the Automation thing, but it seems really complex for something as simple as running SQL scripts. Is there any better or at least easier way to do this ?
I was under the impression that there was a scheduller for that for I can't find it.
Thanks
There are several ways to run a scheduled Task/job on the azure sql database for your use case -
If you are comfortable using the existing on-premise sql sever agent you can connect to your azure sql db(using linked servers) and execute jobs the same way we used to on on-premise sql server.
Use Automation Account/Runbooks to create sql jobs. If you see marketplace you can find several examples on azure sql db(backup,restore,indexing jobs..). I guess you already tried it and does not seem a feasible solution to you.
Another not very famous way could be to use the webjobs(under app service web app) to schedule tasks(can use powershell scripts here). The disadvantage of this is you cannot change anything once you create a webjob
As #jayendran suggested Azure functions is definitely an option to achieve this use case.
If some how out of these if you do not have options to work with the sql directly , there is also "Scheduler Job Collection" available in azure to schedule invocation of HTTP endpoints, and the sql operation could be abstracted/implemented in that endpoint. This would be only useful for less heavy sql operations else if the operation takes longer chances are it might time out.
You can use Azure Functions to Run the T-SQL Queries for Schedule use Timely Trigger.
You can use Microsoft Flow (https://flow.microsoft.com) in order to create a programmed flow with an SQL Server connector. Then in the connector you set the SQL Azure server, database name, username and password.
SQL Server connector
There are many options but the ones that you can use to run a T-SQL query daily are these:
SQL Connector options
Execute a SQL Query
Execute stored procedure
You can also edit your connection info in Data --> Connections menu.
I need to create a connection between ##MongoDB## and ##SQL Server## where I want to replicate a subset of my Database from SQL Server into MongoDB. Can anyone suggest for feasibility of the same and how ?
Right now I am using symmetricDS for the replication but unable to...
Please suggest if symmetricDS is able to serve for this purpose.
Here is how you target MongoDB:
http://www.symmetricds.org/doc/3.8/html/user-guide.html#_mongodb
If you need more flexibility than straight table to table mapping, then you would write your own data loader using the MongoDatabaseWriter as a pattern.
https://github.com/JumpMind/symmetric-ds/tree/0c5cc1c24b42a64405f4b79c3cb6b594a35467f2/symmetric-client/src/main/java/org/jumpmind/symmetric/io
Got an easy way around for the Data Exchange from SQL to MongoDB using:
SQLtoMongo C# Tool
KNIME Analytics Platform (way easy to implement - Opensource)
But still looking for something with triggers to easily replicate the things.
I have a project in which i need to replace the SQL DB with REDIS. Its a job scheduling system. There are tables like JobInfo, TaskInfo, Result, BatchInfo etc.
What is the best way to map DB tables in REDIS server key value pair?
There are join and group by kind of queries used in the project.
What is the best way to replace the sql server with the redis server? Also does redis provides a way with which i can query the data like i can in join and group by queries?
Redis is basically a key-value store (a bit more sophisticated than just a simple one, but yet - a key-value db). the value may be a document that follows some schema, but Redis isn't optimized to search for those documents and query them like other Document Databases or like relational database such as SQL Server.
I dont know why you're trying to migrate from SQL Server to Redis, but you need to re-check yourself if that's the right design choice. If you need fixed schema and join operations - it may suggest that Redis isn't the right solution.
If all you're looking for is caching, you can cache in the application layer, or use other solution to integrate your Redis and SQL Server (I wrote simple open-source project that does that: http://redisql.ishahar.net ).
Hope this helps.
I guess its not possible though you can see below post to implement JOIN like feature in Redis.
Can we take join in Redis?
Please refer below post as well:
Redis database table desing like sql?
I have users entering data in SharePoint (Running on SQL Server), but my application to view that data will be an Oracle Apex app running on Oracle, obviously. How do I have the data be pushed into the Oracle db automatically?
First off, are you sure that you need to replicate the data to Oracle? Oracle Heterogeneous Services allows you to create a database link in Oracle that connects to a non-Oracle database using ODBC (assuming you use the Transparent Gateway for ODBC which is free). Your APEX application could then query and report on data that is in SQL Server by issuing queries that run over the database link. Tim Hall has a good article (though it's a bit dated and some of the components have been renamed, the general approach is still the same) on configuring Heterogeneous Services.
If you do need to replicate the data, you can create materialized views in Oracle that query the objects in SQL Server using the database link you created with Heterogeneous Services and schedule those materialized views to refresh on a regular basis. The materialized views will need to do a complete refresh, though, which means that every row will need to be copied from SQL Server to Oracle every time there is a refresh. That generally limits the frequency with which you can realistically have refreshes happen. If you need the data to be replicated to the Oracle database and you need to send incremental changes so that the Oracle side doesn't lag too far behind, you can use Streams from a non-Oracle database to an Oracle database but that involves a lot more work.
In SQL Server you can setup linked servers that allow you to view data from other db's. You might see if Oracle has something similar, if not the same. Alternatively, you could use the sql's integration services to push the data over to an oracle table. Unfortunately I only know how to setup linked servers in SQL Server and I don't have a lot of experience with ssis to tell you how to do that, but those are the first two options I can think of that you might explore further.
Here's a link I found that might be helpful as well: http://www.dba-oracle.com/t_connecting_sql_server_oracle.htm
There's no way to do it "automatically" that I know of that will work across DBMS. ETL tools like Sql Server Integration Services might help but there's going to be a loading delay (as it will have to poll for changes). You could build some update triggers on the SharePoint database tables but that's going to turn into a support nightmare.
I need to query the Total Physical Memory, Available Physical Memory and Total Commit Charge of the server. Basically values circled in the picture. Is it possible using SQL Server 2005?
alt text http://www.angryhacker.com/toys/task.png
You can try using the sys.dm_os_sys_info table. Wich returns a miscellaneous set of useful information about the computer, and about the resources available to and consumed by SQL Server.
USE [master];
SELECT * FROM sys.dm_os_sys_info
Bye.
It's not entirely clear what you're asking. You can use a subset of SQL called WQL to get information from WMI, and I'm pretty sure all the data you're asking for is available via WMI, so you should be able to get it all via a SQL query. That SQL query won't be talking to the actual SQL server at the time though, it'll be talking to the WMI provider via the WQL adapter.
I'm not sure about the entire box, but you can use DBCC MemoryStatus to get the consumption of SQL Server itself.
Here's an article about it.
I don't think you really mean SQL as in Database information, it looks to me like you're trying to query the operating system for performance information. Is that right?
You'd need to perform WMI queries for that, instead of SQL queries (which are designed for database access)
Here's an example for getting memory information:
http://www.computerperformance.co.uk/vbscript/wmi_memory.htm#Scenario_-_When_to_use_this_WMI_Memory_Script_
The web site included in the link above has all kinds of samples, and I think you'd be able to get to what you want by researching there.