Since Netweaver ship with its own DestinationDataProvider, we can't register our own customized destination data provider. Does means we have to use the Netweaver's destination manager to define a destination and use it in our application? Is there a way to connect to any SAP server and create our own destination without using the Netweaver's destination manager?
The way this is done in JCO 3 is slightly different. The idea is create Properties & place from where it can be retrieved (flat file, ldap etc) and then retrieve the same when you want to connect to a sap server.
// So you First use the following code to create the connection parameters
static String DESTINATION_NAME1 = "ABAP_AS_WITHOUT_POOL";
static String DESTINATION_NAME2 = "ABAP_AS_WITH_POOL";
static
{
Properties connectProperties = new Properties();
connectProperties.setProperty(DestinationDataProvider.JCO_ASHOST, "sap.dsc.com");
connectProperties.setProperty(DestinationDataProvider.JCO_SYSNR, "76");
connectProperties.setProperty(DestinationDataProvider.JCO_CLIENT, "800");
connectProperties.setProperty(DestinationDataProvider.JCO_USER, "dsc007");
connectProperties.setProperty(DestinationDataProvider.JCO_PASSWD, "passwd");
connectProperties.setProperty(DestinationDataProvider.JCO_LANG, "en");
createDestinationDataFile(DESTINATION_NAME1, connectProperties);
connectProperties.setProperty(DestinationDataProvider.JCO_POOL_CAPACITY, "3");
connectProperties.setProperty(DestinationDataProvider.JCO_PEAK_LIMIT, "10");
createDestinationDataFile(DESTINATION_NAME2, connectProperties);
}
static void createDestinationDataFile(String destinationName, Properties connectProperties)
{
File destCfg = new File(destinationName+".jcoDestination");
try
{
FileOutputStream fos = new FileOutputStream(destCfg, false);
connectProperties.store(fos, "for tests only !");
fos.close();
}
catch (Exception e)
{
throw new RuntimeException("Unable to create the destination files", e);
}
}
// The following code snippet can then be used elsewhere to connect to the sap server
static String DESTINATION_NAME1 = "ABAP_AS_WITHOUT_POOL";
static String DESTINATION_NAME2 = "ABAP_AS_WITH_POOL";
public static void step1Connect() throws JCoException
{
JCoDestination destination = JCoDestinationManager.getDestination(DESTINATION_NAME1);
System.out.println("Attributes:");
System.out.println(destination.getAttributes());
System.out.println();
}
You have to use NetWeaver's Destination Service. Either create the RFC destinations via the NetWeaver Admin UI or programmatically via the offered APIs for managing the destination configurations.
Nevertheless, you can also create JCoCustomDestination instances based on the JCoDestination instances retrieved from the JCoDestinationManager.
Related
So I'm trying out the cascading framework and I was able to run apps in local. As a next step I want to use Cascading to download files from S3 to local file system. I'm planning to use FileTap. When I google I found S3FS and looks like its deprecated. Can someone point me in the right direction. In other words, what is the right way to create a FileTap for Amazon S3 with access id and secret key like stuff.
Though this code is for the HadoopFlowConnector this will work with Local Flow connector if using the FileTap.
public class Main {
public void run(String[] args) {
Properties properties = new Properties();
String accessKey = args[0];
String secretKey = args[1];
// better put these keys to hadoop xml file
// for block file system
properties.setProperty("fs.s3.awsAccessKeyId", accessKey);
properties.setProperty("fs.s3.awsSecretAccessKey", secretKey);
// for s3 native file system
// properties.setProperty("fs.s3n.awsAccessKeyId", accessKey);
// properties.setProperty("fs.s3n.awsSecretAccessKey", secretKey);
// properties.setProperty("fs.defaultFS", "hdfs://localhost:8020/");
// properties.setProperty("fs.permissions.umask-mode", "007");
AppProps.setApplicationJarClass(properties, Main.class);
HadoopFlowConnector flowConnector = new HadoopFlowConnector(
properties);
String input = "s3://my-bucket/my-log.csv";
// If using the native S3
// String input = "s3n://my-bucket/my-log.csv";
Tap inTap = new Hfs(new TextDelimited(false, ";"), input);
Pipe copyPipe = new Pipe("copy");
Tap outTap = new Hfs(new TextDelimited(false, ";"),
"data/output");
FlowDef flowDef = FlowDef.flowDef()
.addSource(copyPipe, inTap)
.addTailSink(copyPipe, outTap);
flowConnector.connect(flowDef).complete();
}
public static void main(String[] args) {
new Main().run(args);
}
}
Code courtesy http://whiteboxdeveloper.blogspot.com/2015/01/processing-data-from-aws-s3-using.html.
I just added the S3N string input commented line. Since I was using S3N based accessing I uncommented the S3N features and used it.
I am currently developing a utility to help automate our report deployment process. Multiple files, in multiple folders, to multiple servers.
I am using the reportservice2010.asmx web service, and I am deploying my files to the server - so most of the way there.
My issue is that I have shared data sets and shared data sources, which are deployed to individual folders, separate to the report folders. When the deployment occurs the web service looks locally for the data source rather than in the data source folder, giving an error like:
The dataset ‘CostReduction’ refers to the shared data source ‘CostReduction’, which is not
published on the report server. The shared data source ‘CostReduction’ must be published
before this report can run.
The data source/set has been deployed and the report functions correctly but I need to suppress these error messages as they may be hiding other actual errors.
I can hard code a lookup that checks if the data source/set exists and manually filter them via that, but it seems very in-efficient. Is there any way I can tell the web service where to look for these files or another approach that other people have used?
I'm not looking at changing the reports so the data source is read from
/DataSources/DataSourceName
as there are lots of reports and that's not how our existing projects are configured.
Many thanks in advance.
I realize you are using VB, but perhaps this will give you a clue if you convert it from C# to VB, using one of the translators on the web.
Hopefully this will give you a lead in the right direction.
When All the reports in a particular folder, referred to here as the 'parent folder', all use the same Shared Data source, I use this to set all the reports to the same shared Data Source (in this case "/DataSources/Shared_New")
using GetPropertiesSample.ReportService2010;
using System.Diagnostics;
using System.Collections.Generic; //<== required for LISTS
using System.Reflection;
namespace GetPropertiesSample
{
class Program
{
static void Main(string[] args)
{
GetListOfObjectsInGivenFolder_and_ResetTheReportDataSource("0_Contacts"); //<=== This is the parent folder
}
private static void GetListOfObjectsInGivenFolder_and_ResetTheReportDataSource(string sParentFolder)
{
// Create a Web service proxy object and set credentials
ReportingService2010 rs = new ReportingService2010();
rs.Credentials = System.Net.CredentialCache.DefaultCredentials;
CatalogItem[] reportList = rs.ListChildren(#"/" + sParentFolder, true);
int iCounter = 0;
foreach (CatalogItem item in reportList)
{
iCounter += 1;
Debug.Print(iCounter.ToString() + "]#########################################");
if (item.TypeName == "Report")
{
Debug.Print("Report: " + item.Name);
ResetTheDataSource_for_a_Report(item.Path, "/DataSources/Shared_New"); //<=== This is the DataSource that I want them to use
}
}
}
private static void ResetTheDataSource_for_a_Report(string sPathAndFileNameOfTheReport, string sPathAndFileNameForDataSource)
{
//from: http://stackoverflow.com/questions/13144604/ssrs-reportingservice2010-change-embedded-datasource-to-shared-datasource
ReportingService2010 rs = new ReportingService2010();
rs.Credentials = System.Net.CredentialCache.DefaultCredentials;
string reportPathAndName = sPathAndFileNameOfTheReport;
//example of sPathAndFileNameOfTheReport "/0_Contacts/207_Practices_County_CareManager_Role_ContactInfo";
List<ReportService2010.ItemReference> itemRefs = new List<ReportService2010.ItemReference>();
ReportService2010.DataSource[] itemDataSources = rs.GetItemDataSources(reportPathAndName);
foreach (ReportService2010.DataSource itemDataSource in itemDataSources)
{
ReportService2010.ItemReference itemRef = new ReportService2010.ItemReference();
itemRef.Name = itemDataSource.Name;
//example of DataSource i.e. 'itemRef.Reference': "/DataSources/SharedDataSource_DB2_CRM";
itemRef.Reference = sPathAndFileNameForDataSource;
itemRefs.Add(itemRef);
}
rs.SetItemReferences(reportPathAndName, itemRefs.ToArray());
}
}
To Call it I use this in the 'Main' Method:
GetListOfObjectsInGivenFolder_and_ResetTheReportDataSource("0_Contacts");
In this case "0_Contacts" is the parent folder, itself located in the root directory, that contains all the reports for which I want to reset their DataSources to the new Shared DataSource. Then that Method calls the other method "ResetTheDataSource_for_a_Report" which actually sets the DataSource for the report.
I have researched on how to export BLOBs to image. A DB has an IMAGE column storing several thousand images. I thought of exporting the table but I get a BLOB file error in EMS SQL Manager for InterBase and Firebird.
There have been good posts, but I have still not been able to succeed.
SQL scripts to insert File to BLOB field and export BLOB to File
This example has appeared on numerous pages, including Microsoft's site. I am using INTERBASE (Firebird). I have not found anything related to enabling xp_shell for Firebird, or EMS SQL Manager for InterBase and Firebird (which I have also installed). My guess is: its not possible. I also tried Installing SQL Server Express, SQL Server 2008, and SQL Server 2012. I am at a dead end without having even connected to the server. The reason being I have not managed to start the server. Followed the guide at technet.microsoft: How to: Start SQL Server Agent but there are no services on the right pane to me.
PHP file to download entire column (may not post link due to rep limitation).
It has a MySQL connect section that daunts me. There on my computer is the DB as a GDB file, I also have XAMPP. I can figure out a way to use this as a localhost environment. I hope this is making sense.
Last solution is to use bcp, an idea posted on Stack Overflow titled: fastest way to export blobs from table into individual files. I read the documentation, installed it, but cannot connect to server. I use -S PC-PC -U xxx -P xxx (The server must be wrong) But the information I find all uses -T (Windows Authentication)
Summing up. I am using Firebird, as EMS SQL Manager. I try to extract all images from images table into individual files. These tools both have SQL script screens, but it appears to be in conjunction with xp shell. What would you suggest? Am I using the wrong SQL manager to accomplish this?
There are several ways:
Use isql command BLOBDUMP to write a blob to file,
Use a client library (eg Jaybird for Java, Firebird .net provider for C#) to retrieve the data,
With PHP you can use ibase_blob_get in a loop to get bytes from the blob, and write those to a file.
I don't use nor know EMS SQL Manager, so I don't know if (and how) you can export a blob with that.
The example you link to, and almost all tools you mention are for Microsoft SQL Server, not for Firebird; so it is no wonder those don't work.
Example in Java
A basic example to save blobs to disk using Java 8 (might also work on Java 7) would be:
/**
* Example to save images to disk from a Firebird database.
* <p>
* Code assumes a table with the following structure:
* <pre>
* CREATE TABLE imagestorage (
* filename VARCHAR(255),
* filedata BLOB SUB_TYPE BINARY
* );
* </pre>
* </p>
*/
public class StoreImages {
// Replace testdatabase with alias or path of database
private static final String URL = "jdbc:firebirdsql://localhost/testdatabase?charSet=utf-8";
private static final String USER = "sysdba";
private static final String PASSWORD = "masterkey";
private static final String DEFAULT_FOLDER = "D:\\Temp\\target";
private final Path targetFolder;
public StoreImages(String targetFolder) {
this.targetFolder = Paths.get(targetFolder);
}
public static void main(String[] args) throws IOException, SQLException {
final String targetFolder = args.length == 0 ? DEFAULT_FOLDER : args[0];
final StoreImages storeImages = new StoreImages(targetFolder);
storeImages.store();
}
private void store() throws IOException, SQLException {
if (!Files.isDirectory(targetFolder)) {
throw new FileNotFoundException(String.format("The folder %s does not exist", targetFolder));
}
try (
Connection connection = DriverManager.getConnection(URL, USER, PASSWORD);
Statement stmt = connection.createStatement();
ResultSet rs = stmt.executeQuery("SELECT filename, filedata FROM imagestorage")
) {
while (rs.next()) {
final Path targetFile = targetFolder.resolve(rs.getString("FILENAME"));
if (Files.exists(targetFile)) {
System.out.printf("File %s already exists%n", targetFile);
continue;
}
try (InputStream data = rs.getBinaryStream("FILEDATA")) {
Files.copy(data, targetFile);
}
}
}
}
}
Example in C#
Below is an example in C#, it is similar to the code above.
class StoreImages
{
private const string DEFAULT_FOLDER = #"D:\Temp\target";
private const string DATABASE = #"D:\Data\db\fb3\fb3testdatabase.fdb";
private const string USER = "sysdba";
private const string PASSWORD = "masterkey";
private readonly string targetFolder;
private readonly string connectionString;
public StoreImages(string targetFolder)
{
this.targetFolder = targetFolder;
connectionString = new FbConnectionStringBuilder
{
Database = DATABASE,
UserID = USER,
Password = PASSWORD
}.ToString();
}
static void Main(string[] args)
{
string targetFolder = args.Length == 0 ? DEFAULT_FOLDER : args[0];
var storeImages = new StoreImages(targetFolder);
storeImages.store();
}
private void store()
{
if (!Directory.Exists(targetFolder))
{
throw new FileNotFoundException(string.Format("The folder {0} does not exist", targetFolder), targetFolder);
}
using (var connection = new FbConnection(connectionString))
{
connection.Open();
using (var command = new FbCommand("SELECT filename, filedata FROM imagestorage", connection))
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
string targetFile = Path.Combine(targetFolder, reader["FILENAME"].ToString());
if (File.Exists(targetFile))
{
Console.WriteLine("File {0} already exists", targetFile);
continue;
}
using (var fs = new FileStream(targetFile, FileMode.Create))
{
byte[] filedata = (byte[]) reader["FILEDATA"];
fs.Write(filedata, 0, filedata.Length);
}
}
}
}
}
}
I installed a TFS2012 as a test system and doing some tests before we go productive.
This includes to define many BuildDefinitions which was a lot of work.
After the tests are successful, an new server will be installed with TFS2012 on it.
For this new server - which operates then as the productive system - i would like to restore the BuildDefinitions from the test system. But only the BuildDefinitions, not the whole TeamCollections. Because i ran test checkins and i don`t want these on my productive server.
Now, is it possible to backup and restore BuildDefinitions only?
Maybe it is possible directly throught the Sql database?, but i`am a little affraid of references there, pointing on some other tables.
Best Regards, Peter Bucher
Build definitions are not source controlled. The only option is relying on the TFS database backup where can restore or view the tbl_BuildDefinition* tables in the Tfs_DefaultCollection database.
There is a user voice for this feature and also you can use TFS API to do it.
Add a vote on uservoice:
provide a way to version-control build definitions
Using TFS API
How can I copy a TFS 2010 Build Definition?
Finally i decided not to touch the database, because there are references to a lot of other tables.
I used the TFS API v11 (TFS2012) and a bit C# Code, which i fitted to my needs from this base: How can I copy a TFS 2010 Build Definition?
It copies all Build Definitions from one TFS2012 Server to another. For both servers there is the need to specifiy a TeamCollection and a TeamProject.
So, the copy-task has to be done per TeamProject.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.VersionControl.Client;
namespace TFSBuildDefinitionCreator
{
internal class Program
{
private static void Main(string[] args)
{
// Copies build definitions from one server to another.
// Uses the TeamFoundation API V11 (TFS2012).
// Code was used to copy b uild definitions from a test server to a productive.
string sourceServer = "http://testTfs:8080/tfs/MyTeamCollection";
string sourceTeamProject = "MyTeamProject";
string targetServer = "https://productiveTfs:8080/tfs/MyTeamCollection";
string targetTeamProject = "MyTeamProject";
// DropLocation for defininitions: Share on which the build should be dropped.
string defaultDropLocation = "\\\\MyBuildserver\\Builds$";
// Change the DefaultProcessTemplate in the following method below: GetDefaultProcessTemplateByServerPathFromBuildServer.
CopyBuildDefinitions(sourceServer, sourceTeamProject, targetServer, targetTeamProject, defaultDropLocation);
Console.Read();
}
private static IBuildServer GetBuildServerFromServerUrl(string serverUrl)
{
var tfs = TeamFoundationServerFactory.GetServer(serverUrl);
return (IBuildServer)tfs.GetService(typeof(IBuildServer));
}
private static IBuildController GetDefaultBuildControllerFromBuildServer(IBuildServer buildServer)
{
return buildServer.QueryBuildControllers()[0];
}
private static IProcessTemplate GetDefaultProcessTemplateByServerPathFromBuildServer(IBuildServer buildServer, string teamProject)
{
var processTemplates = buildServer.QueryProcessTemplates(teamProject);
var result = processTemplates.First(t => t.ServerPath.Contains("/BuildProcessTemplates/MyDefaultTemplate.xaml"));
return result;
}
private static void CopyBuildDefinitions(string sourceServer, string sourceTeamProject, string targetServer,
string targetTeamProject, string defaultDropLocation)
{
var sourceBuildServer = GetBuildServerFromServerUrl(sourceServer);
var sourceBuildDetails = sourceBuildServer.QueryBuildDefinitions(sourceTeamProject);
foreach (var sourceBuildDetail in sourceBuildDetails)
{
CopyBuildDefinition(sourceBuildDetail, targetServer, targetTeamProject, defaultDropLocation);
}
}
private static void CopyBuildDefinition(IBuildDefinition buildDefinition, string targetServer, string targetTeamProject, string defaultDropLocation)
{
var targetBuildServer = GetBuildServerFromServerUrl(targetServer);
var buildDefinitionClone = targetBuildServer.CreateBuildDefinition(targetTeamProject);
buildDefinitionClone.BuildController = GetDefaultBuildControllerFromBuildServer(targetBuildServer);
buildDefinitionClone.ContinuousIntegrationType = buildDefinition.ContinuousIntegrationType;
buildDefinitionClone.ContinuousIntegrationQuietPeriod = buildDefinition.ContinuousIntegrationQuietPeriod;
// Noch ändern.
//buildDefinitionClone.DefaultDropLocation = buildDefinition.DefaultDropLocation;
buildDefinitionClone.DefaultDropLocation = defaultDropLocation;
buildDefinitionClone.Description = buildDefinition.Description;
buildDefinitionClone.Enabled = buildDefinition.Enabled;
//buildDefinitionClone.Name = String.Format("Copy of {0}", buildDefinition.Name);
buildDefinitionClone.Name = buildDefinition.Name;
//buildDefinitionClone.Process = buildDefinition.Process;
buildDefinitionClone.Process = GetDefaultProcessTemplateByServerPathFromBuildServer(targetBuildServer, targetTeamProject);
buildDefinitionClone.ProcessParameters = buildDefinition.ProcessParameters;
foreach (var schedule in buildDefinition.Schedules)
{
var newSchedule = buildDefinitionClone.AddSchedule();
newSchedule.DaysToBuild = schedule.DaysToBuild;
newSchedule.StartTime = schedule.StartTime;
newSchedule.TimeZone = schedule.TimeZone;
}
foreach (var mapping in buildDefinition.Workspace.Mappings)
{
buildDefinitionClone.Workspace.AddMapping(
mapping.ServerItem, mapping.LocalItem, mapping.MappingType, mapping.Depth);
}
buildDefinitionClone.RetentionPolicyList.Clear();
foreach (var policy in buildDefinition.RetentionPolicyList)
{
buildDefinitionClone.AddRetentionPolicy(
policy.BuildReason, policy.BuildStatus, policy.NumberToKeep, policy.DeleteOptions);
}
buildDefinitionClone.Save();
}
}
}
Hope that helps others.
I'm running JBoss AS 7.1.0.CR1b. I've got several datasources defined in my standalone.xml e.g.
<subsystem xmlns="urn:jboss:domain:datasources:1.0">
<datasources>
<datasource jndi-name="java:/MyDS" pool-name="MyDS_Pool" enabled="true" use-java-context="true" use-ccm="true">
<connection-url>some-url</connection-url>
<driver>the-driver</driver>
[etc]
Everything works fine.
I'm trying to access the information contained here within my code - specifically the connection-url and driver properties.
I've tried getting the Datasource from JNDI, as normal, but it doesn't appear to provide access to these properties:
// catches removed
InitialContext context;
DataSource dataSource = null;
context = new InitialContext();
dataSource = (DataSource) context.lookup(jndi);
ClientInfo and DatabaseMetadata from a Connection object from this Datasource also don't contain these granular, JBoss properties either.
My code will be running inside the container with the datasource specfied, so all should be available. I've looked at the IronJacamar interface org.jboss.jca.common.api.metadata.ds.DataSource, and its implementing class, and these seem to have accessible hooks to the information I require, but I can't find any information on how to create such objects with these already deployed resources within the container (only constructor on impl involves inputting all properties manually).
JBoss AS 7's Command-Line Interface allows you to navigate and list the datasources as a directory system. http://www.paykin.info/java/add-datasource-programaticaly-cli-jboss-7/ provides an excellent post on how to use what I believe is the Java Management API to interact with the subsystem, but this appears to involve connecting to the target JBoss server. My code is already running within that server, so surely there must be an easier way to do this?
Hope somebody can help. Many thanks.
What you're really trying to do is a management action. The best way to is to use the management API's that are available.
Here is a simple standalone example:
public class Main {
public static void main(final String[] args) throws Exception {
final List<ModelNode> dataSources = getDataSources();
for (ModelNode dataSource : dataSources) {
System.out.printf("Datasource: %s%n", dataSource.asString());
}
}
public static List<ModelNode> getDataSources() throws IOException {
final ModelNode request = new ModelNode();
request.get(ClientConstants.OP).set("read-resource");
request.get("recursive").set(true);
request.get(ClientConstants.OP_ADDR).add("subsystem", "datasources");
ModelControllerClient client = null;
try {
client = ModelControllerClient.Factory.create(InetAddress.getByName("127.0.0.1"), 9999);
final ModelNode response = client.execute(new OperationBuilder(request).build());
reportFailure(response);
return response.get(ClientConstants.RESULT).get("data-source").asList();
} finally {
safeClose(client);
}
}
public static void safeClose(final Closeable closeable) {
if (closeable != null) try {
closeable.close();
} catch (Exception e) {
// no-op
}
}
private static void reportFailure(final ModelNode node) {
if (!node.get(ClientConstants.OUTCOME).asString().equals(ClientConstants.SUCCESS)) {
final String msg;
if (node.hasDefined(ClientConstants.FAILURE_DESCRIPTION)) {
if (node.hasDefined(ClientConstants.OP)) {
msg = String.format("Operation '%s' at address '%s' failed: %s", node.get(ClientConstants.OP), node.get(ClientConstants.OP_ADDR), node.get(ClientConstants.FAILURE_DESCRIPTION));
} else {
msg = String.format("Operation failed: %s", node.get(ClientConstants.FAILURE_DESCRIPTION));
}
} else {
msg = String.format("Operation failed: %s", node);
}
throw new RuntimeException(msg);
}
}
}
The only other way I can think of is to add module that relies on servers internals. It could be done, but I would probably use the management API first.