How to query for namespace dynamically using Linq to XML - vb.net

I have 290 Group Policy Backup xml files which I need to enumerate in separate folders.
With each Group Policy backup xml file, I need to query the Policy settings.
Anyone who's looked at a Group Policy xml backup file before would know they're chock-a-block full of Namespace declarations.
I want to know, using Linq to XML, as I query each xml file, how can I dynamically query the XML the Namespace and then append the Namespace into the Linq query for the child nodes/values?
Here are some examples of the xml structure.
<User>
<ExtensionData>
<Extension xmlns:q1="http://www.microsoft.com/GroupPolicy/Settings/Scripts" xsi:type="q1:Scripts">
<q1:Script>
<ExtensionData>
<Extension xmlns:q1="http://www.microsoft.com/GroupPolicy/Settings/IE" xsi:type="q1:InternetExplorerSettings">
<q1:PreferenceMode>true</q1:PreferenceMode>
<ExtensionData>
<Extension xmlns:q2="http://www.microsoft.com/GroupPolicy/Settings/Registry" xsi:type="q2:RegistrySettings">
<q2:Policy>
<q2:Name>Disable changing accessibility settings</q2:Name>
<q2:State>Enabled</q2:State>
<ExtensionData>
<Extension xmlns:q1="http://www.microsoft.com/GroupPolicy/Settings/DriveMaps" xsi:type="q1:DriveMapSettings">
<q1:DriveMapSettings clsid="{8FDDCC1A-0C3C-43cd-A6B4-71A6DF20DA8C}">
My initial code looks like this:
Dim NS As XNamespace = "http://www.microsoft.com/GroupPolicy/Settings"
NodeValue = XDoc.Descendants(NS + NodeName).First().Value
As you can see I'm going to face literally dozens of different Namespaces, at this stage I don't even know what they all are.
My end-task is to trawl through 290 directories, each containing one Group Policy xml backup file. I then need to read the Policy Name from each of the settings contained within the backup file.
Because I don't know what Policy settings each xml flie will contain, I don't know what Namespace(s) I need to use when attempting to read the xml values. Each xml file may even contain multiple namespaces.
How do I dynamically read the Namespace in Linq so I can read the values?
Thanks

Do you care about these namespaces? i.e. Do you distinguish them and do different kind of processing depending on the namespace-uri? If you don't (e.g. you just display something) you can do something like this:
XDoc.Descendants().Where(e => e.Name.LocalName == "Extension")
This should select all Extension elements regardless of the namespace. Alternatively, if you need to query elements using namespaces instead of hardcoding one you could do something like this:
foreach(XElement extensionElement in XDoc.Descendants().Where(e => e.Name.LocalName == "Extension")
{
var ns = extensionElement.Name.NamespaceName;
Console.WriteLine(extensionElement(ns + "DriveMapSettings"));
}

From e In XDoc.Descendants Group By e.Name.Namespace.NamespaceName should find all distinct namespaces, but it is untested.

Here's what I ended up with:
Dim ListOfNamespaces = z.Root.DescendantsAndSelf.Attributes().Where(Function(a) a.IsNamespaceDeclaration).GroupBy(Function(a) If(a.Name.[Namespace] = XNamespace.None, [String].Empty, a.Name.LocalName), Function(a) XNamespace.[Get](a.Value)).ToDictionary(Function(g) g.Key, Function(g) g.First())

Related

Select values from XML with multiple namespaces

I need to read a value of an attribute from an XML column. The data is an XML with multiple namespaces declared:
<sd:objectData xmlns:sd="http://sd-uri">
<sd:object sourceKey="FC5A0A51-7FB6-4C64-A13E-D4B00649E80E">
<do:properties xmlns:do="http://do-uri">
<do:property name="DECISION">
<do:propertyValues clearExistingValues="true">
<do:propertyValue action="add" valueInteger="1000142" tag="Approve" />
</do:propertyValues>
</do:property>
</do:properties>
</sd:object>
</sd:objectData>
I want to read the value of valueInteger, namely in this example 1000142. I tried with WITH XMLNAMESPACES() but I am not able to get it together to define both aliases.
Does this work for you?
DECLARE #XML xml = '
<sd:objectData xmlns:sd="http://sd-uri">
<sd:object sourceKey="FC5A0A51-7FB6-4C64-A13E-D4B00649E80E">
<do:properties xmlns:do="http://do-uri">
<do:property name="DECISION">
<do:propertyValues clearExistingValues="true">
<do:propertyValue action="add" valueInteger="1000142" tag="Approve" />
</do:propertyValues>
</do:property>
</do:properties>
</sd:object>
</sd:objectData>';
WITH XMLNAMESPACES ('http://sd-uri' AS sd,
'http://do-uri' AS do)
SELECT #XML.value('(/sd:objectData/sd:object/do:properties/do:property/do:propertyValues/do:propertyValue/#valueInteger)[1]','int') AS valueInteger;
In addition to Larnu's answer (which is the best and correct answer) just some alternative shortcuts, if you just want to get one value:
This query fetches the needed value in four different approaches
SELECT #XML.value(N'(//*/#valueInteger)[1]',N'int') AS Super_easy_with_double_wildcard
,#XML.value(N'(//*:propertyValue/#valueInteger)[1]',N'int') AS Easy_with_namespace_wildcard
,#XML.value(N'declare namespace do="http://do-uri";
(//do:propertyValue/#valueInteger)[1]',N'int') AS with_local_declaration
,#XML.value(N'declare namespace do="http://do-uri";
declare namespace sd="http://sd-uri";
(/sd:objectData/sd:object/do:properties/do:property/do:propertyValues/do:propertyValue/#valueInteger)[1]',N'int') AS with_full_local_declaration;
The general advise is: Be as specific as possible to avoid hassels. If you do no bother and you just need a readable, quick catch, you can take one of the alternatives.
UPDATE Add a predicate
With a predicate you can place a filter:
SELECT #XML.value(N'(//*:property[#name="DECISION"]//*:propertyValue/#valueInteger)[1]',N'int') AS Example_with_predicate

XML parsing with namespace SQL Server

We are cleaning up data in our database and a column has XML details inside of it which we want to be able to convert into plain text.
Below is the sample XML in the table column.
<FlowDocument PagePadding="5,5,5,5" Name="RTDocument" AllowDrop="True" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
<Paragraph>FILE DESTROYED - MAY 21st, 2008</Paragraph>
<Paragraph>todo</Paragraph>
</FlowDocument>
I am using this query, but it is not rendering the desired output due to the presence of Namespace (if I remove the namespace from the XML, I am able to render the output successfully).
SELECT
CAST(CAST(Comments AS XML).query('data(/FlowDocument/Paragraph)') AS VARCHAR(7000)) AS activity
FROM
dbo.Activities
WHERE
ActivityID = 1
Kindly help in this matter.
Thanks
You can also declare your namespace like this:
;WITH xmlnamespaces(DEFAULT 'http://schemas.microsoft.com/winfx/2006/xaml/presentation')
SELECT
CAST(CAST(Comments AS XML).query('data(/FlowDocument/Paragraph)') AS VARCHAR(7000)) AS activity
FROM [dbo].Activities where ActivityID=1
Other options are given here: parsing xml using sql server
You need to use namespace declaration in your Query as per: https://msdn.microsoft.com/en-us/library/ms191474.aspx
so your query portion would look something like:
query('
declare namespace NS="http://schemas.microsoft.com/winfx/2006/xaml/presentation";
data(/NS:FlowDocument/NS:Paragraph)
')

Bulk Insert Multiple (More than 1000 XML file) XML files with SSIS 2008 R2

I have below one of XML files that i need to bulk insert them into SQL server, i have more than 1000 XML and i want to use SSIS for this purpose, any ideas please?
all XML having the same attribute with different data content.
<?xml version = "1.0" encoding="Windows-1252" standalone="yes"?>
<VFPData>
<cr_attributes>
<headerexpression>1-108-201403-18-A01276</headerexpression>
<timesheetfield>cc2</timesheetfield>
<sessioninstance>d425d70e-7baf-4f6f-8302-a327b7ded320</sessioninstance>
<originvalue/>
<pagenum>1</pagenum>
<pageaddress>81.1865.11.78</pageaddress>
<fieldid>860002635</fieldid>
<fieldname>TS6CCode2</fieldname>
<interpretaccuracy>100</interpretaccuracy>
<timestamp>4/1/2014 3:53:28 PM</timestamp>
<pen>AJX-AA7-D98-UX</pen>
<validationlink/>
<cfieldvalue>5300330</cfieldvalue>
<crowkey>18-A01276</crowkey>
<calias>PayCard</calias>
<lvaluechanged>true</lvaluechanged>
</cr_attributes>
<cr_attributes>
<headerexpression>1-108-201403-18-A01276</headerexpression>
<timesheetfield>cc2_sub</timesheetfield>
<sessioninstance>d425d70e-7baf-4f6f-8302-a327b7ded320</sessioninstance>
<originvalue/>
<pagenum>1</pagenum>
<pageaddress>81.1865.11.78</pageaddress>
<fieldid>860002642</fieldid>
<fieldname>TS6SCode2</fieldname>
<interpretaccuracy>100</interpretaccuracy>
<timestamp>4/1/2014 3:53:32 PM</timestamp>
<pen>AJX-AA7-D98-UX</pen>
<validationlink/>
<cfieldvalue>000002</cfieldvalue>
<crowkey>18-A01276</crowkey>
<calias>PayCard</calias>
<lvaluechanged>true</lvaluechanged>
</cr_attributes>
<cr_attributes>
<headerexpression>1-108-201403-18-A01276</headerexpression>
<timesheetfield>hr_cc2</timesheetfield>
<sessioninstance>d425d70e-7baf-4f6f-8302-a327b7ded320</sessioninstance>
<originvalue>.0</originvalue>
<pagenum>1</pagenum>
<pageaddress>81.1865.11.78</pageaddress>
<fieldid>860002658</fieldid>
<fieldname>TS6HRCC21</fieldname>
<interpretaccuracy>100</interpretaccuracy>
<timestamp>4/1/2014 3:53:20 PM</timestamp>
<pen>AJX-AA7-D98-UX</pen>
<validationlink/>
<cfieldvalue>08</cfieldvalue>
<crowkey>T24112-FAROOQ AHMAD</crowkey>
<calias>PayIn</calias>
<lvaluechanged>true</lvaluechanged>
</cr_attributes>
</VFPData>
Components required:
To iterate through 1000 xml file - Use Foreach loop container
To read each xml file - Use XML Source component
To Load xml into database - Use OLE DB Destination component
To Change xml file path dynamically - Use expression and variable.
While iterating through each XML file, the XML Source connection needs to be changed each time. So to make this dynamic, you need to create variable to store XML file-path(get from Foreach loop container),and set this variable to XML Source connection manager property.
Below resources will provide you step by step walk-through to understand above component.
Working with Variables
SSIS Expressions Basics
Working with Property Expressions
Step 1 SSIS BASICS: Introducing the Foreach Loop Container
Step 2 MSDN - XML Source
Step 3 Loading XML Using SSIS
Step 4 Loading multiple XML file from the file system into a SQL Server table using SSIS.
Since you are handling 1000 xml files with same schema but varying data length range, so there might be changes that the auto generated schema/XSD by SSIS won't support data content(length and type), you need to tweak auto generated XSD to cover of data-type and data length. You can find guideline in this thread (last part of article) on how to manually change XSD.

Export SQL query results to XML format using powershell

I need to make an XML file based on the SQL query that I run using powershell. I already know the schema for the XML that I need to create. The query results need to be looped through and I want to add each data value to specific XML node as per the schema.
I am able to run the query and get the results I need but I am having issues placing the data as per prescribed format.
Here's an example of how I am trying to accomplish this:
**Parsing the XMl Template
$XmlTemplate= [xml](get-content $xml) ($xml is the schema I have from the client)
***Parsing through XML Template and jumping to tag
$PlanIDXML= $XmlTemplate.NpiLink.PlanProvider.PlanID (to get to the node I need to enter data into)
**Parsing through XML Template and jumping to tag
$PlannameXML= $XmlTemplate.NpiLink.PlanProvider.PlanName (to get to the node I need to enter data into)
sample qry;
select PlanID,PlanName from plan
**Assuming I ran my query and saved the results as $qryresults***
foreach($result in $qryresults)
{
$PlanID=$result.PlanID
$PlanName=$result.PlanName
**Make Clone
$NewPlanIDXML=$PlanIDXML.Clone()
**Make Changes to the data
$NewPlanIDXML=$PlanID
***Append
$PlanIDXML.AppendChild($NewPlanIDXML)
* Do the same thing for Plan Name **
$PlanNameXML=$result.PlanName
}
$XmlTemplate.Save('filepath')
My concern is that I need to do this for each plan or planid that I get in my query results and I need to keep generating tags and tags even and append them to orginal nodes and save the schema.
So, if my query results have 10 Plan IDs it should continue to generate new Plan ID tags and Plan Name tags.
Its not letting me append (because system.string can not be converted to system.xml). I am really stuck and if you have a better approach on how to handle this, I am all ears.
Thanks much in advance!!!
You might be overengineering this a bit. If you have a template for the XML node, just treat it as a string, popping your values in at the appropriate place. Generate some array of these nodes as strings, then join them together and save to disk.
Let's say your template looks like this (type in some tokens yourself where generated values should go):
--Template.xml---
<Node attr="##VALUE1##">
<Param>##VALUE2##</Param>
</Node>
And you want to run some query to generate a bunch of these nodes, filling in VALUE1 and VALUE2. Then something like this could work:
$template = (gc .\Template.xml) -join "`r`n"
$val1Token = '##VALUE1##'
$val2Token = '##VALUE2##'
$nodes = foreach( $item in Run-Query)
{
# simple string replace
$result = $template
$result = $result.Replace($val1Token, $item.Value1)
$result = $result.Replace($val2Token, $item.Value2)
$result
}
# you have all nodes in a string array, just join them together along with parent node
$newXml = (#("<Nodes>") + $nodes + "</Nodes>") -join "`r`n"
$newXml | out-file .\Results.xml

Modify entry in OpenLDAP directory

I have a large Openldap directory. In the directory the display name property for every is filled but i need to modify these entry and make it like "givenName + + sn". Is there are way i can do it directly in the directory just like sql queries (update query). I have read about the ldapmodify but could not find the way to use it like this.
Any help in this regard will be appreciated.
There is no way to do this with a single LDAP API call. You'll always have to use one LDAP search operation to get givenname and sn attributes, and one LDAP modify operation to modify the displayName attribute.
If you use the command line ldaptools "ldapsearch" and "ldapmodify", you can do this easily with some shell scripting, but you'll have to be careful: sometimes ldapsearch(1) can return LDIF data in base64 format, with UTF-8 strings that contain characters beyond ascii. For instance: 'sn:: Base64data' (note the double ':')
So, if I were you I would use a simple script in my language of choice, that has an LDAP API, instead of using shell commands. This would save me the troubles of base64 decoding that the ldaptools sometimes impose.
For instance, with php-cli, your script would be roughly like this (perhaps some more error checking would be appropriate):
<?php
$ldap = ldap_connect('host');
ldap_bind($ldap, ...);
$sr = ldap_search($ldap, 'ou=people,...', 'objectclass=*');
$entries= ldap_get_entries($ldap, $sr);
for($i=0; $i<$entries['count']; $i++) {
$modify = array('displayname' => $entries[$i]['givenname'] . ' ' . $entries[$i]['sn']);
ldap_modify($ldap, $entries[$i]['dn'], $modify);
}
Addendum: if you want to keep this data up to date without any intervention, you will probably need to use a specialized OpenLDAP module that keeps "virtual" attributes, or even a virtual directory, such as Penrose or Oracle Virtual Directory, on top of OpenLDAP. However this might be overkill for a simple concatenation of attributes.