i need to optimize the CQ5 lucene indexing configuration for my application.
I want to provide a custom search configuration but i struggle to really understand the default configuration.
Source: https://helpx.adobe.com/experience-manager/kb/SearchIndexingConfig.html)
First question:
Are the "include"-tags used in the default configuration correct?
For example:
The default configuration uses the tag "include" to include the Property "jcr:content/jcr:lastModified" for the nt:file-Aggregate
<aggregate primaryType="nt:file">
<include>jcr:content</include>
<include>jcr:content/jcr:lastModified</include>
</aggregate>
Compare this to the Jackrabbit wiki which uses the "include-property" for the exact same case. Source: http://wiki.apache.org/jackrabbit/IndexingConfiguration
<aggregate primaryType="nt:file">
<include>jcr:content</include>
<include-property>jcr:content/jcr:lastModified</include-property>
</aggregate>
I only can assume it doesn't matter but i can't find any source to confirm this.
Second question: for the nodeType "cq:PageContent" all properties of four levels are aggregated.
<aggregate primaryType="cq:PageContent">
<include>*</include>
<include>*/*</include>
<include>*/*/*</include>
<include>*/*/*/*</include>
</aggregate>
I assume that because of the aggregation all properties are indexed which are contained within these 4 levels.
Or do i must consider the index rules for the nodeType nt:base which basicly only includes properties which are matching the pattern ".:.".
<index-rule nodeType="nt:base">
<property nodeScopeIndex="false">analyticsProvider</property>
<property nodeScopeIndex="false">analyticsSnippet</property>
...
<property isRegexp="true">.*:.*</property>
</index-rule>
Best regards
The default configuration is ideed incorrect as confirmed by the Adobe CQ5 Support.
For the aggegate to work correctly properties must be included by the "include-property"-Tag
So the default search configuration (or atleast the documentation) is not correct https://helpx.adobe.com/experience-manager/kb/SearchIndexingConfig.html)
I'm using last version of Jasig CAS server (4.0.0) with an LDAP server.
Users are stored under this LDAP structure : ou=Users,ou=SSOTEST,dc=mycompany,dc=com
What I want is to search an user from a top level (example : ou=SSOTEST,dc=mycompany,dc=com).
CAS server has an LdapPersonAttributeDao bean which is looking for an object matching a search filter. Here is the code for this bean :
<bean id="ldapPersonAttributeDao"
class="org.jasig.cas.persondir.LdapPersonAttributeDao"
p:connectionFactory-ref="searchPooledLdapConnectionFactory"
p:baseDN="ou=SSOTEST,dc=company,dc=com"
p:searchControls-ref="searchControls"
p:searchFilter="uid={0}">
<property name="resultAttributeMapping">
<map>
<!--
| Key is LDAP attribute name, value is principal attribute name.
-->
<entry key="memberOf" value="userMemberOf" />
<entry key="cn" value="userCn" />
</map>
</property>
</bean>
And now the searchControls bean which do a lookup at SUBTREE_SCOPE (2) level (according toSearchControls scope level values).
<bean id="searchControls"
class="javax.naming.directory.SearchControls"
p:searchScope="2"
p:countLimit="10" />
When I run my CAS server and I try to authenticate, everything works but there are no extra attributes returned.
I think the problem comes from searchScope, which don't seems to be set to wanted value.
Here is output log from the server :
<execute request=[org.ldaptive.SearchRequest#-1312441815::baseDn=ou=SSOTEST,dc=mycompany,dc=com, searchFilter=[org.ldaptive.SearchFilter#-3391
91059::filter=uid={0}, parameters={0=myuser}], returnAttributes=[], searchScope=null, timeLimit=0, sizeLimit=10 [...]
I know its been some time since this question was asked. But I managed to fix this problem by adding:
<bean class="org.springframework.context.annotation.CommonAnnotationBeanPostProcessor" />
to deployerConfigContext.xml.
The cause of this issue was that the initalize method in LdapPersonAttributeDao was not being invoked because the #PostConstruct annotation wasn't being executed. For this reason the searchScope variable was never set.
I'm using Apache Camel 2.13.1 to poll a database table which will have upwards of 300k rows in it. I'm looking to use the Idempotent Consumer EIP to filter rows that have already been processed.
I'm wondering though, whether the implementation is really scalable or not. My camel context is:-
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route id="main">
<from
uri="sql:select * from transactions?dataSource=myDataSource&consumer.delay=10000&consumer.useIterator=true" />
<transacted ref="PROPAGATION_REQUIRED" />
<enrich uri="direct:invokeIdempotentTransactions" />
<!-- Any processors here will be executed on all messages -->
</route>
<route id="idempotentTransactions">
<from uri="direct:invokeIdempotentTransactions" />
<idempotentConsumer
messageIdRepositoryRef="jdbcIdempotentRepository">
<ognl>#{request.body.ID}</ognl>
<!-- Anything here will only be executed for non-duplicates -->
<log message="non-duplicate" />
<to uri="stream:out" />
</idempotentConsumer>
</route>
</camelContext>
It would seem that the full 300k rows are going to be processed every 10 seconds (via consumer.delay parameter) which seems very inefficient. I would expect some sort of feedback loop as part of the pattern so that the query that feeds the filter could take advantage of the set of rows already processed.
However, the messageid column in the CAMEL_MESSAGEPROCESSED table has the pattern of
{1908988=null}
where 1908988 is the request.body.ID I've set the EIP to key on so this doesn't make it easy to incorporate into my query.
Is there a better way of using the CAMEL_MESSAGEPROCESSED table as a feedback loop into my select statement so that the SQL server is performing most of the load?
Update:
So, I've since found out that it was my ognl code that was causing the odd message id column value. Changing it to
<el>${in.body.ID}</el>
has fixed it. So, now that I have a usable messageId column, I can now change my 'from' SQL query to
select * from transactions tr where tr.ID IN (select cmp.messageid from CAMEL_MESSAGEPROCESSED cmp where cmp.processor = 'transactionProcessor')
but I still think I'm corrupting the Idempotent Consumer EIP.
Does anyone else do this? Any reason not to?
Yes, it is. But you need to use scalable storage for holding sets of already processed messages. You can use either Hazelcast - http://camel.apache.org/hazelcast-idempotent-repository-tutorial.html or Infinispan - http://java.dzone.com/articles/clustered-idempotent-consumer - depending on which solution is already in your stack. Of course, JDBC repository would work, but only if it meets performance criteria selected.
I'm using Jboss 7.1.1.final and I would like to define 2 different DLQ's, one for a certain queue and the other for all the rest of the queues.
I found this configuration :
<address-settings>
<address-setting match="jms.queue.exampleQueue">
<dead-letter-address>jms.queue.deadLetterQueue</dead-letter-address>
<max-delivery-attempts>3</max-delivery-attempts>
<redelivery-delay>5000</redelivery-delay>
<expiry-address>jms.queue.expiryQueue</expiry-address>
<last-value-queue>true</last-value-queue>
<max-size-bytes>100000</max-size-bytes>
<page-size-bytes>20000</page-size-bytes>
<redistribution-delay>0</redistribution-delay>
<send-to-dla-on-no-route>true</send-to-dla-on-no-route>
<address-full-policy>PAGE</address-full-policy>
</address-setting>
</address-settings>
The match attribute can be used to match a certain queue, I have a couple of questions regarding this configuration:
If I define 2 address-setting, one with a wild card to match all and one that matches only one queue, does the one queue definition take precedence? Do i need to put it before the match all definition or it does not matter?
In the example they match a queue jms.queue.exampleQueue, i have a queue defined as:
<jms-queue name="MissionResult">
<entry name="queue/MissionResult"/>
</jms-queue>
what should i put in the match attribute in order to match it?
Found the answer:
The 2 definitions can co-exist. Jboss will find the best match.
You need to define a queue like:
<jms-queue name="exampleQueue">
<entry name="queue/exampleQueue" />
</jms-queue>
and then to match this queue, use jms.queue.exampleQueue.
(Context: I'm an experienced programmer, but new to LDAP, AD and Spring.)
We are a Windows shop, so all of our authentication is done with Active Directory. We are attempting to integrate a third-party product that is written in Java, so it does all of its authentication using Spring Security. So far, so good -- they've done that integration before, and there's a good deal online about how to set things up.
The problem is, our AD setup is a bit complex: in particular, our user accounts exist in various nodes in the AD/LDAP tree. To give a simplified example, say the LDAP tree looks like this:
DC=my-domain,DC=com
+ CN=Users
++ CN=user1,CN=Users,DC=my-domain,DC=com
+ CN=Staff
++ CN=user2,CN=Staff,DC=my-domain,DC=com
The thing is, all of the examples I have found let me authenticate either user1 or user2, but not both. That is, the following XML snippet will work to authenticate user1 against roles defined under "Groups":
<security:ldap-server url="ldap://my-domain.com:389" manager-dn="CN=manager_svc,OU=System Users,DC=my-domain,DC=com" manager-password="MyPa55w0rd"/>
<security:ldap-authentication-provider
user-dn-pattern=""
user-search-base="CN=Users,DC=my-domain,DC=com"
user-search-filter="(&(sAMAccountName={0})(objectclass=user))"
group-search-base="OU=Groups,DC=mydomain,DC=com"
group-search-filter="member={0}"
/>
but that won't authenticate user2, since he doesn't match the user-search-base. Contrariwise, I can change user-search-base to CN=Staff,DC=my-domain,DC=com, which will work for user2, but then it won't work for user1.
So the question is, how do I make this search work for user accounts that are scattered across the AD/LDAP tree? I can imagine two possibilities, but I haven't figured out how to do either yet:
On the one hand, if I can make user-search-base multi-valued, that solves my problem easily and correctly: I just put in all of the locations where user accounts might be found. So far, all of my attempts to do this have met with one error or another, but I'm still experimenting.
OTOH, there is Subtree scoping of the search. I can see in the interactive LDAP tools that search can be either single-level or subtree. Far as I can tell, Spring out of the box is doing single-level. I can see that the underlying FilterBasedLdapUserSearch class has a setSearchSubtree() method, which looks like what I want, but I can't find a way to set that to true from the XML. (For now, let's assume that it isn't feasible to change the underlying Java program.)
The first option would be ideal, since it is probably much more efficient, but if that isn't possible and the second is, I suspect we can make it work.
I have a suspicion that the second approach is possible using thorny bean hackery, but I know next to nothing about beans, so I'd rather not wade into those thickets by myself. Does anybody have a good recipe to recommend?
Thanks much for any guidance you can provide...
I solved this by using a searchBase with the empty string value (this uses the root as the searchbase, just like prule's answer), but I also had to set the property "referral" to "follow", otherwise I got a PartialResultException!
DefaultSpringSecurityContextSource contextSource = new DefaultSpringSecurityContextSource(...);
contextSource.setReferral("follow");
You could try searching from the domain root, if that is feasible, though that can cause problems with AD.
Alternatively, use of explicit bean configuration is probably your best option. You can inject a custom LdapUserSearch implementation into the BindAuthenticator bean, which searches under all the necessary locations. If you look at the example in the docs, it shows a FilterBasedLdapUserSearch configuration. You could either use a couple of these, or implement the interface yourself from scratch. Here's a quick hack as an example:
public class CustomLdapSearch implements LdapUserSearch {
public static final String SAM_FILTER="(&(sAMAccountName={0})(objectclass=user))"
final LdapUserSearch users;
final LdapUserSearch staff;
public CustomLdapSearch(BaseLdapPathContextSource contextSource) {
users = new FilterBasedLdapUserSearch("CN=Users,DC=my-domain,DC=com", SAM_FILTER, contextSource);
staff = new FilterBasedLdapUserSearch("CN=Staff,DC=my-domain,DC=com", SAM_FILTER, contextSource);
}
public DirContextOperations searchForUser(String username) {
try {
return users.searchForUser(username);
} catch(UsernameNotFoundException e) {
return staff.searchForUser(username);
}
}
}
Then change the BindAuthenticator configuration to:
<bean class="org.springframework.security.ldap.authentication.BindAuthenticator">
<constructor-arg ref="contextSource"/>
<property name="userSearch" ref="customSearch"/>
</bean>
<bean id="customSearch" class="CustomLdapSearch">
<constructor-arg ref="contextSource"/>
</bean>
I've done something similar with spring-security-2.0.x using FilterBasedLdapUserSearch - where users were spread across multiple nodes:
<bean id="ldapUserSearch" class="org.springframework.security.ldap.search.FilterBasedLdapUserSearch">
<constructor-arg value=""/> <!-- optional sub-tree here -->
<constructor-arg value="(&(sAMAccountName={0})(objectclass=user))"/>
<constructor-arg ref="contextSource"/>
</bean>
<bean id="ldapAuthProvider"
class="org.springframework.security.providers.ldap.LdapAuthenticationProvider">
<constructor-arg>
<bean class="org.springframework.security.providers.ldap.authenticator.BindAuthenticator">
<constructor-arg ref="contextSource"/>
<property name="userSearch" ref="ldapUserSearch"/>
</bean>
</constructor-arg>
<property name="userDetailsContextMapper" ref="userDetailsContextMapper"/>
</bean>
<bean id="contextSource"
class="org.springframework.security.ldap.DefaultSpringSecurityContextSource">
<constructor-arg value="ldap://localhost:10389/CN=Users,DC=my-domain,DC=com"/>
<!-- you may or may not need to connect with an account that can search -->
<!--<property name="userDn" value="uid=admin,ou=system"/>-->
<!--<property name="password" value="secret"/>-->
</bean>