My requirement Is to each index per tenant , I do already have hibernate configured as MultiTenant , I need to index the database per tenant to different Index. I have seen to dynamicSharding Strategy.. But requirement is dynamic. it can have n number of shards without being any pre information about each tenant existence. Even My Indexer should work in a way that. Whenever it finds the hibernate request fetching the index it should index that tenant first and then search over it.
How i can do it..??
Can anybody give me some example.
Please don't give hibernate doc links... or even Jboss doc links for hibernate search.
To index over a specific tenant you should add the properties of hibernate search of section Entitymanager on your application context, as follow as an example:
<bean depends-on="dataSource" id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="databasePlatform" value="org.hibernate.dialect.SQLServerDialect" />
<property name="generateDdl" value="true"></property>
<property name="showSql" value="true" />
</bean>
</property>
<property name="packagesToScan" value="xxx.xxxx..xxx." />
<property name="persistenceUnitName" value="xxxxxx" />
<property name="dataSource" ref="dataSource" />
<property name="jpaProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.SQLServerDialect</prop>
<prop key="hibernate.search.default.directory_provider">filesystem</prop>
<prop key="hibernate.search.default.indexBase">C:\xxxxx\indexes</prop>
</props>
</property>
</bean>
Now you have to use org.hibernate.Session to identify your tenant to perform a search to a specific tenant as follow:
EntityManager manager = managerFactory.createEntityManager();
SessionImpl i = (SessionImpl) manager.getDelegate();
SessionFactory session = i.getSessionFactory();
Session s = session.withOptions().tenantIdentifier(xxxxx).openSession();
FullTextSession fullTextSession = org.hibernate.search.Search.getFullTextSession(s);
return fullTextSession;
This way each search will use a specifc tenant that you provide, but for me seems that you have another problem that is how to get the tenant, in my case I use the url to identify the tenant, if you have anyway that a user choose te tenant to use, get that information in a static variable to use when necessary.
Related
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="mycache"/>
<!-- Configure query entities -->
<property name="queryEntities">
<list>
<bean class="org.apache.ignite.cache.QueryEntity">
<!-- Setting indexed type's key class -->
<property name="keyType" value="java.lang.Long"/>
<!-- Setting indexed type's value class -->
<property name="valueType"
value="org.apache.ignite.examples.Person"/>
<!-- Defining fields that will be either indexed or queryable.
Indexed fields are added to 'indexes' list below.-->
<property name="fields">
<map>
<entry key="id" value="java.lang.Long"/>
<entry key="name" value="java.lang.String"/>
<entry key="salary" value="java.lang.Long "/>
</map>
</property>
<!-- Defining indexed fields.-->
<property name="indexes">
<list>
<!-- Single field (aka. column) index -->
<bean class="org.apache.ignite.cache.QueryIndex">
<constructor-arg value="id"/>
</bean>
<!-- Group index. -->
<bean class="org.apache.ignite.cache.QueryIndex">
<constructor-arg>
<list>
<value>id</value>
<value>salary</value>
</list>
</constructor-arg>
<constructor-arg value="SORTED"/>
</bean>
</list>
</property>
</bean>
</list>
</property>
</bean>
I understand the above XML configuration can be used to define an SQL entity in ignite with indexes. The documentation is better understandable from a code perspective either Java or NET because API is available. As we do most of the development C++ and API is not available , we would like to know few more details to use the XML configuration. Could anyone please answer below points?
1.Where does this configuration file can be used? Server side or client side (thin & thick) or both side.
2.Is it possible to change the field names, types and indexes once it has been created and loaded data in the same entity?#
3.<property name="valueType" value="org.apache.ignite.examples.Person"/> If not mistaken, we understand the value here is taken from a namespace and from a DLL (for example in c#) but How does ignite knows about the location of DLL or namespace to get load from? where does the binaries to be kept?
4.In the case of C++ , what binary file can be used to define the value type? .lib or .dll or some other way.
C++ Thick Client can use the XML config, see IgniteConfiguration.springCfgPath.
Think about the CacheConfiguration as the "starting" config for a cache. Most of it can't be changed later. A few things, like the set of SQL columns or indexes, can be changed via SQL DDL: ALTER TABLE..., CREATE INDEX..., etc. If something isn't available in the DDL, assume that it can't be changed without recreating the cache.
Check out this. The value type name will be mapped by each platform component - Java, C++, .NET - accordingly to the binary marshaller configuration. For example, it's common to use BinaryBasicNameMapper that will map all platform type names (with namespaces/packages) to simple names, so that different namespace/package naming conventions don't create a problem. When a class is needed to deserialize a value, it will be loaded via the regular platform-specific mechanism to load code. For Java, it'll be the classpath. For C++, I guess it's LD_LIBRARY_PATH. In any case, Ignite has nothing to do with that really.
Again, Ignite has nothing to do with that. Whatever works on your platform to load code can be used.
After few experiments, I found the solution and actually it is easy.
The value given at the valueType property is directly mapped to the binary object name when it create from the code.
for e.g below configuration
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="TEST"/>
<property name="cacheMode" value="PARTITIONED"/>
<property name="atomicityMode" value="TRANSACTIONAL"/>
<property name="writeSynchronizationMode" value="FULL_SYNC"/>
<!-- Configure type metadata to enable queries. -->
<property name="queryEntities">
<list>
<bean class="org.apache.ignite.cache.QueryEntity">
<property name="keyType" value="java.lang.Long"/>
<property name="valueType" value="TEST"/>
<property name="fields">
<map>
<entry key="ID" value="java.lang.Long"/>
<entry key="DATE" value="java.lang.String"/>
</map>
</property>
</bean>
</list>
</property>
</bean>
the below C++ code works
template<>
struct BinaryType<examples::TEST> : BinaryTypeDefaultAll<examples::TEST>
{
static void GetTypeName(std::string& dst)
{
dst = "TEST";
}
static void Write(BinaryWriter& writer, const examples::RHO& obj)
{
writer.WriteInt64("ID", obj.Id);
writer.WriteString("DATE", obj.dt);
}
static void Read(BinaryReader& reader, examples::RHO& dst)
{
dst.Id = reader.ReadInt64("Id");
dst.dt = reader.ReadString("dt");
}
};
I'm using Ignite 2.5 and have deployed a couple of servers like this:
One computer acts as DB server with persistence enabled.
Three other computers are compute servers with same cache as on DB server but without persistence.
I have classes like this:
public class Address implements Serializable
{
String streetName;
String houseNumber;
String cityName;
String countryName;
}
public class Person implements Serializable
{
#QuerySqlField
String firstName;
#QuerySqlField
String lastName;
#QuerySqlField
Address homeAddress;
}
The cache is configured on all servers with this XML:
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="Persons" />
<property name="cacheMode" value="PARTITIONED" />
<property name="backups" value="0" />
<property name="storeKeepBinary" value="true" />
<property name="atomicityMode" value="TRANSACTIONAL"/>
<property name="writeSynchronizationMode" value="FULL_SYNC"/>
<property name="indexedTypes">
<list>
<value>java.lang.String</value>
<value>Person</value>
</list>
</property>
</bean>
On the DB server in addition there is persistence enabled like this:
<property name="dataStorageConfiguration">
<bean class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="storagePath" value="/data/Storage" />
<property name="walPath" value="/data/Wal" />
<property name="walArchivePath" value="/data/WalArchive" />
<property name="defaultDataRegionConfiguration">
<bean class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="initialSize" value="536870912" />
<property name="maxSize" value="1073741824" />
<property name="persistenceEnabled" value="true" />
</bean>
</property>
</bean>
</property>
<property name="binaryConfiguration">
<bean class="org.apache.ignite.configuration.BinaryConfiguration">
<property name="compactFooter" value="false" />
</bean>
</property>
The cache is used with put/get but also with SqlQuery and SqlFieldsQuery.
From time to time I have to update the class definitions, i.e. add another field or so. I'm fine to shut down the whole cluster for updating the classes as it requires an application update anyway.
I believe the above configuration is generally OK to use for Ignite?
Do I understand this other question (Apache Ignite persistent store recommended way for class versions) correctly that on the DB server I shall not have the Person classes in the classpath? Wouldn't then the XML config fail because it's missing the index classes?
On compute servers I shall also not use the Person classes but instead read from cache into BinaryObject? Is the idea to manually fill my Person class from the BinaryObject?
Currently when I update a field in the Person class I get strange errors like:
Unknown pair [platformId=0, typeId=1968448811]
Sorry if there are multiple questions here, I somehow am lost with the "Unknown pair" issues and am now questioning if my complete setup is right.
Thanks for any advise.
I believe the above configuration is generally OK to use for Ignite?
No, you can't configure persistence only for one node only. So in your case, all nodes will store data, but only one node will persist its data, so only part of data will be persisted and this can lead to unpredictable consequences. If you want only one node to store data you need to configure node filter.
With the node filter, the cache will be located only on one node and this node will store data, however in this case your compute nodes would have to do network IO to read from cache.
Do I understand this other question (Apache Ignite persistent store
recommended way for class versions) correctly that on the DB server I
shall not have the Person classes in the classpath? Wouldn't then the
XML config fail because it's missing the index classes?
You don't need classes of your model to be in the classpath, but please, make sure that you work with BinaryObjects only on the server side, so all compute tasks should use BinaryObjects. Also as you mentioned, this configuration won't work, you need to use Query Entity instead for index configuration.
On compute servers I shall also not use the Person classes but instead read from cache into BinaryObject? Is the idea to manually fill my Person class from the BinaryObject?
Well, if you don't have the Person class on the server side you just can't create Person class, you need to use BinaryObject in your compute jobs.
Currently when I update a field in the Person class I get strange errors like: Unknown pair [platformId=0, typeId=1968448811]
Could you please provide the full stacktrace and say on what operation you get this error?
I have created an Ignite cache "contact" and added "Person" object to it.
When I use Ignite JDBC Client mode I am able to query this cache. But when I implement JDBC Thin Client, it says that the table Person does not exist.
I tried the query this way:
Select * from Person
Select * from contact.Person
Both did not work with Thin Client. I am using Ignite 2.1.
I appreciate your help as how to query an existing cache using Thin Client.
Thank you.
Cache Configuration in default-config.xml
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<!-- Enabling Apache Ignite Persistent Store. -->
<property name="persistentStoreConfiguration">
<bean class="org.apache.ignite.configuration.PersistentStoreConfiguration"/>
</property>
<property name="binaryConfiguration">
<bean class="org.apache.ignite.configuration.BinaryConfiguration">
<property name="compactFooter" value="false"/>
</bean>
</property>
<property name="memoryConfiguration">
<bean class="org.apache.ignite.configuration.MemoryConfiguration">
<!-- Setting the page size to 4 KB -->
<property name="pageSize" value="#{4 * 1024}"/>
</bean>
</property>
<!-- Explicitly configure TCP discovery SPI to provide a list of initial nodes. -->
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>127.0.0.1:55500..55502</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
</beans>
Cache Configuration in the Server Side of the Code
CacheConfiguration<Long, Person> cc = new CacheConfiguration<>(cacheName);
cc.setCacheMode(CacheMode.REPLICATED);
cc.setRebalanceMode(CacheRebalanceMode.ASYNC);
cc.setIndexedTypes(Long.class, Person.class);
cache = ignite.getOrCreateCache(cc);
Thin Client JDBC URL
Class.forName("org.apache.ignite.IgniteJdbcThinDriver");
// Open the JDBC connection.
Connection conn = DriverManager.getConnection("jdbc:ignite:thin://192.168.1.111:10800");
Statement st = conn.createStatement();
If you want to query data from an existing cache using SQL, you should specify an SQL schema in the cache configuration. Add the following code before the cache creation:
cc.setSqlSchema("PUBLIC");
Note that you have persistence configured, so when you do ignite.getOrCreateCache(cc); the new configuration won't be applied, if a cache with this name is already persisted. You should, for example, remove persistence data or use createCache(...) method instead.
I have one use case where I have to support multiple persistence store for my ignite cluster,For example Cache A1 should be primed from Database db1 and Cache B1 should be primed from database db2. can this be done?.In ignite Configuration XML I can only provide one persistence store details,
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<!-- Datasource for Persistence. -->
<bean name="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#localhost:1521:roc12c" />
<property name="username" value="test" />
<property name="password" value="test" />
</bean>
In my CacheStore implementation I can only access this Database right?.
I've not tried this, but if its similar to other bean-configured systems. You should be able to create another bean with a different name and configuration. Then in your cache configuration for A1 and B1 specify the different data sources. That being said, I'm guessing that theoretically.
It may be that you are already doing so, but I can't tell from your question. If you instead choose to implement your caches in this manner https://apacheignite.readme.io/docs/persistent-store you can definitely configure two caches to have different data sources. This is how I'm currently implementing multiple caches. In the cache store I use I specifically call out which database to go to.
Here is a cache configuration I use for mine.
<property name="cacheConfiguration">
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<!-- Set a cache name. -->
<property name="name" value="recordData"/>
<property name="rebalanceMode" value="ASYNC"/>
<property name="cacheMode" value="PARTITIONED"/>
<property name="backups" value="1"/>
<!-- Enable Off-Heap memory with max size of 10 Gigabytes (0 for unlimited). -->
<property name="memoryMode" value="OFFHEAP_TIERED"/>
<property name="offHeapMaxMemory" value="0"/>
<property name="swapEnabled" value="false"/>
<property name="cacheStoreFactory">
<bean class="javax.cache.configuration.FactoryBuilder" factory-method="factoryOf">
<constructor-arg value="com.company.util.MyDataStore"/>
</bean>
</property>
<property name="readThrough" value="true"/>
<property name="writeThrough" value="true"/>
</bean>
</property>
Cache store is configured per cache, so you just need to inject different data sources to different stores. What you showed is just a standalone data source bean, it's not even a part of IgniteConfiguration. You can have multiple data source beans with different IDs.
I changed the discovery.xml file as described in the documentation to add a new facet over dc.type to our DSpace. When I finished reindexing and deleting the cache I see the new search filter at advanced search but not as a facet.
These are the changes I made to discovery.xml:
Added filter to sidbarFacets and SearchFilter:
<ref bean="searchFilterType" />
and this is the filter:
<bean id="searchFilterType" class="org.dspace.discovery.configuration.DiscoverySearchFilterFacet">
<property name="indexFieldName" value="type"/>
<property name="metadataFields">
<list>
<value>dc.type</value>
</list>
</property>
</bean>
Thanks in advance
The following modifications to discovery.xml on the latest DSpace master branch worked on my local setup:
https://github.com/bram-atmire/DSpace/commit/3f084569cf1bbc6c6684d114a09a1617c8d3de5d
One reason why the facet wouldn't appear in your setup, could be that you omitted to add it to both the "defaultconfiguration" as well as the specific configuration for the DSpace homepage.
After building and deploying, a forced discovery re-index using the following command made the facet appear:
./dspace index-discovery -f
Here is an example facet that I have configured in our instance. Try setting the facetLimit, sortOrder, and splitter. Re-index and see if that resolves the issue.
<bean id="searchFilterGeographic"
class="org.dspace.discovery.configuration.HierarchicalSidebarFacetConfiguration">
<property name="indexFieldName" value="geographic-region"/>
<property name="metadataFields">
<list>
<value>dc.coverage.spatial</value>
</list>
</property>
<property name="facetLimit" value="5"/>
<property name="sortOrder" value="COUNT"/>
<property name="splitter" value="::"/>
</bean>