Extend a Custom Apache Knox Service to query multiple HBase tables - apache

Question

I'm currently working on extending the functionality of Apache Knox interacting with HBase on HDP 2.3.2. I've created a new gateway on Apache Knox called Decode, to query HBase. The Decode Gateway was built using the HBase gateway as a template. Topologies have been edited such that the following query:
curl -ku admin:admin-password -H "Accept: application/json" https://sandbox.hortonworks.com:8443/gateway/default/decode/hbase/MyHBaseTable/HBaseRowKey123*
Will return the data from Row 123 (In Base 64)
Is there way to change the Decode gateway's rewrite.xml and service.xml so that the query would not need to go through Hbase, e.g:
curl -ku admin:admin-password -H "Accept: application/json" https://sandbox.hortonworks.com:8443/gateway/default/decode/MyHBaseTable/HBaseRowKey123*
I understand this may sound strange to not use in inbuilt HBase gateway but the overall aim is to extend the Decode gateway such that It would be able to query multiple tables and/or multiple rows of data from HBase, rather than one row at a time.
Currently my Decode rewrite.xml is:
<rules>
<rule dir="IN" name="DECODE/decode/inbound" pattern="*://*:*/**/decode/{path=**}?{**}">
<rewrite template="{$serviceUrl[DECODE]}/{path=**}?{**}"/>
</rule>
<rule dir="IN" name="DECODE/decode/inbound" pattern="*://*:*/**/decode{**}">
<rewrite template="{$serviceUrl[DECODE]}/{path=**}?{**}"/>
</rule>
<filter name="WEBHBASE/webhbase/status/outbound">
<content type="*/json">
<apply path="$[LiveNodes][*][name]" rule="WEBHBASE/webhbase/address/outbound"/>
</content>
<content type="*/xml">
<apply path="/ClusterStatus/LiveNodes/Node/#name" rule="WEBHBASE/webhbase/address/outbound"/>
</content>
</filter>
</rules>
and the service.xml:
<service role="DECODE" name="decode" version="0.0.1">
<routes>
<route path="/decode/**"/>
<route path="/decode/?**">
<rewrite apply="WEBHBASE/webhbase/headers/outbound" to="response.headers"/>
</route>
<route path="/decode/**?**">
<rewrite apply="WEBHBASE/webhbase/headers/outbound" to="response.headers"/>
</route>
<route path="/decode/status/cluster?**">
<rewrite apply="WEBHBASE/webhbase/status/outbound" to="response.body"/>
</route>
<route path="/decode/*/regions?**">
<rewrite apply="WEBHBASE/webhbase/regions/outbound" to="response.body"/>
</route>
</routes> <dispatch classname="org.apache.hadoop.gateway.hbase.HBaseDispatch"/> </service>

Answer 1

Just like #lmccay I don't see anything necessarily wrong with what you have shown so I decided to try it for myself.
In the default topology file I copied the WEBHBASE service to a DECODE service as shown.
/usr/hdp/current/knox-server/conf/topologies/default.xml
<service>
<role>WEBHBASE</role>
<url>http://my-hbase-hostname:60080</url>
</service>
<service>
<role>DECODE</role>
<url>http://my-hbase-hostname:60080</url>
</service>
I then copied the hbase service.xml and rewrite.xml from
/usr/hdp/current/knox-server/data/services/hbase/0.98.0
to a new directory
/usr/hdp/current/knox-server/data/services/decode/0.0.1
and modified them to look like the versions shown below. I basically just replaced all occurrences of hbase with decode using the matching case.
/usr/hdp/current/knox-server/data/services/decode/0.0.1/service.xml
<service role="DECODE" name="decode" version="0.0.1">
<routes>
<route path="/decode/?**">
<rewrite apply="DECODE/decode/headers/outbound" to="response.headers"/>
</route>
<route path="/decode/**?**">
<rewrite apply="DECODE/decode/headers/outbound" to="response.headers"/>
</route>
<route path="/decode/status/cluster?**">
<rewrite apply="DECODE/decode/status/outbound" to="response.body"/>
</route>
<route path="/decode/*/regions?**">
<rewrite apply="DECODE/decode/regions/outbound" to="response.body"/>
</route>
</routes>
<dispatch classname="org.apache.hadoop.gateway.hbase.HBaseDispatch"/>
<testURLs>
<testURL>/decode/version</testURL>
<testURL>/decode/version/cluster</testURL>
<testURL>/decode/status/cluster</testURL>
<testURL>/decode</testURL>
</testURLs>
</service>
/usr/hdp/current/knox-server/data/services/decode/0.0.1/rewrite.xml
<rules>
<rule dir="IN" name="DECODE/decode/root/inbound" pattern="*://*:*/**/decode/?{**}">
<rewrite template="{$serviceUrl[DECODE]}/?{**}"/>
</rule>
<rule dir="IN" name="DECODE/decode/path/inbound" pattern="*://*:*/**/decode/{path=**}?{**}">
<rewrite template="{$serviceUrl[DECODE]}/{path=**}?{**}"/>
</rule>
<rule name="DECODE/decode/location/outbound">
<match pattern="*://*:*/{path=**}?{**}"/>
<rewrite template="{$frontend[url]}/decode/{path=**}?{**}"/>
</rule>
<rule name="DECODE/decode/address/outbound">
<match pattern="{host}:{port}"/>
<rewrite template="{$frontend[url]}/hbase-region?host={host}?port={port}"/>
<encrypt-query/>
</rule>
<filter name="DECODE/decode/headers/outbound">
<content type="application/x-http-headers">
<apply path="Location" rule="DECODE/decode/location/outbound"/>
</content>
</filter>
<filter name="DECODE/decode/status/outbound">
<content type="*/json">
<apply path="$[LiveNodes][*][name]" rule="DECODE/decode/address/outbound"/>
</content>
<content type="*/xml">
<apply path="/ClusterStatus/LiveNodes/Node/#name" rule="DECODE/decode/address/outbound"/>
</content>
</filter>
<filter name="DECODE/decode/regions/outbound">
<content type="*/json">
<apply path="$[Region][*][location]" rule="DECODE/decode/address/outbound"/>
</content>
<content type="*/xml">
<apply path="/TableInfo/Region/#location" rule="DECODE/decode/address/outbound"/>
</content>
</filter>
</rules>
I was then able to query a table with the curl command below. This does not contain hbase in the URL I believe answering your question.
curl -k -u guest:******** -H 'Accept: application/json' https://localhost:8443/gateway/default/decode/t1/*
{"Row":[{"key":"ZjE6djE=","Cell":[{"column":"ZjI6djI=","timestamp":1453139991995,"$":"ZjM6djM="}]}]}