AWS Gateway + x-www-form-urlencoded breaks with empty field - velocity

I followed this awesome resource here -- https://forums.aws.amazon.com/thread.jspa?messageID=673012 -- but it throws an internal server error if the data has empty fields (which mine will, and its beyond my control).
Then went searching and implemented this more recently updated library from Christian E Willman - https://github.com/christianewillman/aws-api-gateway-bodyparser
Despite being more terse, it's still failing when I send empty fields (via Postman). Any tips on getting it working would be greatly appreciated!
Here's the mapping template for reference:
## Parses x-www-urlencoded data to JSON for AWS' API Gateway
##
## Author: Christian E Willman <christian#willman.io>
#if ( $context.httpMethod == "POST" )
#set( $requestBody = $input.path('$') )
#else
#set( $requestBody = "" )
#end
#set( $keyValuePairs = $requestBody.split("&") )
#set( $params = [] )
## Filter empty key-value pairs
#foreach( $kvp in $keyValuePairs )
#set( $operands = $kvp.split("=") )
#if( $operands.size() == 1 || $operands.size() == 2 )
#set( $success = $params.add($operands) )
#end
#end
{
## add in the API stage data
"stage": "$context.stage",
#foreach( $param in $params )
#set( $key = $util.urlDecode($param[0]) )
#if( $param.size() > 1 )
#set( $value = $util.urlDecode($param[1]) )
#else
#set( $value = "" )
#end
"$key": "$value"#if( $foreach.hasNext ),#end
#end
}
Edit: This is the data I'm sending in:
WSO_SIGNATURE=1d9ce78c2778ecf795a93009a09f8102dd4ef38a&WP_BUYER_NAME=Neil+calabroso&WP_ITEM_NAME=Welcome+to+the+Vault&SHIPTONAME=Neil+calabroso&WSO_PRODUCT_ID=wp_product_3&WP_AFFID=&WSO_SALE_CURRENCY=USD&EMAIL=nn%2Bpaypal%40freelancer.com&WP_ITEM_NUMBER=wp_product_3&PAYMENTSTATUS=COMPLETED&WP_BUYER_EMAIL=nn%2Bpaypal%40freelancer.com&WSO_PRODUCT_NAME=Welcome+to+the+Vault&WP_SALEID=wp_sale_3&AMT=43&WSO_CUSTOMER_EMAIL=nn%2Bpaypal%40freelancer.com&WSO_SALE_ACTION=SALE&WP_SALE_AMOUNT=43&WSO_TXN_ID=35&saleid=wp_sale_3&WSO_AFF_AMOUNT=0.00&WP_ACTION=sale&WSO_IPN_VERSION=1&WP_TXNID=35&WSO_SALE_ID=wp_sale_3&payer_email=nn%2Bpaypal%40freelancer.com&WP_SALE_CURRENCY=USD&WSO_SALE_AMOUNT=43&WSO_AFF_EMAIL=&TRANSACTIONID=35&WSO_CUSTOMER_NAME=Neil+calabroso
The expected output is a series of JSON key-value pairs:
{
"WSO_SIGNATURE": "1d9ce78c2778ecf795a93009a09f8102dd4ef38a",
"WP_BUYER_NAME": "Neil calabroso",
"WP_ITEM_NAME": "Welcome to the Vault",
"SHIPTONAME": "Neil calabroso",
"WSO_PRODUCT_ID": "wp_product_3",
"WP_AFFID": "",
"WSO_SALE_CURRENCY": "USD",
"EMAIL": "nn+paypal#freelancer.com",
"WP_ITEM_NUMBER": "wp_product_3",
"PAYMENTSTATUS": "COMPLETED",
"WP_BUYER_EMAIL": "nn+paypal#freelancer.com",
"WSO_PRODUCT_NAME": "Welcome to the Vault",
"WP_SALEID": "wp_sale_3",
"AMT": "43",
"WSO_CUSTOMER_EMAIL": "nn+paypal#freelancer.com",
"WSO_SALE_ACTION": "SALE",
"WP_SALE_AMOUNT": "43",
"WSO_TXN_ID": "35",
"saleid": "wp_sale_3",
"WSO_AFF_AMOUNT": "0.00",
"WP_ACTION": "sale",
"WSO_IPN_VERSION": "1",
"WP_TXNID": "35",
"WSO_SALE_ID": "wp_sale_3",
"payer_email": "nn+paypal#freelancer.com",
"WP_SALE_CURRENCY": "USD",
"WSO_SALE_AMOUNT": "43",
"WSO_AFF_EMAIL": "",
"TRANSACTIONID": "35",
"WSO_CUSTOMER_NAME": "Neil calabroso"
}

Ended up finding this mapping template from #marcus-whybrow solved the problem -- How to pass a params from POST to AWS Lambda from Amazon API Gateway
{
#foreach( $token in $input.path('$').split('&') )
#set( $keyVal = $token.split('=') )
#set( $keyValSize = $keyVal.size() )
#if( $keyValSize >= 1 )
#set( $key = $util.urlDecode($keyVal[0]) )
#if( $keyValSize >= 2 )
#set( $val = $util.urlDecode($keyVal[1]) )
#else
#set( $val = '' )
#end
"$key": "$val"#if($foreach.hasNext),#end
#end
#end
}

Related

Using VTL Velocity, how can I create VALUES array to use in SQL Statement

I am trying to create an SQL Statement in my AWS resolver which uses Values.
For example:
#set( $inserts = [] )
#foreach ( $item in $ctx.args.input.listItems)
$util.qr($inserts.add( ['$item.item_id', '$item.item_name', 'item.item_image_id'] ))
#end
And my SQL Statement is:
#set( $sql = "INSERT INTO items (item_id, item_name, item_image_id) VALUES $inserts" )
However this is not working. I there another way to create this type of SQL Statement?
Thanks
I'd do this way:
#set( $sql = "INSERT INTO items (item_id, item_name, item_image_id) VALUES " )
#foreach ( $item in $ctx.args.input.listItems)
#set( $sql = $sql + "($item.item_id, $item.item_name, $item.item_image_id)")
#if( $foreach.hasNext )
#set( $sql = $sql + ", ")
#else
#set( $sql = $sql + ";")
#end
#end
$sql
Do this solutions work for you?
Considering $ctx as
{
ctx: {
args: {
input: {
listItems: [
{
item_id: "itemId1",
item_name: "itemName1",
item_image_id: "itemImageId1"
},
{
item_id: "itemId2",
item_name: "itemName2",
item_image_id: "itemImageId2"
}
]
}
}
}
}
the output is equals to INSERT INTO items (item_id, item_name, item_image_id) VALUES (itemId1, itemName1, itemImageId1), (itemId2, itemName2, itemImageId2);.

Errors while debugging code using HTML::TreeBuilder::XPath

I am getting some errors when I try to debug the following code.
Note that it fetches the data from approx 6,000 fields from the http://europa.eu/youth/volunteering/evs-organisation#open
After parsing each page, check for the existence of the next › link at the bottom.
View-source is a browser based command. It tells the browser to output the response in plain text rather than render it based on its actual content type, HTML in this case. You should not need to include view-source in your URL.
Here we have a script that extracts the data out of each block and cleans it up a little. The browse function is generic. It takes an input reference which contains the URL and XPaths of the parent and children in order to construct the output ref. It is just an approach: it does not yet navigate across each page,
In a rough script I tested, I fetched the total results using //span[#class="ey_badge"] then the max page using
my $page_max = $results / 21;
$page_max = int( $page_max ) == $page_max ? $page_max-- : int( $page_max ) ;
See the errors
martin#linux-3645:~/dev/perl> perl eu.pl
syntax error at eu.pl line 81, near "our "
Global symbol "$iterator_organizations" requires explicit package name at eu.pl line 81.
Can't use global #_ in "my" at eu.pl line 84, near "= #_"
Missing right curly or square bracket at eu.pl line 197, at end of line
Execution of eu.pl aborted due to compilation errors.
martin#linux-3645:~/dev/perl> ^C
martin#linux-3645:~/dev/perl>
It fetches the data from approx 6,000 fields from http://europa.eu/youth/volunteering/evs-organisation#open
See the code
use strict;
use warnings FATAL => qw#all#;
use LWP::UserAgent;
use HTML::TreeBuilder::XPath;
use Data::Dumper;
my $handler_relurl = sub { q#https://europa.eu# . $_[0] };
my $handler_trim = sub { $_[0] =~ s#^\s*(.+?)\s*$#$1#r };
my $handler_val = sub { $_[0] =~ s#^[^:]+:\s*##r };
my $handler_split = sub { [ split $_[0], $_[1] ] };
my $handler_split_colon = sub { $handler_split->( qr#; #, $_[0] ) };
my $handler_split_comma = sub { $handler_split->( qr#, #, $_[0] ) };
my $conf = {
url => q#https://europa.eu/youth/volunteering/evs-organisation_en#,
parent => q#//div[#class="vp ey_block block-is-flex"]#,
children => {
internal_url => [ q#//a/#href#, [ $handler_relurl ] ],
external_url => [ q#//i[#class="fa fa-external-link fa-lg"]/parent::p//a/#href#, [ $handler_trim ] ],
title => [ q#//h4# ],
topics => [ q#//div[#class="org_cord"]#, [ $handler_val, $handler_split_colon ] ],
location => [ q#//i[#class="fa fa-location-arrow fa-lg"]/parent::p#, [ $handler_trim ] ],
hand => [ q#//i[#class="fa fa-hand-o-right fa-lg"]/parent::p#, [ $handler_trim, $handler_split_comma ] ],
pic_number => [ q#//p[contains(.,'PIC no')]#, [ $handler_val ] ],
}
};
print Dumper browse( $conf );
sub browse {
my $conf = shift;
my $ref = [ ];
my $lwp_useragent = LWP::UserAgent->new( agent => q#IE 6#, timeout => 10 );
my $response = $lwp_useragent->get( $conf->{url} );
die $response->status_line unless $response->is_success;
my $content = $response->decoded_content;
my $html_treebuilder_xpath = HTML::TreeBuilder::XPath->new_from_content( $content );
my #nodes = $html_treebuilder_xpath->findnodes( $conf->{parent} );
for my $node ( #nodes ) {
push #$ref, { };
while ( my ( $key, $val ) = each %{ $conf->{children} } ) {
my $xpath = $val->[0];
my $handlers = $val->[1] // [ ];
$val = ( $node->findvalues( qq#.$xpath# ) )[0] // next;
$val = $_->( $val ) for #$handlers;
$ref->[-1]->{$key} = $val;
}
}
return $ref;
}
{
'internal_url' => 'https://europa.eu/youth/volunteering/organisation/948417016_en',
'external_url' => 'http://www.apd.ge',
'location' => 'Tbilisi, Georgia',
'title' => '"Academy for Peace and Development" Union',
'topics' => [
'Access for disadvantaged',
'Youth (Participation, Youth Work, Youth Policy)',
'Intercultural/intergenerational education and (lifelong)learning'
],
'pic_number' => '948417016',
'hand' => [
'Receiving',
'Sending'
]
}
our $iterator_organizations = sub {
my ( $browser, $parent ) = #_;
my $url = q#https://europa.eu/youth/volunteering/evs-organisation_en#;
my $nodes = $browser->nodes( url => $url );
my $iterator = sub {
return shift #$nodes;
};
return ( $iterator, 1 );
our $iterator_organizations_b = sub {
my ( $browser, $parent ) = #_;
my $url = q#https://europa.eu/youth/volunteering/evs-organisation_en#;
my $uri = URI->new( $url );
my $xpath = q#//div[#class="vp ey_block block-is-flex"]#;
my $nodes = [ ];
my $page = 0;
my $results = $parent->{results};
my $page_max = $results / 21;
$page_max = int($page_max) == $page_max ? $page_max-- : int($page_max);
my $iterator_uri = sub {
$uri->query_form( page => $page++ );
return $page > 2 ? undef : $uri ; # $page_max;
};
my $iterator_node = sub {
unless ( #$nodes ) {
my $uri = $iterator_uri->( ) // return undef;
my $options = $page == 1 ? { tree => $parent->{_node} } : { url => $uri->as_string };
$nodes = $browser->nodes( %$options, xpath => $xpath );
}
return shift #$nodes;
};
return ( $iterator_node, 0 );
};
our $iterator_organization = sub {
my ( $browser, $parent ) = #_;
my $url = $parent->{internal_url};
my $nodes = $browser->nodes( url => $url );
my $iterator = sub {
return shift #$nodes;
};
return ( $iterator, 1 );
};
sub organizations {
my ( $self, $options ) = ( shift, { #_ } );
my $map = [
$Massweb::Browser::Europa::iterator_organizations,
results => q#.//span[#class="ey_badge"]#,
organizations => [
$Massweb::Browser::Europa::iterator_organizations_b,
internal_url => [ q#.//a/#href#, $Massweb::Browser::Europa::handler_url ],
external_url => [ q#.//i[#class="fa fa-external-link fa-lg"]/parent::p//a/#href#, $Massweb::Browser::handler_trim ],
title => q#.//h4#,
topics => [ q#.//div[#class="org_cord"]#, $Massweb::Browser::handler_val, $Massweb::Browser::handler_list_colon ],
location => [ q#.//i[#class="fa fa-location-arrow fa-lg"]/parent::p#, $Massweb::Browser::handler_trim ],
hand => [ q#.//i[#class="fa fa-hand-o-right fa-lg"]/parent::p#, $Massweb::Browser::handler_trim, $Massweb::Browser::handler_list_comma ],
pic_number => [ q#.//p[contains(.,'PIC no')]#, $Massweb::Browser::handler_val ],
recruiting => [ q#boolean(.//i[#class="fa fa-user-times fa-lg"])#, $Massweb::Browser::handler_bool_rev ],
_ => \&organization,
],
];
my $organizations = $self->browse( map => $map );
return $organizations;
}
sub organization {
my ( $self, $options ) = ( shift, { #_ } );
my $map = [
sub { $Massweb::Browser::Europa::iterator_organization->( $_[0], $options ) },
#title => q#.//h1#,
description => q#.//div[#class="ey_vp_detail_page"]/p#,
];
my $organization = $self->browse( map => $map );
return $organization;
}
The problem appears to be the block/anonymous hash starting 'internal_url'. I can't imagine what you intend there but it is a syntax error and would have no effect if you fixed it
Why are you declaring so many subroutine references like our $iterator_organizations = sub { ... } instead of using standard subroutines? It is a very strange approach

CDbCriteria throwing column name is ambigous

I have the following method in a controller of my Yii app.
public function actionManage($typeid=0, $locationid=0, $page=1, $rows=12, $sidx='date_input', $sord='desc', $kategori='')
{
if (Yii::app()->request->isAjaxRequest) {
// Jika dilakukan operasi 'edit' pada row
if (isset($_REQUEST['oper']))
{
$oper = $_REQUEST['oper'];
$id = $_REQUEST['id'];
if ($oper == 'edit')
{
$value = $_REQUEST['value'];
$record = InputData::model()->findByPk($id);
$record->value = $value;
$record->update();
}
if($oper == 'delete'){
$model = InputData::model()->findByPk($id);
$model->delete();
}
}
// inisialisasi criteria query
$criteria = new CDbCriteria();
$criteria->order = "$sidx $sord";
// filter lokasi
if (is_numeric($locationid) && $locationid !== 0)
{
$criteria->with = array('data'=>array(
'condition'=>'data.locationid=:locationid',
'params'=>array(':locationid'=>$locationid)
));
} else {
if (is_numeric($typeid) && $typeid !== 0)
{
$criteria->with = array('data.location'=>array(
'with'=>array(
'type'=>array(
'condition'=>'type.typeid=:typeid',
'params'=>array(':typeid'=>$typeid)
)
)
));
} else {
$criteria->with = array('data.location'=>array(
'with'=>array(
'type'=>array(
'condition'=>'type.type_desc=:type_desc',
'params'=>array(':type_desc'=>$kategori)
)
)
));
}
}
// filter range tanggal
if (isset($_REQUEST['startdate'], $_REQUEST['enddate']))
{
$startdate = $_REQUEST['startdate'];
$enddate = $_REQUEST['enddate'];
$criteria->condition = 'date_input <= :enddate AND date_input >= :startdate';
$criteria->params = array(':startdate'=>$startdate, ':enddate'=>$enddate);
}
if(isset($_REQUEST['dataid'])){
$dataid = $_REQUEST['dataid'];
$criteria->addCondition("dataid = $dataid");
}
$dataProvider = new CActiveDataProvider('InputData', array(
'criteria'=>$criteria,
'pagination'=>array(
'currentPage'=>$page-1,
'pageSize'=>$rows
)
));
$count = $dataProvider->totalItemCount;
$total_pages = $count > 0 ? ceil($count/$rows) : 0;
if ($page > $total_pages) $page=$total_pages;
// generate response untuk jqgrid
$response = new stdClass();
$response->page = $page;
$response->total = $total_pages;
$response->records = $count;
foreach($dataProvider->getData() as $row)
{
$response->rows[] = array(
'id'=>$row->inputdataid,
'cell'=>array(
$row->inputdataid,
$row->date_input,
$row->time_input,
$row->data->location->location_name,
$row->data->data_name,
$row->data->variable->var_name,
round($row->value, 3),
$row->data->variable->unit->uom_name,
($row->inputOfficer !== NULL ? $row->inputOfficer->officer_name:''),
'<i class="fa fa-pencil-square-o"></i> Edit <i class="fa fa-trash-o"></i> Delete'
)
);
}
echo json_encode($response);
} else {
$url = $this->createUrl("dataAir/manage");
$delurl = $this->createUrl("dataAir/deleteRow");
$startdate = '2013-01-01';
$enddate = date_format(new DateTime(), 'Y-m-d');
$this->render('jqgrid', array(
'kategori'=>$kategori,
'url'=>$url,
'delurl'=>$delurl,
'startdate'=>$startdate,
'enddate'=>$enddate
));
}
}
this line causing the problem
if(isset($_REQUEST['dataid'])){
$dataid = $_REQUEST['dataid'];
$criteria->addCondition("dataid = $dataid");
}
when, I remove those lines the method work just fine. what could be the problem causing ambigous colum name? here is the error log
SELECT COUNT(DISTINCT "t"."inputdataid") FROM "app_inputdata" "t" LEFT OUTER JOIN "app_ref_periodicdata" "data" ON ("t"."dataid"="data"."dataid") WHERE ((date_input <= :enddate AND date_input >= :startdate) AND (dataid = 7)) AND (data.locationid=:locationid). Bound with :startdate='2013-01-01', :enddate='2015-02-02', :locationid='6'
You need to add the table alias t to your condition:
$criteria->addCondition("t.dataid = $dataid");
Also, since $dataid is being obtained from a $_REQUEST it is best to pass it as a parameter. This can be done in two ways:
$criteria->addCondition("t.dataid = :dataid", [":dataid" => $dataid]);
$criteria->compare("t.dataid", $dataid);
you have used table aliases,
'with'=>array(
'type'=>array(
'condition'=>'type.type_desc=:type_desc', <<- I mean here you have used alias : type
'params'=>array(':type_desc'=>$kategori)
)
you just have to remember that the main model that you are working with, will always need alias t to unambigufy! (not sure if that is an actual word :D )

jquery DataTables erroring on search box

Trying to get DataTables to work with PDO. I found this script online and it works fine, BUT, when I set ATTR_EMULATE_PREPARES to false the search capability does not work and reports back this error.
I cannot view the json response as there is none to view when this error happens, however, in all other cases other than using the search the json is returned properly and it works perfectly fine. Since the error only happens when emulation is set to false I am thinking this has something to do with binding? I cannot figure this one out as I don't see anything wrong that is sticking out at me.
Also, I am not looking to turn on emulation as a solution either. Help would be very appreciated.
the get in firebug:
http://www.example.com/assets/data-tables/test-pdo.php?sEcho=3&iColumns=4&sColumns=&iDisplayStart=0&iDisplayLength=10&mDataProp_0=0&mDataProp_1=1&mDataProp_2=2&mDataProp_3=3&sSearch=d&bRegex=false&sSearch_0=&bRegex_0=false&bSearchable_0=true&sSearch_1=&bRegex_1=false&bSearchable_1=true&sSearch_2=&bRegex_2=false&bSearchable_2=true&sSearch_3=&bRegex_3=false&bSearchable_3=true&iSortCol_0=2&sSortDir_0=asc&iSortingCols=1&bSortable_0=false&bSortable_1=true&bSortable_2=true&bSortable_3=true&_=1388479579319
Error in firebug:
<br />
<b>Fatal error</b>: Uncaught exception 'PDOException' with message 'SQLSTATE[HY093]: Invalid parameter number' in /home/test/public_html/assets/data-tables/test-pdo.php:107
Stack trace:
#0 /home/test/public_html/assets/data-tables/test-pdo.php(107): PDOStatement->execute()
#1 /home/test/public_html/assets/data-tables/test-pdo.php(155): TableData->get('accounts', 'account_id', Array)
#2 {main}
thrown in <b>/home/test/public_html/assets/data-tables/test-pdo.php</b> on line <b>107</b><br />
db connection:
$db = new PDO("mysql:host=$db_host;dbname=$db_database;charset=utf8", $db_user, $db_pass, array(PDO::ATTR_EMULATE_PREPARES => false, PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, PDO::ATTR_PERSISTENT => true));
processing:
<?php
/*
* Script: DataTables server-side script for PHP and MySQL
* Copyright: 2012 - John Becker, Beckersoft, Inc.
* Copyright: 2010 - Allan Jardine
* License: GPL v2 or BSD (3-point)
*/
define('INCLUDE_CHECK',true);
// These files can be included only if INCLUDE_CHECK is defined
require '/home/test/public_html/assets/functions/connect.php';
//inject db connection into class
class TableData {
/** #var \PDO */
protected $_db;
public function __construct(\PDO $_db) {
$this->_db = $_db;
}
public function get($table, $index_column, $columns) {
// Paging
$sLimit = "";
if ( isset( $_GET['iDisplayStart'] ) && $_GET['iDisplayLength'] != '-1' ) {
$sLimit = "LIMIT ".intval( $_GET['iDisplayStart'] ).", ".intval( $_GET['iDisplayLength'] );
}
// Ordering
$sOrder = "";
if ( isset( $_GET['iSortCol_0'] ) ) {
$sOrder = "ORDER BY ";
for ( $i=0 ; $i<intval( $_GET['iSortingCols'] ) ; $i++ ) {
if ( $_GET[ 'bSortable_'.intval($_GET['iSortCol_'.$i]) ] == "true" ) {
$sortDir = (strcasecmp($_GET['sSortDir_'.$i], 'ASC') == 0) ? 'ASC' : 'DESC';
$sOrder .= "`".$columns[ intval( $_GET['iSortCol_'.$i] ) ]."` ". $sortDir .", ";
}
}
$sOrder = substr_replace( $sOrder, "", -2 );
if ( $sOrder == "ORDER BY" ) {
$sOrder = "";
}
}
/*
* Filtering
* NOTE this does not match the built-in DataTables filtering which does it
* word by word on any field. It's possible to do here, but concerned about efficiency
* on very large tables, and MySQL's regex functionality is very limited
*/
//need this change to only show correct responses from db
//$test = 100;
//$sWhere = ""; OR $sWhere = "WHERE account_id < ".$test;
$sWhere = "";
if ( isset($_GET['sSearch']) && $_GET['sSearch'] != "" ) {
// changes for correct display from db plus searching
if ($sWhere == ""){
$sWhere = "WHERE (";
}
else {
$sWhere .= " AND (";
}
//$sWhere = "WHERE (";
for ( $i=0 ; $i<count($columns) ; $i++ ) {
if ( isset($_GET['bSearchable_'.$i]) && $_GET['bSearchable_'.$i] == "true" ) {
$sWhere .= "`".$columns[$i]."` LIKE :search OR ";
}
}
$sWhere = substr_replace( $sWhere, "", -3 );
$sWhere .= ')';
}
// Individual column filtering
for ( $i=0 ; $i<count($columns) ; $i++ ) {
if ( isset($_GET['bSearchable_'.$i]) && $_GET['bSearchable_'.$i] == "true" && $_GET['sSearch_'.$i] != '' ) {
if ( $sWhere == "" ) {
$sWhere = "WHERE ";
}
else {
$sWhere .= " AND ";
}
$sWhere .= "`".$columns[$i]."` LIKE :search".$i." ";
}
}
// SQL queries get data to display
$sQuery = "SELECT SQL_CALC_FOUND_ROWS `".str_replace(" , ", " ", implode("`, `", $columns))."` FROM `".$table."` ".$sWhere." ".$sOrder." ".$sLimit;
$statement = $this->_db->prepare($sQuery);
// Bind parameters
if ( isset($_GET['sSearch']) && $_GET['sSearch'] != "" ) {
$statement->bindValue(':search', '%'.$_GET['sSearch'].'%', PDO::PARAM_STR);
}
for ( $i=0 ; $i<count($columns) ; $i++ ) {
if ( isset($_GET['bSearchable_'.$i]) && $_GET['bSearchable_'.$i] == "true" && $_GET['sSearch_'.$i] != '' ) {
$statement->bindValue(':search'.$i, '%'.$_GET['sSearch_'.$i].'%', PDO::PARAM_STR);
}
}
$statement->execute();
$rResult = $statement->fetchAll();
$iFilteredTotal = current($this->_db->query('SELECT FOUND_ROWS()')->fetch());
// Get total number of rows in table
$sQuery = "SELECT COUNT(`".$index_column."`) FROM `".$table."`";
//$sQuery = "SELECT COUNT(`".$index_column."`) FROM `".$table."` WHERE account_id < 100";
$iTotal = current($this->_db->query($sQuery)->fetch());
// Output
$output = array(
"sEcho" => intval($_GET['sEcho']),
"iTotalRecords" => $iTotal,
"iTotalDisplayRecords" => $iFilteredTotal,
"aaData" => array()
);
// Return array of values
foreach($rResult as $aRow) {
$row = array();
for ( $i = 0; $i < count($columns); $i++ ) {
//else if ( $aColumns[$i] != ' ' )
if ( $columns[$i] != ' ' )
{
/* General output */
//if column is empty give it n/a
$row[] = ($aRow[ $columns[$i] ]=="") ? 'n/a' : $aRow[ $columns[$i] ];
}
}
$output['aaData'][] = $row;
}
echo json_encode( $output );
}
}
header('Pragma: no-cache');
header('Cache-Control: no-store, no-cache, must-revalidate');
// Create instance of TableData class
$table_data = new TableData($db);
// Get the data
//$table_data->get('table_name', 'index_column', array('column1', 'column2', 'columnN'));
$table_data->get('accounts', 'account_id', array('account_id', 'account_username', 'account_password', 'account_email'));
?>
I doubt that anyone will be interested, but I finally figured this out. The script was trying to use the same binding, :search, multiple times in the statement.
Even though it will always be the same actual value the error was being thrown as it was the same binding. How I did not see this earlier I do not know, but it is obvious to me now.
//$sWhere .= "`".$columns[$i]."` LIKE :search OR ";
$sWhere .= "`".$columns[$i]."` LIKE :searchm".$i." OR ";
// Bind parameters
//if ( isset($_GET['sSearch']) && $_GET['sSearch'] != "" ) {
// $statement->bindValue(':search', '%'.$_GET['sSearch'].'%', PDO::PARAM_STR);
//}
if ( isset($_GET['sSearch']) && $_GET['sSearch'] != "" ) {
for ( $i=0 ; $i<count($columns) ; $i++ ) {
$statement->bindValue(':searchm'.$i, '%'.$_GET['sSearch'].'%', PDO::PARAM_STR);
}
}

Generate JSON from nested sets (perl, sql, jquery)

I have content pages in the database (using nested sets) and I need to show it by jQuery jsTree plugin. It's need to return JSON with data like this:
[
{
data: 'node1Title',
children: [
{
data: 'subNode1Title',
children: [...]
},
{
data: 'subNode2Title',
children: [...]
}
]
},
{
data: 'node2Title',
children: [...]
}
]
What I need for do it?
I can transform an array of hashes to JSON but I don't understand how to generate an array.
Sample data:
**'pages'table**
id parent_id level lkey rkey name
1 0 1 1 14 index
2 1 2 2 7 info
3 1 2 8 13 test
4 2 3 3 4 about
5 2 3 5 6 help
6 3 3 9 10 test1
7 3 3 11 12 test2
I need to get:
[
{
data: 'index',
children: [
{
data: 'info',
children: [
{
data: 'about'
},
{
data: 'help',
}
]
},
{
data: 'test',
children: [
{
data: 'test1'
},
{
data: 'test2'
}
]
}
]
}
]
I had exactly the same problem and here is what I wrote in Perl to convert my nested set tree into a JSON object for jsTree plugin (I'm using DBIx::Tree::NestedSet to access the MySQL database tree). I know my code is ugly from a Perl perspective, but it works for me.
sub get_json_tree {
my $json = '[';
my $first = 1;
my $last_level = 1;
my $level = 1;
my $tree = DBIx::Tree::NestedSet->new(dbh => $dbh);
my $ancestors = $tree->get_self_and_children_flat(id => $tree->get_root);
foreach (#{$ancestors}) {
my $name = $_->{'name'};
$last_level = $level;
$level = $_->{'level'};
if ($level > $last_level) {
$json .= ',' if ($json =~ /}$/);
} elsif ($level < $last_level) {
$json .= ']}';
for (my $i = 0; $i < $last_level - $level; $i++) {
$json .= ']}';
}
$json .= ',';
} elsif ($level == $last_level && !$first) {
$json .= ']},';
}
$json .= '{"attr":{"id":'.$_->{'id'}.',"rel":"folder"},"data":"'.$name.'","children":[';
$first = 0;
}
$json .= ']}';
for (my $i = 1; $i < $level; $i++) {
$json .= ']}';
}
$json .= ']';
return $json;
}
I'm looking for it. Perhaps DataTable plugin examples offer a solution. I'm looking on the plugin directory /examples/server_side/scripts/ssp.class.php. You can download it here.
Take a look about simplest way of using it at "Server-side script" label in this documentation.
This is very simple. You need to write a recursive function. I wrote it in Perl. $list - this is your array sorted by 'left_key'.
sub make_tree {
my $list = shift;
my #nodes;
while (my $node = shift #$list) {
if (#$list and $node->{level} < $list->[0]{level}) {
$node->{data} = make_tree($list);
push #nodes, $node;
}
last if #$list and $node->{level} > $list->[0]{level};
}
return \#nodes;
}
my $hash = make_tree($list);
Recently I was looking for a similar solution. I didn't find this until after posting my own question. The final code I posted on question I think would answers your question nicely.
I am using the following code with a modified version of DBIx::Tree::NestedSet. I use this code to create a JSON output of the nested sets tree.
Convert a flat datastructure into a tree
sub get_jsonTree {
my ($array_of_hashes_ref) = #_;
my $roots;
my %recs_by_name;
my %children_by_parent_name;
my %count;
for my $row (#$array_of_hashes_ref) {
my $name = $row->{position_id};
my $parent_name = $row->{placement_id};
my $rec = {
name => $name,
};
## Added to loop through all key,value pairs and add them to $rec
while ( my ($key, $value) = each(%$row) ) {
$rec->{$key} = $value;
}
##Added To Count Child Nodes
$count{$parent_name} = 0 if (!$count{$parent_name});
$rec->{'child_count'} = $count{$parent_name};
$count{$parent_name}++;
push #{ $children_by_parent_name{$parent_name // 'root'} }, $rec;
$recs_by_name{$name} = $rec;
}
$roots = delete($children_by_parent_name{root}) || [];
for my $name (keys(%children_by_parent_name)) {
my $children = $children_by_parent_name{$name};
if ( my $rec = $recs_by_name{$name} ) {
$rec->{children} = $children;
} else {
$util{'test'} .= "Parent $name doesn't exist.\n<BR>";
push #$roots, #$children;
}
}
use JSON;
my $json_str = encode_json(#{$roots}[0]);
return $json_str;
}
my $array_of_hashes_ref = [
{ position_id => 123, placement_id => undef },
{ position_id => 456, placement_id => 123 },
{ position_id => 789, placement_id => 123 },
# ...
];
my $json_str = &get_jsonTree($array_of_hashes_ref);