How to use "POST url" JSON file in RoR website? - ruby-on-rails-3

I am using Ruby on Rails and Kibana to show visualisation in my website(Ruby2.6.0, Rails5.2.3, Kibana6.6.1). By using Xpack->Share->Generate PDF/PNG, I find 'copy POST url'. I can get the following JSON file from the POST url by using 'net/http', but how can I use POST url/JSON file in my RoR website to download PNG file in 'assets' folder?
JSON file I got from POST url:
{'path': '/api/reporting/jobs/download/jvds4zs10qv79d0062b9cel6',
'job': {'id': 'jvds4zs10qv79d0062b9cel6', 'index':
'.reporting-2019.05.05', 'type': 'esqueue', 'jobtype': 'PNG',
'created_by': False, 'payload': {'type': 'visualization', 'title':
'[eCommerce] Sales by Gender', 'relativeUrl':
"/app/kibana#/visualize/edit/ed8436b0-b88b-11e8-a6d9-e546fe2bba5f?_g=(refreshInterval:(pause:!t,value:0),time:(from:now-60d,mode:quick,to:now))&_a=(filters:!(),linked:!f,query:(language:lucene,query:''),uiState:(),vis:(aggs:!((enabled:!t,id:'1',params:(),schema:metric,type:count),(enabled:!t,id:'2',params:(field:customer_gender,json:'',missingBucket:!f,missingBucketLabel:Missing,order:desc,orderBy:'1',otherBucket:!f,otherBucketLabel:Other,size:5),schema:segment,type:terms)),params:(addLegend:!t,addTooltip:!t,isDonut:!t,labels:(last_level:!t,show:!t,truncate:100,values:!t),legendPosition:right,type:pie),title:'%5BeCommerce%5D+Sales+by+Gender',type:pie))",
'headers':
'uxY4w6gCRwwLZyFkDX0ujVlANYq7ae5UuNJAN0GpbTOX7vw5aPuCuyr37nNcpz3vtA8kNVaL6Gacs24mPjZfwl4mB9xRKGA62CkCLo8Xz1amuIEthI+BtKPGo5QAk2k2+7zLvLgX3KouVvYdd61U5rLAvfjv4TydcpH9qJ4qrL5OELkzAJGrNdujtDLWaoy2Qj9YXbLnh7gSdI+lrNml6usZIh4pMzz8qdRo597iO/4AeSJRa2JfAruyGB3zYInZRtMuCbA4f3ANvT2WeAn6lpiwThB993uEjwu4LBVUZuo2XL0TcB29YNNa9oMCUEhWjdJ1ase5VQdz8V2IOgecQ1W52V5JuVTvl9SkVvFWSBhXJg==',
'browserTimezone': 'Europe/Zurich', 'layout': {'dimensions':
{'height': 589.234375, 'width': 866}}, 'basePath': '', 'forceNow':
'2019-05-07T12:38:12.768Z'}, 'timeout': 120000, 'max_attempts': 3,
'priority': 10, 'browser_type': 'chromium'}}
I am trying to use 'net/http', What I added in controller:
class MaisonController < ApplicationController
require 'net/http'
def require_post
uri = URI.parse("http://localhost:5601/api/reporting/generate/png?jobParams=(browserTimezone:Europe%2FZurich,layout:(dimensions:(height:589.234375,width:866)),objectType:visualization,relativeUrl:%27%2Fapp%2Fkibana%23%2Fvisualize%2Fedit%2Fed8436b0-b88b-11e8-a6d9-e546fe2bba5f%3F_g%3D(refreshInterval:(pause:!!t,value:0),time:(from:now-60d,mode:quick,to:now))%26_a%3D(filters:!!(),linked:!!f,query:(language:lucene,query:!%27!%27),uiState:(),vis:(aggs:!!((enabled:!!t,id:!%271!%27,params:(),schema:metric,type:count),(enabled:!!t,id:!%272!%27,params:(field:customer_gender,json:!%27!%27,missingBucket:!!f,missingBucketLabel:Missing,order:desc,orderBy:!%271!%27,otherBucket:!!f,otherBucketLabel:Other,size:5),schema:segment,type:terms)),params:(addLegend:!!t,addTooltip:!!t,isDonut:!!t,labels:(last_level:!!t,show:!!t,truncate:100,values:!!t),legendPosition:right,type:pie),title:!%27%255BeCommerce%255D%2BSales%2Bby%2BGender!%27,type:pie))%27,title:%27%5BeCommerce%5D%20Sales%20by%20Gender%27)")
http = Net::HTTP.new(uri.host,uri.port)
request = Net::HTTP::Post.new(uri.request_uri)
response = http.request(request)
render :json => response.body
end
end
Edit:
I try to use 'attachment_fu' gem to download png file with the above JSON file. Modify the code in Controller to the following code:
class MaisonController < ApplicationController
require 'net/http'
require 'open-uri'
def require_post
uri = URI.parse("http://localhost:5601/api/reporting/generate/png?jobParams=(browserTimezone:Europe%2FZurich,layout:(dimensions:(height:589.234375,width:866)),objectType:visualization,relativeUrl:%27%2Fapp%2Fkibana%23%2Fvisualize%2Fedit%2Fed8436b0-b88b-11e8-a6d9-e546fe2bba5f%3F_g%3D(refreshInterval:(pause:!!t,value:0),time:(from:now-60d,mode:quick,to:now))%26_a%3D(filters:!!(),linked:!!f,query:(language:lucene,query:!%27!%27),uiState:(),vis:(aggs:!!((enabled:!!t,id:!%271!%27,params:(),schema:metric,type:count),(enabled:!!t,id:!%272!%27,params:(field:customer_gender,json:!%27!%27,missingBucket:!!f,missingBucketLabel:Missing,order:desc,orderBy:!%271!%27,otherBucket:!!f,otherBucketLabel:Other,size:5),schema:segment,type:terms)),params:(addLegend:!!t,addTooltip:!!t,isDonut:!!t,labels:(last_level:!!t,show:!!t,truncate:100,values:!!t),legendPosition:right,type:pie),title:!%27%255BeCommerce%255D%2BSales%2Bby%2BGender!%27,type:pie))%27,title:%27%5BeCommerce%5D%20Sales%20by%20Gender%27)")
http = Net::HTTP.new(uri.host,uri.port)
request = Net::HTTP::Post.new(uri.request_uri)
response = http.request(request)
render :json => response.body
http.start() { |http|
tempfile = Tempfile.new('test.png')
File.open(tempfile.path, 'w') do |f|
f.write response.body
end
attachment = Attachment.new(:upload_data => LocalFile.new(tempfile.path))
attachment.save
}
end
The error I get is 'uninitialized constant MaisonController::Attachment'.
Expected result: download png file in assets by using the above JSON file.

I will answer my own question about what I tried in my project and close this question.
I got 'copy POST url' in 'Kibana->Visualisation/Dashboard->PDF/PNG'. It is a post url. I sent post request with this url to kibana server. The action will trigger kibana server to 'pending', 'processing' and generate the report. And I will get a JSON file back from the server. There is an important job ID in JSON file, like this 'jvi53ajd21w09d006233kcvo'. I can find all history reports in 'Management->Reporting'. Finally, I use the generated url 'http://localhost:5601/api/reporting/jobs/download/jvi53ajd21w09d006233kcvo' with 'wget' command to download the report in my local folder.
You can find similar answer here: https://discuss.elastic.co/t/how-to-use-post-url-to-share-visualisation-in-my-website/179671/4
Hope my answer can help those who want to use POST URL in Kibana.

Related

Accept an image via Rails 5 API and Active Storage

I am trying to send images to my Rails app and then store them via Active Storage.
I tried Base64 and direct upload and researched for hours but nothing really works.
Can somebody point me to a good way?
My last attempt was to use Base64 like so:
def attach_preview
page = Page.first
content = JSON.parse(request.body.read)
decoded_data = Base64.decode64(content["file_content"].force_encoding("UTF-8"))
begin
file = Tempfile.new('test')
file.write decoded_data
#page.thumbnail = file
filename = "foooo"
page.thumbnail.attach(io: File.read(file), filename: filename)
if page.save
render :json => {:message => "Successfully uploaded the profile picture."}
else
render :json => {:message => "Failed to upload image"}
end
ensure
file.close
file.unlink
end
end
But this results in a "\xAB" from ASCII-8BIT to UTF-8 error.
Dont really care if its Base64 or something else, I just need a way :-)
This works, I use IO directly since ActiveStorage needs it anyway.
def attach_thumbnail
content = JSON.parse(request.body.read.force_encoding("UTF-8"))
decoded_data = Base64.decode64(content["file_content"])
io = StringIO.new
io.puts(decoded_data)
io.rewind
#page.thumbnail.attach(io: io, filename: 'base.png')
#page.save
render json: {
success: #page.thumbnail.attached?,
thumbnail_url: url_for(#page.thumbnail),
page: #page
}
end

Rails 3.2 SSRS and SavonRD

I'm integrating my Rails 3.2 app with SSRS for reporting using Savonrb to do the Soap call to the SSRS web service. Everything is finally working and I can successfully make the following calls:
LoadReport
SetExecutionParameters
Render
The render call takes a format parameter (in my case PDF) and returns an XML response. I thought the call would result in the report being rendered in a pdf, but it doesn't. What do I need to do with the response object to render the report?
code below:
response = #client.call(:load_report, message: {report: "/path/to/report"} )
response = #client.call(:set_execution_parameters, message: { "Parameters" => param1} )
response = #client.call(:render, message: {"Format" => "PDF"})
Thanks!
The answer is that I need to take the xml string that was returned from the render call and decode it to Base64, write it to a file, and then user send_file to stream the file to the browser. This works now, code below:
response = #newClient.call(:render, message: {"Format" => "PDF"})
#results = response.to_hash
#report = #results[:render_response][:result]
#extension = #results[:render_response][:extension]
#mime_type = #results[:render_response][:mime_type]
#report_name = "MyReportName.pdf" # this should be dynamically generated from a param passed in the call to run_report
File.open(#report_name, "wb+") do |f|
f.puts Base64.decode64(#report)
end
#file = File.open(#report_name, 'r')
send_file #file, :filename => #report_name, :type => "application/pdf; charset=utf-8", :disposition => "attachment"

Paperclip and Phusion Passenger NoHandlerError

I followed this guide to get drag and drop file uploads through AJAX: http://dannemanne.com/posts/drag-n-drop_upload_that_works_with_ror_and_paperclip
Everything was working fine on my development environment with WebBrick but if I deploy to PhusionPassenger then I get:
Paperclip::AdapterRegistry::NoHandlerError (No handler found for #<PhusionPassenger::Utils::RewindableInput:0x000000041aef38 #io=#<PhusionPassen...
I'm using this in my controller:
before_filter :parse_raw_upload, :only => :bulk_submissions
def bulk_submissions
...
#submission = Submission.create!(url: "", file: #raw_file, description: "Please edit this description", work_type: "other", date_completed: DateTime.now.to_date)
...
end
private
def parse_raw_upload
if env['HTTP_X_FILE_UPLOAD'] == 'true'
#raw_file = env['rack.input']
#raw_file.class.class_eval { attr_accessor :original_filename, :content_type }
#raw_file.original_filename = env['HTTP_X_FILE_NAME']
#raw_file.content_type = env['HTTP_X_MIME_TYPE']
end
end
Looking at the request itself all the headers are set (X_MIME_TYPE, X_FILE_NAME) etc.
Any ideas?
Thanks in advance!
The example you're cribbing from expects the file stream to be a StringIO object, but Passenger is giving you a PhusionPassenger::Utils::RewindableInput object instead.
Fortunately, a RewindableInput is duckalike to StringIO for this case, so Paperclip's StringioAdapter can be used to wrap your upload stream.
Inside the if block in your parse_raw_upload, at the end, do:
if #raw_file.class.name == 'PhusionPassenger::Utils::RewindableInput'
#raw_file = Paperclip::StringioAdapter.new(#raw_file)
end

encoding error when uploading image via paperclip

I'm trying to upload an image to paperclip and save it to s3. However, I get the following error in my console
!! Unexpected error while processing request: invalid byte sequence in UTF-8
There are a few responses on StackOverflow about how to resolve this problem, though most point to the original solution being an update to Rack. However, I'm using Ruby 1.9.3 and Rails 3.1.3, and believe I don't have Rack (I haven't installed it as a Gem, should I??).
The filenames that I've been trying are fairly simple, so I'm assuming the issue is in the actual file, but I'm not sure how to debug which upload variable the error is coming from. Rails isn't putting any of these errors in the log files, so I can't seem to get more details.
My controller is fairly simple, just like the example on the paperclip github documentation
def create
wine_photo = WinePhoto.create(params[:wine_photo])
return render :json => wine_photo
end
though originally I used the more common
wine_photo - WinePhoto.new(params[:wine_photo])
if wine_photo.save
return render :json => wine_photo
else
return render :json => wine_photo.errors
end
My model (which I doubt is very helpful) is
class WinePhoto true
validates_with AttachmentPresenceValidator, :attributes => :photo
belongs_to :wine
belongs_to :user
def photo_url
photo.url
end
end
based on this response on stackoverflow, Ruby Invalid Byte Sequence in UTF-8, I've tried the below in my controller
def create
wine_photo = WinePhoto.new(params[:wine_photo])
wine_photo.photo = IO.read(wine_photo.photo).force_encoding("ISO-8859-1").encode("utf-8", replace: nil)
...
but still got the error.
Any suggestions on how to get past this encoding issue? Is there a way to confirm that the error is coming from the file being uploaded?
My upload code (ajax) is
save_photo: function(){
var file = document.getElementById('file_api').files[0];
console.log(file);
var xhr = new XMLHttpRequest();
if (xhr.upload && file.type == "image/jpeg" ) {
// create progress bar
var o = document.getElementById("progress");
var progress = o.appendChild(document.createElement("p"));
progress.appendChild(document.createTextNode("upload " + file.name));
// progress bar
xhr.upload.addEventListener("progress", function(e) {
var pc = parseInt(100 - (e.loaded / e.total * 100));
progress.style.backgroundPosition = pc + "% 0";
}, false);
// file received/failed
xhr.onreadystatechange = function(e) {
if (xhr.readyState == 4) {
progress.className = (xhr.status == 200 ? "success" : "failure");
}
};
// start upload
xhr.open("POST", document.getElementById("add_photo").action, true);
xhr.setRequestHeader("X_FILENAME", file.name);
xhr.send(file);
}
}
and the params for file are
File {webkitRelativePath: "", lastModifiedDate: Thu Nov 10 2011 09:40:39 GMT+1100 (AUS Eastern Summer Time), name: "WP_000012.jpg", type: "image/jpeg", size: 1344450}
After two days of messing with this, it turns out the problem was in my ajax upload, rails wasn't getting the file field at all.
I followed this blog post to get the ajax upload working,
https://github.com/newbamboo/example-ajax-upload/blob/master/public/index.html
and changed the name in my input file to < input type="file" name="wine_photo[photo]" > where before the name was just photo.
the html form now also has
<input name="utf8" type="hidden" value="✓">

encoding error with ajax upload (qqfile) and paperclip

I'm trying to get an ajax upload working with rails 3.1.3 and paperclip.
I found this solution to my problem Rails 3 get raw post data and write it to tmp file, but using this, I get an 'encoding undefined conversion error "\xFF" from ASCII-8BIT to UTF-8.
The error occurs at the line #user.photo = #user.photo = QqFile.parse(params[:qqfile], request)
I have not edited the code supplied in the previous answer, but I'll include it here so you don't have to switch back and forth.
the gem list paperclip, returns 2.5.2, 2.4.5, 2.3.8
my controller
def create
#user = User.new(params[:user])
#user.photo = QqFile.parse(params[:qqfile], request)
if #user.save
return render :json => #user
else
return render :json => #user.errors
end
end
qq_file.rb
# encoding: utf-8
require 'digest/sha1'
require 'mime/types'
# Usage (paperclip example)
# #asset.data = QqFile.new(params[:qqfile], request)
class QqFile < ::Tempfile
def initialize(filename, request, tmpdir = Dir::tmpdir)
#original_filename = filename
#request = request
super Digest::SHA1.hexdigest(filename), tmpdir
fetch
end
def self.parse(*args)
return args.first unless args.first.is_a?(String)
new(*args)
end
def fetch
self.write #request.raw_post
self.rewind
self
end
def original_filename
#original_filename
end
def content_type
types = MIME::Types.type_for(#request.content_type)
types.empty? ? #request.content_type : types.first.to_s
end
end
This was an encoding error related to Ruby 1.9.2 (or I believe Ruby 1.9+).
this github post lead to the answer
https://github.com/lassebunk/webcam_app/issues/1
You must specify raw_post.force_encoding("UTF-8") when reading an upload as far as I could tell (I'm not a great programmer).