My application scrapes information from various sites using Mechanize. Naturally, each site requires custom Mechanize code. Each site is stored in my database, including the url to scrape and the string name of an .rb file containing that site's Mechanize code. For this case, let's assume the scripts are available in the assets folder.
I would like to call http://example.com/site/:id, then have the show action dynamically choose which Mechanize script to run (say, #site.name + ".rb" ). The script will massage the data into a common model, so all sites can use the same show template.
I can't find a way to dynamically load a .rb script within an action and obtain the result. It may be easier to have the scripts return a JSON string, which I can parse before passing on to the template, but I can't see a solution for that either. Ideally, the script will run in the action's scope. The ugly solution is an enormous if-else chain (testing the site name to determine which code block to run), but there must be a better way.
Any suggestions would be greatly appreciated, as would any general solutions to running different code dependent upon the properties of database objects.
If you have all the code in your app already why are you eval'ing Ruby code?
Create classes like:
class GoogleSpider < Spider; end
class NewYorkTimesSpider < Spider; end
class SomeOtherSpider < Spider; end
And the site class will hold the class name that will be used, so you would be able to easily do something like this in your controller action:
def show
#site = Site.find(params[:id])
# name contains SomeOtherSpider
#process_output = #site.name.constantize.new.process
# do something with the output here
end
And then you don't need to mess around evaluating Ruby code, just call the class needed. You can even make them all singletons or keep them all in a hash for faster access.
Related
I am currently trying to extract property values from my properties file, but am running into some problems. I can't test this in ML query console, because the properties file doesn't exist there. I am currently trying to grab the values of the file like this
let $port := #{#properties["ml.properties-name"]}
I've also looked at
xdmp:document-get-properties(
$uri as xs:string,
$property as xs:QName
however that is limited to .xml files I believe. Does anyone have a way/work-around of accessing these values? I can't seem to find one I've looked at some documentation on Marklogic's website, but can't seem to get anything to work. The way I was accessing before was in ruby, through monkey-patching allowing me to access those private fields.The problem with that is the ruby script I call is only called once, while my .xqy file is ran every minute that sends args to another function. I need to access those args from the properties file, right now I just have them hard-coded in. Any thoughts?
Thanks
You cannot access deployment properties like that, but you can pass them along with deployment. If you create a new REST app with latest Roxy, you should get a copy of this config.xqy added to src/config/:
https://github.com/marklogic-community/roxy/blob/master/deploy/sample/custom-config.xqy
That file is treated specially when deployed to the modules database. Properties references are replaced inside there. In your case, add another variable, and give it a string value following the #ml.xyz pattern:
declare variable $c:port := "#ml.property-name";
You can then import the config lib, and use it in your code.
These so-called Deployer Substitutions are described in more detail on the Roxy wiki:
https://github.com/marklogic-community/roxy/wiki/Deployer-Substitutions
I am attempting to build a little modification in our code to allow easier A/B testing.
I'd like to know if I can somehow
have my regular code under the /templates directory
have any a/b code under /templates/_abtests/, but also follow the same hierarchy as the regular code. for example... an ab test can overwrite a file like '/templates/foo.tpl', and use instead '/templates/_abtests/testfoo/foo.tpl'
I tried changing the template directory when in a test. Right before calling the display method, I would check if a user is in a test, and if so, set up the template_dir accordingly. I'd assign an array with the 'ab' directory first, then the default. I am using Smarty2.
the problem with this is that it caches the first instance, and uses that as the template for the baseline and ab test case. ie: i have a parameter to force me into a test bucket, but the template is the same.
thoughts on how to achieve this? goal is to not have to add a bunch of template hooks (if/else) in the templates. and achieve this by simple template/file includes.
I believe that the solution to my problem could be to put templates into folders. ie: /templates/base/, /templates/test_foo/, etc.". then in my template_dir setting, set the array up based on what test we are in.
I had tried this with mobile/desktop before, and forgot about this solution.
I can extend the smarty_template class and override the display method to change the template_dir. adding the test directory first.
I have heard about both terms Injectable and Embeddable many times, but I am not getting actual meaning of it.
Please help me to understand both clearly.
Injectable means that something can be created and added to the main script while the script is running.
Embeddable means something can be added to a script or code before running it i.e before compilation or running of the script.
For better understanding lets take a website with a textbox as a context.
Now, In the textbox, suppose its very basic one. So, I can add a javascript into the textbox and when I will submit, it will run my JS script. This way, I am injecting my own script into the main page.
Now, suppose, I add an Iframe of another website to the HTML file of my website. In this way, when the website will be viewed, it contains the iframe. In this way, the Iframe is embedded to the website.
Injectable means that the object can be created and injected at run time. This is a hint to the compiler that this object will be managed outside the scope of the compilation and can be used at runtime to determine if the object was intended to be injected.
Embeddable means that the object can be serialized and stored in a column instead of as a separate table when the containing object is persisted. That also implies the lifetime of the embedded object is the same as the lifetime of the containing object.
I have a multi-step form where the user fills out info on several different pages. In conventional rails, you keep each resource separate in its own controller and you use the REST actions to manipulate the data.
In the conventional system I would have 3-5 different controllers (some steps are optional) for a single multi-step form. There's no real sense of "order" in the controllers if I do it the conventional way. A new developer coming on to the project has to learn what steps map to what steps and so forth.
On the other hand, I have thought about breaking convention and having a single controller that organizes the entire multi-step form. This controller would be full of methods like:
def personal_info
# code...
end
def person_info_update
# code...
end
def residence_info
# code...
end
def residence_info_update
# code...
end
# many more coupled methods like the above...
This single controller will get fairly long, but it's essentially a bunch of coupled methods: one for showing the step (form) and the other for updating and redirecting to the next step.
This would be breaking rails convention and I would have to setup my own routing.
But I'm curious how others have solved this problem? I know both CAN work, but I would like to know which is easier to maintain and code with in the long run.
A resource does not equal a page. I suspect that both ways would break a constraint on REST.
All of your interests have been with the View domain, which resides in your browser. If you want to display a single form in multiple parts you should do so using HTML, CSS etc.
Otherwise your just creating temporary storage on your servers for the forms progress.
I did something like this with https://github.com/pluginaweek/state_machine
The idea was to have one state per step of the form and simply render a different form partial depending on which state the actual resource has. The above gem let's you specify validations and callbacks for each states.
Like this, you can use the standard REST controller actions.
I need some help figuring out the best way to proceed with creating a Rails 3 engine(or plugin, and/or gem).
Apologies for the length of this question...here's part 1:
My company uses an email service provider to send all of our outbound customer emails. They have created a SOAP web service and I have incorporated it into a sample Rails 3 app. The goal of creating an app first was so that I could then take that code and turn it into a gem.
Here's some of the background: The SOAP service has 23 actions in all and, in creating my sample app, I grouped similar actions together. Some of these actions involve uploading/downloading mailing lists and HTML content via the SOAP WS and, as a result, there is a MySQL database with a few tables to store HTML content and lists as a sort of "staging area".
All in all, I have 5 models to contain the SOAP actions (they do not inherit from ActiveRecord::Base) and 3 models that interact with the MySQL database.
I also have a corresponding controller for each model and a view for each SOAP action that I used to help me test the actions as I implemented them.
So...I'm not sure where to go from here. My code needs a lot of DRY-ing up. For example, the WS requires that the user authentication info be sent in the envelope body of each request. So, that means each method in the model has the same auth info hard coded into it which is extremely repetitive; obviously I'd like for that to be cleaner. I also look back now through the code and see that the requests themselves are repetitive and could probably be consolidated.
All of that I think I can figure out on my own, but here is something that seems obvious but I can't figure out. How can I create methods that can be used in all of my models (thinking specifically of the user auth part of the equation).
Here's part 2:
My intention from the beginning has been to extract my code and package it into a gem incase any of my ESP's other clients could use it (plus I'll be using it in several different apps). However, I'd like for it to be very configurable. There should be a default minimal configuration (i.e. just models that wrap the SOAP actions) created just by adding the gem to a Gemfile. However, I'd also like for there to be some tools available (like generators or Rake tasks) to get a user started. What I have in mind is options to create migration files, models, controllers, or views (or the whole nine yards if they want).
So, here's where I'm stuck on knowing whether I should pursue the plugin or engine route. I read Jordan West's series on creating an engine and I really like the thought of that, but I'm not sure if that is the right route for me.
So if you've read this far and I haven't confused the hell out of you, I could use some guidance :)
Thanks
Let's answer your question in parts.
Part One
Ruby's flexibility means you can share code across all of your models extremely easily. Are they extending any sort of class? If they are, simply add the methods to the parent object like so:
class SOAPModel
def request(action, params)
# Request code goes in here
end
end
Then it's simply a case of calling request in your respective models. Alternatively, you could access this method statically with SOAPModel.request. It's really up to you. Otherwise, if (for some bizarre reason) you can't touch a parent object, you could define the methods dynamically:
[User, Post, Message, Comment, File].each do |model|
model.send :define_method, :request, proc { |action, params|
# Request code goes in here
}
end
It's Ruby, so there are tons of ways of doing it.
Part Two
Gems are more than flexible to handle your problem; both Rails and Rake are pretty smart and will look inside your gem (as long as it's in your environment file and Gemfile). Create a generators directory and a /name/name_generator.rb where name is the name of your generator. The just run rails g name and you're there. Same goes for Rake (tasks).
I hope that helps!