Hi I have wanted to learn Scrapy and use it to scrape dynamic data off of webpages, and use it in a website backend.
When I went to the official docs, I go to know that V.2.0 just came out.
Given that I'm new to scrappy, and plan to develop an autonomous hosted application, I was wondering whether I should choose v 1.8 over v 2.0 because bugs would've been worked out better and there'll be more tutorials etc. But on the other hand, I'll end up learning 2.0 anyway in the future, so maybe I should start with 2.0 itself.
So I have two questions:
Are there any major changes from v1.8 to v.2.0 (I am aware that there are release notes that accompany each version, but the only thing that I can really understand is that Python 2 support was removed; everything else uses terminology that I don't understand.)
I'd be grateful for your advice on which one I should opt for.
I have worked with Selenium & BeatifulSoup4 on 1 project before hand, which involved scraping stock price and relative strength index, and using that as a part of Flask backed web app.
Always use the latest Scrapy release for a new project, unless you cannot for some reason.
There are no major changes in how Scrapy works between 1.8 and 2.0; upgrading from 1.8 to 2.0 should be as easy as upgrading from 1.7 to 1.8.
Related
In casting my eye around for a good, stable, javascript UI framework I've been drawn to Dojo. In particular, the Dijit set of UI components look really good, and are very good as far as accessibility is concerned.
My question is this: is it worth learning Dojo 1.10 (or at least dabbling and familiarising myself with it) before 2.0 comes out? Or is 2.0 going to be a "throw the baby out the with bath water" complete rewrite?
I could probably learn TypeScript as that would be a useful investment of my time. I'm just eager to start doing something productive in Dojo as I really think 2.0 is going to see a major resurgence of interest and it looks like something I'll want to commit to for 5, 10 years or more (hopefully!). Dojo seems to have stood the test of time well since it came out, and there's no reason to think that 2.0 won't be as stable, either (from what I can tell).
Thanks.
You could get familiar with Dojo Toolit 1.10 and later get updated with the new version.
In my opinion some concept behind Dojo will be remain very similar in the new version, for example:
Modules/Amd, Ajax Deferreds, Promises DOM Basics.
As mention in the Ken's comment usefull source of onformation on dojo 2 can be found here:
dojotoolkit.org/community/roadmap
I'm currently trying to upgrade a custom app from sdk 1.x to new version 2.0rc3. I saw the Task Board app or Custom Board app in the AppCatalogs have ability to group the tasks by WorkProduct, however I have unable to make it work.
I have copied/pasted exactly the given example here and it does not work also.
In the screenshot from this link: https://rally1.rallydev.com/apps/x/doc/#!/example/groupable-board
it shows the cardboard grouped by the owner but when I try it, cards item is not sorted into group as expected.
if anyone know how to make it work, please help me.
Thanks!
The swimlanes feature that rowConfig enables is not available in 2.0rc3. The version x SDK and documentation are not officially released, but we will be releasing a new 2.0 version early in 2015 which will definitely include support for this feature.
In the meantime you can use /apps/x/sdk.js to build apps with the usual caveat that this is not a stable version of the API yet and we make no guarantee that any apps written against it will continue to work.
I have an idea of what the Jvcl is..it's a set of components and you install them, but what I'm really interested in is the Jedi win32 API conversion stuff. I'm unsure what to do with them or how to use them.
You don't install them do you? Say, for instance, I want to use the API SendInput or similar; how do I find how to use it within Jedi API? Is that what Jedi API is even for?
I've looked all over their site and searched for tutorials with no luck. I even downloaded all the help files I could find but I'm still lost.
The JEDI API is made up of a number of header translations of the Windows API. The Delphi RTL has a good portion of the Windows API translated. This is implemented in a number of units, the main one being the Windows unit.
However, the Delphi header translations are incomplete. What's more each new version of Windows comes with a swathe of new APIs. Embarcadero catches up slowly and in some cases chooses not to translate.
The JEDI API project attempts to be a more complete set of header translations. It is still incomplete, but it has more coverage than the units shipped with Delphi. It is particularly useful if you are using an older version of Delphi where the supplied header translations are very out of date.
I have been asked to develop a new, small, custom-specific CRM (Customer Relationship Manager) that will be used mainly on Linux desktops (compatibility with Windows and Mac OS X would be appreciated but it is not required).
This seems to be a good opportunity to try the new Vala language and some of its libraries (most notably libgda and the rest of Gnome-DB) but, of course, I still have to deliver a working product to the customer in time so... I'm still scratching my head and wondering.
To develop this application I would need:
A "glue" language (Vala itself). This is OK.
A GUI Library (GKT+ 2.X or 3.X). This is OK.
A database abstraction layer (libgda). Here I have some doubt.
Maybe a MVC framework like Bakery (Bakery 2.6 seems to be working
with GTKmm 2.4 only. It does not work with the GObject-enabled GTKmm
3, as long as I can see.).
Maybe a ORM like Hiberlite (libgda supplies data-aware widgets
and other tools but it is not a full-blown ORM, as long as I know).
At the moment, I'm confident about the first two items, only. Even the real amount of Vala support for libgda is not very clear to me (The ValaDoc describes as supported the interface of a old version of LibGDA while the Gnome-DB website says that the new 4.2 and 5.X versions of the library are GObject- and Vala- enabled). Most likely, Bakery and Hiberlite would not be available any time soon for Vala.
The nearest alternative seems to be:
C++
GTKmm (2.X)
Maybe Bakery 2.6
libgda
Maybe Hiberlite
A more mature stack but... maybe so mature to be fated.
Hence: would you try Vala for a new database-centric project like this?
Or would you wait for a more mature and more rich Vala ecosystem?
Thanks
Vala just means native compilation without requiring a framework (and versions) to bother about. Connecting to database still looks premature and definitely undocumeted (that's how I came to this post). Besides, there is no IDE. Glade is not really and IDE, but an interface designer.
Try out Lazarus and you will be in for a surprise, how conveniently database front ends can be developed. Pretty mature, native compilation, ready to use third party components, database support right through the IDE, options of using Gtk or Qt.
And it gives native exe's on Windows, Linux and Mac. Nothing comes even remotely close if you are developing cross-platform database front ends. Development time would be a fraction and performance comparable to C, if not equal.
We are in mid of a project where we have used an extended Dojo 1.1.1 to meet the customer requirement and add richness.
But there are quite some bugs and performance problems with this version of Dojo and
we are looking ahead to migrate the Dojo version to overcome both the issues, but the migration cycle seems to be quite painful and may not be yield expected result.
The concern we have is with the various extension which we have created with the version of Dojo for components that were provided in 1.1.1 and the impact on them after migration.But, the advantage we see are equally important.
As per Dojo , they have kept some level of compatibility with version 1.1.1 but i have not seen any discussion around this anywhere.
Has any body else previously done
migrated between Dojo version?
Will the components like Grid will
work as expected or will i need to
carry out a refactoring exercise?
Do we have any commercial support
available as the forum seems to
deprecated?
Any help or suggestions are welcome
Dojo has had a policy of freezing and supporting public APIs since 1.0. Migrations prior to 1.0 were extremely painful. Now, it should be much better, provided you use only public APIs. Code written for stable JS APIs in Dojo or Dijit in 1.1 should largely still work. Exceptions are noted in the release notes, which you should explore (good luck finding them... unfortunately the site is a bit of a mess)
If you wrote any custom widgets, you're probably in for some extra work. dojox.grid was not particularly stable at that point, and it has also seen a major rewrite since then (there is an old 'compat' layer you may wish to use)
Regarding for forum, like the note says, you can either use the active dojo-interest mailing list or post questions here at SO. There are some firms which offer commercial support, but that's outside of the scope of Dojo as an open source project. (try googling 'Dojo commercial support' or asking on dojo-interest)
I have done 5 dojo migrations now (from 0.2 -> 1.4) over the last few years. Although the API does not change, you will often have coded in workarounds that no longer work after upgrading. Things I have noticed:
quality in 1.4 is VERY good and worth
upgrading to (even from 1.3)
although
the API does not change, little
things that are not public often
change slightly (diji.Tree
itemNodeMap -> itemNodesMap in 1.4)
build options are usually added each
release but not always publicised -
strage really as they are always
useful improvement
since you are 1.1.1, you should change all your set attribute calls to 'attr' - this could take a while to do.
As for commercial support, you could try Sitepen