Data flow visualisation - data-visualization

I'm searching for a program where I can visualize all my data flow. I've made a simple example:
Is there a way how I can visualize all the data. I've found examples of "Prefuse" and "Flare". But there's too much to add to the examples. Isn't there a nice program wich has it all?'
it's preffered when I can connect the data to a database. So the whole process will be automated.
it's preffered when I can collaps/uncollaps the systems and application flow(e.g. when I click on an application, the arrows appear).
I hope someone can help.

you can use Graphviz (http://www.graphviz.org/) for creating the vizualization.
for the database connection, you could use a programming language to generate the input to the graphviz program.
I had this same problematic and used a combination of SAS and Graphviz to solve it.
hope that helps.

Related

What are the main points that must be documented for a data integration project?

I am working on a data integration project using Talend.
I have many input sources heterogeneous, I make transformations and I save output data to many output sources. Actually, I am doing Extraction, Transformation and Load (ETL).
My Talend Job is executing everyday on a linux server (the production). I have a development and test environment on a windows VM, ... In fact, I have many things I want to document and I don't really know how. I used to document web development projects (just the frontend), but not data integration projects.
Can you guys help me with some keywords, examples, templates, so that I can make a clear documentation for my client ?
Thanks in advance :)
While I can't speak to your organizational needs, Talend is designed to be largely self documenting. If you were diligent in filling out the documentation and descriptions in your job, you can right click the job and select 'Generate Doc as HTML'.

Is there any recommended way to automate module port connection?

I'm trying to understand or research about the best practices of ASIC design in verilog. I'm working on a medium size block with ~20 sub modules (each ~1000 lines of code). It's a painstaking job to manually instantiate all the sub-module and do port connection to create top level RTL.
I want to write a script to automate this. As long as we can define the input/outputs of all the sub-modules and how each sub-modules are connected with each other, it shouldn't be super hard to auto-generate the top level. I don't have enough expertise in design automation though. I was wondering if anyone can give me some pointers about how to get started.
Is there any open source tool to achieve what I'm trying to do? I didn't find any so far.
Is there any standardize way to generate synthesizable code of this sort?
I'll highly appreciate any sort of input or advice.
Depending on what text editor you're using, you might be able to use some pre-existing tools. There is an add-on for Emacs that supports auto-instantiating and connecting signals, assuming that you follow certain naming conventions:
http://www.veripool.org/wiki/verilog-mode/Verilog-mode_veritedium
For the vim users out there, there are a couple of plugins that allow using the Emacs script, such as this one:
http://www.vim.org/scripts/script.php?script_id=1875

Automation scripts: autoitscript vs ptfbpro

I try to use this 2 projects for primitive gui testing automation:
http://www.ptfbpro.com/
http://www.autoitscript.com/
And I can't make my choice.
Can somebody explain me: why(in 2 or 3 lines) he use one of them(or other please specify)?
I use AutoIt...
because it's free, well documented (not only) from inside of the Scite Editor and you can easily compile your script into a small executable or even create a complete GUI and there is a very good community in the forums and around here. And its Basic-Like Syntax is really easy to understand, there are functions and even a foreach-syntax, dynamic arrays and lots of additional functions from other users... There's good integration with other programming languages and from the use of so many WinAPI functions you lack of very little possibilities. It can automate IE usage without even displaying a browser window and send network packages, you can send Keystrokes like a user sitting in front of your screen and there's the AU3Record Tool which allows you to just record a Macro and replay it or save it as a script and then you can easily optimize it and edit it for your needs. Or use the AutoIt Window Info tool to see all the possible handlings for your application, you can interact with any kind of program output/display according to different algorithms you may invent.
Enough facts? ;-)
Go with Autoit3. It 's a lot more reliable, and you have a complete script language. Ptfbpro is only a tool (not free), nothing more. AUtoit3 has a lot of contributors that can help you in your process, Ptfbpro is dead.
If you want a script taht really do what you want, just go for AutoIt. Ptfbpro can't be used as a professional tool.
Autoit3 as well. You really can't beat it for being free and so easy to use.

Script or automate feature class creation in ESRI/ArcSDE

I'm looking for info on how to write SQL scripts to automate the creation of a versioned feature class in ArcSDE I want to be able to automate the process itself as well as put the scripts under version control. Can anyone point me to a resource that explains how to do this?
Is this even possible? It seems like there are lots of interrelationships between tables and data when a feature class is added.
P.S. It doesn't have to be pure SQL, but it should be some kind of scripting so we can save to version control and run outside of ESRI desktop tools.
It would be exceedingly difficult to do this in SQL without breaking your database. As you indicated, there are a lot of relationships between the tables, and if you don't get it right, then your database is messed up.
If you're looking at a scripted solution, you might want to look at creating a Python script to create the versioned feature class. There are a few ways to do it, from creating a raw Python script in Notepad, to creating a geoprocessing model in ArcCatalog and exporting the model to a Python script.
Here's a link to the ESRI help on geoprocessing scripts: http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?TopicName=An_overview_of_writing_geoprocessing_scripts

3d files in vb.net

I know this will be a difficult question, so I am not necessarily looking for a direct answer but maybe a tutorial or a point in the right direction.
What I am doing is programing a robot that will be controlled by a remote operator. We have a 3D rendering of the robot in SolidWorks. What I am looking to do is get the 3D file into VB (probably using DX9) and be able to manipulate it using code so that the remote operator will have a better idea of what the robot is doing. The operator will also have live video to look at, but that doesn't really matter for this question.
Any help would be greatly appreciated. Thanks!
Sounds like a tough idea to implement. Well, for VB you are stuck with MDX 1.1(Comes with DirectX SDK) or SlimDX (or other 3rd party Managed DirectX wrapper). The latest XNA (replacement for MDX 1.1/2.0b) is only available for C# coder. You can try some workaround but it's not recommended and you won't get much community support. These are the least you need to get your VB to display some 3d stuffs.
If you want to save some trouble, you could use ready made game engine to simplified you job. Try Ogre, and it's managed wrapper MOgre. It was one of the candidate for my project. But I ended up with SlimDX due to Ogre not supporting video very well. But since video is not your requirement, you can really consider it. Most sample would be in C# also, so you need to convert to VB.Net to use. It won't be hard.
Here comes the harder part, you need to export your model exported from SolidWorks to DirectX Format (*.x). I did a quick search in google and only found a few paid tools to do that. You might need to spend a bit on that or spend more time looking for free converter tools.
That's about it. If you have more question, post again. Good Luck
I'm not sure what the real question is but what I suspect that you are trying to do is to be able to manipulate a SW model of a robot with some sort of a manual input. Assuming that this is the correct question, there are two aspects that need to be dwelt with:
1) The Solidworks module: Once the model of the robot is working properly in SW, a program can be written in VB.Net that can manipulate the positional mates for each of the joints. Also using VB, a window can be programmed with slide bars etc. that will allow the operator to be able to "remotely" control the robot. Once this is done, there is a great opportunity to setup a table that could store the sequencial steps. When completed, the VB program could be further developed to allow the robot to "cycle" through a sequence of moves. If any obstacles are also added to the model, this would be a great tool for collission detection and training off line.
2) If the question also includes the incorporation of a physical operator pendent there are a number of potential solutions for this. It would be hoped that the robot software would provide a VB library for communicating and commanding the Robot programatically. If this is the case, then the VB code could then be developed with a "run" mode where the SW robot is controlled by the operator pendent, instead of the controls in the VB window, (as mentioned above). This would then allow the opertor to work "offline" with a virtual robot.
Hope this helps.