I want to kick myself to learning network programming, starting with implementing existing network protocols. I've finished the (rudimentary) design and will start coding soon. The problem I haven't been able to figure out solution to is related to testing: I only have one Windows laptop running Windows 7 Pro with only a recovery disc (no installation disc) that obviously cannot be used on a VM.
Hard-coding input/output data clearly isn't a good way to test any sort of program. So, what solutions can I look into?
Thanks for your time.
P.S.: In case this matters, I'll do the coding in C++.
You can run a client and a server on the same machine. When accessing the network layer, just use the local callback loop (127.0.0.1 for ipv4 or ::1 for ipv6) to connect to your server when you run the client.
If you provide the APIs that you will be using (wininet, APR, Boost etc) a more detailed answer would be available.
What about a VM with Ubuntu or some other distro of Linux?
Related
I have a Python / REDIS service running on my desk that I want to move to my Blue-Domino-hosted site. I've got Python available on the server, but not REDIS. They don't give me root access to my Debian VM so I can't git, extract, and install myself from a Unix prompt.
Their tech support might do the install for me, but they need me to point them to server requirements, which I don't see on the REDIS download page.
I could probably FTP binaries to the site if they were available, but that's dicey.
Has anyone dealt with this?
Installing Redis is actually quite easy, from source. It doesn't have any dependencies, so just download the tarball, unzip it, and follow the install instructions. I'm always afraid of doing that sort of stuff, but with Redis it really was a breeze. If you don't dare to do it their tech support should be able to do it.
If it is Intel/AMD server, you can compile the Redis somewhere (32 bit version for example), and upload it as binary. Then start it with Python. I did this myself couples weeks ago.
For port you will need to use something over 1000. I don't recommend to use default port. Remember to change LogLevel too. Daemonize works well as non-root too.
Some servers blocks all external ports, so you will not be able to connect to Redis from outside, but this will be a problem only if you connect from different machine. For same machine should be OK, since is "internal".
However, I am unsure how hosting administrator will react when he sees the process running :) I personally will kill it immediately.
There is other option as well - check service like Redis4you.com . But their free account is small, you probably will need to spend some money for more RAM.
Is your hosting provider looking for a minimum set of system requirements for running Redis? This is indeed not listed on the Redis website. Probably because there aren't many exotic requirements. Also it depends a lot on your use case. Basically what you need to run Redis is:
Operating system: Unix like, Linux is recommended (one reason to favor Linux I've heard of is the performance of its TCP/IP stack)
Tools: GCC, make, (git).
Memory: lots (no seriously this depends on your use case, but because Redis keeps everything in-memory you need a least more RAM than the size of your dataset).
Disk: disk access for making snapshots.
The problem seems to be dealing with something non-traditional with my BlueDomino hosting. Since this project is a new venture, I think the best course for me is to rent a small Linux VM from rackspace and forget about the BD hosting.
Currently, I am writing an application that utilizes WMI to scan all the computers on our Active Directory network.
I'm interested in testing the program against all flavors of Windows machines in a testing environment.
Is there a way to similuate this environment in VMware or something?
Any ideas?
VMWare works well and can host many virtual computers on a single physical computer. You can also put the virtual computers on your active directory network.
If your goal is to set up a separate large network for testing that has it's own AD server you can look into Amazon EC2 for testing. The advantage here is once you setup your set of servers, you can turn them on and off as needed and only pay for the time actually used ($0.12 per hour).
http://aws.amazon.com/
You can use network simulation: http://en.wikipedia.org/wiki/Network_simulation
and good GPL tool is http://www.nsnam.org/
You have two options.
You probably have it right, with VMWare this is easy, try looking for cloning tools. If you plan on copying and pasting the image, you will get several problems (computer Guids repeated, Network Computer Names repeated, etc)
You can also "mock" the WMI response by wrapping the WMI methods that you want to call and implementing an interface, using Rhino Mock or NMock if you are working in .NET (which I assume you are).
Simplified, I have an application where data is intended to flow over the internet between two servers. Ideally, I'd like to test at what point the software ceases to function. At what lowerbound limit (bandwidth, latency, dropped packets) do things stop working to test the reliability of the software.
What I thought I would do was the following:
Setup up 3 machines (VMware instances)
Install the 2 applications on two of the servers.
Setup up the 3rd server to sit between the two machines by doing some sort of magic with Routing and Remote Access on Windows 2003
Install either Traffic Shaper XP or NetLimiter to limit the bandwidth
Run something like TMnetSim Network Simulator to simulate a bad connection.
Does this sound like a good idea or are there easier/better ways of doing this? I'm not that comfortable on Linux and my team mates are even less so.
WANem does exactly this. We have used it both in a virtual machine on the desktop and on a dedicated old pc and it worked great. It can simulate all sorts of broken connectivity.
FreeBSDs ipfw has provisions to simulate links with a given bandwith, latency or error rate. You could use that FreeBSD machine as your machine "in the middle" in your above setup.
You probably can also run at least one of the endpoints on the same machine if you want to reduce the amount of servers involved.
Someone actually packaged up the settings and whatnot necessary for the FreeBSD solution to this problem and they call it DUMMYNET.
It simulates/enforces queue and bandwidth limitations, delays, packet losses, and multipath effects. It also implements a variant of Weighted Fair Queueing called WF2Q+. It can be used on user's workstations, or on FreeBSD machines acting as routers or bridges.
It can simulate exactly what you want, and its free and will boot onto commodity hardware. They even have a canned install of it that is small enough to put on a floppy disk (!) that you can download at that link.
Maybe it is time to learn a bit about Linux because adding a 50ms delay on every outgoing packet can be done in typing just one line:
tc qdisc add dev eth0 root netem delay 50ms
For more see the Linux Traffic Control HOWTO
We had a similar requirement some ten years ago - I'll see if I can recall how we managed it.
If I remember, we wrote a socket proxy program which was controlled by inetd on a UNIX box. This socket would accept connections from a client and open equivalent sessions through to the server. It would then loop, passing messages in both directions.
The way we achieved WAN characteristics was to introduce random delays (with upper and lower limits) in both the connection establishment and the passing of data once the link was up.
It also had the feature to drop the link occasionally as WAN links were less reliable for us than local traffic.
I recall we had to make it threaded to stop the delays from affecting reverse traffic on the link.
There is a very good (and free) Microsoft solution for that, we use it for quite some time and it works great, it can very easily simulate every thing(packet loss, low bandwidth, disconnection, latency....)
This is the best solution i found for a windows environment
More information and a download link can be found here: MARCO blog post
this product has gone some evolution and it is now integrated into visual studio as part of the automation testing, but i found the use of the standalone(that is quite hard to find, so keep a local copy) to work much better. keep in mind that you need at least two computers(or VMs) since you need to pass through a network adapter in order for the application to work its magic.
I recently build myself a semi beef up PC (Q9450, 8GB DDR2 1066, 1TB HDD, Dual 8600GT, Vista Ultimate and Dual 22' Monitors) and I'm evaluating whether i should develop on a VPC/VMWare session on top of Vista or not?
One benefit I can see is that I can run the same VM on my Vista laptop so my development environment is the same on any of my machines. I also plan on purchasing a MBP before the end of the year as well.
Found a couple of articles online that semi-help Here
Any other thoughts would be really appreciated?
For webdevelopment I like to have the serverpart separeted out into a VM. My current setup is a Macbook Pro with several Debian VM's inside. I like the isolation aspect of it. I can try new software on the servers and have the ability to revert them back if something is messed up.
I do the programming via network-share (samba) in Textmate on the host system.
Another advantage of a VM is having a clean installed base. I use my desktop and laptop for lots of things aside from development. You never know when a piece of software you install is going to conflict, or if the little tweaks and what not you play around with are going to trash your OS. Reinstalling/configuring all your tools so they are exactly the way you want them can take quite some time. If you have a backup of your Development VM Image you can mess up your PC as much as you want but still be able to code without downtime. It also allows you to run Win/Visual Studio/Etc on a box that you would otherwise prefer Linux or MacOS on.
You can also make multiple copies of the same Image and use each one for a separate project.
Being able to transition between a laptop/desktop/server/remote connection, and always be in the same environment is also very helpful.
One problem I found (at least when using VMWare Server) is that no matter how fast your machine is, the screen refresh rate is still around ~30hz. That makes for a slightly unpleasant experience after using it for a while.
Where I'm working at now I use a VM for all of my development because I don't have admin rights to my base copy of XP.
Pros:
I like using a VM's because it give you some flexibility - you can switch between machines - have programs running on both and have a cool environment to work on.
Cons:
You have to boot up multiple operating systems. This takes time, memory and resources.
Clipboard operations on VM's can be interesting at times. Sometimes copying to clipboard does not work or gets mixed up between VM's. (Using VMWare).
File operations can be interesting when you plug in USB drives and other external devices. VM's sometimes do not see the devices, sometimes it does.
If your VM image become corrupt - you can easily loose everything in it.... unless it is backed up.....
It's great for presenting development talks, you can revert to a snapshot and give the talk from the exact same starting point each time.
Bulk-up your RAM on your future MacBookPro if VMWare will be used. I haven't (yet) and the performance with several other (mac-side) apps open really starts to feel sluggish.
All the best.
I was doing some work with Visual Studio recently with a Windows XP vm on Linux and somehow the guys who made the vm (vmware) made the windows machine actually run faster. We did some time tests to make sure and it wasn't major, but a few things (autocomplete for example) really did pop up faster.
If you are on Windows, Virtual PC is pretty decent for development work. VMWare Virtual Server is not really designed for use as a desktop and you will get very tired of it with any prolonged use. Sun's VirtualBox is another option competing with Virtual PC. VMWare has a workstation product but it is not free.
Typically, I do development on the real desktop (non-virtual) and then deploy or test to virtual machines which I can snapshot and roll back easily.
For a long time, we were developing on very early versions of Visual Studio 2005 and the associated .Net bits that went along with it. To protect our real machines from the various problems associated with pre-release software, we did all of our development work inside virtual machines. It worked amazingly well. I've been considering moving back to that model as it makes upgrading the physical hardware a snap (not to mention making it easier to deal with hardware failures by just replacing the entire machine): you just copy the VM image over.
On my current machine (A Core2Duo with 4GB of RAM), the performance drop when running one VM is almost not noticeable. Running two VMs, however, is painful.
I also can't figure out how to get VMWare Server to work across two monitors well.
I wouldnt want to develop in a VM so much as test things in a VM. For instance, it might be nice to set up a couple VM's to emulate an n-tier architecture, or a client-server setup or finally simply to test code on multiple OSs
It depends what you are developing and in what language.
VM's tend to take a fairly hard hit on disk access, so compiling may slow down significantly, especially for large C/C++ projects. Not sure if this would be such an issue with .NET/Java.
If you are doing anything that is graphics intensive (3D, video, etc) then I would steer clear of a VM too.
I don't know if it is so useful as a development platform unless you are doing something that ties into software you don't want to have installed on your regular working machine or that needs to work around a certain event that you need to be able to reset on a regular basis. It can also be handy when you are working with code that risks crashing your computer as it will at least only crash your VM.
It is brilliant for testing different configurations and setups- working with installers and so on, that is where virtualisation really shines as far as I am concerned, being able to roll things back whenever you need to and run through stuff repeatedy is amazingly useful for identifying problems before your end users run into them.
While doing development at home, I have to VPN into my company to be able to use the collaborative tools that are on the intranet. I also have a desktop + laptop that are hooked together through Synergy.
The problem that I have is that our VPN software wants things to be so secure that it will force all network routing through the VPN gateway -- even if I'm using additional NICs to network my desktop and laptop through a separate private network. The end result is that I can't use Synergy between my desktop and laptop and VPN into my company at the same time.
The solution suggested to me by a co-worker was to setup a VM instance on my desktop and use that for all my VPN needs. Works like a charm!
Speaking from personal experience developing java in an Ubuntu VM on Windows 7, I've found this to be quite productive. Mainly because my local IT support on the ground supports Windows 7, so I can do things like access all the local file shares and printers in Windows, and then config my Ubuntu VM to my heart's content.
Huge productivity benefits around remote access and desktop sharing. Windows allowed me to very quickly and easily use tools like logmein.com and join.me to access my machine from home and to desktop share the VM with other people in the company (both work seamlessly with the VM in a nearly full screen window). Neither of these services are supported on Linux, and I wouldn't want to deal with all the associated VNC/X setup and network config on Ubuntu.
My machine is fairly beefy. Quad core, with 16Gb RAM - 8Gb for the VM. Java dev in the VM is pretty quick.
Heres the problem. I use around three different machines for development. My partner is using two. We have to go through the same freaking set up procedure on all five machines to get to work.
Working with a php project here, so:
Install and configure, PDT, a php debugger, and some version of XAMPP.
Then possible install an svn client, and any other tools.
Again, to each of the five machines.
What if, instead, we did all of this once, in a virtual machine that is set up with the same stack, same versions, as the production server. Then each of us could grab a copy of the VM image, run that image on each of the five machines and do all of our development in that VM. Put Eclipse, apache, mysql, the works, all in that vm.
The only negative of this approach, and please correct me on the only part, is performance. Is it really that big of an issue though? The slowest machine out of the five is a Samsung NC10 powered by an Intel Atom 1.6 ghz processor.
Do you think this is possible and practically usable? Or am I crazy?
I use a VM for development (running on my laptop) and have never had performance problems. Another approach that you could take would be to image the drive in the state that you want. Use Acronis or Ghost to re-image each machine when you need to. Only takes about 5-10 minutes to restore an image on any modern PC.
I use a VM for all my "work" as it keeps it away from my "play". This set up allows me to use the office VPN without exposing my whole machine to the office environment (which I trust about as much as the internets. ;-) Also I don't have to worry about messing up my development environment by trying games or other software. My work VM is currently running inside VirtualBox but I have used VMWare in the past. I have only noticed performance issues when using graphic intensive programs like Webex or the Terminal Server Client.
It can certainly be done. What turns me off is the size of the VM image, which would normally be several GBs. Having it on a network share means it can take longer to transfer then your current setup process takes. I guess an external hard drive would be the easiest way to move it around.
Performance wouldn't be an issue with any web development.
I have to ask why your current machines need to be "re-imaged" each time you sit down for work?
If you're using Windows you'll probably want to use SYSPREP on the master image so that the 'mini-setup' runs when you boot up the virtual machines for the first time.
Otherwise in terms of Windows' point of view, the machines have the exact same SID, hostname and other things - running multiple machines with the same SID on the same network can cause tons of headaches. Even more if you want them to communicate with each other.
I've run websphere for zSeries on a vmware virtual machine with no problem and websphere is more resource intensive then any PHP stack. I find that having a multi core machine or at least hyper threading makes it run a lot faster.
With vmware, disk operations are slower. For PHP development I doubt it would be a problem, but you'd definitely notice it if you are compiling a large C++ project. There is also Sun's VirtualBox which is free, and the latest version is rather nice (but I haven't looked at how slow disk operations are yet).
I am using that idea in practice. Virtual machines are generally great for development.
To run on multiple operating systems and multiple separate development environments.
Preserver older development environments for later support.
Can be easily backed up, when hard drive crashes no need to start from beginning.
Can be copied from developer to another, so everyone don't have to do tedious installations and configurations.
Down sides are:
Virtual machines are slower, you need more powerful computers than you would need otherwise. I would recommend having at least 4 G of ram, but preferably more like 16, fast multi core processors and fast hard drives.
Copying Windows OS virtual machines, each used copy of virtual machine should have it's own product key. When you make a copy, it needs to be registered with new product key.
Did you think about a software configuration manager like ansible, chef or puppet? With such software automation of such tasks is very easy! It can even create fresh vm and then configure it.