Virtual drives on linux

One of the issues I was facing in my linux migration efforts was how to create and use virtual drives, so that I can mount ISO images of CDs etc.

So here is how to do it..

Virtual devices are achieved in linux using loop devices.
To create a new loop device, we use the losetup command. For eg. say we need to mount the cd image file /temp/varmathe/damn-small-linux.iso ad a cd drive. Here is how to do it.

1. Locate the first available loop device

opensuse103-vm:/temp/varmathe # losetup -f
/dev/loop0

2. Associate this device node to the iso file

opensuse103-vm:/temp/varmathe # losetup /dev/loop0 damn-small-linux-3.4.4.iso

3. Now mount the cd somewhere on our filesystem hierarchy.

opensuse103-vm:/temp/varmathe # mount -t iso9660 /dev/loop0 /mnt/vcd
opensuse103-vm:/temp/varmathe # cd /mnt/vcd
opensuse103-vm:/mnt/vcd # ls
boot  index.html  KNOPPIX  lost+found

Additionally, it might be a wise idea to mount cd/dvd images etc as read only..

4. Use the volume.. (he he :D )

5. Now that we are done and want to unmount the device and release the file

opensuse103-vm:/temp/varmathe # umount /mnt/vcd
opensuse103-vm:/temp/varmathe # losetup -d /dev/loop0

The -d option detaches the file from the loop device

Steps 1,2, and 3 can be combined into a single step as

opensuse103-vm:/home/varmathe # mount -t iso9660 -o loop /temp/varmathe/damn-small-linux-3.4.4.iso /mnt/vcd

While the above single step approach looks like it should have been explained first; I deliberately explained the broken approach for a reason. If you only need the device node, say for eg, to pass that on to a virtual machine; then you don't need to do the mount.. The "losetup" steps alone would do.

.

C++ language standards

Find the official standards for the language at

http://www.open-std.org/jtc1/sc22/wg21/

A GPL licensed thorough c++ reference

http://www.icce.rug.nl/documents/cplusplus/

A bunch of open source c++ libraries

http://www.boost.org

.

Some concerns on Computer Science education today

Where are the Software Engineers of Tomorrow

http://www.stsc.hill.af.mil/CrossTalk/2008/01/0801DewarSchonberg.html

http://mitpress.mit.edu/sicp/

.

Setting up qemu with networking on a linux host

I wanted to install Qemu with networking enabled on my linux box (Which incidentally is on a Virtual Machine built on vmware). Through the setting up process, I had a tough time figuring out how to go about with the whole thing as I could not find any single document which precisely described what I wanted. But anyways, now I've figured it all out.. And so I've decided to share whatever I've garnered with you through this post.

Objective: To set up Microsoft Windows XP on a Qemu simulated PC with an OpenSUSE 10.3 linux box serving as host.

Host machine: OpenSUSE 10.3 linux on Intel i386 based PC

Packages used:
qemu-0.9.1-i386.tar.gz : http://fabrice.bellard.free.fr/qemu/
vde2-2.1.6.tar.gz : http://vde.sourceforge.net/

I wanted to compile qemu from source, but apparently qemu needs the gcc3.x compilers whereas OpenSUSE 10.3 I have on my system has the gcc4.x version.

Step1: extract qemu-0.9.1-i386.tar.gz int '/' as root.

Step2: extract vde2-2.1.6.tar.gz into a temporary location and build it

./configure
make

and then as root

make install

Step3: (As root) set up a VDE switch with one port connected to the host using a tap interface

host$ vde_switch -s /tmp/switch -tap tap0 -m 666 [-daemon]

The "-daemon" part is option; it will cause vde_switch to run in the background, whereas, if you do not give it, you get access to the switch console

Step4: (As root) set up an ip address for the newly created tap interface on the host

host$ ifconfig tap0 10.0.0.1

Step5: set up our virtual machine. For this,

5.1 Create a working directory for the machine and copy bios files into it.

host$ mkdir qemu-vm
host$ cd qemu-vm
host$ cp /usr/local/share/qemu/{bios,vgabios-cirrus}.bin ./

5.2 Create a disk for the machine.

host$ qemu-img -f qcow win-xp.qcow 4G

5.3 Now start the virtual machine as follows, with the windows-XP CD in the /dev/cdrom drive. Have the Product key ready at hand.

host$ vdeqemu -L . -m 128 -boot d -hda win-xp.qcow \
-cdrom /dev/cdrom -soundhw all -localtime -M pc \
-net nic,vlan=0 -net vde,vlan=0,sock=/tmp/switch

here, "-m 128" gives the virtual machine 128 mb RAM
"-boot d" makes cdrom as the default boot device.. Later you can set this to "-boot c" once you have an OS installed on the hard disk, to make hard disk as the default
"-net nic,vlan=0" adds one network interface card onto the virtual machine
"-net vde,vlan=0,sock=/tmp/switch" connects the above NIC to the switch we created.

5.4 When the system boots up, install windows on this guest machine. You will notice that it has one ethernet adapter for which the cable is already plugged (to our switch as we specified).

5.5 Configure IP address on this interface so that this interface is on the same Network (subnet) as the host's tap interface. In our case, we use "10.0.0.2" as the IP. Also set the guest's default gateway as 10.0.0.1 and provide your service provider's DNS ip to the guest.

Step6: Now you can try pinging 10.0.0.1 from the guest and 10.0.0.2 from the host. You will see that the pings go through successfully. However, the guest still cannot access the internet. For this we need to set up NAT (Network Address Translation) on the host ( or some form of bridging b/w the tap0 interface and the host's eth0 interface)

Step7: (As root) Set up the NAT.

host$ echo "1" > /proc/sys/net/ipv4/ip_forward
host$ iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
host$ iptables -A FORWARD -i tap0 -o eth0 -j ACCEPT
host$ iptables -A FORWARD -i eth0 -o tap0 \
-m state --state RELATED,ESTABLISHED -j ACCEPT

Now we are up and running!!



References:
[1] http://wiki.virtualsquare.org/index.php/VDE_Basic_Networking

.

FreeBSD

Check out the freeBSD operating system.

Another member of the family of UNIX operating systems.

Ports to most common comp architectures available.

http://www.freebsd.org

.

qemu

A sexy emulator !

Qemu: http://fabrice.bellard.free.fr/qemu/
Qemu for Windows: http://www.h7.dion.ne.jp/~qemu-win/

This program can emulate x86, MIPS, Sparc, ARM, PPC etc. I tried both the windows and linux x86 versions.. on the windows version, I tried loading a linux livecd.. And on the linux version, I tried out Windows XP

The project is still under development though.. Some bugs here and there are evident. I didn't like the networking support for the thingy. And I hate having to type out configs.. vmware virtualPC etc are gorgeous for their GUI alone, I'd say. Yet, neither vmware nor virtualPC support that entire gamut of processors, and best of all, this thing is FREE !

------------------------------------
Related links I found to be useful..

The OpenVPN tool which can be used for host-only networking / bridged networking on a windows host: http://openvpn.net/
- a how to for the same: http://www.h7.dion.ne.jp/~qemu-win/TapWin32-en.html

VDE - Virtual Distributed Ethernet, to use for host-only / bridged networking on linux hosts: http://vde.sourceforge.net/
- how to: http://wiki.virtualsquare.org/index.php/VDE_Basic_Networking

.

autobook

A free online book on Autoconf Automake and Libtool..
http://sourceware.org/autobook/

For all those people who have been mystified by

./configure
make
make install

I'm yet to read the book.. Placing this post lest I lose the url

--------- --------- --------- --------- --------- --------- --------- ---------
(2008-05-12)
Another book on auto-tools

http://www.freesoftwaremagazine.com/books/autotools_a_guide_to_autoconf_automake_libtool

.

Linux from the scratch !

Learn to
- install linux from source yourself
- build liveCDs and distributions
- build embedded linux systems
and lot more

http://www.linuxfromscratch.org/

.

PowerPC Simulators

PearPC http://pearpc.sourceforge.net/

PSIM http://sourceware.org/psim/

.

Coolest bugs

The URL gives a list of 10 cool bugs caused by software/hardware glitches..

http://news.zdnet.com/2424-9595_22-177729.html

.

Damn Small Linux, aka DSL

Ever wanted a really small Linux live CD? This is the perfect one.
http://damnsmalllinux.org/
It supports the X windowing system, and comes with Firefox web browser, a VNC viewer, basic word processors, media players, USB support and in short, everything you will need out of a basic computer.
And the punch line is its amazingly small size of ~ 50mb!

.

BonaFide OS Development

A place all you folks interested in operating systems should check out !
Some good documents on Operating Systems, oriented at the lesser discussed practical side.

http://www.osdever.net/goal.php

.

The Minix3 Operating System

This is an Operating System created and maintained by Prof. Andrew S Tanenbaum, for use in the academia to provide students with first hand experience on OS code. Also among the design goals thankfully are readability, modularity, and control over code size.

http://www.minix3.org/

A good blog on minix
http://www.minixtips.com
Earliest posts:
http://www.minixtips.com/2006_06_01_archive.html

.

UIUC Choices

This is a research project going on at the University of Illinois, Urbana-Champaign, into developing an Object Oriented Operating System, called Choices. The various links of interest are

• http://choices.cs.uiuc.edu/
• http://choices.cs.uiuc.edu/uChoices.html

.

Python References

[This post has been translocated from my python blog to this blog, as I'm planning to close down my python blog.. It's not seeing much attention from me anyways :D ]

Hm.. Now where do I learn python from.

Well, I started with the book "A Byte of Python" [pdf]. This book is a concise and succinct tutorial for someone who has a background in some or other programming language. I would recommend it to anyone who wishes to learn python, and you can learn from this book in under two weeks even at a relaxed pace of learning.

Likewise, the book "Dive into Python" also seems to be a good resource. I must admit I haven't dived into the book much to make a more accurate comment.

The python official website can be found at http://www.python.org . You can download the version of python for your particular operating system. Python is free !

You will also find Python documentation available for download at the site. [http://www.python.org/doc/] There's one particular tutorial in this documentation [online copy @ http://docs.python.org/tut/tut.html]. The tutorial is very exhaustive and will make you pretty much the guru of Python. However, on the flip side, I found the tutorial a little too involved. If you can bolster the patience to pore through it, there's nothing like it.

Have fun charming the python !

The Python programming language

[This post has been translocated from my python blog to this blog, as I'm planning to close down my python blog.. It's not seeing much attention from me anyways :D ]

Python was a buzzword around that time. Like an intelligent babe, this language was being exalted as simple and yet tremendously powerful. Well I never bothered; but instead just cleaved to the faith that the prodigious C++ was _the_ panacea for all programming imbroglio.

That went on until I stumbled upon this book: 'Byte of Python' by Swaroop C H [http://www.byteofpython.info/]
And that day, I was bitten.

Well, my old buddy C++ is still my best buddy because she is the most adaptable and reliable programming language that I've learned. In fact C++ and her object oriented lineage teaches you to think of programming situations from an entirely different perspective than what used to be conventional. C++ made you think more like you normally would.

But then Python too borrows heavily from the same Object oriented approaches and additionally allows you the flexibility of more expressive expressions and concise statements. Also somewhere in her lineage python had some functional programming ancestry which allows scope for expressing certain situations more flexibly.

So.. What exactly is this python?

Well I'd qualify python by the following salient features
- It's an interpreted programming language.
- It's a language which uses type inference and dynamic typing, which means you needn't specify the type of variables at compile time, and variables can be assigned values of different types without having to worry too much about the risks involved.
- It's an object oriented programming language.
- Seasoned with condiments from functional programming, it delivers that flexibility too, without the concomitant obscurity associated with functional languages.
- It has a _huge_ standard library base catering to almost all conceivable requirements.
- It is concise and succinct, reducing code size to probably 30% of the C++ analogue.

On the whole, it's one hell of a language.

[python] image converter

[This post has been translocated from my python blog to this blog, as I'm planning to close down my python blog.. It's not seeing much attention from me anyways :D ]

The following is a code for an image format converter cum resizing program. It takes a source directory, a destination directory and a target file format as parameters and converts all images in the source directory to the target format and writes them at the destination, after resizing them.

This is only a prototypical version and is rather rigid. You might have to modify it to suit your requirements.

Also note that the program requires Python Image Library (PIL) installed in addition to python.

import os
import Image

def imageConvert (source, dest, ext):
    if os.path.isdir(source) and os.path.isdir(dest):
        result = [True,[]]
        for i in os.listdir(source):
            if os.path.isdir(source + os.sep + i) == False:
                outfile = dest + os.sep + \
                    os.path.splitext(i)[0] + ext
                infile = source + os.sep + i
                if infile != outfile:
                    try:
                        im = Image.open(infile)
                        if ( im.size == (1024,768) ):
                            # Landscape Image
                            im = im.resize( (640,480) )
                        elif ( im.size == (768,1024) ):
                            # Portrait Image
                            im = im.resize( (480,640) )
                        im.save(outfile)
                        #Image.open(infile).save(outfile)
                    except :
                        result[0] = False
                        result[1].append('cannot convert %s'\
                                    % infile)
        return result
    else:
        return [False, ['Error, either the source or the' + \
            'destination given is not a directory']]

def main ():
    source = raw_input('Enter source directory: ')
    dest = raw_input('Enter destination directory: ')
    ext = '.' + raw_input('Enter file format extension: ')
    r,l = imageConvert ( source, dest, ext )
    if r == False:
        for i in range(len(l)):
            print l[i]

if __name__ == '__main__':
    main()

[Python] Permutations of a string

[This post has been translocated from my python blog to this blog, as I'm planning to close down my python blog.. It's not seeing much attention from me anyways :D ]

Given a string of any length, this python program generates all permutations of the characters in the string. The program is capable of taking care of repeated occurrences of the same character

count = 0  # a global counter to give a unique id to each permutation
        # that is generated


def constructDict (s):
 """Constructs the dictionary.

 Constructs the dictionary of
 unique character to no-of-occurance mapping.
 """
 d = {}
 for i in range(0,len(s)):
     try:                           # if the character is already present
         d[s[i]] = d[s[i]] + 1      # increment its count
     except:                        # if the character is not already presetn
         d[s[i]] = 1                # put it in and set its count as 1
 return d


def recursivePermuter ( d, strSoFar, maxDepth, thisDepth = 0 ):
 """Constructs all permutations.

 d: the dictionary
 strSoFar: the prefix part of the string which
           has been constructed so far
 maxDepth: this is same as the length of the
           user input string
 thisDepth: this level of recursion (Default value 0)

 Each place in the permutation can be occupied by
 an "available" character. An "available" character is
 one whose value in the dictionary d is not zero.
 When we use/borrow an available character, we decrement
 its value in the dictionary; and when we backtrace/return
 the available character, we restore its value by
 incrementing once.
 """
 global count
 if (thisDepth == maxDepth):         # if we have completed one permutation
     count = count + 1
     print str(count) + ": " + strSoFar
     return
 for key,value in d.items():
     if value == 0:
         continue
     # all elements in d with value > 0 are "candidates" to go
     # into this place.. We give each a chance systematically..
     d[key] = value - 1
     recursivePermuter ( d, strSoFar + key, maxDepth, thisDepth + 1 )
     d[key] = value


def __main__ ():
 """The perpetrator of all evil.
 """
 s = raw_input("Enter String: ")     # get the user input string
 d = constructDict(s)                # construct the dictionary
 recursivePermuter(d, "", len(s))    # despatch to the guy
                                     # who churns out permutations


# you have to explicitly call the main
# main, in python is a convention rather than
# a mandate..
__main__()

How it works..

1. get all the unique characters in a pool of pairs, each pair (symbol, no_of_occurances)
   
2. consider each permutation as a set of n blanks, n = strlen(given string)
   
3. for each position/blank in the permutations,
   
   
   
   1. we pick a symbol to go into that blank.. In picking, we reduce the number of occurance..
      So obviously also note that we may not pick a symbol whose number of occurances is already zero.
      
   2. now we go down the line to the next blank and repeat 3 //this is a recursive step..
      
   3. once the recursive function returns, we return the symbol we had borrowed for this instance
      of the recursive function.. We do so, by incrementing the number of occurances by one ..