This took some time and I came out with a better understanding of device drivers.

Some tidbits -

lspci, rfkill list all, lsusb, modprobe (-r) and a few more commands helped me triage what was happening. In general when you want to install a driver, first find out the device - "lspci -nn" -> gives hardware id.

Armed with this hardware id, you can go around and search for working device drivers. For me, the wifi was wrecking havoc, it is Broadcom 4312 chipset and my lubuntu was 12.04. Steps i followed -

1. Removed the existing drivers sudo apt-get remove bcmwl-kernel-source

2. Install this - sudo apt-get install firmware-b43-lpphy-installer b43-fwcutter (lpphy driver for lower power wireless chipsets - bcm4312)

3. Check if bcm43xx drivers are blacklisted -  cat /etc/modprobe.d/* | egrep 'bcm'

4. If they are, uncomment the line where it says 'blacklist bcmxx' in the following file - sudo vim /etc/modprobe.d/blacklist.conf