Category Archives: OSX

Offline Browsing in Linux: wget and some tricks

Ever since I joined Hostgator.com, I’ve been learning a lot of Linux in the hopes that I switch my career into Linux. Hopefully Forensics related.

So this new dilemma I had was to download a website for offline browsing. I went on the hunt for an offline file browser for Linux…. I found that I could use wget to mirror a whole website.

For example, I want to make a copy of blackberrysimunlockcode.com, Here’s how:

wget -m http://blackberrysimunlockcode.com

Here the -m option is telling wget to mirror the website. This is the basic command. But say I need some advanced options. What do I do?

I was trying to get all the script files off of a website to save for later learning and all it was downloading was the index.html and robots.txt
The robots.txt file was blocking user agent wget. To confirm this I used the debug option in wget:

wget -m -d http://blackberrysimunlockcode.com

You’ll get something like:

Not following http://blackberrysimunlockcode.com/privacy.shtml because robots.txt forbids it.

or

Rejecting path sh/eg/talk.sh.txt because of rule `sh’

or

no-follow in index.html

I tried using the option –user-agent “Mozilla” ….. no luck

I tried adding the following in .wgetrc :

## Local settings (for a user to set in his $HOME/.wgetrc). It is
## *highly* undesirable to put these settings in the global file, since
## they are potentially dangerous to “normal” users.
##
## Even when setting up your own ~/.wgetrc, you should know what you
## are doing before doing so.
##

header = Accept-Language: en-us,en;q=0.5
header = Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
header = Accept-Encoding: gzip,deflate
header = Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
header = Keep-Alive: 300
user_agent = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6
referer = http://www.google.com

…Still no luck.

The trick is to use option -e robots=off

So my new command became:

wget -m -k -e robots=off -w 2 --random-wait -U "Mozilla" -np http://blackberrysimunlockcode.com

Heres what the options do:

-m mirrors website
-k fix links so you don’t get directed to www.blackberrysimunlockcode.com/sh/eg instead of /sh/eg (relative vs absolute)
-e executes command robots=off
-w 2 sets wait time as 2 seconds so you don’t overload server and get ip blocked
–random-wait can be random in 2 secon increments
-U sets user agent
-np no parent, so if the current subdirectory/page links to parent pages, it might crawl whole website

Getting root on your new Nexus One Android Google Phone

The Nexus one was rooted before it even got released. Paul at modaco forums showed us how to perform this hack. I’ll try to simplify the method here.

1. First unlock your bootloader if you have a T-mobile Nexus One (this voids the warranty) or get an unlocked version from Google for like $550:

a. download fastboot
b. open terminal or command line and type
– Windows fastboot-windows oem unlock
– Mac ./fastboot-mac oem unlock
– Linux ./fastboot-linux oem unlock

2. Rewrite the Bootloader to obtain root privileges:

a. Download either SuperBoot or InsecureBoot and extract. (The difference is that with the insecure boot you won’t make any changes to the system and you would have to launch su manually)
b. Put the device in bootloader mode:
– Turn off the phone and restart holding the trackball.
c. Flash the bootloader:
– Windows = double click the windows bat file
– Mac = open terminal and cd to the directory containing the file. type
chmod +x install-superboot-mac.sh
./install-superboot-mac.sh

– Linux = open terminal and cd to the directory containing the file. type
chmod +x install-superboot-linux.sh
./install-superboot-linux.sh

Now you should have root!

Rooted Nexus One Google Phone
Pic courtesy of Stericson.

Automatic website backup without SSH enabled over FTP

Last time I backed up my website with rsync and ssh, but on my new host they disabled SSH. Rsync does not work over ftp. I do not want to do incremental backups with delta files like is done with rdiff-backup or duplicity. I need to have an exact mirror of my site. But remember that your sql databases won’t be backed up.

Curlftpfs is the key! In this tutorial I will show you how to backup from one server to a backup location which can be your hard drive, another web host, dropbox folder, Box.net via webdav, etc.

Download curlftpfs, rsync and ncftp:
sudo apt-get install curlftpfs ncftp rsync

make directories to mount your ftp server:
sudo mkdir /media/hydtechblog
sudo mkdir /media/hydtechbackupserver

edit fstab to mount the ftp servers using curlftpfs:
sudo gedit /etc/fstab

add the lines and modify them according to your server:
curlftpfs#username:password@hydtechblog.com /media/hydtechblog fuse rw,allow_other,uid=root 0 0
curlftpfs#username:password@hydtechbackupserver /media/hydtechbackupserver fuse rw,allow_other,uid=root 0 0

One thing to remember is that these two will not mount automatically because when the computer restarts, the fstab is done while you are not connected to the network. To fix this we can just add the mount commands in our crontab.

Edit crontab:
sudo crontab -e
enter the following lines and modify accordingly:
00 09 * * * mount /media/hydtechblog
00 09 * * * mount /media/hydtechbackup
01 09 * * * rsync -avz –rsync-path=/usr/bin/rsync /media/hydtechblog/public_html /media/hydtechbackup/public_html

ctrl + o to write and ctrl + x to save

This will tell cron to mount the folders at 9:00 am and start rsync at 9:01 am. You can replace the backup location to another folder on your hard drive or your dropbox or ubuntu one folder. You can also mount box.net with webdav and use this method.

For encrypted incremental backups checkout duplicity, it also works with webdav and ftp.

Dual, Triple, Quad Boot a Macbook with Mac OS X, Ubuntu Linux, Windows XP, and Windows Vista

In this  tutorial I will be showing you how to Quad boot your macbook with OS X, Windows Vista, Windows XP, and Ubuntu Linux.  You can leave out the OSes you don’t want or replace them with something else.  This is just a general but detailed guide for noobz!

QUADBOOT SETUP

We will be installing OS X from scratch so all your setting will be deleted. You might want to back up the data with Time machine or another backup software. Insert the OS X DVD and reboot the macbook holding down the C key.

QUADBOOT SETUP

Select your Language.

QUADBOOT SETUP

Before we install the OSes, we will partition the hard drive into 5 partitions.  Go to Utilities and click on Disk Utility.

QUADBOOT SETUP

Select the Hard drive on the left.  Mine has a 120GB drive.

QUADBOOT SETUP

Click on the partition tab and select 5 Partitions from the Volume Scheme.  Label them as follows:

1. Vista (MS-DOS FAT and atleast 10GB)

2. Storage (MS-DOS FAT and however much space you want as a storage location)

3. XP (MS-DOS FAT and atleast 7GB)

4. OSX (Extended Journaled and atleast 20GB)

5. Free space (Chose from the drop down.  will be used for our Ubuntu Linux)

QUADBOOT SETUP

Click on Options and make sure GUID Partition Table is selected. Hit Apply

QUADBOOT SETUP

Click Partition

QUADBOOT SETUP

Now we are going to Install our first OS.  Select the OSX partition and continue.

QUADBOOT SETUP

Install

QUADBOOT SETUP

You can skip the check if you know the DVD is in good condition and without scratches.

QUADBOOT SETUP

Your Installing your first OS. excited?

Be patient.  This will take a while.

QUADBOOT SETUP

Restart your computer.

QUADBOOT SETUP

It will ask you for the second Disc.  Insert the second disc.

QUADBOOT SETUP

Installation Complete!! now pat yourself on the back.

QUADBOOT SETUP

Once you setup your OSX with your settings, open up the safari web browser and goto http://refit.sourceforge.net/ and download the Mac disk image

QUADBOOT SETUP

downloading….

QUADBOOT SETUP

once the download finishes, double click the image and then on the installer to install it.

QUADBOOT SETUP

After rEFIt is installed, insert the VISTA disc and reboot holding the  C key.

QUADBOOT SETUP

Select language.

QUADBOOT SETUP

Install.

QUADBOOT SETUP

Accept the terms.

QUADBOOT SETUP

Custom (advanced)

QUADBOOT SETUP

Select the Vista partition and format with NTFS before continuing with Install.

QUADBOOT SETUP

QUADBOOT SETUP

QUADBOOT SETUP

QUADBOOT SETUP

After restarting, you will see the rEFIt GUI.  select the second partition to boot into VISTA.

QUADBOOT SETUP

Goto My computer.

QUADBOOT SETUP

select the CD/DVD drive.

QUADBOOT SETUP

Eject the CD and insert the OSX disc.

QUADBOOT SETUP

Select Open folder.

QUADBOOT SETUP

Go into Boot Camp folder.

QUADBOOT SETUP

click setup.exe

QUADBOOT SETUP

Continue to Install the BootCamp drivers.

QUADBOOT SETUP

QUADBOOT SETUP

QUADBOOT SETUP

QUADBOOT SETUP

QUADBOOT SETUP

Select No to restart.  We need to mark our XP partition as active first.

QUADBOOT SETUP

Goto Control Panel.

QUADBOOT SETUP

System and maintenance

QUADBOOT SETUP

Click ‘Create and format hard disk partitions’ under Administrative tools.

QUADBOOT SETUP

right click the XP partition and formate with NTFS.

QUADBOOT SETUP

QUADBOOT SETUP

QUADBOOT SETUP

after formatting is complete, right click the XP partition and mark as active.

QUADBOOT SETUP

Hell yes.  Now eject the OSX disc and insert the XP disc.

QUADBOOTING A MACBOOK

Upon entering rEFIt, select boot from CD.

QUADBOOTING A MACBOOK

Select partition 4

QUADBOOTING A MACBOOK

Let it do its thang.

QUADBOOTING A MACBOOK

upon restart, Boot from partition 4.

QUADBOOTING A MACBOOK

QUADBOOTING A MACBOOK

Awesome!! Now you have 3 OSes. Take a break.

QUADBOOTING A MACBOOK

In XP, goto my computer and eject the CD.  Insert the OSX disc.

QUADBOOTING A MACBOOK

Install Boot Camp drivers.

QUADBOOTING A MACBOOK

QUADBOOTING A MACBOOK

QUADBOOTING A MACBOOK

Select no to the restart.  Eject the OSX disc and insert the Ubuntu CD.  Now restart

QUADBOOTING A MACBOOK

Boot from CD.

QUADBOOTING A MACBOOK

Select language

QUADBOOTING A MACBOOK

Install Ubuntu

QUADBOOTING A MACBOOK

Select language

QUADBOOTING A MACBOOK

Pick your timezone.

QUADBOOTING A MACBOOK

Select USA – Macintosh for the keyboard.

QUADBOOTING A MACBOOK

QUADBOOTING A MACBOOK

Select the free space and click on new partition.

QUADBOOTING A MACBOOK

Now we will make a swap space for Linux.

Size: more than your RAM to be able to hibernate. My RAM is 2GB so I choose 2001MB.

Location: End

Use as: Swap area

QUADBOOTING A MACBOOK

Create another partition.

Size: Use the remaining disk space.

Location: Beginning

Use as: Ext3

Mount point: /

QUADBOOTING A MACBOOK

QUADBOOTING A MACBOOK

type in your username and password.  Don’t forget your password.

QUADBOOTING A MACBOOK

nothing to migrate.  so leave this blank.

QUADBOOTING A MACBOOK

Click on advanced and in the drop down box type in (hd0,2)

QUADBOOTING A MACBOOK

proceed normally

QUADBOOTING A MACBOOK

Restart.

Ubuntu will ask you to remove the disc and press ENTER.

QUADBOOTING A MACBOOK

You will come back to rEFIt

QUADBOOTING A MACBOOK

Start the partitioning tool.

QUADBOOTING A MACBOOK

Type Y to update the MBR.

QUADBOOTING A MACBOOK

QUADBOOTING A MACBOOK

Boot from the Linux partition

QUADBOOTING A MACBOOK

Boot into Vista.

QUADBOOTING A MACBOOK

You should get an error.  If you get this error, Follow the directions and enter the Vista CD. restart the computer and select Boot from CD from the rEFIt menu.

QUADBOOTING A MACBOOK

repair and restart.

QUADBOOTING A MACBOOK

now we want our Ubuntu to start automatically without waiting for a timeout when we select the Ubuntu partition, instead of showing us the Grub menu.  To do this, boot into ubuntu and open a terminal shell.

QUADBOOTING A MACBOOK

type sudo gedit /boot/grub/menu.lst

enter your password.

QUADBOOTING A MACBOOK

you will be presented with the grub configuration file in a text editor. change the timeout to 0.

Enjoy your QUAD BOOT MACBOOK !!!!

You can download this tutorial as a pdf here, but please give me credit.
Reliable Shared Hosting