HydTech

Popularity: 13% [?]

Ever since I joined Hostgator.com, I’ve been learning a lot of Linux in the hopes that I switch my career into Linux. Hopefully Forensics related.

So this new dilemma I had was to download a website for offline browsing. I went on the hunt for an offline file browser for Linux…. I found that I could use wget to mirror a whole website.

For example, I want to make a copy of blackberrysimunlockcode.com, Here’s how:

wget -m http://blackberrysimunlockcode.com

Here the -m option is telling wget to mirror the website. This is the basic command. But say I need some advanced options. What do I do?

I was trying to get all the script files off of a website to save for later learning and all it was downloading was the index.html and robots.txt
The robots.txt file was blocking user agent wget. To confirm this I used the debug option in wget:

wget -m -d http://blackberrysimunlockcode.com

You’ll get something like:

Not following http://blackberrysimunlockcode.com/privacy.shtml because robots.txt forbids it.

or

Rejecting path sh/eg/talk.sh.txt because of rule `sh’

or

no-follow in index.html

I tried using the option –user-agent “Mozilla” ….. no luck

I tried adding the following in .wgetrc :

## Local settings (for a user to set in his $HOME/.wgetrc). It is
## *highly* undesirable to put these settings in the global file, since
## they are potentially dangerous to “normal” users.
##
## Even when setting up your own ~/.wgetrc, you should know what you
## are doing before doing so.
##

header = Accept-Language: en-us,en;q=0.5
header = Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
header = Accept-Encoding: gzip,deflate
header = Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
header = Keep-Alive: 300
user_agent = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6
referer = http://www.google.com

…Still no luck.

The trick is to use option -e robots=off

So my new command became:

wget -m -k -e robots=off -w 2 --random-wait -U "Mozilla" -np http://blackberrysimunlockcode.com

Heres what the options do:

-m mirrors website
-k fix links so you don’t get directed to www.blackberrysimunlockcode.com/sh/eg instead of /sh/eg (relative vs absolute)
-e executes command robots=off
-w 2 sets wait time as 2 seconds so you don’t overload server and get ip blocked
–random-wait can be random in 2 secon increments
-U sets user agent
-np no parent, so if the current subdirectory/page links to parent pages, it might crawl whole website

Popularity: 8% [?]

Popularity: 7% [?]

Popularity: 3% [?]

The Nexus one was rooted before it even got released. Paul at modaco forums showed us how to perform this hack. I’ll try to simplify the method here.

1. First unlock your bootloader if you have a T-mobile Nexus One (this voids the warranty) or get an unlocked version from Google for like $550:

a. download fastboot
b. open terminal or command line and type
– Windows fastboot-windows oem unlock
– Mac ./fastboot-mac oem unlock
– Linux ./fastboot-linux oem unlock

2. Rewrite the Bootloader to obtain root privileges:

a. Download either SuperBoot or InsecureBoot and extract. (The difference is that with the insecure boot you won’t make any changes to the system and you would have to launch su manually)
b. Put the device in bootloader mode:
– Turn off the phone and restart holding the trackball.
c. Flash the bootloader:
– Windows = double click the windows bat file
– Mac = open terminal and cd to the directory containing the file. type
chmod +x install-superboot-mac.sh
./install-superboot-mac.sh
– Linux = open terminal and cd to the directory containing the file. type
chmod +x install-superboot-linux.sh
./install-superboot-linux.sh

Now you should have root!

Pic courtesy of Stericson.

Popularity: 9% [?]

Last time I backed up my website with rsync and ssh, but on my new host they disabled SSH. Rsync does not work over ftp. I do not want to do incremental backups with delta files like is done with rdiff-backup or duplicity. I need to have an exact mirror of my site. But remember that your sql databases won’t be backed up.

Curlftpfs is the key! In this tutorial I will show you how to backup from one server to a backup location which can be your hard drive, another web host, dropbox folder, Box.net via webdav, etc.

Download curlftpfs, rsync and ncftp:
sudo apt-get install curlftpfs ncftp rsync

make directories to mount your ftp server:
sudo mkdir /media/hydtechblog
sudo mkdir /media/hydtechbackupserver

edit fstab to mount the ftp servers using curlftpfs:
sudo gedit /etc/fstab

add the lines and modify them according to your server:
curlftpfs#username:password@hydtechblog.com /media/hydtechblog fuse rw,allow_other,uid=root 0 0
curlftpfs#username:password@hydtechbackupserver /media/hydtechbackupserver fuse rw,allow_other,uid=root 0 0

One thing to remember is that these two will not mount automatically because when the computer restarts, the fstab is done while you are not connected to the network. To fix this we can just add the mount commands in our crontab.

Edit crontab:
sudo crontab -e
enter the following lines and modify accordingly:
00 09 * * * mount /media/hydtechblog
00 09 * * * mount /media/hydtechbackup
01 09 * * * rsync -avz –rsync-path=/usr/bin/rsync /media/hydtechblog/public_html /media/hydtechbackup/public_html

ctrl + o to write and ctrl + x to save

This will tell cron to mount the folders at 9:00 am and start rsync at 9:01 am. You can replace the backup location to another folder on your hard drive or your dropbox or ubuntu one folder. You can also mount box.net with webdav and use this method.

For encrypted incremental backups checkout duplicity, it also works with webdav and ftp.

Popularity: 3% [?]

In this  tutorial I will be showing you how to Quad boot your macbook with OS X, Windows Vista, Windows XP, and Ubuntu Linux.  You can leave out the OSes you don’t want or replace them with something else.  This is just a general but detailed guide for noobz!

We will be installing OS X from scratch so all your setting will be deleted. You might want to back up the data with Time machine or another backup software. Insert the OS X DVD and reboot the macbook holding down the C key.

Select your Language.

Before we install the OSes, we will partition the hard drive into 5 partitions.  Go to Utilities and click on Disk Utility.

Select the Hard drive on the left.  Mine has a 120GB drive.

Click on the partition tab and select 5 Partitions from the Volume Scheme.  Label them as follows:

1. Vista (MS-DOS FAT and atleast 10GB)

2. Storage (MS-DOS FAT and however much space you want as a storage location)

3. XP (MS-DOS FAT and atleast 7GB)

4. OSX (Extended Journaled and atleast 20GB)

5. Free space (Chose from the drop down.  will be used for our Ubuntu Linux)

Click on Options and make sure GUID Partition Table is selected. Hit Apply

Click Partition

Now we are going to Install our first OS.  Select the OSX partition and continue.

Install

You can skip the check if you know the DVD is in good condition and without scratches.

Your Installing your first OS. excited?

Be patient.  This will take a while.

Restart your computer.

It will ask you for the second Disc.  Insert the second disc.

Installation Complete!! now pat yourself on the back.

Once you setup your OSX with your settings, open up the safari web browser and goto http://refit.sourceforge.net/ and download the Mac disk image

downloading….

once the download finishes, double click the image and then on the installer to install it.

After rEFIt is installed, insert the VISTA disc and reboot holding the  C key.

Select language.

Install.

Accept the terms.

Custom (advanced)

Select the Vista partition and format with NTFS before continuing with Install.

After restarting, you will see the rEFIt GUI.  select the second partition to boot into VISTA.

Goto My computer.

select the CD/DVD drive.

Eject the CD and insert the OSX disc.

Select Open folder.

Go into Boot Camp folder.

click setup.exe

Continue to Install the BootCamp drivers.

Select No to restart.  We need to mark our XP partition as active first.

Goto Control Panel.

System and maintenance

Click ‘Create and format hard disk partitions’ under Administrative tools.

right click the XP partition and formate with NTFS.

after formatting is complete, right click the XP partition and mark as active.

Hell yes.  Now eject the OSX disc and insert the XP disc.

Upon entering rEFIt, select boot from CD.

Select partition 4

Let it do its thang.

upon restart, Boot from partition 4.

Awesome!! Now you have 3 OSes. Take a break.

In XP, goto my computer and eject the CD.  Insert the OSX disc.

Install Boot Camp drivers.

Select no to the restart.  Eject the OSX disc and insert the Ubuntu CD.  Now restart

Boot from CD.

Select language

Install Ubuntu

Select language

Pick your timezone.

Select USA – Macintosh for the keyboard.

Select the free space and click on new partition.

Now we will make a swap space for Linux.

Size: more than your RAM to be able to hibernate. My RAM is 2GB so I choose 2001MB.

Location: End

Use as: Swap area

Create another partition.

Size: Use the remaining disk space.

Location: Beginning

Use as: Ext3

Mount point: /

type in your username and password.  Don’t forget your password.

nothing to migrate.  so leave this blank.

Click on advanced and in the drop down box type in (hd0,2)

proceed normally

Restart.

Ubuntu will ask you to remove the disc and press ENTER.

You will come back to rEFIt

Start the partitioning tool.

Type Y to update the MBR.

Boot from the Linux partition

Boot into Vista.

You should get an error.  If you get this error, Follow the directions and enter the Vista CD. restart the computer and select Boot from CD from the rEFIt menu.

repair and restart.

now we want our Ubuntu to start automatically without waiting for a timeout when we select the Ubuntu partition, instead of showing us the Grub menu.  To do this, boot into ubuntu and open a terminal shell.

type sudo gedit /boot/grub/menu.lst

enter your password.

you will be presented with the grub configuration file in a text editor. change the timeout to 0.

Enjoy your QUAD BOOT MACBOOK !!!!

You can download this tutorial as a pdf here, but please give me credit.

Popularity: 28% [?]