[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[ale] If you own (almost) ANY HDD, repartition with 4k boundaries

You're welcome.  Glad to help.  Maybe it will help someone else avoid 
some potholes.  I'm sure I'll need the help on other issues later.

Sincerely,

Ron

On 1/3/2013 9:25 PM, Jim Kinney wrote:
> Nice write-up. Thanks!
>
> On Thu, Jan 3, 2013 at 9:00 PM, Ron Frazier (ALE) 
> <atllinuxenthinfo at techstarship.com 
> <mailto:atllinuxenthinfo at techstarship.com>> wrote:
>
>     Hi all,
>
>     I recently posted a message about misaligned partitions, to which
>     a couple of people replied with info about 4k sectors for HDD's. 
>     I decided to post this message with a different subject because
>     this can have a much broader impact than many people probably
>     know.  Some of you may already know this.  Others may not.  I
>     haven't purchased any new drives lately, so I'm just learning
>     about it.
>
>     To reiterate what happened, I replaced a 500 GB hybrid SSD HDD
>     with a new 750 GB one of newer vintage.  I cloned the hard drive
>     using a 3 year old version of Acronis True Image, which had always
>     worked before.  When I brought up the new drive in Ubuntu 11.10
>     Disk Utility, for each partition I clicked on, it said the
>     partition boundary was misaligned, and performance would suffer. 
>     Based on the responses to the other threads, I started doing research.
>
>     I'm not the best at explaining this, so bear with me.  See also
>     the links and quotes below.
>
>     Essentially, all HDD's made after January 2011 are now employing
>     4096 byte or 4 KB sectors instead of the traditional 512 byte
>     sectors.  The firmware in the drive reports sectors as though they
>     were 512 so legacy software will work.  Also, traditionally, the
>     first partition of an MBR formatted HDD started on sector 63,
>     after the MBR.  So, if the data from a 512 byte sector drive is
>     cloned onto a 4 KB sector drive, the partition boundaries will be
>     offset and the data will not align properly with the new physical
>     media.  You might think, so what, the firmware is translating, no
>     problem.  Wrong!  As it turns out, misaligned partitions can lead
>     to write performance which is 10 times slower.
>
>     There are the equivalent of 8 - 512 byte sectors in 1 - 4 KB sector.
>
>     Let's say you want to read 512 bytes, what your software thinks is
>     1 sector.  It sends the request to the HDD.  The firmware finds
>     the actual 4 KB sector with what you want in it.  It then extracts
>     the 512 bytes you want and sends that to you.  That works pretty
>     well.  Writing is a whole other matter.
>
>     Here's a quote from the Seagate article below:
>
>     "A write process can be more complicated, particularly when data
>     the host computer attempts to write is a subset of a physical 4K
>     sector. In these cases, the hard drive must first read the entire
>     4K sector containing the targeted location of the host write
>     request, merge the existing data with the new data and then
>     rewrite the entire 4K sector. (On the next revolution.)"
>
>     So, if you're writing 512 bytes, the firmware finds and reads the
>     4 KB sector that contains the area you want to write, replaces the
>     512 bytes you're writing while retaining the rest, and writes the
>     sector back to the disk.  It has to wait for the platter to spin
>     around again before it can rewrite the sector.  At 7200 RPM, this
>     takes an additional 8.3 mS.  Lets say you're writing 1024 bytes
>     and that bridges two 4 KB sectors.  The system has to read both
>     sectors, merge 512 bytes into one, then 512 bytes into the other,
>     then write them back on the next revolution.  You get the idea.
>
>     The IBM article below indicates that this can lead to a 10 TIMES
>     speed degradation if doing small writes.
>
>     You fix at least part of the problem by realigning both the start
>     and end locations of each partition on the disk to 4 KB
>     boundaries.  The standard industry practice seems to be to align
>     them on on 1 MiB boundaries, which is a multiple of 4 KB.
>
>     So, you want your first partition to start at 1 MiB, which is
>     sector number 2048 rather than 63.  I would leave 1 MiB between
>     partitions, and make the next partition start and end on 1 MiB
>     boundaries.
>
>     I should note that you don't want to do this with Windows XP from
>     what I've read.  It expects the 1st partition to be at 63, period.
>
>     In my case, I cloned a disk with 512 byte sectors to a disk with 4
>     K sectors, but I kept the partition arrangement exactly the same,
>     which had always worked before.  So, everything on the clone ended
>     up being misaligned.
>
>     Here's why I worded the subject the way I did.
>
>     If you have an SSD, you MUST align the partitions properly for
>     optimum performance.  They have 4K sectors.
>
>     If you have a HDD with 4K sectors, you MUST align it properly for
>     optimum performance.
>
>     If you have a hybrid SSD HDD with 4K sectors, you MUST align it
>     properly for optimum performance.
>
>     If you have a hybrid SSD HDD with 512 byte sectors on the
>     platters, you MOST LIKELY need to align it properly for optimum
>     performance.  Very hard to find any data on this.
>
>     AND, if you have a lowly old HDD with 512 byte sectors, and you
>     EVER MIGHT clone it to any of the others, you probably need to
>     align it properly for optimum performance of the drive that you
>     clone it to.
>
>     As was pointed out in the other thread, some cloning software
>     understands the 4K sectors.  The older stuff, including my 3 year
>     old Acronis True Image, does not.  However, I'm pretty sure that
>     if I align the partitions of an old HDD to 1 MiB boundaries, then
>     do an identical clone, that should be preserved.  If I let it
>     expand the partitions on the new HDD, I don't know.  Even with new
>     software that understands, you might have to turn on an option
>     such as align to 4K, align to 1 MiB, align to target, etc.
>
>     I looked at the manual for Terabyte Unlimited's cloning software
>     and I may replace Acronis with that.  The product looks pretty
>     impressive, and the manual mentions 4K sectors.  I'm not afraid to
>     pay for something this complex and critical if I have to.  I've
>     heard good things about Paragon software but don't know anything
>     about it.
>
>     You can check to see if you have an alignment conflict with Disk
>     Utility (in Ubuntu).  I don't know what program that actually
>     runs.  It will only show a message if the drive is partitioned the
>     old way and it has 4K sectors, apparently.  If the drive is
>     partitioned the old way and you have 512 byte sectors, it won't
>     complain.  The version of Disk Utility in Ubuntu 11.10 does not
>     partition the drive properly by default.
>
>     Because of the potential problems of having ANY drives around with
>     512 byte sector alignment (also called cylinder alignment, I
>     think), I've decided to repartition all my boot drives and am
>     considering doing the archival drives.  I'm putting everything on
>     1 MiB boundaries whether the actual drive has 512 byte or 4K sectors.
>
>     The only way I could get into trouble with that is if I partition
>     a drive with old software or if I clone a drive with old software
>     and resize the partitions.
>
>     You can use the latest version of Gparted from gparted.org
>     <http://gparted.org> to repartition the drive non destructively
>     with data on it.  You should backup the data first.  Don't use the
>     one in Ubuntu, for example.  It's old and buggy.  Download the ISO
>     for the Gparted live CD and use that so you have the newest version.
>
>     This lifehacker article tells how to do it.
>
>     http://lifehacker.com/5837769/make-sure-your-partitions-are-correctly-aligned-for-optimal-solid-state-drive-performance
>
>     It's a little dated.  The UI for Gparted has changed slightly. 
>     There is a drop down box for alignment.  Set this to MiB.  There
>     is no longer an align to cylinders check box.
>
>     So, here's what I did to all my boot drives.
>
>     For the first partition, I simultaneously reduce the size and move
>     the boundaries to allow 2 MiB before and after.
>
>     Then, I move the partition to the left by 1 MiB as the article
>     indicates.
>
>     I then move each other partition that has at least a 2 MiB
>     boundary to the left to within 1 MiB.  If any partition doesn't
>     have at least 2 MiB to it's left, I first move it to the right up
>     to 2 MiB, then back to 1 MiB.  This is due to an anomaly in the
>     way Gparted calculates space, as mentioned in the article.
>
>     When, I'm done, there is 1 MiB before the first partition,
>     although you cannot see it on the Gparted drive map.  I have 1 MiB
>     unused spaces between partitions.  And, every partition begins and
>     ends on a 1 MiB boundary.
>
>     This will optimize performance if the drive has 4K sectors, or if
>     it is ever cloned to a drive that does.
>
>     Doing this, particularly moving the boot partition away from
>     sector 63 will probably break your boot loader.  The article talks
>     about how you can fix a Windows installation with a Windows
>     install disk.  If you don't have one handy, you may not want to
>     try this.
>
>     See the Gparted faq (below) for info on fixing a Linux boot loader.
>
>     Here's how to check if you've finished the job properly using Gparted.
>
>     Click on the first partition, right click, and select information.
>
>     The first sector should be at 2048.
>     Look at the total sectors.  In one case, for me, it's 307,183,616.
>     Get out an engineering calculator that can handle big numbers. 
>     Enter this in and divide by 2048.
>     In my case, the answer is 149,992.  If this is a whole number,
>     with no decimal, you're good.
>
>     The unused space between each partition should have 2048 total
>     sectors, or a multiple of that.
>
>     For each other partition, check that the <first sector> / 2048 is
>     a whole number
>     and that the <total sectors> / 2048 is a whole number.
>
>     If all the numbers check out, you have optimized your partition
>     structure for drives with 4K sectors.
>
>     Below are a number of links and some quotes that I found helpful.
>
>     I hope this info is helpful to you.
>
>     Sincerely,
>
>     Ron
>
>     ---------------------------------------
>
>     http://en.wikipedia.org/wiki/Advanced_Format
>     http://www.zdnet.com/blog/storage/are-you-ready-for-4k-sector-drives/731
>
>     http://wdc.custhelp.com/app/answers/detail/a_id/5655
>     https://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues (geeky)
>     http://gparted.org/faq.php
>     http://gparted.org/display-doc.php?name=help-manual
>     http://gparted.org/display-doc.php?name=help-manual#gparted-fix-grub-boot-problem
>
>
>     ---------------------------------
>
>     http://www.seagate.com/tech-insights/advanced-format-4k-sector-hard-drives-master-ti/
>     (good)
>
>     "As all hard drive manufacturers have agreed to transition to the
>     Advanced Format sector design by January 2011, the industry must
>     adapt to and embrace this change to minimize potential negative
>     side effects. "
>
>     "A write process can be more complicated, particularly when data
>     the host computer attempts to write is a subset of a physical 4K
>     sector. In these cases, the hard drive must first read the entire
>     4K sector containing the targeted location of the host write
>     request, merge the existing data with the new data and then
>     rewrite the entire 4K sector. (On the next revolution.) "
>
>     "Fdisk: GNU Fdisk is a command line utility that partitions hard
>     drives. Versions 1.2.3 and above support Advanced Format drives. "
>
>     "Parted: GNU Parted is a graphical utility for partitioning hard
>     drives. Versions 2.1 and above support Advanced Format drives. "
>
>     ------------------------------
>
>     http://www.terabyteunlimited.com/index.htm - cloning software
>     looks cool
>     http://goinglinux.com/ - Just did a podcast, # 195, on SSD's.
>
>     from prior thread - http://jrfom.com/2010/05/03/4kb-sectors/
>     from prior thread - http://lwn.net/Articles/377895/
>
>     -----------------------
>
>     from prior thread -
>     http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/index.html?ca=dgr-lnxw074KB-Disksdth-LX
>     (good) (dated)
>
>     "Most modern file systems use data structures that are 4096 bytes
>     or larger in size. "
>
>     "Writes of misaligned data structures, on the other hand, require
>     the disk's firmware to first read two sectors, modify portions of
>     both sectors, and then write two sectors.
>     The biggest write performance effects occurred with small file
>     creation (extracting the kernel tarball). Effects on tarball
>     extraction ranged from 1.04 (for ext4fs) to 25.53 (for ReiserFS),
>     with a mean of 10.9. The second-best performer in this test was
>     XFS, with a figure of 1.82. Because these figures are ratios of
>     unaligned to aligned performance, a value of 10.9 means that a
>     tarball extraction that takes 10 seconds on a properly aligned
>     partition takes 109 seconds on an improperly aligned partition?a
>     huge difference! The XFS figure of 1.82 means that this 10-second
>     operation takes 18.2 seconds on an improperly aligned partition. "
>
>     -- 
>
>     (To whom it may concern.  My email address has changed.  Replying to former
>     messages prior to 03/31/12 with my personal address will go to the wrong
>     address.  Please send all personal correspondence to the new address.)
>
>     (PS - If you email me and don't get a quick response, you might want to
>     call on the phone.  I get about 300 emails per day from alternate energy
>     mailing lists and such.  I don't always see new email messages very quickly.)
>
>     Ron Frazier
>     770-205-9422  <tel:770-205-9422>  (O)   Leave a message.
>     linuxdude ATtechstarship.com  <http://techstarship.com>
>          
>
>
>     _______________________________________________
>     Ale mailing list
>     Ale at ale.org <mailto:Ale at ale.org>
>     http://mail.ale.org/mailman/listinfo/ale
>     See JOBS, ANNOUNCE and SCHOOLS lists at
>     http://mail.ale.org/mailman/listinfo
>
>
>
>
> -- 
> -- 
> James P. Kinney III
> ////
> ////Every time you stop a school, you will have to build a jail. What 
> you gain at one end you lose at the other. It's like feeding a dog on 
> his own tail. It won't fatten the dog.
> - Speech 11/23/1900 Mark Twain
> ////
> http://electjimkinney.org
> http://heretothereideas.blogspot.com/
> ////
>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>    

-- 

(To whom it may concern.  My email address has changed.  Replying to former
messages prior to 03/31/12 with my personal address will go to the wrong
address.  Please send all personal correspondence to the new address.)

(PS - If you email me and don't get a quick response, you might want to
call on the phone.  I get about 300 emails per day from alternate energy
mailing lists and such.  I don't always see new email messages very quickly.)

Ron Frazier
770-205-9422 (O)   Leave a message.
linuxdude AT techstarship.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130103/a6c49a1c/attachment-0001.html>