Helpful ReplyShould I format Hard Drive at 64k Cluster (Block) size?

Author
SongCraft
Max Output Level: -36 dBFS
  • Total Posts : 3902
  • Joined: 2007/09/19 17:54:46
  • Status: offline
2012/05/12 19:15:01 (permalink)

Should I format Hard Drive at 64k Cluster (Block) size?


Should I format new Hard Drive at 64k Cluster (Block) size? 

Just asking because my additional new Hard Drive will be used for all my audio projects and video editing thereby handling large files. 
Yeah I'm curious to know what others here use (Block Size =?) 

Any other related information about installing/formatting a new hard drive is welcome. Thanks! 

I found it very easy to install the new hard drive but that's only after getting the right accessories because my case did not have spare drive trays to mount the new additional hard drive in. 

After installation I booted up and got an alarm (Beep Code) that indicated failed fan, overheating, smoke, fire, fire, fire; Jim, quick grab the fire extinguisher (hahahaha just joking) .    I found out it was a false alarm and no further issues; all fans and systems check = OK, temperatures are way within safety operations and windows located the new hard drive immediately eagerly waiting for me to format it :) 

-




 
 
#1
Jim Roseberry
Max Output Level: 0 dBFS
  • Total Posts : 9871
  • Joined: 2004/03/23 11:34:51
  • Location: Ohio
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/13 12:09:16 (permalink) ☄ Helpfulby glennstanton 2017/07/09 16:20:29
Should I format new Hard Drive at 64k Cluster (Block) size?

 
Yes.

Best Regards,

Jim Roseberry
jim@studiocat.com
www.studiocat.com
#2
craigb
Max Output Level: 0 dBFS
  • Total Posts : 41704
  • Joined: 2009/01/28 23:13:04
  • Location: The Pacific Northwestshire
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/19 20:52:36 (permalink)
I've got a general question about this topic...  I just got two new harddrives (one about two months ago and one last week - 2TB and 3TB) and they are formatted with whatever came up as the default (Allocation unit size = 4096 bytes).  This is also the smallest choice.  Is this the same thing as the Cluster (Block) size being discussed above AND should I be using one of the larger selections?

Right now these are being used for general PC storage and backups. There's two partitions on each with one being used to hold the backups for the partition in use on the other harddrive - in other words, on the 2TB drive I have a partition that holds all of my non-music data and another partition that holds backups from the 3TB and OS drives - then, on the 3TB drive, I have one partition with all my music on it and a second partition that holds all the backups from the 2TB and OS drives.  This way, if any drive fails, I can restore everything from another hard drive (yes, I make occasional offline copies in case of theft or fire, but that's not really part of my question here).

Note that the OS is on its own, separate drive and this isn't my DAW.  It does have 500 GB of music and 300 GB of documents and pictures on it though.

So the basic question is whether I should format these data/backup drives with a larger Allocation Unit Size to get better performance.  If so, what are the Pros and Cons of using a larger size and what would be the recommended size to move to?

Thank you!
post edited by craigb - 2012/05/19 20:55:24

 
Time for all of you to head over to Beyond My DAW!
#3
slartabartfast
Max Output Level: -22.5 dBFS
  • Total Posts : 5289
  • Joined: 2005/10/30 01:38:34
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/20 03:50:18 (permalink)
Why are you worried about performance on these drives? Unless speed is critical to the use of the drive, why not just use the default value? You can certainly spend hours understanding the issues involved with optimizing drive efficiency here, but did you not say these are storage and backup drives? Why spend hours and risk problems to save milliseconds?
 
 
http://technet.microsoft.com/en-us/library/cc767961.aspx
 
#4
craigb
Max Output Level: 0 dBFS
  • Total Posts : 41704
  • Joined: 2009/01/28 23:13:04
  • Location: The Pacific Northwestshire
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/20 05:45:31 (permalink)
That's part of the question - whether it makes a difference.  I haven't had drives this size before or Windows 7 (just got it a couple of months ago) so I'm asking basic questions.

These drives are what I work on every day.  I edit many large projects in PhotoShop so if changing these settings brings them in quicker, great.  If changing these settings helps me stream my music in the background, great (I've got something like 60,000 songs - enough to bog down iTunes even with a quick PC so I'm just curious).

Obviously, if we're only talking about a few milliseconds then it's not useful - getting the memory up to 12GB helped more than that.  I just didn't want to be doing something dumb only because I never asked.
 
EDIT:  Didn't even think about this the first time, but if I can cut down the time it takes to create the backups or move them around it would be a huge benefit.  Some of those large files take between 45 minutes and three hours to move!
post edited by craigb - 2012/05/20 05:56:40

 
Time for all of you to head over to Beyond My DAW!
#5
Alegria
Max Output Level: -54.5 dBFS
  • Total Posts : 2075
  • Joined: 2008/11/07 12:57:49
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/20 11:19:50 (permalink)
"craigb"
That's part of the question - whether it makes a difference. I haven't had drives this size before or Windows 7 (just got it a couple of months ago) so I'm asking basic questions.

Most articles I've read on this subject had one conclusion in common. It really doesn't make a noticeable difference, although I have never been satisfied with the methodologies used <shrugs>. I believe if memory serves me right, that cluster size was more an issue on older OSs where proper alignment was more critical than it is today (had to do with software still writing in 512 byte chunks..., and still do today). But I have to say that this was a while ago and I'm simply not very sure about the above.

If you google "ntfs format cluster size" for example, I'm sure you'll find some interesting articles from reputable sites such as "Toms Hardware". Good luck!

@ Jim

Could you elaborate a tad more as to why this would be beneficial with todays hardware/software?
#6
Jonbouy
Max Output Level: 0 dBFS
  • Total Posts : 22562
  • Joined: 2008/04/14 13:47:39
  • Location: England's Sunshine South Coast
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/20 14:34:04 (permalink)
slartabartfast


Why are you worried about performance on these drives? Unless speed is critical to the use of the drive, why not just use the default value? You can certainly spend hours understanding the issues involved with optimizing drive efficiency here, but did you not say these are storage and backup drives? Why spend hours and risk problems to save milliseconds?
 
 
http://technet.microsoft.com/en-us/library/cc767961.aspx
 


It's not really complex.  Using the larger block size will reduce the I/O overhead quite simply.  Now that drive space is cheap losing 8Gb (in a bad case scenario) of space over a 1Tb drive isn't really a concern especially when you'll need to keep a space overhead anyway (10%) to keep the drive running sweetly.  So why not save the milliseconds rather than the cheap space. The equation since NTFS came out has always been balancing performance against efficient storage.  Now the storage is hardly a consideration for most of us why not go for the best performance.

All my data drives are formatted with 64k clusters and that took slightly less time than formatting to the default size.  I didn't spend hours at all and it hasn't caused a problem either.

If they were already formatted to the default cluster size I probably wouldn't bother changing them because of the amount of time it would take getting the data off and back on again but a new drive I'd format 64k everytime.

This is the key line though.

Using the larger block size will reduce the disk I/O overhead

Not to any big degree but on a new drive why not? A glitch or a pop happens in a millisecond.

@Alegria

The problem on older FAT volumes was that you HAD to use large cluster sizes to access larger volumes.  There was no choice.
post edited by Jonbouy - 2012/05/20 15:08:52

"We can't do anything to change the world until capitalism crumbles.
In the meantime we should all go shopping to console ourselves" - Banksy
#7
slartabartfast
Max Output Level: -22.5 dBFS
  • Total Posts : 5289
  • Joined: 2005/10/30 01:38:34
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/21 00:16:20 (permalink)
64KiB clusters probably do produce some theoretical enhanced performance so long as all your data files are large. That it will do so for a large variety of smaller files is not immediately obvious. You should recognize that drive compression under Windows NTFS will not work at greater than 4 KiB clusters. That is why Windows defaults to that value when formatting. The speed of fast processors relative to disk reads make compression almost invisible for data uses that are not real time critical. The cost in terms of lost storage by forgoing drive compression can be very large compared to the loss of storage in unfilled clusters.
 
And, although it does not matter in the case of backup/storage drives, you do not want to format a partition to which you intend to install windows with a nonstandard cluster size. It will not install. Also it is possible to run into other software that will not understand the large clusters.  Some older imaging programs could choke on nonstandard clusters for example resulting in lost images. Everything is simple until it stops working or runs into an unanticipated complication. It is always best to assume that most software is designed and tested with the least inventive option in mind.
post edited by slartabartfast - 2012/05/21 00:20:06
#8
Jonbouy
Max Output Level: 0 dBFS
  • Total Posts : 22562
  • Joined: 2008/04/14 13:47:39
  • Location: England's Sunshine South Coast
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/21 08:49:40 (permalink)
I'd never use NTFS compression since in a previous version of Sonar it wouldn't even recognize compressed data files...presets and the like.  I decided that using Windows compression was a bad idea then.

I already mentioned with the prices and availability of storage these makes the space savings moot.

There is a tangible performance improvement with using the larger cluster size, timed and tested at this end.  My data drives contain 1000's of small files, midi, patches alongside many large files also and like I said it works out at around 8 Gb of 'lost' space on a single Tb drive on that average, given that I normally would keep at least 10% breathing space anyway it isn't an issue.  If space is that short another Terabyte or so is just £45

The reduced seek times, sustained transfer rate, less need for defrags, and quicker when you do need a defrag makes things sing a lot nicer in the context of a DAW, and anyone here asking if a 64k cluster size for a new format for a data drive and stating how it is to be used as in the OP my answer would be the same as Jim's, a simple 'Yes'.

My 2 OS's dual boot from a single 500Gb drive which is formatted at the default cluster size, everything else gets the 64k treatment.  As far as imaging goes I don't know of a current imaging app. that isn't 64k cluster able.  One thing I found with imaging as a single use instance I'm getting a 20% improvement in the time it takes.  I'd say that was more than a few milliseconds.  It's two whole minutes on a previously 10 minute job albeit on a task that particularly seems to favour the larger cluster size.

I've tried it, tested it, use it and am happy with it and therefore recommend it.  As I already said I also wouldn't fret about it if I had a drive already formatted at the 'default' setting that was currently occupied with stuff.  On a new data drive however I'd format at 64k everytime.
post edited by Jonbouy - 2012/05/21 09:09:24

"We can't do anything to change the world until capitalism crumbles.
In the meantime we should all go shopping to console ourselves" - Banksy
#9
SongCraft
Max Output Level: -36 dBFS
  • Total Posts : 3902
  • Joined: 2007/09/19 17:54:46
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/21 14:24:26 (permalink)
Thanks guys, appreciate the input. 

So anyway.... I take that as a yes with a cautionary note from Slartabartfast, on that note I plan to use old software which I purchased a long time ago (Photoshop 6) and plan to assign (scratch disk) to the new hard drive. I guess it should run well particularly on larger files. Can anyone else confirm this?

As for video editing and playback; I read elsewhere that 64k cluster formatted drive will handle larger files better; less likely to get stuttering, snap, crackle and pops. Will see how that all pans out in the arena. 




 
 
#10
Jonbouy
Max Output Level: 0 dBFS
  • Total Posts : 22562
  • Joined: 2008/04/14 13:47:39
  • Location: England's Sunshine South Coast
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/21 14:37:57 (permalink)
I use PS CS3 (V8?) with no problems. And yup faster load times.

"We can't do anything to change the world until capitalism crumbles.
In the meantime we should all go shopping to console ourselves" - Banksy
#11
SongCraft
Max Output Level: -36 dBFS
  • Total Posts : 3902
  • Joined: 2007/09/19 17:54:46
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/21 15:09:56 (permalink)
Jonbouy


I use PS CS3 (V8?) with no problems. And yup faster load times.

Cool, on my much older (non-DAW load of trash PC) it handles OK but on larger files with tons of layers it gets a weee bit stoned. But on my much more beefed up PC (DAW) with the new hard drive (formatted at 64k cluster) assigned primarily for; video/audio and scratch disk it should run at lightening speed. I'll report later if I run into issues. 


I think the first CS started at version 8 thereby CS3 is version 10. 

 
 
#12
Jonbouy
Max Output Level: 0 dBFS
  • Total Posts : 22562
  • Joined: 2008/04/14 13:47:39
  • Location: England's Sunshine South Coast
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/21 15:57:46 (permalink)

I think the first CS started at version 8 thereby CS3 is version 10.


Ah, OK I didn't realise I was that up to date. 6 was a good vintage I think before much of the bloat appeared.

I don't suspect for a minute that Jim would have sold you down the river with his succinct reply, although you could be forgiven for treating my expertise with a pinch of salt.

It's just that the homework I've done on the subject seems to bear out Jim's recommendation 100%

Good luck anyway.
post edited by Jonbouy - 2012/05/21 16:02:55

"We can't do anything to change the world until capitalism crumbles.
In the meantime we should all go shopping to console ourselves" - Banksy
#13
SongCraft
Max Output Level: -36 dBFS
  • Total Posts : 3902
  • Joined: 2007/09/19 17:54:46
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2012/05/21 17:40:29 (permalink)
Jonbouy



I think the first CS started at version 8 thereby CS3 is version 10.


Ah, OK I didn't realise I was that up to date. 6 was a good vintage I think before much of the bloat appeared.

I don't suspect for a minute that Jim would have sold you down the river with his succinct reply, although you could be forgiven for treating my expertise with a pinch of salt.

It's just that the homework I've done on the subject seems to bear out Jim's recommendation 100%

Good luck anyway.



No, no, not at all.... not rowing down the river without an iPaddle LOL!! 

Everything is fine thanks :) , I sincerely appreciate your advise and Jim's. 

I was concerned about older software and some other imaging software I have. Also 64k cluster is something I need to come up to speed.... to better understand.  


Your contribution to this topic is most helpful not only to me but to anyone else who happens to stumble upon this thread. 


All the best! 




 
 
#14
^Gecko^
Max Output Level: -90 dBFS
  • Total Posts : 1
  • Joined: 2017/07/09 04:18:22
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2017/07/09 04:23:57 (permalink)
I know this is years overdue, but there is another benefit to using NTFS cluster sizes larger than 4k;
4k limits you to partition and file sizes no greater than 16 terabytes.  If you need more than 16tB, use 64k clusters, lol.
#15
abacab
Max Output Level: -30.5 dBFS
  • Total Posts : 4464
  • Joined: 2014/12/31 19:34:07
  • Status: offline
Re:Should I format Hard Drive at 64k Cluster (Block) size? 2017/07/09 19:14:19 (permalink)

DAW: CbB; Sonar Platinum, and others ... 
#16
Jump to:
© 2024 APG vNext Commercial Version 5.1