Custom Search
Join the PC homebuilding revolution! Read the all-new, FREE 200-page online guide: How to Build Your Own PC!
NOTE: Using robot software to mass-download the site degrades the server and is prohibited. See here for more.
Find The PC Guide helpful? Please consider a donation to The PC Guide Tip Jar. Visa/MC/Paypal accepted.
Results 1 to 20 of 20

Thread: Format, scandisk, defrag a USB flash drive?

  1. #1
    Join Date
    Nov 2002
    Location
    The State of Independence
    Posts
    889

    Format, scandisk, defrag a USB flash drive?

    I searched posts in this forum and didn't see this question posed before, but I can't believe I'm the first one to wonder if I could or should do routine maintenance on a flash drive like I do on a HD, floppy or Zip disk. Can it be reformatted? How about scandisk and defrag? I'll wait for a reply before performing any of these actions.

  2. #2
    Join Date
    Aug 2003
    Location
    Northern California
    Posts
    13,442
    No you should not need to defragment, a USB flash drive is a digital memory device, not a spinning disk.

  3. #3
    Join Date
    Dec 2001
    Posts
    320
    Careful how you play. USB drives can written to by various utilities. It is possible, that you could get unexpected results. Depending on what drive you want to play with, be sure you check the vendor's website and use their utilities........
    Missing Operating System

  4. #4
    Join Date
    Oct 2001
    Location
    N of the S of Ireland
    Posts
    20,504
    It really depends on the file system rather than the fact that such drives don't contain a spinning disk inside them. Mostly FAT16 seems to be used but bear in mind that some of these flash drives are getting into big capacities and other file systems are used particularly for use on MACs, for example.

    The main "problem area" is to do with non standard mbrs (usually only one non-bootable primary partition) on those that emulate HDDs. I have accessed many of these drives with hex editors without problems and have run chkdsk and defrag and formatted them from windows without problems - which doesnt mean no problems will ever occur.
    Take nice care of yourselves - Paul - ♪ -
    Help to start using BiNG. Some stuff about Boot CDs & Data Recovery Basics & Back-up using Knoppix.

  5. #5
    Join Date
    Mar 2002
    Posts
    12,211
    Blog Entries
    2
    First of all, you don't really want to do defrag a flash drive because flash memory doesn't last forever. The onboard controller in a flash drive is designed to stagger the use of the sections of flash memory on the chip(s) so that they age evenly. By running defrag, you're aging certain portions of your flash memory faster than others.

    Scandisk doesn't hurt a flash drive, but it doesn't necessarily help, either. Again, the onboard controller keeps track of bad sectors by verifying all data after a write cycle. If the verification fails, the onboard controller quarantines that section and never uses it again.

    Formatting is also done by the onboard controller, which uses some of the total space for the file system. For instance, my 128MB flash drive uses 5MB for the file system and core routines, leaving 123MB of usable space.

    The whole reason for having the USB & drive controller built into a flash drive is to take care of all this maintainance and to allow flash drives to be bootable. Don't try to usurp its authority!

  6. #6
    Join Date
    Oct 2001
    Location
    N of the S of Ireland
    Posts
    20,504
    you're aging certain portions of your flash memory faster than others
    Now that's interesting and particularly interesting if one is going to be booting to an OS on such drives. What I am getting at is the VM in such a situation. Since (with or without a fixed size VM) there is going to be an awful lot more read/writes made to the same or similar areas of the drive. I guess it would thus be advisable to try to avoid a pagefile altogether and to only boot to such drives when there is loads of RAM on the system.


    Formatting is also done by the onboard controller
    Also interesting and something I had not really ever thought about. How does this compare with the role of the onbard controller when formatting magnetic media?
    Take nice care of yourselves - Paul - ♪ -
    Help to start using BiNG. Some stuff about Boot CDs & Data Recovery Basics & Back-up using Knoppix.

  7. #7
    Join Date
    Mar 2002
    Posts
    12,211
    Blog Entries
    2
    Quote Originally Posted by Paul Komski
    How does this compare with the role of the onbard controller when formatting magnetic media?
    Normal hard drives have a shared controller architecture, where both the hard drive and the system have a piece of the puzzle, so to speak. And hard drives are based on many many years of universal drivers for both Windows and Mac OS, which can natively handle hard drives of all sorts (including USB external ones).

    However, there's no such set of universal drivers for the various types & capacities of flash memory. Flash drives have a USB controller for handling the USB bus interface (which is universal for the most part) and of course most USB implementations have a bootable spec. But on the inside, the flash memory controller itself is in full control. It shares it duty with nothing else. This is a good thing, because there's no set rule for how flash memory works. There are many types of flash memory, many voltage schemes, many sizes, many chip configurations, etc. A full internal controller for the flash memory itself is the only solution. This also makes them highly flexible, because the file structure is internal and is thus fully compatible with any system capable of interfacing with a USB removable drive. The files themselves may not make any sense from Windows to Mac OS, but the drive itself is still usable.

    As to the aging effect, this is really only a concern if you exceed the write cycles of the flash memory (which is in the range of tens of thousands of rewrites). Small embedded systems have been using flash memory for hard drive duties for years, and of course tiny Linux-based boxes using CompactFlash cards for hard drive duties are also popular. Running an OS off a flash drive is of little concern as long as you realize that you have to replace it every now and then. But to intentionally make things worse by defragging a flash drive is unnecessary.

  8. #8
    Join Date
    Oct 2001
    Location
    N of the S of Ireland
    Posts
    20,504
    The files themselves may not make any sense from Windows to Mac OS, but the drive itself is still usable.
    That I don't follow unless you are talking about sectors and not files as such. As I had understood things, any drive controller (following a true factory low level format) presents the usable disk space to any operating system as an "organisation of usable bytes". The geometry may vary but under modern systems this space is usually addressable as LBA sectors containing 512 bytes (of data and which doesnt include any metadata) per sector.

    For most (but not all) operating systems such raw space needs to be initialised with one or more partitions which then have file systems written to them by normal formating using an operating system file such as format.com for example.

    When you say "Formatting is also done by the onboard controller" and "The files themselves may not make any sense from Windows" this states or implies that such controller-formatting is organising the file system - as distinct from the addressable space - and that is what I am having trouble grasping. I have variously formatted USB flash drives with FAT, FAT32 and NTFS file systems (and I believe that Mac users can format with OSX). I have used WinXP, BootItNG and the HP download from http://h18007.www1.hp.com/support/fi...oad/20306.html at different times and with different results.

    We may well be talking at cross purposes but this is an area of great interest to me so any help in understanding is much appreciated - particularly when it relates to data recovery.
    Take nice care of yourselves - Paul - ♪ -
    Help to start using BiNG. Some stuff about Boot CDs & Data Recovery Basics & Back-up using Knoppix.

  9. #9
    Join Date
    Aug 2003
    Location
    Northern California
    Posts
    13,442
    I am treating my thumb drive with a bit more respect after reading this thread.

  10. #10
    Join Date
    Oct 2001
    Location
    N of the S of Ireland
    Posts
    20,504
    Have read up a few articles on Flash File Systems and there's a pretty good Wikipedia about Flash Memory at http://en.wikipedia.org/wiki/Flash_memory.

    I now see that there are two important problem areas. One is that erasure is in blocks and is slow and the other is to distribute as much wear levelling as possible.

    In reading about the flash file systems such as FFS2, FTL JFFS etc I guess that is what you, Saphalline, was referring to. I also noted the comment: "However, in practice most flash media is used with the old FAT filesystem for compatibility purposes."

    I suppose that since read/write is best minimised on such media that an operating system using FAT would be better than NTFS (when installed on such memory) since the latter is a journaling system with many more i/o operations. On the other hand some figures on the expected life of such memory under different real life conditions would be nice to get one's hands on. I mean - hard drives dont last forever either so would one expect to get a comparable life out of such media.

    As ever it amazes me how much more there is to learn - but that is also the beauty of the way technology is developing.
    Take nice care of yourselves - Paul - ♪ -
    Help to start using BiNG. Some stuff about Boot CDs & Data Recovery Basics & Back-up using Knoppix.

  11. #11
    Join Date
    Jan 2004
    Location
    Here
    Posts
    10,487
    Paul if I remember right when I bought my flash drives they said "data retention up to 10 years".

    I would take it that means 10 years is the max. So I would guess maybe 7-8 years for normal operations. Maybe 5 for heavy use. That's just my guess though.

    It would be interesting to see some real world stats.
    8.1 Pro 64bit
    AMD FX 8350
    Asus Sabertooth 990FX R2.0
    24GB GSkill/Corsair 1866MHz
    2x XFX 6870 1GB
    SSD - OCZ V4 128GB/Kingston 120GB
    HDD - 3TB/2x 750GB/500GB/250GB
    Corsair TX850M
    Cooler Master HAF 932 Red
    CM Hyper 212 EVO w/2x SickleFlow 120mm Red
    12x BD-ROM
    Logitech X540 5.1 Surround
    2X Acer 23" LED - Eyefinity: 3840x1080
    Microsoft Sidewinder x4 KB

    "The significant problems we face cannot be solved at the same level of thinking we were at when we created them."
    - Albert Einstein

  12. #12
    Join Date
    Mar 2002
    Posts
    12,211
    Blog Entries
    2
    Quote Originally Posted by saphalline
    The files themselves may not make any sense from Windows to Mac OS, but the drive itself is still usable.
    Haha, sorry, I guess this one was confusing. What I meant by this is that you can put files on a flash drive using a Windows system, and then put that same flash drive into a Mac system and transfer the files over. However, Windows and Mac don't always use the same file types! A VXD driver file from a Windows system would mean nothing to a Mac system. That's all I was saying.

    Yes, flash memory is quite interesting, isn't it? NOR gates vs NAND gates, different topologies, block erasures, limited insulator lifespan, quantum-tunneling, all sorts of cool stuff. One of the reasons that you have to be gentle with flash memory is specifically because of the block erasures inherent in the NAND gate usage. Since you can only erase entire blocks, making a small change to a file essentially means erasing a block and then rewriting the portion of the block where the file resides. In that sense, I hadn't thought about the overhead of NTFS vs FAT, but that's a good point. Less wear & tear is always good.

    Number of write cycles for each type of flash memory? That would be a HUGE list! The problem here is that they're all different. Even among the same manufacturer. For instance, a normal SD card from Kingston may have x number of write cycles for its lifespan, but a high-speed SD card from Kingston might have 0.8x for its lifespan. Faster flash memory generally ages faster because the extra speed and/or voltage wears out the insulator faster. And of course the oldies like slow CompactFlash have really long lifespans, because they use larger gate elements and thus thicker layers of insulator and they're slower. Newer types of flash memory like xD use multi-level gate technologies where each gate element can store multiple bits. This level of precision, coupled with thinner layers of insulator would seem to make it age faster, but the much lower voltages and smaller eV capacitancies increase its lifespan.

    Even the manufacturers have no set formula for figuring out the lifespan of a flash memory product. They essentially just take a sample, set it up for constant rewriting, and let them go! Then they measure the death toll. That's about it! But of course this doesn't take into account actual aging over time, as in how does it store data over years and years if untouched? All forms of solid-state memory suffer some form of leakage over time, but how do you truly measure this? My advice is not to leave any sensitive data on a flash drive in storage for long periods of time but theoretically, you could. At least for some types of flash memory.

    For the formatting, the onboard flash memory controller is responsible for translating the file system and for keeping track of evening out the wear & tear, but I'm not entirely certain about the implications of organizing the memory itself into sectors while at the same time the memory is orgainized into blocks and layers. I suppose in that case the file system installed would lay on top of the native file system of the onboard controller. It would have to be something like that because flash memory is solid-state memory, based on chips capable of random access. It's not a platter of expansive and contiguous bits. The only "order" to any type of RAM/ROM chip is implied. It's like the difference between a race track with definitive circular elements and a direction that you're supposed to follow, and an office building. Finding a car on a race track is easy! Finding the "Associate VP" in an office building is not quite as intuitive!

  13. #13
    Join Date
    Jan 2004
    Location
    Here
    Posts
    10,487
    Very interesting. The first flash drive I got(Kingmax 128MB) I formatted it with NTFS just to play around. It was unusable. It wouldn't transfer anything at all. Tried FAT, same thing. I had to use the recovery utility to get it back to normal. I haven't tried to format with my 1.5GB(Rosewill) one yet. Actually I don't see any reason to format it. They work on any computer I have put them in.
    8.1 Pro 64bit
    AMD FX 8350
    Asus Sabertooth 990FX R2.0
    24GB GSkill/Corsair 1866MHz
    2x XFX 6870 1GB
    SSD - OCZ V4 128GB/Kingston 120GB
    HDD - 3TB/2x 750GB/500GB/250GB
    Corsair TX850M
    Cooler Master HAF 932 Red
    CM Hyper 212 EVO w/2x SickleFlow 120mm Red
    12x BD-ROM
    Logitech X540 5.1 Surround
    2X Acer 23" LED - Eyefinity: 3840x1080
    Microsoft Sidewinder x4 KB

    "The significant problems we face cannot be solved at the same level of thinking we were at when we created them."
    - Albert Einstein

  14. #14
    Join Date
    Apr 2005
    Location
    fairview park, ohio (near cleveland)
    Posts
    666
    I nominate saphalline to be the first human to have his brain cloned.

  15. #15
    hi,

    i beginer here, hope everyone can help......thx 1st........
    i'm currently implementing the FAT structure on to the nand flash and i found some difficulties, hope anyone able to help me.

    i had this nand flash -> 256MB: 1page = (2046 + 64)bytes, 64pages = 1 block, total block number = 2048.
    izzit 1 sector = 1 page, and 1 cluster = 1 block? if this is the case, my cluster size is larger than 65535bytes which the microsoft claim this may cause problems.

    besides, 1 logical address represent 1 block? so my total logical address is 2000?

    all the information i get is 1 sector = 512+16 byte where the structure can be known but 64byte redundancy area,what is the stucture???

    if i need to transfer data from pc to nand flash through the usb, izzit my nand flash mush had the fat file structure? or i can implement another type of file management system to the nand flash?

    anyone able to help can direct email me oso ->ycherjier[AT]yahoo.com
    Last edited by PrntRhd; 05-04-2006 at 12:10 AM. Reason: Munged email address

  16. #16
    Join Date
    Jul 2005
    Location
    Newburgh, NY
    Posts
    315
    Quote Originally Posted by jcnoernberg
    I nominate saphalline to be the first human to have his brain cloned.
    I second that nomination

  17. #17
    Join Date
    Dec 2001
    Location
    NEW ZEALAND
    Posts
    3,593
    I 3rd it!!!
    "An expert is someone who will tell you why you can't do something." - Alec Issigonis (designer of the Mini)

  18. #18

    Solution to override not being able run ckdsk on USB drives

    Problem:

    I asked disk keeper to run boot time defragmentation on my external drive, however when I rebooted my computer it completely ignored the scheduled event, leaving it scheduled for next boot time. I now have the problem of needing to run check disk on my drive before I can do any defragmentation on it (the computer refuses to defragment a drive that has disk checking scheduled). Is there any way to make windows recognise the disk and perform the disk check?


    Yes I found the solution:

    Here is a solution to override this issue of not being able to run the ckdsk when you boot up for any external USB drive. Since the USB drivers arent loaded before the Checkdisk can run .. you need to do it manually.

    Here is what you do..

    1. Have the drive plugged into your computer.
    2. Under "My Computer" find your external drive, usually E: or whatever...
    3. RIGHT CLICK on that drive
    4. Select PROPERTIES
    5. Select TOOLS
    6. Click CHECK NOW button under the ERROR CHECKING option
    7. The you will be asked for check 2 choices..
    8. Check BOTH options..( this will take awhile depending on how bad your drive is...)
    9. Then you can go back to the orginal TOOLS tab and choose the DEFRAGMENT option
    10. Choose that option.. it should use the defrag software you have installed on your computer to do the defragmentation on that drive.
    11. Let it run..it will take awhile and then close it.

    It should have your external drive check disked and defragment when your finished.

    I did this and it worked for me.. I hope it works for anyone else who uses this tip!!

    Let me know if it does or doesnt.. I would be happy to know if it works for anyone else!!

  19. #19
    Join Date
    Mar 2002
    Location
    west Lothian, Scotland.
    Posts
    13,308
    "Format, scandisk, defrag a USB flash drive?"
    At THIS THREAD sburtchin showed me how to add USB4DOS drivers to an "Emergency Boot CD" [EBCD].
    I'm now able to run most of the MANY utilities on the EBCD so as to check not only the internal HDD, but also the external USB devices including my external 80 GB USB HDD and 1 GB USB memory stick.

    I noticed when I ran "Microsoft Scandisk" that it checked ALL the drives [file system and cluster checking] including the USB HDD and memory stick.
    [Cluster checking froze the software part way through the 1st partition on the external USB HDD ]

    Things it might do [haven't checked them all]...
    1. Browse contents with "File Manager".
    2. Scan drives for faults in file system and clusters.
    3. Un-Format.
    4. Un-Erase.
    5. FDISK partition.
    6. Save image backups.
    7. Zero-fill.

    Still trying to figure out just how much customisation is possible, and what is necessary to make various programs work on external USB devices.

    e.g. Attempting at this time to make the "Drive Fitness Test Utility" work with external USB devices, as seen HERE.
    Last edited by Sylvander; 11-19-2006 at 05:44 AM.

  20. #20
    Join Date
    Nov 2006
    Location
    Cornwall
    Posts
    5
    8 gig micro disk, usb drive you can defrag them.........

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •