Jul
2
So a few months ago I was visiting Georgia and had a pleasant dinner with a few guys from the Georgia Team. Somehow the topic of Windows Vista came up and some of the “cool” features that it had, one of which being an ability to add available space from an inserted USB drive to the virtual memory (swap). Well, I quickly outlined a plan in my head on how it *could* work, but never sat down to actually try it. Today I finally made time to try it.
After twenty minutes I had a basic implementation. Not very flexible, but it worked on the USB drive I plugged it. The first implementation took only 43 lines of bash code, much of that simply being echo output, so it could have been smaller.
After some more testing and expanding it now seems a little more solid. It’s a bit over 100 lines of bash code, with plenty of room to grow but I wanted to get some feedback from the general public. If you would like to help alpha test this application you can get a copy of the script below. If you’d like to help on this project I think there is plenty of improvement and could make for a good team project as well.
In any event, here is a basic outline of what it does and how to use it at this point. It may not be completely intuitive, but that is part of what I need feedback on. Below are three steps to increase your swap size by using excess space on a pluggable USB disk.
- Insert USB disk and allow gnome / KDE to auto-mount the device.
- Run ./swapboost.sh -n to create a new swap addition
- Check your system memory. Your swap should now be extended by the amount of free space on the device. (swapon -s will show you all available swap areas).
When you would like to make use of the disk again for that other than swap you can run the cleanup portion of the script by doing the following.
- Run ./swapboost.sh -d to delete added swap and safely unmount your USB disk.
- Check your system memory. Your swap should now be reduced by the amount of space previously added. (swapon -s will show you only your default swap area).
I’d love to get some feedback on this script. I know there are some features that need improving. A GUI would be great. The ability to select the size to expand the swap, instead of *only* the full size. A more intuitive launcher.. etc. The list can go on and on. Again, if you’d like to join this project I think it could be fun. If you’d simply like to give some feedback check out the v0.1alpha release below.
This release goes out to my boys in Georgia. Thanks for planting the seed.
Remember, this script is in alpha stage and the user assumes all responsibility for its potential lack of function. If this messes up your machine it is not my fault. Use at your own risk. (I had to get that out of the way. It most likely won’t really screw things up, but you never know at this stage.)
I post tutorials very regularly on this site. You may want to consider subscribing to the RSS feed. Or if you'd prefer these tips sent to your inbox you can use Email Subscriptions.
Related Posts
Random Posts
-->Comments
43 Responses to “SwapBoost v0.1alpha - early testers wanted”
Leave a Reply
Loading ...-
Blogroll
OSCON
Subscribe to the RSS feed!
[...] of course, read more here at Ubuntu Tutorials. These icons link to social bookmarking sites where readers can share and discover new web [...]
Cool — note that if you are going to swap, it will drastically reduce the lifespan of your flash media; current flash media is not meant to be rapidly overwritten to in this fashion. Of course, with flash media so cheap and people owning several sticks (and losing them way before 100,000 rewrites), I doubt many people care
I just looked at your code. Even I could understand it.
Is this ALL that readyboost does? There has to be more.
It’s hard to believe that it could be this simple.
Martins
From my understanding of this technology on Vista, it is nothing at all like swap space.
What happens is that commonly needed, yet infrequently modified files are copied to the usb drive.
These files are then used during boot and application startups. The additional drive takes load off of the disk. Since the system is already bottlenecked by IO from the hard disk, having the additional data source makes things faster.
Plus, since the files on the usb disk are infrequently changed, this has no ill effects on the lifespan of the usb drive.
Agreed with jdong, this will kill your flash drive.
I know you have a disclaimer already, but you might want to make a note of it being bad for the drive.
That is nothing like readyboost, but might still be useful. Readyboost does pre-emptive caching, copies stuff that has not been used yet but likely will, to the usb stick.
Sure, you can mimick that with building a statistics gathering daemon and some fs redirecting to the stick…
Well preload certainly works for your “statistics gathering daemon”.
Indeed, I was aware of that. All that is missing is some plumbing to make it so that the applications will access the certain files from the flash disk instead of the normal hdd..
Dont’ want to offend you, but I don’t really see the point of this. It’s surely useless from the point of view of providing additional space, and I’m really not sure whether it could provide any speed boost by using the flash memory.
Linux swap is not suitable for flash drives at all. Please do not encourage its misuse.
Well /I/ think it’s a good idea. ReadyBoost is just swap + preload. That’s /really/ all this is too. ReadyBoosts performance boost is also questionable, it’s just proof of concept of how simple MS readyboost is. nj zelut.
I wonder what would happen if you (intentionally or accidentally) pulled out the flash drive when it was being used as swap.
I wouldn’t be surprised to get a kernel panic.
Cool app, I had this idea as well when I read about Vista’s ReadyBoost ‘technology’. ..
some ideas and concerns:
1. Would it be possible to offer to backup the data, and restore it when the swap part wasn’t needed? I would hate to plug in my drive for this purpose and not realize i was losing everything.
2. It would be ‘desasterous’ if the USB was a vfat32 partition and this left unknowing users with a stick that didn’t function on their closed OSes (unless drivers were installed, but you get the idea)
3. Do well-setup Linux systems really need an additional 128 or whatever of Swap? Memory shortfalls are a Windows issue, and it looks like Vista hasn’t fixed it, they just allow you to dynamically add space. Adding swap space from a hard drive is easy enough (but this doesn’t solve the hard drive bottleneck issues as discussed).
4. There are graphical tools meant for Bash which may help you; an example is Zenity (Automatix uses it)
I like the concept though and it may be useful…
Sooooo stupid idea… Ubuntu, when Windows does something ugly, please do not waste your time trying copying it… You want to boost boot ? Fix apps. Fix HAL, CUPS, X.org, Gnome, … Read “Why userspace sucks” here : http://trends.newsforge.com/article.pl?sid=06/07/21/192241
“on boot, FC5 was touching 79,000 files and opening 26,000 of them. On shutdown, 23,000 files were touched, of which 7,000 were opened.”
Arrggghhhh… 79.000 files !!!
Vista ReadyBoost has nothing to do with swap or paging — it uses flash media as a cache basically. Linux does not use swap to perform any caching.
The ONLY time I would do this, is if I had 512MB of RAM and a 1GB flash drive, my root fs was damaged, and fsck requires 800MB of total memory to rebuild my filesystem.
[...] executable. SwapBoost is a free download, but it’s still in alpha, so proceed with caution. SwapBoost [Ubuntu [...]
[...] executable. SwapBoost is a free download, but it’s still in alpha, so proceed with caution. SwapBoost [Ubuntu [...]
[...] SwapBoost [Ubuntu Tutorials] [...]
I’m sorry for this, but this is a horrendous idea.
Flash media poorly suited for use as normal swap space (very slow, limited life span).
Vista is doing something quite particular and different than what you do here.
[...] executable. SwapBoost is a free download, but it’s still in alpha, so proceed with caution. SwapBoost [Ubuntu [...]
[...] SwapBoost es un script de Bash que nos permite utilizar el espacio libre que tengamos en nuestra memoria usb, para utilizarlo como memoria Swap incrementando en algunos casos el rendimiento del sistema. [...]
[...] [Source] [...]
[...] SwapBoost es un script experimental escrito en Bash que nos permite utilizar el espacio libre que tengamos en nuestra memoria USB o pendrive, para utilizarlo como memoria Swap incrementando en algunos casos el rendimiento del sistema. Se podría decir que funciona de una forma similar al ReadyBoost de Windows Vista. [...]
[...] SwapBoost v0.1alpha - early testers wanted : Ubuntu Tutorials : Breezy - Dapper - Edgy - Feisty Basically an automated way to put a page file on your flash drive. Still kinda cool, though. [...]
[...] SwapBoost es un script experimental escrito en Bash que nos permite utilizar el espacio libre que tengamos en nuestra memoria USB o pendrive, para utilizarlo como memoria Swap incrementando en algunos casos el rendimiento del sistema. Se podría decir que funciona de una forma similar al ReadyBoost de Windows Vista. [...]
for mount points those included spaces it cannot works.
Bad for flash, agreed, and dangerous for swap safety (unplugging while active?). However, pre-emptive caching is not necessarily available only with flash. I have a shell script that runs when I log in to Gnome, containing the line:
( cd /usr/lib ; find firefox* openoffice.org2.0/program -type f -name \*.so ) | nice xargs cat > /dev/null &
This finds all regular files and reads them, effectively loading their contents into file cache. This speeds up the launch of Firefox and OpenOffice.org noticeably. You can substitute your own files/directories.
Note: this really works only if you don’t normally use swap during regular system usage. I have 768M, so I want to use filesystem cache. On a 256M system, it probably won’t work as well for lots of file data.
Oh, and by the way:
I started doing this a couple years ago, before Vista even got its name.
Nyah nyah nyah, Redmond!
[...] 9th, 2007 SwapBoost é un script experimental escrito en Bash que nos permite utilizar o espazo libre que teñamos na [...]
[...] Swapboost è uno script per Linux che permette di implementare su un dispositivo di memoria USB una caratteristica molto simile al Readyboost di Windows Vista! Swapboost è uno script di dimensioni molto limitate, e non fa altro che trasformare lo spazio libero sulla chiavetta USB in spazio disponibile per la memoria virtuale (file di swap, appunto). E’ sicuramente un risultato apprezzabile, ma la domanda è: Linux, con la sua eccellente gestione della memoria, ne ha davvero bisogno? Su sistemi con 2GB di memoria è altamente improbabile che Linux acceda al file di swap. Su sistemi con 1GB gli accessi si possono contare sulle dita di una mano. Il discorso è completamente diverso per i sistemi Windows, dove anche task di infimo livello accedono al file di swap. [...]
[...] Windows Vista turi funkcij? ReadyBoost, kuri, pasitelkdama j?s? USB atmintin?s talp?, teigia galinti paspartinti sistem?. Rezultatai apie real? spartos padid?jim? labai nevienareikšmiški, bet jei norite išbandyti analogišk? dalyk? savo Linux sistemoje, tai galite pasinaudoti nedideliu bash scenarijumi, kurio alfa versij? galima parsisi?sti iš Ubuntu Tutorials. [...]
[...] SwapBoost adl? yaz?l?m ile ReadyBoost benzeri bir uygulaman?n Linux alt?nda nas?l uygulanabilece?ini anlatm??. Faydas? ve pratikli?i tart??ma konusu olsa da, böyle bir?eyin yap?labiliyor olmas? [...]
It would make more sense to modify readahead or “preload” for example to cache the often readaccessed files (e.g. libc,libstdc++, etc.). This best resembles “ReadyBoost” technology.
Anyway, the swap-prefetch patches from Con Kolivas, which prefetches swap data back to RAM, makes much more sense than this script…
With your static swap in fstab give it a low priority. It will still be just as fast but allow you to sneak in higher priority swap later on. In the script make sure you can set which priority the USB swap device will be when you insert it.
Sure, this will zap your flash drive eventually - whats with the new U3 flash tech however, what makes that more stable?
As I understand it, ReadyBoost is a cached copy of part of the swap intended to speed up small, random accesses. Paging in of an entire app, for example, would still come from the HD, but individual pages could be read from the flash drive as needed.
If you have an app that requires more memory than your available RAM (think heavy number-crunching), and the memory accesses are random enough, then using a FLASH drive for swap should speed things up.
Hey, i feel a lot of people are being kind of dickish on this forum. Hell, he made it and I’d be interested to use it. it needs a little more fine tuning yes, maybe employing preload to tell it what to store (that way it acts basically just like readyboost). The only trouble I’m having is running the script which seems to be having several conflicting permission problems, both OS and harddrive. Good job man, way to try and come up with something new for linux!
my recommendations:
-interact with preload to boost preload space that will be fast
-add a gui, basic is all thats needed, if a noob wanted to use this, they couldn’t
-definitely adjustable space option
-backup swap and in case of sudden removal, sub in back up so kernel doesn’t freak
Flash drives will also be made better and more reliable in the next few years so software made on flash should be started now. I’d keep this project, despite the naysayers
Does linux use swap evenly? If so, how long does it take to write 200 terabytes(*) of data to it? If not, patching it should be a big deal, right?
*) 50k*4G, is this number way off for some reason?
[...] SwapBoost es un script experimental escrito en Bash que nos permite utilizar el espacio libre que tengamos en nuestra memoria USB o pendrive, para utilizarlo como memoria Swap incrementando en algunos casos el rendimiento del sistema. Se podría decir que funciona de una forma similar al ReadyBoost de Windows Vista. [...]
Considering just how inexspensive flash has become I feel the risk of killing the drive is a minimal bump to cross. RAM costs much more and we are limited by just how much can be added. If the usb swap can speed up my pc, even by an small amount, it is worth it; If for nothing less that saving on the wear and tear of my hard drive. And as an added bonus, I dont have to listen to the thrashing as often.
Good deal
I like that idea of something simple, a little script that will work with moreorless any linux, not just ubuntu, that perhaps wears out flashdrives, but they’re cheap, I’d rather wear ten of those out than one hard drive. Perhaps it’s the first in essentially a tree of dependancies, where a GUI is added, and it’s automated as part of whatever automates USB disk mounting and unmounting? Those are the next steps, but they can’t happen without someone trying it in the first place. Thankyou, because it has to start somewhere.
Your SwapBoost solution is not really readyboost at all. You see ReadyBoost mirrors some of the info from the swap file into a file on the memory stick. Thus the info on the memory stick is still in the swap file on the HD in case the memory stick is pulled out. SwapBoost may crash your system if you pull the memory stick without unmounting.
I don’t think it would be hard to make a Linux kernel patch that does this.
I’ve got a couple of machines using CF Flash as hard drives. They’ve been working for about a year now without any problems. I am seeing comments about flash sticks not being able to handle the rewrites in this thread. Is this a weakness of the flash in USB sticks? If so, will your batch work with a CF? My laptop has a CF port and I’d like to try this.