aboutsummaryrefslogtreecommitdiff
path: root/block/elevator.c
AgeCommit message (Collapse)Author
2008-04-29block: Skip I/O merges when disabledAlan D. Brunelle
The block I/O + elevator + I/O scheduler code spend a lot of time trying to merge I/Os -- rightfully so under "normal" circumstances. However, if one were to know that the incoming I/O stream was /very/ random in nature, the cycles are wasted. This patch adds a per-request_queue tunable that (when set) disables merge attempts (beyond the simple one-hit cache check), thus freeing up a non-trivial amount of CPU cycles. Signed-off-by: Alan D. Brunelle <alan.brunelle@hp.com> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2008-04-29block/elevator.c:elv_rq_merge_ok() mustn't be inlineAdrian Bunk
This patch fixes the following build error with UML and gcc 4.3: <-- snip --> ... CC block/elevator.o /home/bunk/linux/kernel-2.6/git/linux-2.6/block/elevator.c: In function ‘elv_merge’: /home/bunk/linux/kernel-2.6/git/linux-2.6/block/elevator.c:73: sorry, unimplemented: inlining failed in call to ‘elv_rq_merge_ok’: function body not available /home/bunk/linux/kernel-2.6/git/linux-2.6/block/elevator.c:103: sorry, unimplemented: called from here /home/bunk/linux/kernel-2.6/git/linux-2.6/block/elevator.c:73: sorry, unimplemented: inlining failed in call to ‘elv_rq_merge_ok’: function body not available /home/bunk/linux/kernel-2.6/git/linux-2.6/block/elevator.c:495: sorry, unimplemented: called from here make[2]: *** [block/elevator.o] Error 1 make[1]: *** [block] Error 2 <-- snip --> Signed-off-by: Adrian Bunk <bunk@kernel.org> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2008-04-29block: make queue flags non-atomicNick Piggin
We can save some atomic ops in the IO path, if we clearly define the rules of how to modify the queue flags. Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2008-02-19elevator: make elevator_get() attempt to load the appropriate moduleJens Axboe
Currently we fail if someone requests a valid io scheduler, but it's modular and not currently loaded. That can happen from a driver init asking for a different scheduler, or online switching through sysfs as requested by a user. This patch makes elevator_get() request_module() to attempt to load the appropriate module, instead of requiring that done manually. Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2008-02-01block: make elevator lib checkpatch compliantJens Axboe
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2008-01-28block: implement drain buffersJames Bottomley
These DMA drain buffer implementations in drivers are pretty horrible to do in terms of manipulating the scatterlist. Plus they're being done at least in drivers/ide and drivers/ata, so we now have code duplication. The one use case for this, as I understand it is AHCI controllers doing PIO mode to mmc devices but translating this to DMA at the controller level. So, what about adding a callback to the block layer that permits the adding of the drain buffer for the problem devices. The idea is that you'd do this in slave_configure after you find one of these devices. The beauty of doing it in the block layer is that it quietly adds the drain buffer to the end of the sg list, so it automatically gets mapped (and unmapped) without anything unusual having to be done to the scatterlist in driver/scsi or drivers/ata and without any alteration to the transfer length. Signed-off-by: James Bottomley <James.Bottomley@HansenPartnership.com> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2008-01-24Kobject: rename kobject_init_ng() to kobject_init()Greg Kroah-Hartman
Now that the old kobject_init() function is gone, rename kobject_init_ng() to kobject_init() to clean up the namespace. Cc: Kay Sievers <kay.sievers@vrfy.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2008-01-24Kobject: rename kobject_add_ng() to kobject_add()Greg Kroah-Hartman
Now that the old kobject_add() function is gone, rename kobject_add_ng() to kobject_add() to clean up the namespace. Cc: Kay Sievers <kay.sievers@vrfy.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2008-01-24Kobject: convert block/elevator.c to use kobject_init/add_ng()Greg Kroah-Hartman
This converts the code to use the new kobject functions, cleaning up the logic in doing so. Cc: Jens Axboe <axboe@kernel.dk> Cc: Kay Sievers <kay.sievers@vrfy.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2007-12-18block: let elv_register() return voidAdrian Bunk
elv_register() always returns 0, and there isn't anything it does where it should return an error (the only error condition is so grave that it's handled with a BUG_ON). Signed-off-by: Adrian Bunk <bunk@kernel.org> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2007-10-20fix typo "insted" -> "instead"Uwe Kleine-König
Signed-off-by: Uwe Kleine-König <ukleinek@informatik.uni-freiburg.de> Signed-off-by: Adrian Bunk <bunk@kernel.org>
2007-10-16block: Initial support for data-less (or empty) barrier supportJens Axboe
This implements functionality to pass down or insert a barrier in a queue, without having data attached to it. The ->prepare_flush_fn() infrastructure from data barriers are reused to provide this functionality. Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2007-10-16block: add end_queued_request() and end_dequeued_request() helpersJens Axboe
We can use this helper in the elevator core for BLKPREP_KILL, and it'll also be useful for the empty barrier patch. Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2007-10-12kobjects: fix up improper use of the kobject name fieldGreg Kroah-Hartman
A number of different drivers incorrect access the kobject name field directly. This is not correct as the name might not be in the array. Use the proper accessor function instead.
2007-07-24[BLOCK] Get rid of request_queue_t typedefJens Axboe
Some of the code has been gradually transitioned to using the proper struct request_queue, but there's lots left. So do a full sweet of the kernel and get rid of this typedef and replace its uses with the proper type. Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2007-07-17Slab allocators: Replace explicit zeroing with __GFP_ZEROChristoph Lameter
kmalloc_node() and kmem_cache_alloc_node() were not available in a zeroing variant in the past. But with __GFP_ZERO it is possible now to do zeroing while allocating. Use __GFP_ZERO to remove the explicit clearing of memory via memset whereever we can. Signed-off-by: Christoph Lameter <clameter@sgi.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-07-10Use list_for_each_entry() instead of list_for_each() in the block deviceMatthias Kaehlcke
elevator Signed-off-by: Matthias Kaehlcke <matthias.kaehlcke@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2007-04-30[PATCH] elevator: elv_list_lock does not need irq disablingJens Axboe
It's never grabbed from irq context, so just make it plain spin_lock(). Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2007-03-27make elv_register() output atomicThibaut VARENE
Booting 2.6.21-rc3-g45592145 I noticed the following on one of my machines in the bootlog: io scheduler noop registered<6>Time: jiffies clocksource has been installed. io scheduler deadline registered (default) Looking at block/elevator.c, it appears that elv_register() uses two consecutive printks in a non-atomic way, leading to the above glitch. The attached trivial patch fixes this issue, by using a single printk. Signed-off-by: Thibaut VARENE <varenet@parisc-linux.org> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2007-02-11elevator: don't sort reads between writesJens Axboe
Don't allow elv_dispatch_sort() to mix reads and writes together, it's rarely a good idea. Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2007-02-11elevator: abstract out the activate and deactivate functionsJens Axboe
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2007-01-23[PATCH] elevator: move clearing of unplug flag earlierLinas Vepstas
A flag was recently added to the elevator code to avoid performing an unplug when reuests are being re-queued. The goal of this flag was to avoid a deep recursion that can occur when re-queueing requests after a SCSI device/host reset. See http://lkml.org/lkml/2006/5/17/254 However, that fix added the flag near the bottom of a case statement, where an earlier break (in an if statement) could transport one out of the case, without setting the flag. This patch sets the flag earlier in the case statement. I re-discovered the deep recursion recently during testing; I was told that it was a known problem, and the fix to it was in the kernel I was testing. Indeed it was ... but it didn't fix the bug. With the patch below, I no longer see the bug. Signed-off by: Linas Vepstas <linas@austin.ibm.com> Signed-off-by: Jens Axboe <axboe@suse.de> Cc: Chris Wright <chrisw@sous-sol.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2006-12-21[PATCH] elevator: fixup typo in merge logicJens Axboe
The recent io scheduler allow_merge commit left the block layer with no merging, oops. This patch fixes that up. That means the CFQ change needs to be verified again, it might not fix the original bug now. But that's a seperate thing, I'll double check that tomorrow. Signed-off-by: Jens Axboe <jens.axboe@oracle.com> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-12-20[PATCH] cfq-iosched: don't allow sync merges across queuesJens Axboe
Currently we allow any merge, even if the io originates from different processes. This can cause really bad starvation and unfairness, if those ios happen to be synchronous (reads or direct writes). So add a allow_merge hook to the io scheduler ops, so an io scheduler can help decide whether a bio/process combination may be merged with an existing request. Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2006-12-01[BLOCK] Cleanup unused variable passingJens Axboe
- ->init_queue() does not need the elevator passed in - ->put_request() is a hot path and need not have the queue passed in - cfq_update_io_seektime() does not need cfqd passed in Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2006-10-12[PATCH] block layer: elv_iosched_show should get elv_list_lockVasily Tarasov
elv_iosched_show function iterates other elv_list, hence elv_list_lock should be got. Signed-off-by: Vasily Tarasov <vtaras@openvz.org> Signed-off-by: Vasily Tarasov <jens.axboe@oracle.com>
2006-10-12[PATCH] block layer: elevator_find function cleanupVasily Tarasov
We can easily produce search through the elevator list without introducing additional elevator_type variable. Signed-off-by: Vasily Tarasov <vtaras@openvz.org> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2006-09-30[PATCH] Update axboe@suse.de email addressJens Axboe
As people often look for the copyright in files to see who to mail, update the link to a neutral one. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2006-09-30[PATCH] Make sure all block/io scheduler setups are node awareJens Axboe
Some were kmalloc_node(), some were still kmalloc(). Change them all to kmalloc_node(). Signed-off-by: Jens Axboe <axboe@suse.de>
2006-09-30[PATCH] Drop useless bio passing in may_queue/set_request APIJens Axboe
It's not needed for anything, so kill the bio passing. Signed-off-by: Jens Axboe <axboe@suse.de>
2006-09-30[PATCH] Remove ->waiting member from struct requestJens Axboe
As the comments indicates in blkdev.h, we can fold it into ->end_io_data usage as that is really what ->waiting is. Fixup the users of blk_end_sync_rq(). Signed-off-by: Jens Axboe <axboe@kernel.dk>
2006-09-30[PATCH] elevator: abstract out the rbtree sort handlingJens Axboe
The rbtree sort/lookup/reposition logic is mostly duplicated in cfq/deadline/as, so move it to the elevator core. The io schedulers still provide the actual rb root, as we don't want to impose any sort of specific handling on the schedulers. Introduce the helpers and rb_node in struct request to help migrate the IO schedulers. Signed-off-by: Jens Axboe <axboe@suse.de>
2006-09-30[PATCH] elevator: move the backmerging logic into the elevator coreJens Axboe
Right now, every IO scheduler implements its own backmerging (except for noop, which does no merging). That results in duplicated code for essentially the same operation, which is never a good thing. This patch moves the backmerging out of the io schedulers and into the elevator core. We save 1.6kb of text and as a bonus get backmerging for noop as well. Win-win! Signed-off-by: Jens Axboe <axboe@suse.de>
2006-09-30[PATCH] Split struct request ->flags into two partsJens Axboe
Right now ->flags is a bit of a mess: some are request types, and others are just modifiers. Clean this up by splitting it into ->cmd_type and ->cmd_flags. This allows introduction of generic Linux block message types, useful for sending generic Linux commands to block devices. Signed-off-by: Jens Axboe <axboe@suse.de>
2006-08-22elv_unregister: fix possible crash on module unloadOleg Nesterov
An exiting task or process which didn't do I/O yet have no io context, elv_unregister() should check it is not NULL. Signed-off-by: Oleg Nesterov <oleg@tv-sign.ru> Acked-by: Jens Axboe <axboe@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2006-06-30Remove obsolete #include <linux/config.h>Jörn Engel
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de> Signed-off-by: Adrian Bunk <bunk@stusta.de>
2006-06-23[PATCH] remove dead code from elevator switchingDave Jones
We already drop the refcount in elevator_exit(), and as we're setting 'e' to NULL, we'll never take that branch anyway. Finally, as 'e' is a local var that isn't referenced afterwards, setting it to NULL is pointless. Signed-off-by: Dave Jones <davej@redhat.com> Signed-off-by: Jens Axboe <axboe@suse.de>
2006-06-08[PATCH] elevator switching raceJens Axboe
There's a race between shutting down one io scheduler and firing up the next, in which a new io could enter and cause the io scheduler to be invoked with bad or NULL data. To fix this, we need to maintain the queue lock for a bit longer. Unfortunately we cannot do that, since the elevator init requires to be run without the lock held. This isn't easily fixable, without also changing the mempool API. So split the initialization into two parts, and alloc-init operation and an attach operation. Then we can preallocate the io scheduler and related structures, and run the attach inside the lock after we detach the old one. This patch has survived 30 minutes of 1 second io scheduler switching with a very busy io load. Signed-off-by: Jens Axboe <axboe@suse.de> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-05-11[BLOCK] limit request_fn recursionJens Axboe
Don't recurse back into the driver even if the unplug threshold is met, when the driver asks for a requeue. This is both silly from a logical point of view (requeues typically happen due to driver/hardware shortage), and also dangerous since we could hit an endless request_fn -> requeue -> unplug -> request_fn loop and crash on stack overrun. Also limit blk_run_queue() to one level of recursion, similar to how blk_start_queue() works. This patch fixed a real problem with SLES10 and lpfc, and it could hit any SCSI lld that returns non-zero from it's ->queuecommand() handler. Signed-off-by: Jens Axboe <axboe@suse.de> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-04-20[PATCH] block/elevator.c: remove unused exportsAdrian Bunk
This patch removes the following unused EXPORT_SYMBOL's: - elv_requeue_request - elv_completed_request They are only used by the block core, hence they need not be exported. Signed-off-by: Adrian Bunk <bunk@stusta.de> Signed-off-by: Jens Axboe <axboe@suse.de>
2006-03-31[PATCH] Don't pass boot parameters to argv_init[]OGAWA Hirofumi
The boot cmdline is parsed in parse_early_param() and parse_args(,unknown_bootoption). And __setup() is used in obsolete_checksetup(). start_kernel() -> parse_args() -> unknown_bootoption() -> obsolete_checksetup() If __setup()'s callback (->setup_func()) returns 1 in obsolete_checksetup(), obsolete_checksetup() thinks a parameter was handled. If ->setup_func() returns 0, obsolete_checksetup() tries other ->setup_func(). If all ->setup_func() that matched a parameter returns 0, a parameter is seted to argv_init[]. Then, when runing /sbin/init or init=app, argv_init[] is passed to the app. If the app doesn't ignore those arguments, it will warning and exit. This patch fixes a wrong usage of it, however fixes obvious one only. Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-03-24BUG_ON() Conversion in block/elevator.cEric Sesterhenn
this changes if() BUG(); constructs to BUG_ON() which is cleaner, contains unlikely() and can better optimized away. Signed-off-by: Eric Sesterhenn <snakebyte@gmx.de> Signed-off-by: Adrian Bunk <bunk@stusta.de>
2006-03-23[PATCH] Block queue IO tracing support (blktrace) as of 2006-03-23Jens Axboe
Signed-off-by: Jens Axboe <axboe@suse.de>
2006-03-18[PATCH] fix rmmod problems with elevator attributes, clean them upAl Viro
2006-03-18[PATCH] elevator_t lifetime rules and sysfs fixesAl Viro
2006-03-18[PATCH] stop elv_unregister() from rogering other iosched's data, fix lockingAl Viro
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2006-02-08[PATCH] block: implement elv_insert and use it (fix ordcolor flipping bug)Tejun Heo
q->ordcolor must only be flipped on initial queueing of a hardbarrier request. Constructing ordered sequence and requeueing used to pass through __elv_add_request() which flips q->ordcolor when it sees a barrier request. This patch separates out elv_insert() from __elv_add_request() and uses elv_insert() when constructing ordered sequence and requeueing. elv_insert() inserts the given request at the specified position and does nothing else. Signed-off-by: Tejun Heo <htejun@gmail.com> Acked-by: Jens Axboe <axboe@suse.de> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-02-05[PATCH] block: request_queue->ordcolor must not be flipped on SOFTBARRIERTejun Heo
q->ordcolor must not be flipped on SOFTBARRIER. Signed-off-by: Tejun Heo <htejun@gmail.com> Acked-by: Jens Axboe <axboe@suse.de> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-01-24[BLOCK] elevator: allow default scheduler to potentially be modularNate Diller
Jens has decided that allowing the default scheduler to be a module is a bug, and should not be allowed under kconfig. However, I find that scenario useful for debugging, and wish for the kernel to be able to handle this situation without OOPSing, if I enable such an option in the .config directly. This patch dynamically checks for the presence of the compiled-in default, and falls back to no-op, emitting a suitable error message, when the default is not available Tested for a range of boot options on 2.6.16-rc1-mm2. Signed-off-by: Nate Diller <nate.diller@gmail.com> Signed-off-by: Jens Axboe <axboe@suse.de>
2006-01-24[BLOCK] elevator: default choice selectionNate Diller
My previous default iosched patch did a poor job dealing with the 'elevator=' boot-time option. The old behavior falls back to the compiled-in default if the requested one is not registered at boot time. This patch dynamically evaluates which default to use, and emits a suitable error message when the requested scheduler is not available. It also does the 'as' -> 'anticipatory' conversion before elevator registration, which along with a modified registration function, allows it to correctly indicate which default scheduler is in use. Tested for a range of boot options on 2.6.16-rc1-mm2. Signed-off-by: Nate Diller <nate.diller@gmail.com> Signed-off-by: Jens Axboe <axboe@suse.de>