aboutsummaryrefslogtreecommitdiff
path: root/drivers/dma/ioat_dma.c
AgeCommit message (Collapse)Author
2009-05-28i7300_idle: allow testing on i5000-series hardware w/o re-compileLen Brown
Testing the i7300_idle driver on i5000-series hardware required an edit to i7300_idle.h to "#define SUPPORT_I5000 1" and a re-build of both i7300_idle and ioat_dma. Replace that build-time scheme with a load-time module parameter: "7300_idle.forceload=1" to make it easier to test the driver on hardware that while not officially validated, works fine and is much more commonly available. By default (no modparam) the driver will continue to load only on the i7300. Note that ioat_dma runs a copy of i7300_idle's probe routine to know to reserve an IOAT channel for i7300_idle. This change makes ioat_dma do that always on the i5000, just like it does on the i7300. Signed-off-by: Len Brown <len.brown@intel.com> Acked-by: Andrew Henroid <andrew.d.henroid@intel.com>
2009-05-12ioatdma: fix "ioatdma frees DMA memory with wrong function"Maciej Sosnowski
as reported by Alexander Beregalov <a.beregalov@gmail.com> ioatdma 0000:00:08.0: DMA-API: device driver frees DMA memory with wrong function [device address=0x000000007f76f800] [size=2000 bytes] [map ped as single] [unmapped as page] The ioatdma driver was unmapping all regions (either allocated as page or single) using unmap_page. This patch lets dma driver recognize if unmap_single or unmap_page should be used. It introduces two new dma control flags: DMA_COMPL_SRC_UNMAP_SINGLE and DMA_COMPL_DEST_UNMAP_SINGLE. They should be set to indicate dma driver to do dma-unmapping as single (first one for the source, tha latter for the destination). If respective flag is not set, the driver assumes dma-unmapping as page. Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Reported-by: Alexander Beregalov <a.beregalov@gmail.com> Tested-by: Alexander Beregalov <a.beregalov@gmail.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-03-25dmaengine: initialize tx_list in dma_async_tx_descriptor_initDan Williams
Centralize this common initialization (and one case where ipu_idmac is duplicating ->chan initialization). Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-03-04I/OAT: fail self-test if callback test reaches timeoutDan Williams
If we miss interrupts in the self test then fail registration of this channel as it is unsuitable for use as a public channel. Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-03-04I/OAT: update driver version and copyright datesMaciej Sosnowski
Together with new fixes update driver version and extend copyright dates ranges. Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Acked-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-03-04I/OAT: list usage cleanupEric Sesterhenn
Trivial cleanup, list_del(); list_add_tail() is equivalent to list_move_tail(). Semantic patch for coccinelle can be found at www.cccmz.de/~snakebyte/list_move_tail.spatch Signed-off-by: Eric Sesterhenn <snakebyte@gmx.de> Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Acked-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-03-04I/OAT: cancel watchdog before dma removeMaciej Sosnowski
Channel watchdog should be canceled before the rest of dma remove stuff. Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Acked-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-03-04I/OAT: fail initialization on zero channels detectionMaciej Sosnowski
On some systems with I/OAT ver.2 when DCA is disabled in BIOS situations have been observed that zero DMA channels are detected instead of four. To avoid kernel panic driver should fail gracefully with appropriate message. Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Acked-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-03-04I/OAT: do not set DCACTRL_CMPL_WRITE_ENABLE for I/OAT ver.3Maciej Sosnowski
Flag DCACTRL_CMPL_WRITE_ENABLE is valid only for I/OAT ver.2 so it should not be set for I/OAT ver.3. Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Acked-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-01-06ioat: fix self test for multi-channel caseDan Williams
In the multiple device case we need to re-arm the completion and protect against concurrent self-tests. The printk from the test callback is removed as it can arbitrarily delay completion of the test. Cc: <stable@kernel.org> Cc: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2009-01-06dmaengine: kill struct dma_client and supporting infrastructureDan Williams
All users have been converted to either the general-purpose allocator, dma_find_channel, or dma_request_channel. Reviewed-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2008-12-03ioat: wait for self-test completionDan Williams
As part of the ioat_dma self-test it performs a printk from a completion callback. Depending on the system console configuration this output can take longer than a millisecond causing the self-test to fail. Introduce a completion with a generous timeout to mitigate this failure. Cc: <stable@kernel.org> Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2008-11-10[3/4] I/OAT: fix async_tx.callback checkingMaciej Sosnowski
async_tx.callback should be checked for the first not the last descriptor in the chain. Cc: <stable@kernel.org> Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2008-11-10[1/4] I/OAT: fix channel resources free for not allocated channelsMaciej Sosnowski
If the ioatdma driver is loaded but not used it does not allocate descriptors. Before it frees channel resources it should first be sure that they have been previously allocated. Cc: <stable@kernel.org> Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Tested-by: Tom Picard <tom.s.picard@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2008-10-25Merge branch 'i7300_idle' into releaseLen Brown
2008-10-24i7300_idle: Fix compile warning CONFIG_I7300_IDLE_IOAT_CHANNEL not definedVenki Pallipadi
When I7300_idle driver is not configured, there is a compile time warning about IDLE_IOAT_CHANNEL not defined. Fix it. Reported-by: Suresh Siddha <suresh.b.siddha@intel.com> Reported-by: Arjan van de Ven <arjan@linux.intel.com> Signed-off-by: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com> Signed-off-by: Len Brown <len.brown@intel.com>
2008-10-24i7300_idle: Disable ioat channel only on platforms where ile driver can loadVenki Pallipadi
Based on input from Andi Kleen: share the platform detection code with ioat_dma and disable the channel in dma engine only for specific platforms. Signed-off-by: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com> Signed-off-by: Len Brown <len.brown@intel.com>
2008-10-23Merge branch 'linus' into testLen Brown
Conflicts: MAINTAINERS arch/x86/kernel/acpi/boot.c arch/x86/kernel/acpi/sleep.c drivers/acpi/Kconfig drivers/pnp/Makefile drivers/pnp/quirks.c Signed-off-by: Len Brown <len.brown@intel.com>
2008-10-21i7300_idle driver v1.55Andy Henroid
The Intel 7300 Memory Controller supports dynamic throttling of memory which can be used to save power when system is idle. This driver does the memory throttling when all CPUs are idle on such a system. Refer to "Intel 7300 Memory Controller Hub (MCH)" datasheet for the config space description. Signed-off-by: Andy Henroid <andrew.d.henroid@intel.com> Signed-off-by: Len Brown <len.brown@intel.com> Signed-off-by: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com>
2008-09-13drivers/dma/ioat_dma.c: drop code after returnJulia Lawall
The break after the return serves no purpose. Signed-off-by: Julia Lawall <julia@diku.dk> Reviewed-by: Richard Genoud <richard.genoud@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2008-08-07list.h: Add list_splice_tail() and list_splice_tail_init()Luis R. Rodriguez
If you are using linked lists for queues list_splice() will not do what you would expect even if you use the elements passed reversed. We need to handle these differently. We add list_splice_tail() and list_splice_tail_init(). Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> Signed-off-by: Luis R. Rodriguez <lrodriguez@atheros.com> Signed-off-by: John W. Linville <linville@tuxdriver.com>
2008-07-22I/OAT: I/OAT version 3.0 supportMaciej Sosnowski
This patch adds to ioatdma and dca modules support for Intel I/OAT DMA engine ver.3 (aka CB3 device). The main features of I/OAT ver.3 are: * 8 single channel DMA devices (8 channels total) * 8 DCA providers, each can accept 2 requesters * 8-bit TAG values and 32-bit extended APIC IDs Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2008-07-22I/OAT: tcp_dma_copybreak default value dependent on I/OAT versionMaciej Sosnowski
I/OAT DMA performance tuning showed different optimal values of tcp_dma_copybreak for different I/OAT versions (4096 for 1.2 and 2048 for 2.0). This patch lets ioatdma driver set tcp_dma_copybreak value according to these results. [dan.j.williams@intel.com: remove some ifdefs] Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2008-07-22I/OAT: Add watchdog/reset functionality to ioatdmaMaciej Sosnowski
Due to occasional DMA channel hangs observed for I/OAT versions 1.2 and 2.0 a watchdog has been introduced to check every 2 seconds if all channels progress normally. If stuck channel is detected, driver resets it. The reset is done in two parts. The second part is scheduled by the first one to reinitialize the channel after the restart. Signed-off-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2008-07-08dmaengine: add DMA_COMPL_SKIP_{SRC,DEST}_UNMAP flags to control dma unmapDan Williams
In some cases client code may need the dma-driver to skip the unmap of source and/or destination buffers. Setting these flags indicates to the driver to skip the unmap step. In this regard async_xor is currently broken in that it allows the destination buffer to be unmapped while an operation is still in progress, i.e. when the number of sources exceeds the hardware channel's maximum (fixed in a subsequent patch). Acked-by: Saeed Bishara <saeed@marvell.com> Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Acked-by: Haavard Skinnemoen <haavard.skinnemoen@atmel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2008-07-08dmaengine: Add dma_client parameter to device_alloc_chan_resourcesHaavard Skinnemoen
A DMA controller capable of doing slave transfers may need to know a few things about the slave when preparing the channel. We don't want to add this information to struct dma_channel since the channel hasn't yet been bound to a client at this point. Instead, pass a reference to the client requesting the channel to the driver's device_alloc_chan_resources hook so that it can pick the necessary information from the dma_client struct by itself. [dan.j.williams@intel.com: fixed up fsldma and mv_xor] Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Haavard Skinnemoen <haavard.skinnemoen@atmel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2008-04-17dmaengine: ack to flags: make use of the unused bits in the 'ack' fieldDan Williams
'ack' is currently a simple integer that flags whether or not a client is done touching fields in the given descriptor. It is effectively just a single bit of information. Converting this to a flags parameter allows the other bits to be put to use to control completion actions, like dma-unmap, and capture results, like xor-zero-sum == 0. Changes are one of: 1/ convert all open-coded ->ack manipulations to use async_tx_ack and async_tx_test_ack. 2/ set the ack bit at prep time where possible 3/ make drivers store the flags at prep time 4/ add flags to the device_prep_dma_interrupt prototype Acked-by: Maciej Sosnowski <maciej.sosnowski@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2008-04-17async_tx: kill ->device_dependency_addedDan Williams
DMA drivers no longer need to be notified of dependency submission events as async_tx_run_dependencies and async_tx_channel_switch will handle the scheduling and execution of dependent operations. [sfr@canb.auug.org.au: extend this for fsldma] Acked-by: Shannon Nelson <shannon.nelson@intel.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2008-03-04ioat: fix 'ack' handling, driver must ensure that 'ack' is zeroDan Williams
Initialize 'ack' to zero in case the descriptor has been recycled. Prevents "kernel BUG at crypto/async_tx/async_xor.c:185!" Signed-off-by: Dan Williams <dan.j.williams@intel.com> Acked-by: Shannon Nelson <shannon.nelson@intel.com> Cc: stable@kernel.org
2008-02-06async_tx: replace 'int_en' with operation preparation flagsDan Williams
Pass a full set of flags to drivers' per-operation 'prep' routines. Currently the only flag passed is DMA_PREP_INTERRUPT. The expectation is that arch-specific async_tx_find_channel() implementations can exploit this capability to find the best channel for an operation. Signed-off-by: Dan Williams <dan.j.williams@intel.com> Acked-by: Shannon Nelson <shannon.nelson@intel.com> Reviewed-by: Haavard Skinnemoen <hskinnemoen@atmel.com>
2008-02-06async_tx: kill tx_set_src and tx_set_dest methodsDan Williams
The tx_set_src and tx_set_dest methods were originally implemented to allow an array of addresses to be passed down from async_xor to the dmaengine driver while minimizing stack overhead. Removing these methods allows drivers to have all transaction parameters available at 'prep' time, saves two function pointers in struct dma_async_tx_descriptor, and reduces the number of indirect branches.. A consequence of moving this data to the 'prep' routine is that multi-source routines like async_xor need temporary storage to convert an array of linear addresses into an array of dma addresses. In order to keep the same stack footprint of the previous implementation the input array is reused as storage for the dma addresses. This requires that sizeof(dma_addr_t) be less than or equal to sizeof(void *). As a consequence CONFIG_DMADEVICES now depends on !CONFIG_HIGHMEM64G. It also requires that drivers be able to make descriptor resources available when the 'prep' routine is polled. Signed-off-by: Dan Williams <dan.j.williams@intel.com> Acked-by: Shannon Nelson <shannon.nelson@intel.com>
2007-12-17I/OAT: fix null device in call to dev_err()Shannon Nelson
We can't use the device in a dev_err() after a kzalloc failure or after the kfree, so simplify it to the pdev that was originally passed in. Cc: Eric Sesterhenn <snakebyte@gmx.de> Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-12-17I/OAT: fixups from code commentsShannon Nelson
A few fixups from Andrew's code comments. - removed "static inline" forward-declares - changed use of min() to min_t() - removed some unnecessary NULL initializations - removed a couple of BUG() calls Fixes this: drivers/dma/ioat_dma.c: In function `ioat1_tx_submit': drivers/dma/ioat_dma.c:177: sorry, unimplemented: inlining failed in call to '__ioat1_dma_memcpy_issue_pending': function body not available drivers/dma/ioat_dma.c:268: sorry, unimplemented: called from here Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Cc: "Williams, Dan J" <dan.j.williams@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-11-14I/OAT: Add support for version 2 of ioatdma deviceShannon Nelson
Add support for version 2 of the ioatdma device. This device handles the descriptor chain and DCA services slightly differently: - Instead of moving the dma descriptors between a busy and an idle chain, this new version uses a single circular chain so that we don't have rewrite the next_descriptor pointers as we add new requests, and the device doesn't need to re-read the last descriptor. - The new device has the DCA tags defined internally instead of needing them defined statically. Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Cc: "Williams, Dan J" <dan.j.williams@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-18I/OAT: Add completion callback for async_tx interface useShannon Nelson
The async_tx interface includes a completion callback. This adds support for using that callback, including using interrupts on completion. [akpm@linux-foundation.org: various fixes] Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Cc: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-18I/OAT: Tighten descriptor setup performanceShannon Nelson
The change to the async_tx interface cost this driver some performance by spreading the descriptor setup across several functions, including multiple passes over the new descriptor chain. Here we bring the work back into one primary function and only do one pass. [akpm@linux-foundation.org: cleanups, uninline] Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Cc: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-18I/OAT: clean up error handling and some print messagesShannon Nelson
Make better use of dev_err(), and catch an error where the transaction creation might fail. Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Cc: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-18I/OAT: clean up of dca provider start and stopShannon Nelson
Don't start ioat_dca if ioat_dma didn't start, and then stop ioat_dca before stopping ioat_dma. Since the ioat_dma side does the pci device work, This takes care of ioat_dca trying to use a bad device reference. Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Cc: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-18I/OAT: cleanup pci issuesShannon Nelson
Reorder the pci release actions Letting go of the resources in the right order helps get rid of occasional kernel complaints. Fix the pci_driver object name [Randy Dunlap] Rename the struct pci_driver data so that false section mismatch warnings won't be produced. Cc: Randy Dunlap <randy.dunlap@oracle.com> Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Cc: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-16I/OAT: Add support for MSI and MSI-XShannon Nelson
Add support for MSI and MSI-X interrupt handling, including the ability to choose the desired interrupt method. Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Acked-by: David S. Miller <davem@davemloft.net> [bunk@kernel.org: drivers/dma/ioat_dma.c: make 3 functions static] Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-16I/OAT: Split PCI startup from DMA handling codeShannon Nelson
Split the general PCI startup from the DMA handling code in order to prepare for adding support for DCA services and future versions of the ioatdma device. [Rusty Russell] Removal of __unsafe() usage. Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Acked-by: David S. Miller <davem@davemloft.net> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-16I/OAT: code cleanup from checkpatch outputShannon Nelson
Take care of a bunch of little code nits in ioatdma files Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Acked-by: David S. Miller <davem@davemloft.net> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-16I/OAT: Rename the source fileShannon Nelson
Rename the ioatdma.c file in preparation for splitting into multiple files, which will allow for easier adding new functionality. Signed-off-by: Shannon Nelson <shannon.nelson@intel.com> Acked-by: David S. Miller <davem@davemloft.net> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>