aboutsummaryrefslogtreecommitdiff
path: root/drivers/infiniband/hw/mthca/mthca_mr.c
AgeCommit message (Collapse)Author
2008-07-22IB/mthca: Keep free count for MTT buddy allocatorRoland Dreier
MTT entries are allocated with a buddy allocator, which just keeps bitmaps for each level of the buddy table. However, all free space starts out at the highest order, and small allocations start scanning from the lowest order. When the lowest order tables have no free space, this can lead to scanning potentially millions of bits before finding a free entry at a higher order. We can avoid this by just keeping a count of how many free entries each order has, and skipping the bitmap scan when an order is completely empty. This provides a nice performance boost for a negligible increase in memory usage. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2008-07-14RDMA: Remove subversion $Id tagsRoland Dreier
They don't get updated by git and so they're worse than useless. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2008-04-29IB/mthca: Avoid recycling old FMR R_Keys too soonOlaf Kirch
When a FMR is unmapped, mthca resets the map count to 0, and clears the upper part of the R_Key which is used as the sequence counter. This poses a problem for RDS, which uses ib_fmr_unmap as a fence operation. RDS assumes that after issuing an unmap, the old R_Keys will be invalid for a "reasonable" period of time. For instance, Oracle processes uses shared memory buffers allocated from a pool of buffers. When a process dies, we want to reclaim these buffers -- but we must make sure there are no pending RDMA operations to/from those buffers. The only way to achieve that is by using unmap and sync the TPT. However, when the sequence count is reset on unmap, there is a high likelihood that a new mapping will be given the same R_Key that was issued a few milliseconds ago. To prevent this, don't reset the sequence count when unmapping a FMR. Signed-off-by: Olaf Kirch <olaf.kirch@oracle.com> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2008-02-19IB/mthca: Free correct MPT on error exit from mthca_fmr_alloc()Roland Dreier
When mthca_fmr_alloc() returns an error, it should free the MPT at the index key, not mr->ibmr.lkey, since the lkey has been mangled by hw_index_to_key() and no longer is the real index. This bug causes corruption of the MPT table free bitmap when mthca_fmr_alloc() fails. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2008-02-04IB/mthca: Return proper error codes from mthca_fmr_alloc()Olaf Kirch
If the allocation of the MTT or the mailbox failed, mthca_fmr_alloc() would return 0 (success) no matter what. This leads to crashes a little down the road, when we try to dereference eg mr->mtt, which was really ERR_PTR(-Ewhatever). Signed-off-by: Olaf Kirch <olaf.kirch@oracle.com> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-04-24IB/mthca: Fix mthca_write_mtt() on HCAs with hidden memoryRoland Dreier
Commit b2875d4c ("IB/mthca: Always fill MTTs from CPU") causes a crash in mthca_write_mtt() with non-memfree HCAs that have their memory hidden (that is, have only two PCI BARs instead of having a third BAR that allows access to the RAM attached to the HCA) on 64-bit architectures. This is because the commit just before, c20e20ab ("IB/mthca: Merge MR and FMR space on 64-bit systems") makes dev->mr_table.fmr_mtt_buddy equal to &dev->mr_table.mtt_buddy and hence mthca_write_mtt() tries to write directly into the HCA's MTT table. However, since that table is in the HCA's memory, this is impossible without the PCI BAR that gives access to that memory. This causes a crash because mthca_tavor_write_mtt_seg() basically tries to dereference some offset of a NULL pointer. Fix this by adding a test of MTHCA_FLAG_FMR in mthca_write_mtt() so that we always use the WRITE_MTT firmware command rather than writing directly if FMRs are not enabled. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-04-16IB/mthca: Fix data corruption after FMR unmap on SinaiMichael S. Tsirkin
In mthca_arbel_fmr_unmap(), the high bits of the key are masked off. This gets rid of the effect of adjust_key(), which makes sure that bits 3 and 23 of the key are equal when the Sinai throughput optimization is enabled, and so it may happen that an FMR will end up with bits 3 and 23 in the key being different. This causes data corruption, because when enabling the throughput optimization, the driver promises the HCA firmware that bits 3 and 23 of all memory keys will always be equal. Fix by re-applying adjust_key() after masking the key. Thanks to Or Gerlitz for reproducing the problem, and Ariel Shahar for help in debug. Signed-off-by: Michael S. Tsirkin <mst@dev.mellanox.co.il> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-03-26IB/mthca: Fix thinko in init_mr_table()Michael S. Tsirkin
Commit c20e20ab ("IB/mthca: Merge MR and FMR space on 64-bit systems") swapped the number of MTTs and MPTs when initializing the MR table. As a result, we get a kernel oops when the number of MTT segments allocated exceeds 0x20000. Noted by Troy Benjegerdes <troy@scl.ameslab.gov>, and reproduced by Dotan Barak <dotanb@mellanox.co.il>. This fixes https://bugs.openfabrics.org/show_bug.cgi?id=490 Signed-off-by: Michael S. Tsirkin <mst@dev.mellanox.co.il> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-02-20IB/mthca: Make 2 functions staticAdrian Bunk
This patch makes the needlessly global functions mthca_tavor_write_mtt_seg() and mthca_arbel_write_mtt_seg() static. Signed-off-by: Adrian Bunk <bunk@stusta.de> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-02-12IB/mthca: Always fill MTTs from CPUMichael S. Tsirkin
Speed up memory registration by filling in MTTs directly when the CPU can write directly to the whole table (all mem-free cards, and to Tavor mode on 64-bit systems with the patch I posted earlier). This reduces the number of FW commands needed to register an MR by at least a factor of 2 and speeds up memory registration significantly. Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-02-12IB/mthca: Merge MR and FMR space on 64-bit systemsMichael S. Tsirkin
For Tavor, we currently reserve separate MPT and MTT space for FMRs to avoid abusing the vmalloc space on 32 bit kernels. No such problem exists on 64 bit kernels so let's not do it there. This way we have a shared pool for MR and FMR resources, used on demand. This will also make it possible to write MTTs for regular regions directly from driver. Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-02-12IB/mthca: Fix access to MTT and MPT tables on non-cache-coherent CPUsMichael S. Tsirkin
We allocate the MTT table with alloc_pages() and then do pci_map_sg(), so we must call pci_dma_sync_sg() after the CPU writes to the MTT table. This works since the device will never write MTTs on mem-free HCAs, once we get rid of the use of the WRITE_MTT firmware command. This change is needed to make that work, and is an improvement for now, since it gives FMRs a chance at working. For MPTs, both the device and CPU might write there, so we must allocate DMA coherent memory for these. Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-11-29IB/mthca: Fix section mismatchesRoland Dreier
Commit b3b30f5e ("IB/mthca: Recover from catastrophic errors") introduced some section mismatch breakage, because the error recovery code tears down and reinitializes the device, which calls into lots of code originally marked __devinit and __devexit from regular .text. Fix this by getting rid of these now-incorrect section markers. Reported by Randy Dunlap <randy.dunlap@oracle.com>. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-05-10IB/mthca: FMR ioremap fixMichael S. Tsirkin
Addresses for ioremap must be calculated off of pci_resource_start; we can't directly use the bus address as seen by the HCA. Fix the code that remaps device memory for FMR access. Based on patch by Klaus Smolin. Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-29IB/mthca: Fix section mismatch problemsRoland Dreier
Quite a few cleanup functions in mthca were marked as __devexit. However, they could also be called from error paths during initialization, so they cannot be marked that way. Just delete all of the incorrect annotations. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20IB/mthca: Optimize large messages on Sinai HCAsEli Cohen
Sinai (one-port PCI Express) HCAs get improved throughput for messages bigger than 80 KB in DDR mode if memory keys are formatted in a specific way. The enhancement only works if the memory key table is smaller than 2^24 entries. For larger tables, the enhancement is off and a warning is printed (to avoid silent performance loss). Signed-off-by: Eli Cohen <eli@mellanox.co.il> Signed-off-by: Michael Tsirkin <mst@mellanox.co.il> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20IB: Enable FMR pool user to set page sizeOr Gerlitz
This patch allows the consumer to set the page size of "pages" mapped by the pool FMRs, which is a feature already existing in the base verbs API. On the cosmetic side it changes ib_fmr_attr.page_size field to be named page_shift. Signed-off-by: Or Gerlitz <ogerlitz@voltaire.com> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20IB/mthca: Whitespace cleanupsRoland Dreier
Remove trailing whitespace and fix indentation that with spaces instead of tabs. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20IB/mthca: Get rid of might_sleep() annotationsRoland Dreier
The might_sleep() annotations in mthca are silly -- they all occur shortly before calls that will end up in core functions like kmalloc() that will print the same warning in an unsafe context anyway. In fact, beyond cluttering the source, we're actually bloating text with CONFIG_DEBUG_SPINLOCK_SLEEP and/or CONFIG_PREEMPT_VOLUNTARY set. With both options set, getting rid of the might_sleep()s saves a lot: add/remove: 0/0 grow/shrink: 0/7 up/down: 0/-171 (-171) function old new delta mthca_pd_alloc 132 109 -23 mthca_init_cq 969 946 -23 mthca_mr_alloc 592 568 -24 mthca_pd_free 67 42 -25 mthca_free_mr 219 194 -25 mthca_free_cq 570 545 -25 mthca_fmr_alloc 742 716 -26 Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-11-02[IB] kzalloc() conversionsRoland Dreier
Replace kmalloc()+memset(,0,) with kzalloc(), for a net savings of 35 source lines and about 500 bytes of text. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-08-26[PATCH] IB/mthca: use generic function instead of arbel_ version in ↵Guy German
mthca_free_region() Use the generic key_to_hw_index() function instead of the Arbel-specific version in mthca_free_region(). Signed-off-by: Guy German <guyg@voltaire.com> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-08-26[PATCH] IB: sparse endianness cleanupSean Hefty
Fix sparse warnings. Use __be* where appropriate. Signed-off-by: Sean Hefty <sean.hefty@intel.com> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-08-26[PATCH] IB: Add copyright noticesRoland Dreier
Make some lawyers happy and add copyright notices for people who forgot to include them when they actually touched the code. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-06-27[PATCH] IB/mthca: Align FW command mailboxes to 4KRoland Dreier
Future versions of Mellanox HCA firmware will require command mailboxes to be aligned to 4K. Support this by using a pci_pool to allocate all mailboxes. This has the added benefit of shrinking the source and text of mthca. Signed-off-by: Roland Dreier <roland@topspin.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-06-27[PATCH] IB/mthca: Split off MTT allocationRoland Dreier
Split allocation of MTT range from creation of MR. This will be useful for implementing shared memory regions and userspace verbs. Signed-off-by: Roland Dreier <roland@topspin.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-06-27[PATCH] IB/mthca: Move mthca_is_memfree checksRoland Dreier
Make mthca_table_put() and mthca_table_put_range() NOPs if the device is not mem-free, so that we don't have to have "if (mthca_is_memfree())" tests in the callers of these functions. This makes our code more readable and maintainable, and saves a couple dozen bytes of text in ib_mthca.ko as well. Signed-off-by: Roland Dreier <roland@topspin.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-04-16[PATCH] IB/mthca: encapsulate mem-free check into mthca_is_memfree()Roland Dreier
Clean up mem-free mode support by introducing mthca_is_memfree() function, which encapsulates the logic of deciding if a device is mem-free. Signed-off-by: Roland Dreier <roland@topspin.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-04-16[PATCH] IB/mthca: add fast memory region implementationMichael S. Tsirkin
Implement fast memory regions (FMRs), where the driver writes directly into the HCA's translation tables rather than requiring a firmware command. For Tavor, MTTs for FMR are separate from regular MTTs, and are reserved at driver initialization. This is done to limit the amount of virtual memory needed to map the MTTs. For Arbel, there's no such limitation, and all MTTs and MPTs may be used for FMR or for regular MR. Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il> Signed-off-by: Roland Dreier <roland@topspin.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-04-16[PATCH] IB/mthca: split MR key munging routinesMichael S. Tsirkin
Split Tavor and Arbel/mem-free index<->hw key munging routines, so that FMR implementation can call correct implementation without testing HCA type (which it already knows). Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il> Signed-off-by: Roland Dreier <roland@topspin.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-04-16[PATCH] IB/mthca: encapsulate MTT buddy allocatorMichael S. Tsirkin
Encapsulate the buddy allocator used for MTT segments. This cleans up the code and also gets us ready to add FMR support. Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il> Signed-off-by: Roland Dreier <roland@topspin.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-04-16[PATCH] IB/mthca: fix MTT allocation in mem-free modeRoland Dreier
Fix bug in MTT allocation in mem-free mode. I misunderstood the MTT size value returned by the firmware -- it is really the size of a single MTT entry, since mem-free mode does not segment the MTT as the original firmware did. This meant that our MTT addresses ended up being off by a factor of 8. This meant that our MTT allocations might overlap, and so we could overwrite and corrupt earlier memory regions when writing new MTT entries. We fix this by always using our 64-byte MTT segment size. This allows some simplification of the code as well, since there's no reason to put the MTT segment size in a variable -- we can always use our enum value directly. Signed-off-by: Roland Dreier <roland@topspin.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-04-16[PATCH] IB/mthca: fix MR allocation error pathMichael S. Tsirkin
Fix error handling in MR allocation for mem-free mode: mthca_free must get an MR index, not a key. Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il> Signed-off-by: Roland Dreier <roland@topspin.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-04-16[PATCH] IB/mthca: map MPT/MTT context in mem-free modeRoland Dreier
In mem-free mode, when allocating memory regions, make sure that the HCA has context memory mapped to cover the virtual space used for the MPT and MTTs being used. Signed-off-by: Roland Dreier <roland@topspin.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-04-16Linux-2.6.12-rc2Linus Torvalds
Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!