| Age | Commit message (Collapse) | Author |
|---|
|
After upgrading to the latest kernel on my mpc875 userspace started
running incredibly slow (hours to get to a shell, even!).
I tracked it down to commit 8d30c14cab30d405a05f2aaceda1e9ad57800f36,
that patch removed a work-around for the 8xx. Adding it
back makes my problem go away.
Signed-off-by: Rex Feany <rfeany@mrv.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
|
|
This is an attempt at cleaning up a bit the way we handle execute
permission on powerpc. _PAGE_HWEXEC is gone, _PAGE_EXEC is now only
defined by CPUs that can do something with it, and the myriad of
#ifdef's in the I$/D$ coherency code is reduced to 2 cases that
hopefully should cover everything.
The logic on BookE is a little bit different than what it was though
not by much. Since now, _PAGE_EXEC will be set by the generic code
for executable pages, we need to filter out if they are unclean and
recover it. However, I don't expect the code to be more bloated than
it already was in that area due to that change.
I could boast that this brings proper enforcing of per-page execute
permissions to all BookE and 40x but in fact, we've had that now for
some time as a side effect of my previous rework in that area (and
I didn't even know it :-) We would only enable execute permission if
the page was cache clean and we would only cache clean it if we took
and exec fault. Since we now enforce that the later only work if
VM_EXEC is part of the VMA flags, we de-fact already enforce per-page
execute permissions... Unless I missed something
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
|
|
Since the pte_lockptr is a spinlock it gets optimized away on
uniprocessor builds so using spin_is_locked is not correct. We can use
assert_spin_locked instead and get the proper behavior between UP and
SMP builds.
Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
|
|
The definition for the global structure mmu_gathers, used by generic code,
is currently defined in multiple places not including anything used by
64-bit Book3E. This changes it by moving to one place common to all
processors.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
|
|
That patch used to just add a hook to page table flushing but
pulling that string brought out a whole bunch of issues, so it
now does that and more:
- We now make the RCU batching of page freeing SMP only, as I
believe it was intended initially. We make a few more things compile
to nothing on !CONFIG_SMP
- Some macros are turned into functions, though that forced me to
out of line a few stuffs due to unsolvable include depenencies,
however it's probably better that way anyway, it's not -that-
critical code path.
- 32-bit didn't call pte_free_finish() on tlb_flush() which means
that it wouldn't push out the batch to RCU for delayed freeing when
a bunch of page tables have been freed, they would just stay in there
until the batch gets full.
64-bit BookE will use that hook to maintain the virtually linear
page tables or the indirect entries in the TLB when using the
HW loader.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
|
|
pr_debug() can now result in code being generated even when DEBUG
is not defined. That's not really desirable in some places.
With CONFIG_DYNAMIC_DEBUG=y:
size before:
text data bss dec hex filename
2036 368 8 2412 96c arch/powerpc/mm/pgtable.o
size after:
text data bss dec hex filename
1677 248 8 1933 78d arch/powerpc/mm/pgtable.o
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
|
|
With CONFIG_DEBUG_VM, an assertion is made when changing the protection
flags of a PTE that the PTE is locked. Huge pages use a different pagetable
format and the assertion is bogus and will always trigger with a bug looking
something like
Unable to handle kernel paging request for data at address 0xf1a00235800006f8
Faulting instruction address: 0xc000000000034a80
Oops: Kernel access of bad area, sig: 11 [#1]
SMP NR_CPUS=32 NUMA Maple
Modules linked in: dm_snapshot dm_mirror dm_region_hash
dm_log dm_mod loop evdev ext3 jbd mbcache sg sd_mod ide_pci_generic
pata_amd ata_generic ipr libata tg3 libphy scsi_mod windfarm_pid
windfarm_smu_sat windfarm_max6690_sensor windfarm_lm75_sensor
windfarm_cpufreq_clamp windfarm_core i2c_powermac
NIP: c000000000034a80 LR: c000000000034b18 CTR: 0000000000000003
REGS: c000000003037600 TRAP: 0300 Not tainted (2.6.30-rc3-autokern1)
MSR: 9000000000009032 <EE,ME,IR,DR> CR: 28002484 XER: 200fffff
DAR: f1a00235800006f8, DSISR: 0000000040010000
TASK = c0000002e54cc740[2960] 'map_high_trunca' THREAD: c000000003034000 CPU: 2
GPR00: 4000000000000000 c000000003037880 c000000000895d30 c0000002e5a2e500
GPR04: 00000000a0000000 c0000002edc40880 0000005700000393 0000000000000001
GPR08: f000000011ac0000 01a00235800006e8 00000000000000f5 f1a00235800006e8
GPR12: 0000000028000484 c0000000008dd780 0000000000001000 0000000000000000
GPR16: fffffffffffff000 0000000000000000 00000000a0000000 c000000003037a20
GPR20: c0000002e5f4ece8 0000000000001000 c0000002edc40880 0000000000000000
GPR24: c0000002e5f4ece8 0000000000000000 00000000a0000000 c0000002e5f4ece8
GPR28: 0000005700000393 c0000002e5a2e500 00000000a0000000 c000000003037880
NIP [c000000000034a80] .assert_pte_locked+0xa4/0xd0
LR [c000000000034b18] .ptep_set_access_flags+0x6c/0xb4
Call Trace:
[c000000003037880] [c000000003037990] 0xc000000003037990 (unreliable)
[c000000003037910] [c000000000034b18] .ptep_set_access_flags+0x6c/0xb4
[c0000000030379b0] [c00000000014bef8] .hugetlb_cow+0x124/0x674
[c000000003037b00] [c00000000014c930] .hugetlb_fault+0x4e8/0x6f8
[c000000003037c00] [c00000000013443c] .handle_mm_fault+0xac/0x828
[c000000003037cf0] [c0000000000340a8] .do_page_fault+0x39c/0x584
[c000000003037e30] [c0000000000057b0] handle_page_fault+0x20/0x5c
Instruction dump:
7d29582a 7d200074 7800d182 0b000000 3c004000 3960ffff 780007c6 796b00c4
7d290214 7929a302 1d290068 7d6b4a14 <800b0010> 7c000074 7800d182 0b000000
This patch fixes the problem by not asseting the PTE is locked for VMAs
backed by huge pages.
Signed-off-by: Mel Gorman <mel@csn.ul.ie>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
|
|
Makes code futureproof against the impending change to mm->cpu_vm_mask.
It's also a chance to use the new cpumask_ ops which take a pointer
(the older ones are deprecated, but there's no hurry for arch code).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
|
|
This patch reworks the way we do I and D cache coherency on PowerPC.
The "old" way was split in 3 different parts depending on the processor type:
- Hash with per-page exec support (64-bit and >= POWER4 only) does it
at hashing time, by preventing exec on unclean pages and cleaning pages
on exec faults.
- Everything without per-page exec support (32-bit hash, 8xx, and
64-bit < POWER4) does it for all page going to user space in update_mmu_cache().
- Embedded with per-page exec support does it from do_page_fault() on
exec faults, in a way similar to what the hash code does.
That leads to confusion, and bugs. For example, the method using update_mmu_cache()
is racy on SMP where another processor can see the new PTE and hash it in before
we have cleaned the cache, and then blow trying to execute. This is hard to hit but
I think it has bitten us in the past.
Also, it's inefficient for embedded where we always end up having to do at least
one more page fault.
This reworks the whole thing by moving the cache sync into two main call sites,
though we keep different behaviours depending on the HW capability. The call
sites are set_pte_at() which is now made out of line, and ptep_set_access_flags()
which joins the former in pgtable.c
The base idea for Embedded with per-page exec support, is that we now do the
flush at set_pte_at() time when coming from an exec fault, which allows us
to avoid the double fault problem completely (we can even improve the situation
more by implementing TLB preload in update_mmu_cache() but that's for later).
If for some reason we didn't do it there and we try to execute, we'll hit
the page fault, which will do a minor fault, which will hit ptep_set_access_flags()
to do things like update _PAGE_ACCESSED or _PAGE_DIRTY if needed, we just make
this guys also perform the I/D cache sync for exec faults now. This second path
is the catch all for things that weren't cleaned at set_pte_at() time.
For cpus without per-pag exec support, we always do the sync at set_pte_at(),
thus guaranteeing that when the PTE is visible to other processors, the cache
is clean.
For the 64-bit hash with per-page exec support case, we keep the old mechanism
for now. I'll look into changing it later, once I've reworked a bit how we
use _PAGE_EXEC.
This is also a first step for adding _PAGE_EXEC support for embedded platforms
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
|
|
Refactor the RCU based pte free code that was used on ppc64 to be used
on all powerpc.
Additionally refactor pte_free() & pte_free_kernel() into common code
between ppc32 & ppc64.
Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
|
