aboutsummaryrefslogtreecommitdiffstats
path: root/arch/arm/mm/flush.c
Commit message (Collapse)AuthorAge
* ARM: 6007/1: fix highmem with VIPT cache and DMANicolas Pitre2010-04-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The VIVT cache of a highmem page is always flushed before the page is unmapped. This cache flush is explicit through flush_cache_kmaps() in flush_all_zero_pkmaps(), or through __cpuc_flush_dcache_area() in kunmap_atomic(). There is also an implicit flush of those highmem pages that were part of a process that just terminated making those pages free as the whole VIVT cache has to be flushed on every task switch. Hence unmapped highmem pages need no cache maintenance in that case. However unmapped pages may still be cached with a VIPT cache because the cache is tagged with physical addresses. There is no need for a whole cache flush during task switching for that reason, and despite the explicit cache flushes in flush_all_zero_pkmaps() and kunmap_atomic(), some highmem pages that were mapped in user space end up still cached even when they become unmapped. So, we do have to perform cache maintenance on those unmapped highmem pages in the context of DMA when using a VIPT cache. Unfortunately, it is not possible to perform that cache maintenance using physical addresses as all the L1 cache maintenance coprocessor functions accept virtual addresses only. Therefore we have no choice but to set up a temporary virtual mapping for that purpose. And of course the explicit cache flushing when unmapping a highmem page on a system with a VIPT cache now can go, which should increase performance. While at it, because the code in __flush_dcache_page() has to be modified anyway, let's also make sure the mapped highmem pages are pinned with kmap_high_get() for the duration of the cache maintenance operation. Because kunmap() does unmap highmem pages lazily, it was reported by Gary King <GKing@nvidia.com> that those pages ended up being unmapped during cache maintenance on SMP causing segmentation faults. Signed-off-by: Nicolas Pitre <nico@marvell.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: Fix ptrace accessesRussell King2009-12-14
| | | | Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: add size argument to __cpuc_flush_dcache_pageRussell King2009-12-14
| | | | | | | ... and rename the function since it no longer operates on just pages. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: I-cache: flush executable mappings in flush_cache_range()Russell King2009-12-04
| | | | | | | | | | | | | | | | | | | | | Dirk Behme reported instability on ARM11 SMP (VIPT non-aliasing cache) caused by the dynamic linker changing protection on text pages to write GOT entries. The problem is due to an interaction between the write faulting code providing new anonymous pages which are incoherent with the I-cache due to write buffering, and the I-cache not having been invalidated. a4db94d plugs the hole with the data cache coherency. This patch provides the other half of the fix by flushing the I-cache in flush_cache_range() for VM_EXEC VMAs (which is what we have when the region is being made executable again.) This ensures that the I-cache will be up to date with the newly COW'd pages. Note: if users are writing instructions, then they still need to use the ARM sys_cacheflush API to ensure that the caches are correctly synchronized. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: I-cache: avoid flushing in flush_cache_mm()Russell King2009-12-04
| | | | | | | | | | | | | | | | | | flush_cache_mm() is called in two cases: 1. when a process exits, just before the page tables are torn down. We can allow the stale lines to evict themselves over time without causing any harm. 2. when a process forks, and we've allocated a new ASID. The instruction cache issues are dealt with as pages are brought into the new process address space. Flushing the I-cache here is therefore unnecessary. However, we must keep the VIPT aliasing D-cache flush to ensure that any dirty cache lines are not written back after the pages have been reallocated for some other use - which would result in corruption. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: I-cache: Add invalidation for VIVT ASID tagged cachesRussell King2009-12-04
| | | | Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: Remove __flush_icache_all() from __flush_dcache_page()Russell King2009-12-04
| | | | | | | | Both call sites for __flush_dcache_page() end up calling __flush_icache_all() themselves, so having __flush_dcache_page() do this as well is wasteful. Remove the duplicated icache flushing. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: Move __flush_icache_all() out of flush_pfn_alias()Russell King2009-12-04
| | | | Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: ZERO_PAGE: Avoid flush_dcache_page() for zero pageRussell King2009-12-01
| | | | | | | The zero page is read-only, and has its cache state cleared during boot. No further maintanence for this page is required. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: Avoid evaluating page_address() multiple timesRussell King2009-12-01
| | | | | | | | | | | | page_address() is a function call rather than a macro, and so: if (page_address(page)) do_something(page_address(page)); results in two calls to this function. This is unnecessary; remove the duplication. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: Avoid duplicated implementation for VIVT cache flushingRussell King2009-12-01
| | | | | | | | We had two copies of the wrapper code for VIVT cache flushing - one in asm/cacheflush.h and one in arch/arm/mm/flush.c. Reduce this down to one common copy. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* ARM: Fix errata 411920 workaroundsRussell King2009-10-29
| | | | | | | | | | Errata 411920 indicates that any "invalidate entire instruction cache" operation can fail if the right conditions are present. This is not limited just to those operations in flush.c, but elsewhere. Place the workaround in the already existing __flush_icache_all() function instead. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* cpumask: use mm_cpumask() wrapper: armRusty Russell2009-09-23
| | | | | | | | | Makes code futureproof against the impending change to mm->cpu_vm_mask. It's also a chance to use the new cpumask_ ops which take a pointer (the older ones are deprecated, but there's no hurry for arch code). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* ARM: 5687/1: fix an oops with highmemNicolas Pitre2009-09-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | In xdr_partial_copy_from_skb() there is that sequence: kaddr = kmap_atomic(*ppage, KM_SKB_SUNRPC_DATA); [...] flush_dcache_page(*ppage); kunmap_atomic(kaddr, KM_SKB_SUNRPC_DATA); Mixing flush_dcache_page() and kmap_atomic() is a bit odd, especially since kunmap_atomic() must deal with cache issues already. OTOH the non-highmem case must use flush_dcache_page() as kunmap_atomic() becomes a no op with no cache maintenance. Problem is that with highmem the implementation of kmap_atomic() doesn't set page->virtual, and page_address(page) returns 0 in that case. Here flush_dcache_page() calls __flush_dcache_page() which calls __cpuc_flush_dcache_page(page_address(page)) resulting in a kernel oops. None of the kmap_atomic() implementations uses set_page_address(). Hence we can assume page_address() is always expected to return 0 in that case. Let's conditionally call __cpuc_flush_dcache_page() only when the page address is non zero, and perform that test only when highmem is configured. Signed-off-by: Nicolas Pitre <nico@marvell.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] 5488/1: ARM errata: Invalidation of the Instruction Cache operation ↵Catalin Marinas2009-04-30
| | | | | | | | | | can fail This patch implements the recommended workaround for erratum 411920 (ARM1136, ARM1156, ARM1176). Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] kmap supportNicolas Pitre2009-03-15
| | | | | | | | | | | | | | | | | | | | The kmap virtual area borrows a 2MB range at the top of the 16MB area below PAGE_OFFSET currently reserved for kernel modules and/or the XIP kernel. This 2MB corresponds to the range covered by 2 consecutive second-level page tables, or a single pmd entry as seen by the Linux page table abstraction. Because XIP kernels are unlikely to be seen on systems needing highmem support, there shouldn't be any shortage of VM space for modules (14 MB for modules is still way more than twice the typical usage). Because the virtual mapping of highmem pages can go away at any moment after kunmap() is called on them, we need to bypass the delayed cache flushing provided by flush_dcache_page() in that case. The atomic kmap versions are based on fixmaps, and __cpuc_flush_dcache_page() is used directly in that case. Signed-off-by: Nicolas Pitre <nico@marvell.com>
* [ARM] cachetype: move definitions to separate headerRussell King2008-09-01
| | | | | | | | Rather than pollute asm/cacheflush.h with the cache type definitions, move them to asm/cachetype.h, and include this new header where necessary. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] 5092/1: Fix the I-cache invalidation on ARMv6 and later CPUsCatalin Marinas2008-07-03
| | | | | | | | | This patch adds the I-cache invalidation in update_mmu_cache if the corresponding vma is marked as executable. It also invalidates the I-cache if a thread migrates to a CPU it never ran on. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] Resolve fuse and direct-IO failures due to missing cache flushesRussell King2007-01-08
| | | | | | | | | | | | | | | fuse does not work on ARM due to cache incoherency issues - fuse wants to use get_user_pages() to copy data from the current process into kernel space. However, since this accesses userspace via the kernel mapping, the kernel mapping can be out of date wrt data written to userspace. This can lead to unpredictable behaviour (in the case of fuse) or data corruption for direct-IO. This resolves debian bug #402876 Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] Unuse another Linux PTE bitRussell King2006-12-13
| | | | | | | | | | L_PTE_ASID is not really required to be stored in every PTE, since we can identify it via the address passed to set_pte_at(). So, create set_pte_ext() which takes the address of the PTE to set, the Linux PTE value, and the additional CPU PTE bits which aren't encoded in the Linux PTE value. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] 3853/1: Fix flush_ptrace_access() thinko for nonaliasing VIPT cache caseGeorge G. Davis2006-09-25
| | | | | | | | | | Fix thinko in the flush_ptrace_access() "if (expr)" for the ARM VIPT non-aliasing cache case. We only need to flush cache when VM_EXEC is set in vma->vm_flags but "if (expr) always evaluates to true on UP systems for the ARM VIPT non-aliasing cache case. Signed-off-by: George G. Davis <gdavis@mvista.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] Cleanup arch/arm/mm a littleRussell King2006-09-20
| | | | | | | | | | Move top_pmd into arch/arm/mm/mm.h - nothing outside arch/arm/mm references it. Move the repeated definition of TOP_PTE into mm/mm.h, as well as a few function prototypes. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] 3762/1: Fix ptrace cache coherency bug for ARM1136 VIPT nonaliasing ↵George G. Davis2006-09-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Harvard caches Patch from George G. Davis Resolve ARM1136 VIPT non-aliasing cache coherency issues observed when using ptrace to set breakpoints and cleanup copy_{to,from}_user_page() while we're here as requested by Russell King because "it's also far too heavy on non-v6 CPUs". NOTES: 1. Only access_process_vm() calls copy_{to,from}_user_page(). 2. access_process_vm() calls get_user_pages() to pin down the "page". 3. get_user_pages() calls flush_dcache_page(page) which ensures cache coherency between kernel and userspace mappings of "page". However flush_dcache_page(page) may not invalidate I-Cache over this range for all cases, specifically, I-Cache is not invalidated for the VIPT non-aliasing case. So memory is consistent between kernel and user space mappings of "page" but I-Cache may still be hot over this range. IOW, we don't have to worry about flush_cache_page() before memcpy(). 4. Now, for the copy_to_user_page() case, after memcpy(), we must flush the caches so memory is consistent with kernel cache entries and invalidate the I-Cache if this mm region is executable. We don't need to do anything after memcpy() for the copy_from_user_page() case since kernel cache entries will be invalidated via the same process above if we access "page" again. The flush_ptrace_access() function (borrowed from SPARC64 implementation) is added to handle cache flushing after memcpy() for the copy_to_user_page() case. Signed-off-by: George G. Davis <gdavis@mvista.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] 3356/1: Workaround for the ARM1136 I-cache invalidation problemCatalin Marinas2006-03-10
| | | | | | | | | | | | | | | | Patch from Catalin Marinas ARM1136 erratum 371025 (category 2) specifies that, under rare conditions, an invalidate I-cache by MVA (line or range) operation can fail to invalidate a cache line. The recommended workaround is to either invalidate the entire I-cache or invalidate the range by set/way rather than MVA. Note that for a 16K cache size, invalidating a 4K page by set/way is equivalent to invalidating the entire I-cache. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM SMP] Disable lazy flush_dcache_page for SMPRussell King2005-11-30
| | | | | | | Lazy flush_dcache_page() causes userspace instability on SMP platforms, so disable it for now. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] 2939/1: Fix compilation error in arch/arm/mm/flush.cCatalin Marinas2005-09-30
| | | | | | | | | | | Patch from Catalin Marinas When CONFIG_CPU_CACHE_VIPT is defined, the flush_pfn_alias() function is implicitely declared and it later conflicts with its actual definition. This patch moves the function definition to the beginning of the file. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [ARM] Fix ARMv6 VIPT cache >= 32KRussell King2005-09-08
| | | | | | | This adds the necessary changes to ensure that we flush the caches correctly with aliasing VIPT caches. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [PATCH] ARM: Fix delayed dcache flush for ARMv6 non-aliasing cachesRussell King2005-06-20
| | | | | | | | flush_dcache_page() did nothing for these caches, but since they suffer from I/D cache coherency issues, we need to ensure that data is written back to RAM. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* [PATCH] ARM: Add V6 aliasing cache flushRussell King2005-05-10
| | | | | | | Add cache flushing support for aliased V6 caches to flush_dcache_page. Signed-off-by: Russell King <rmk@arm.linux.org.uk>
* Linux-2.6.12-rc2v2.6.12-rc2Linus Torvalds2005-04-16
Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
generated by cgit v1.2.2 (git 2.25.0) at 2024-12-20 22:26:25 -0500