summaryrefslogtreecommitdiffstats
path: root/drivers/gpu/nvgpu/common/linux/nvgpu_mem.c
Commit message (Collapse)AuthorAge
* gpu: nvgpu: gv100: nvlink endpoint driverThomas Fleury2018-02-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The following changes implements the initial (as per bringup) nvlink driver. (1) SW initialization of nvlink core driver structures (2) Nvlink interrupt handling (3) Device initialization (IOCTRL, pll and clocks, device level intr) (4) Falcon support for minion (5) Minion load and bootstrapping (6) Link initialization and DL PROD settings (7) Device Interface init (and switching HSHUB to nvlink) (8) HS set/get mode for both link and sublink (9) Topology discovery and VBIOS settings. (10) Ensures we get physical contiguous memory when Nvlink is enabled This driver includes a hack for the current single dev/single link limitation. JIRA: EVLR-2331 JIRA: EVLR-2330 JIRA: EVLR-2329 JIRA: EVLR-2328 Change-Id: Idca9a819179376cc655784482b24b575a52fa9e5 Signed-off-by: Thomas Fleury <tfleury@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1656790 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: allocate from coherent poolDavid Nieto2018-01-08
| | | | | | | | | | | | | | | | | | | | Maps memory coherently on devices that are connected to a coherent bus. (1) Add code to be able to get the platform device node. (2) Create a new flag to mark if the device is connected to a coherent bus (3) Map memory coherently on coherent devices. bug 2040331 Change-Id: Ide83a9261acdbbc6e9fef4fc5f38d6f9d0e5ab5b Signed-off-by: David Nieto <dmartineznie@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1633985 Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com> Reviewed-by: Alex Waterman <alexw@nvidia.com> GVS: Gerrit_Virtual_Submit Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Use device_is_iommuable() only for iGPUTerje Bergstrom2017-12-11
| | | | | | | | | | | | | | device_is_iommuable() is defined only in Tegra kernel. There is no explicit config option to check for its existance, so skip building that code when Tegra iGPU is not supported. Change-Id: I50dc47070fa416181d458beabf5a2f2373931331 Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1612649 GVS: Gerrit_Virtual_Submit Reviewed-by: Konsta Holtta <kholtta@nvidia.com> Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Define create_from_phys when nvhost enabledTerje Bergstrom2017-12-01
| | | | | | | | | | | | | | Define __nvgpu_mem_create_from_phys only in systems with nvhost enabled. The calling code is also built only when nvhost is enabled. phys_to_page() also exists only in arm64, so using it in non-arm64 platform causes a build failure. Change-Id: Iee023b55bba863d46079796e1c49c19456c1d229 Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1607581 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Include vmalloc.h in nvgpu_mem.cTerje Bergstrom2017-12-01
| | | | | | | | | | | | | | nvgpu_mem.c uses vmap(), which is defined in linux/vmalloc.h, but does not include it. Add the missing #include. Change-Id: I38a44083f08f91df50e372c7f1732e2ad27c6bb1 Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1607578 Reviewed-by: Automatic_Commit_Validation_User GVS: Gerrit_Virtual_Submit Reviewed-by: Konsta Holtta <kholtta@nvidia.com> Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Make buf alignment genericAlex Waterman2017-11-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Drastically simplify and move the aligment computation for buffers getting mapped into the SGT code. An SGT is all that is needed for computing the alignment. However, this did require that a new SGT op was added: nvgpu_sgt_iommuable() This function returns true if the passed SGT is IOMMU'able and must be implemented by an SGT implementation that has IOMMU'able buffers. If this function is left as NULL then it is assumed that the buffer is not IOMMU'able. Also cleanup the parameter ordering convention among all nvgpu_sgt functions. Previously there was a mishmash of different parameter orderings. This patch now standardizes on the gk20a first approach seen everywhere else in the driver. JIRA NVGPU-30 JIRA NVGPU-246 JIRA NVGPU-71 Change-Id: Ic4ab7b752847cf795c7cfafed5a07818217bba86 Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1583985 Reviewed-by: Automatic_Commit_Validation_User GVS: Gerrit_Virtual_Submit Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* Revert "gpu: nvgpu: fix patch buf count update for vidmem"Timo Alho2017-11-03
| | | | | | | | | | | | | | This reverts commit de399ccb0019513a5f9e8f2bcadb02486f99bc80. Bug 2012077 Change-Id: Ie608c3b41aa91f9aaed3fad240ed882a0c6f1ea2 Signed-off-by: Timo Alho <talho@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1591423 Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com> Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com> Tested-by: Terje Bergstrom <tbergstrom@nvidia.com> GVS: Gerrit_Virtual_Submit
* gpu: nvgpu: fix patch buf count update for vidmemPeter Daifuku2017-11-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | gr_gk20a_ctx_patch_write_begin() updates the patch buffer data_count when the associated graphics context memory buffer has been CPU-mapped; it was doing so by looking for a non-null cpu_va. However, if the graphics context has been allocated from vidmem, cpu_va is always 0, so we can't tell if nvgpu_mem_begin() was called for the context buffer or not. Instead: - add a cpu_accessible flag to the nvgpu_mem struct and set it in nvgpu_mem_begin() - return the value of that flag in nvgpu_mem_cpu_accessible() - gr_gk20a_ctx_patch_write_begin() now calls this new function instead of checking cpu_va. Bug 2012077 JIRA ESRM-74 Change-Id: I8401699f30b4ae7154111721c25c7ec3ff95d329 Signed-off-by: Peter Daifuku <pdaifuku@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1587293 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Abstract IO aperture accessorsTerje Bergstrom2017-10-13
| | | | | | | | | | | | | | | | | | | | | Add abstraction of IO aperture accessors. Add new functions gk20a_io_exists() and gk20a_io_valid_reg() to remove dependencies to aperture fields from common code. Implement Linux version of the abstraction by moving gk20a_readl() and gk20a_writel() to new Linux specific io.c. Move the fields defining IO aperture to nvgpu_os_linux. Add t19x specific IO aperture initialization functions and add t19x specific section to nvgpu_os_linux. JIRA NVGPU-259 Change-Id: I09e79cda60d11a20d1099a9aaa6d2375236e94ce Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1569698 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Reduce usage of nvgpu_vidmem_get_page_allocAlex Waterman2017-10-13
| | | | | | | | | | | | | | | | | Reduce the usage of nvgpu_vidmem_get_page_alloc() and friends as much as possible. This reduces the dependency of nvgpu on Linux SGLs. SGLs still need to be used, however, since sharing buffers in userspace is done by dma_buf FD. The best way to pass the vidmem buf through the dma_buf is by SGL pointer. JIRA NVGPU-30 JIRA NVGPU-138 Change-Id: Ide0e9e5a557f00aa63b063be085042101a5b34ee Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1540709 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Rename vidmem APIsAlex Waterman2017-10-13
| | | | | | | | | | | | | | | Rename the VIDMEM APIs to be prefixed by nvgpu_ to ensure consistency and that all the non-static vidmem functions are properly namespaced. JIRA NVGPU-30 JIRA NVGPU-138 Change-Id: I9986ee8f2c8f95a4b7c5e2b9607bc1e77933ccfc Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1540707 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: add API to create nvgpu_mem from physseshendra Gadagottu2017-10-13
| | | | | | | | | | | | | | | | | | | | Added new memory API _nvgpu_mem_create_from_phys for creating nvgpu_mem from physical memory aperture. With this new API, avoided usage of linux specific "struct page" in general code and moved this code to common linux code. This API internally uses __nvgpu_mem_create_from_pages for creating nvgpu_mem from physical pages. JIRA GPUT19X-2 Change-Id: Iaf0193a7c33e71422e4ddabde01edf46f5a81794 Signed-off-by: seshendra Gadagottu <sgadagottu@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1571073 Reviewed-by: svc-mobile-coverity <svc-mobile-coverity@nvidia.com> Reviewed-by: Alex Waterman <alexw@nvidia.com> GVS: Gerrit_Virtual_Submit Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com>
* gpu: nvgpu: Move PRAMIN functions to nvgpu_memTerje Bergstrom2017-10-10
| | | | | | | | | | | | | | PRAMIN batch access functions are only used by nvgpu_mem. The way the functions are written is Linux specific, so move the implementation from common PRAMIN code. JIRA NVGPU-259 Change-Id: I6e2aba08c98568c651a86fe8ca7f9f5220d67348 Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1569697 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Split VIDMEM support from mm_gk20a.cAlex Waterman2017-10-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | Split VIDMEM support into its own code files organized as such: common/mm/vidmem.c - Base vidmem support common/linux/vidmem.c - Linux specific user-space interaction include/nvgpu/vidmem.h - Vidmem API definitions Also use the config to enable/disable VIDMEM support in the makefile and remove as many CONFIG_GK20A_VIDMEM preprocessor checks as possible from the source code. And lastly update a while-loop that iterated over an SGT to use the new for_each construct for iterating over SGTs. Currently this organization is not perfectly adhered to. More patches will fix that. JIRA NVGPU-30 JIRA NVGPU-138 Change-Id: Ic0f4d2cf38b65849c7dc350a69b175421477069c Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1540705 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: rename ops.mm.get_physical_addr_bitsAlex Waterman2017-10-04
| | | | | | | | | | | | | | | | Rename get_physical_addr_bits and related functions to something that more clearly conveys what they are doing. The basic idea of these functions is to translate from a physical GPU address to a IOMMU GPU address. To do that a particular bit (that varies from chip to chip) is added to the physical address. JIRA NVGPU-68 Change-Id: I536cc595c4397aad69a24f740bc74db03f52bc0a Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1542966 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Remove sg_phys() from GMMU codeAlex Waterman2017-10-04
| | | | | | | | | | | | | | | | | | Remove the last sg_phys() call from the GMMU code and replace it with a generic nvgpu_mem API. This new API, nvgpu_mem_get_phys_addr(), returns the physical address of an nvgpu_mem struct. Also, implement this new API in the Linux specific nvgpu_mem code since it requires access to the underlying SGT/SGL. JIRA NVGPU-68 Change-Id: Idf88701a2a8515464c658c26e0de493c82ff850d Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1542964 Reviewed-by: Automatic_Commit_Validation_User Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: SGL passthrough implementationSunny He2017-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | The basic nvgpu_mem_sgl implementation provides support for OS specific scatter-gather list implementations by simply copying them node by node. This is inefficient, taking extra time and memory. This patch implements an nvgpu_mem_sgt struct to act as a header which is inserted at the front of any scatter- gather list implementation. This labels every struct with a set of ops which can be used to interact with the attached scatter gather list. Since nvgpu common code only has to interact with these function pointers, any sgl implementation can be used. Initialization only requires the allocation of a single struct, removing the need to copy or iterate through the sgl being converted. Jira NVGPU-186 Change-Id: I2994f804a4a4cc141b702e987e9081d8560ba2e8 Signed-off-by: Sunny He <suhe@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1541426 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: nvgpu SGL implementationAlex Waterman2017-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The last major item preventing the core MM code in the nvgpu driver from being platform agnostic is the usage of Linux scattergather tables and scattergather lists. These data structures are used throughout the mapping code to handle discontiguous DMA allocations and also overloaded to represent VIDMEM allocs. The notion of a scatter gather table is crucial to a HW device that can handle discontiguous DMA. The GPU has a MMU which allows the GPU to do page gathering and present a virtually contiguous buffer to the GPU HW. As a result it makes sense for the GPU driver to use some sort of scatter gather concept so maximize memory usage efficiency. To that end this patch keeps the notion of a scatter gather list but implements it in the nvgpu common code. It is based heavily on the Linux SGL concept. It is a singly linked list of blocks - each representing a chunk of memory. To map or use a DMA allocation SW must iterate over each block in the SGL. This patch implements the most basic level of support for this data structure. There are certainly easy optimizations that could be done to speed up the current implementation. However, this patches' goal is to simply divest the core MM code from any last Linux'isms. Speed and efficiency come next. Change-Id: Icf44641db22d87fa1d003debbd9f71b605258e42 Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1530867 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Remove mm.get_iova_addrAlex Waterman2017-08-04
| | | | | | | | | | | | | | | | | | | | | | Remove the mm.get_iova_addr() HAL and replace it with a new HAL called mm.gpu_phys_addr(). This new HAL provides the real phys address that should be passed to the GPU from a physical address obtained from a scatter list. It also provides a mechanism by which the HAL code can add extra bits to a GPU physical address based on the attributes passed in. This is necessary during GMMU page table programming. Also remove the flags argument from the various address functions. This flag was used for adding an IO coherence bit to the GPU physical address which is not supported. JIRA NVGPU-30 Change-Id: I69af5b1c6bd905c4077c26c098fac101c6b41a33 Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: https://git-master.nvidia.com/r/1530864 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: split vidmem_is_vidmemKonsta Holtta2017-06-06
| | | | | | | | | | | | | | | | | | | | | As the vidmem_is_vidmem flag has got two separate meanings in one bit, split it in two bits into the enabled() API: Add NVGPU_MM_HONORS_APERTURE bit, which is the same as vidmem_is_vidmem with its original meaning, and use it to test which aperture bits to write to hardware. Add NVGPU_MM_UNIFIED_MEMORY bit, which has the opposite meaning: that the GPU shares the SoC memory. When this flag is false, the GPU has its own local video memory. Jira NVGPU-86 Change-Id: I2d0bed3b1ede5a712be99323d3035b154bb23c3a Signed-off-by: Konsta Holtta <kholtta@nvidia.com> Reviewed-on: http://git-master/r/1496080 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Add __nvgpu_mem_create_from_pages()Alex Waterman2017-04-25
| | | | | | | | | | | | | | | | | | Add a function to create a nvgpu_mem from a list of arbitrary pages. This API is useful for pages not necessarily allocated by the Linux page allocator. It is useful for making nvgpu_mems that represent carveouts or other things like that. JIRA NVGPU-12 JIRA NVGPU-30 Change-Id: Ibcb6432f077a6b0ecf9d183248e47a1f9ecb3ddd Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: http://git-master/r/1464080 Reviewed-by: Automatic_Commit_Validation_User GVS: Gerrit_Virtual_Submit Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
* gpu: nvgpu: Move and rename gk20a_sgtable*Alex Waterman2017-04-25
| | | | | | | | | | | | | | | | | Move and rename the functions that build sgtables for nvgpu_mems into the Linux specific DMA code. One place outside of the Linux code do include the Linux DMA header. That will be fixed in a subsequent patch. JIRA NVGPU-12 JIRA NVGPU-30 Change-Id: Ie43c752b8f998f122af70f7c7eb727af0b0d98df Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: http://git-master/r/1464078 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Add sub-nvgpu_memAlex Waterman2017-04-25
| | | | | | | | | | | | | | | | | | | | | | Add an API for creating a special sub-nvgpu_mem struct. This struct comes with some fairly important caveats but is very useful for the semaphore code. Also, make sure that in nvgpu_mem_begin() and nvgpu_mem_end() no additional mappings are made if not necessary. More importantly during nvgpu_mem_end() it would be possible to vunmap() a CPU mapping of a DMA allocation that does not expect this to happen. JIRA NVGPU-12 JIRA NVGPU-30 Change-Id: I579429da9ff7288488753a113bafc558e0f17a0f Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: http://git-master/r/1464077 Reviewed-by: Automatic_Commit_Validation_User GVS: Gerrit_Virtual_Submit Reviewed-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-by: svccoveritychecker <svccoveritychecker@nvidia.com>
* gpu: nvgpu: Move Linux nvgpu_mem fieldsAlex Waterman2017-04-20
| | | | | | | | | | | | | | | | | | | | Hide the Linux specific nvgpu_mem fields so that in subsequent patches core code can instead of using struct sg_table it can use mem_desc. Routines for accessing system specific fields will be added as needed. This is the first step in a fairly major overhaul of the GMMU mapping routines. There are numerous issues with the current design (or lack there of): massively coupled code, system dependencies, disorganization, etc. JIRA NVGPU-12 JIRA NVGPU-30 Change-Id: I2e7d3ae3a07468cfc17c1c642d28ed1b0952474d Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: http://git-master/r/1464076 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: Add wrapper nvgpu/bug.hTerje Bergstrom2017-04-13
| | | | | | | | | | | | | Add wrapper header file nvgpu/bug.h. It #includes <linux/bug.h> in Linux. JIRA NVGPU-13 Change-Id: I7bf02ba554333f7cbd79d72bd1cb423c81ebcb49 Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-on: http://git-master/r/1461545 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
* gpu: nvgpu: common: linux: Use new error macrosTerje Bergstrom2017-04-10
| | | | | | | | | | | | | | | | gk20a_err() and gk20a_warn() require a struct device pointer, which is not portable across operating systems. The new nvgpu_err() and nvgpu_warn() macros take struct gk20a pointer. Convert code to use the more portable macros. JIRA NVGPU-16 Change-Id: Ie3cf6f278dd4468832de3c3a6a6b601d55185461 Signed-off-by: Terje Bergstrom <tbergstrom@nvidia.com> Reviewed-on: http://git-master/r/1459818 Reviewed-by: Automatic_Commit_Validation_User Reviewed-by: Alex Waterman <alexw@nvidia.com> GVS: Gerrit_Virtual_Submit
* gpu: nvgpu: rename mem_desc to nvgpu_memAlex Waterman2017-04-06
Renaming was done with the following command: $ find -type f | \ xargs sed -i 's/struct mem_desc/struct nvgpu_mem/g' Also rename mem_desc.[ch] to nvgpu_mem.[ch]. JIRA NVGPU-12 Change-Id: I69395758c22a56aa01e3dffbcded70a729bf559a Signed-off-by: Alex Waterman <alexw@nvidia.com> Reviewed-on: http://git-master/r/1325547 Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com> Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>