Remove unneeded idle commands from the command stream when updating
pagetables and doing TLB flushes. Ths cuts down on the number of
commands in the ringbuffer during switches.
Change-Id: Ic0dedbad7a0d64c39518b2dd2bb6322f4ab74ba0
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Modify GMEM Restore procedure as per A3XX GPU spec to support
multiple GPU revisions.
Change-Id: I037cc32c5e259a2ff83cd579b586e9c1f585236f
Signed-off-by: Rajeev Kulkarni <krajeev@codeaurora.org>
Move almost all of the SOC based GPU identification code out
of the KGSL driver and into the board files. This cleans up
the GPU side and reduces our usage of the cpu_is and machine_is
macros.
Change-Id: Ic0dedbad34eb034316b7baabf05aa7f8e8aa9ced
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The pdata structure and associated structs have no business being
in msm_kgsl.h. Move them to include/mach/kgsl.h where they can
thrive in a non-userspace facing environment.
Change-Id: Ic0dedbadfe0d3d7d44f50c8a9a7f4cc59057d44b
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Modify GMEM Save procedure as per A3XX GPU spec to support
multiple GPU revisions. It also fixes A3XX GPU
hang caused by GMEM save procedure.
CRs-Fixed: 375034
Change-Id: I24a4d6c4fd8304b3f41a167c116a75f2e3680158
Signed-off-by: Rajeev Kulkarni <krajeev@codeaurora.org>
Before returning from recovery routine, wait for the GPU to
become completely idle to ensure that recovery passed
Change-Id: I381b2e62641b0eaf187b410e2d043a6d4f1f4e3b
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
The condition argument to the WARN call in ion_share_dma_buf
is missing. Add the argument to allow correct printing of
warning message.
Change-Id: I6ded3a78d3030f38bf29a1be063364690cbc0ddb
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Add an idle for synchronization after writing the SMMU ASID register.
Change-Id: I1e13c9725e52293b7d4b9034fd1bac045ca5f0d1
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
User memory needs to be zeroed out before it is sent to the user.
To do this, the kernel maps the page, memsets it to zero and then
unmaps it. By virtue of mapping it, this forces us to flush the
dcache to ensure cache coherency between kernel and user mappings.
Originally, the page_alloc loop was using GFP_ZERO (which does a
map, memset, and unmap for each individual page) and then we were
additionally calling flush_dcache_page() for each page killing us
on performance. It is far more efficient, especially for large
allocations (> 1MB), to allocate the pages without GFP_ZERO and
then to vmap the entire allocation, memset it to zero, flush the
cache and then unmap. This process is slightly slower for very
small allocations, but only by a few microseconds, and is well
within the margin of acceptability. In all, the new scheme is
faster than the default for all sizes greater than 16k, and is
almost 4X faster for 2MB and 4MB allocations which are common for
textures and very large buffer objects.
The downside is that if there isn't enough vmalloc room for the
allocation that we are forced to fallback to a slow page by
page memset/flush, but this should happen rarely (if at all) and
is only included for completeness.
CRs-Fixed: 372638
Change-Id: Ic0dedbadf3e27dcddf0f068594a40c00d64b495e
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
This reverts commit 5462907f72ddcc7d337ce9431d54428c8dc486eb.
Change-Id: Ibe6ec76096838b71e43cd9beac1f57ff179c5008
Signed-off-by: Sunil Joseph <sunilj@codeaurora.org>
Update the call sites of cpu_is_msm8930() to include checks
for the MSM8930AA() variant. Relevant drivers will be
updated for more driver-specific specific MSM8930AA checks
at a later time.
Change-Id: Iff1af7a5454ec56c40390682ce2b4b6d1d325c91
Signed-off-by: Stepan Moskovchenko <stepanm@codeaurora.org>
Add EXPORT_SYMBOL to specific functions that should be
allowed to be used from within kernel modules.
Change-Id: I666c1f1a18d5af58906cb2185c5cf29d8596013f
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Per revised design decisions, cpu_is_msm8930() shall only
return true on 8930, and not on the 8627 variant. Modify
the cpu_is_xxx functions to reflect this change, and update
call sites accordingly.
Change-Id: I50b943f80c731717e6cd5d7fffb13aeec0f85a40
Signed-off-by: Stepan Moskovchenko <stepanm@codeaurora.org>
last_expired_ctxt_id was used briefly for per context timestamps and now
can be removed as it is no longer used.
Change-Id: I51d3947fe6da366824d1b947ae3448bee3518477
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
There are two head switches for GPU, one is for the GPU core
itself, the other is for surrounding digital circuits. KGSL
needs to control them both.
The turn on sequence is:
1. Turn on GPU core
2. Turn on digital circuits
The turn off sequence is:
1. Turn off digital circuits
2. Turn off GPU core
Two regulator (actually head switch) names "vdd_dig" and "vdd"
are created in device tree file, representing digital
circuits and GPU core head switches respectively. In struct
kgsl_pwrctrl, adding a new regulator struct "gpu_dig"
which corresponds to vdd_dig. And the existing struct "gpu_reg"
corresponds to vdd.
Change-Id: Idb05caf547ad148885adb9cf498c47635067b70b
Signed-off-by: Pu Chen <puchen@codeaurora.org>
Check that the ops pointer is valid before using it, otherwise we
may fault.
Change-Id: I7f1420638eca53f6d85b8c3ff114033ce77cb408
CRs-fixed: 360810
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Remove unneeded idle commands from the command stream when updating
pagetables and doing TLB flushes. Ths cuts down on the number of
commands in the ringbuffer during switches.
Change-Id: I8f1fa31db33c465ecdbd0ca897f4651f2670c4e4
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
In case of IOMMU memory needs to only be flushed when it is
unmapped and not mappped. Also, when an application is running
the number of unmaps would in most cases be less than number of
maps, so this way the number of flushes wil be fewer.
Change-Id: Id4107631c343045edbc744125035202844405910
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Check global GPU status registers in intervals of 2 seconds,
if the registers did not change after 2 seconds trigger a hang.
When there is an actual GPU hang and GPU is stalled, this reduces
the current hang detection time from 10 seconds to 2 seconds.
Faster GPU hang detection reduces the overall time to recover
from GPU hang.
Change-Id: If432ccacc1b77b4fc7f08b756886bd3ae5edf04f
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
A busy loop is executed when we allocate space from the ringbuffer.
If the GPU is hung then we can wait indefinitely in the busy loop.
Add a wait time to this busy loop, and if the wait time elapses then
report a GPU hang instead of waiting indefinitely.
Change-Id: I035c39063cbfa25380702720f929df5319e73e61
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Pull in Ion patches for 3.4 upgrade. Inclues the following
patches from google:
commit 7191e9ba25
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Mon Jan 30 14:18:08 2012 -0800
ion: Switch map/unmap dma api to sg_tables
Switch these api's from scatterlists to sg_tables
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 6f9e56945d
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Tue Jan 31 09:40:30 2012 -0800
ion: Add reserve function to ion
Rather than requiring each platform call memblock_remove or reserve
from the board file, add this to ion
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 9ae7e01de1
Author: KyongHo Cho <pullip.cho@samsung.com>
Date: Wed Sep 7 11:27:07 2011 +0900
gpu: ion: several bugfixes and enhancements of ION
1. Verifying if the size of memory allocation in ion_alloc() is aligned
by PAGE_SIZE at least. If it is not, this change makes the size to be
aligned by PAGE_SIZE.
2. Unmaps all mappings to the kernel and DMA address spaces when
destroying ion_buffer in ion_buffer_destroy(). This prevents leaks in
those virtual address spaces.
3. Makes the return value of ion_alloc() to be explicit Linux error code
when it fails to allocate a buffer.
4. Makes ion_alloc() implementation simpler. Removes 'goto' statement and
relavant call to ion_buffer_put().
5. Checks if the task is valid before calling put_task_struct() due
to failure on creating a ion client in ion_client_create().
6. Returns error when buffer allocation requested by userspace is failed.
Signed-off-by: KyongHo Cho <pullip.cho@samsung.com>
commit 043a61468f
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Wed Feb 1 11:09:46 2012 -0800
ion: Switch ion to use dma-buf
Ion now uses dma-buf file descriptors to share
buffers with userspace. Ion becomes a dma-buf
exporter and any driver that can import dma-bufs
can now import ion file descriptors.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 0d1259b5f8
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Mon Apr 30 16:45:38 2012 -0700
gpu: ion: Use alloc_pages instead of vmalloc from the system heap
With this change the ion_system_heap will only use kernel address
space when the memory is mapped into the kernel (rare case).
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit be4a1ee79a
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Thu Apr 26 20:44:10 2012 -0700
gpu: ion: support begin/end and kmap/kunmap dma_buf ops
These ops were added in the 3.4 kernel. This patch adds support
for them to ion. Previous ion_map/unmap_kernel api is also
retained in addition to this new api.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 46d71337f9
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Mon May 7 16:06:32 2012 -0700
gpu: ion: Allocate the sg_table at creation time rather than dynamically
Rather than calling map_dma on the allocations dynamically, this patch
switches to creating the sg_table at the time the buffer is created.
This is necessary because in future updates the sg_table will be used
for cache maintenance.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 903f6c716d
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Wed May 23 12:55:55 2012 -0700
gpu: ion: Get an sg_table from an ion handle
This patch adds an interface to return and sg_table given a
valid ion handle.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
The commit also includes fixups needed for MSM specific code.
Change-Id: Idbcfa9d6af0febf06f56daaa6beaa59cc08e4351
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Add a function which adds an event to disable the iommu clocks
based on a timestamp. If timestamp elapses then the event function
will disable the iommu clocks. Remove the disabling of iommu clocks
from the power control code.
Change-Id: Ic1263771f8c532131059283dde05425a73554974
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Make sure that the busy times are calculated at the end of the
current sample and before the next sample starts. This prevents
false alarms of busy timer events being triggered. Applies only
to devices with idlestats power policy.
Change-Id: I1e28fff4128b72c25e8073d845d519867d67a9ca
Signed-off-by: Ranjhith Kalisamy <ranjhith@codeaurora.org>
Add TLB flush for each submit to prevent target freeze
in case of 8x25/7x27a/7x25a targets.
Change-Id: I350e72e4f71fef86c7cf9da0954ef65d5765b88f
Signed-off-by: Rajesh Kemisetti <rajeshk@codeaurora.org>
The pmem_kalloc APIs are now deprecated. A reference to
an old pmem memtype still exists in kgsl code even though
it isn't used. Remove it.
Change-Id: I9b8bd2037f1a49dc543fc766593395fc440b1318
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
SLEEP & SLUMBER should both wake at default power level on targets w/
GPU DCVS. Combine the calls to make this possible.
CRs-Fixed: 369927
Change-Id: I386a9ab04850b0beb249b47398ad623f91fbbfda
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>
We do not need to do a soft reset of the A3XX GPUs after power on
as all the interesting registers appear to be in a reasonable power
on state. This saves 20ms during suspend/resume.
CRs-Fixed: 369927
Change-Id: Ic0dedbad367e8b402aefbf6e7746ae5b17120ac8
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The IOMMU registers should only be accessed by priv bank. Create a separate
pagetable for priv bank and map the IOMMU registers to only this pagetable.
This is only supported for devices which can use the priv bank for
performing in stream pagetable switches.
Change-Id: If9edada5d3a05e9a660c126139e18091b1ab43cc
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Add support to allow ion_cp_heap.c to secure according to new
requirements. This includes adding versioning to the secure
calls to allow other types of securing to take place.
Change-Id: I65e07ebaeefa1d0572b6531753a707a28284aa0d
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
CL contexts turn off the VPC block while running to avoid a hang. The
VPC block needs to be turned back on before any GL calls. This includes
GMEM restore, so set the clock control value back to the original default
before doing any other operations.
CRs-fixed: 363922
Change-Id: Ic0dedbad6b031ccd9c0816013b75ba9effb18fcb
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
When an axi error happens and the GPU is using IOMMU then a warning stack
trace was being reported. This is because during axi error interrupt
handling the IOMMU clocks were being turned on to read the current
pagetable base address. Switching the IOMMU clocks from interrupt context
is not allowed. Fix this by reporting the axi error and displaying 0
as the pagetable base address since we cannot read its value from
the axi error interrupt code.
Change-Id: If98538bc1b2df51a1b3632faf449d5d332bccc3e
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
In the IOMMU heap, the return value of msm_allocate_iova_address
needs to be checked instead of the value of the iova address.
Check the return value instead.
Change-Id: I4554515d17c926a7dd738824aff21eb65d2ba56a
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
When GMEM save is not performed, the events to reuse shader space
can get dropped. This make the CP block to wait indefinitely
causing a hang. To unblock this, issue a packet to set shader bases.
This applies only to A20x/220 GPU.
CRs-fixed: 365465
Change-Id: I3ea6a58821b238706f167845380370d5bd19c4b3
Signed-off-by: Ranjhith Kalisamy <ranjhith@codeaurora.org>
Reference count the calls to ion_{un}secure_heap. The secure SCM
call will only be made when the count goes from 0 -> 1 and the
unsecure call will only be made when the count goes from 1 -> 0.
Change-Id: Ia221b79782c9223d9853ef00c2848e1ab7a73091
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The next generation of content protection has separate SCM
calls into TZ. Add these functions in a dedicated file so they
can be used by any Ion heap.
Change-Id: I197130f6a5fc33efcd6871b0f7c362fa99f18d20
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
kgsl_core_exit() should not call kgsl_sharedmem_uninit_sysfs() and
device_unregister() if those resources have not been allocated.
We check this by seeing if kgsl_driver.virtdev has been
populated or not.
CRs-fixed: 359477
Change-Id: Ic040667b29bd4e006e6f29a50809d94a2bb4b469
Signed-off-by: Harsh Vardhan Dwivedi <hdwivedi@codeaurora.org>
Dump the shader memory (working memory and shadow memory) for A3XX GPUs
in the snapshot.
Change-Id: Ic0dedbadf28733b96323a680b014dbea10b1ceda
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Update the snapshot header to version 2 to return the raw chipid
in the dump.
Change-Id: Ic0dedbad82bbe61395924bf49cb0144e5e2e10e3
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
There are some subtle differences between revisions of GPUs
that are not reflected in the GPU ID value. Return the raw
chipid (derived from the hardware) along with the GPU ID
when queried.
Change-Id: Ic0dedbad035a80726e6fedc066ec11b78b79da2b
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The code seeking out the failing frame for a hang got confused if
the CP read and write pointers were equal resulting in missing
IBs.
Change-Id: Ic0dedbadb32e4a708d0f8ee9e8a853fdfaff10b0
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Use a iterator to cleanup how snapshot GPU objects are printed.
This removes a ton of ugly math, and allows us to do subtle
silly things like add padding at the end of a non dword
aligned object.
Change-Id: Ic0dedbadabeee50f42fe9be62ea0b7dfc1f333da
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Enhance the logic that checks to see if the CP_RPTR is stuck
mid packet. Ensure that when we do packet counting that we
use real type3 packets instead of regular data pretending to
be a type3 packet.
Change-Id: Ic0dedbadb502fa2790df0c6752309da300d6d1ae
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Currently we just check the most significant two bits of a dword
in the ringbuffer or IB are set to decide if the data is a type3
packet header. This has proven to be problematic, as it turns out
that the top two bits are often set in nominal data. Enhance
the type3 packet check to verify that various reserved bits in
the dword are 0 as expected.
Change-Id: Ic0dedbada5eff8353bc64da9eefd9919de127fa4
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
User side drivers are starting to use more indirect buffers
for CP_LOAD_STATE, so expand the snapshot to support grabbing
all of the possible combinations.
Change-Id: Ic0dedbadff743d03b5accb2a0299ab5dc0bc1a61
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
If there is a failure in drawctxt create, then the KGSL core will call
drawctxt destroy to free the memory. If the failure happened soon
enough, there might not a drawctxt to free.
CRs-Fixed: 371852
Change-Id: Ic0dedbad629090ae1827e354b9b94bbd3c5efcdd
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
This reverts commit 609467077ae6b794af47613e05949e8b2fb3a842.
Change-Id: I457634982603c957788b749198219d0974bd1de5
Signed-off-by: Zhoulu Luo <zluo@codeaurora.org>
Conditional interrupts were not getting fired at the correct time when
using per context timestamps. When a context was waiting on a timestamp,
the interrupt was being fired on the global timestamp rather than the per
context timestamp. Now the interrupt is fired on the per context
timestamp.
CRs-Fixed: 340620
Change-Id: I8393ca74aabfe4b711b61efb023d38d6ab852eec
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
Change prevents IOMMU pagefaults which may occur when
indexing reaches 4096/sizeof(kgsl_devmemstore) number
of contexts
CRs-Fixed: 370594
Change-Id: If726c2a1fcf7667b062e890453a71e95f5417c60
Signed-off-by: Richard Ruigrok <rruigrok@codeaurora.org>
Use one code path to dereference shared variables. Simplify code
for readability.
Change-Id: Ifcfe948b7c04a6594c3fa9eac4afea81f65e59af
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
