Add bounds checking to kgsl_sharedmem_find_region to ensure that GPU
addresses supplied by the user via KGSL_IOCTL_SHAREMEM_FREE and
KGSL_IOCTL_CFF_SYNCMEM are within a valid range for the MMU scheme of
choice.
CRs-Fixed: 380296
Change-Id: Ic0dedbadf40fca4a9e04ba075e67e08e617806b5
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Put a reasonable upper limit on the number of indirect buffers allowed
per submission via IOCTL_KGSL_RINGBUFFER_ISSUEIBCMDS.
CRs-Fixed: 380267
Change-Id: Ic0dedbadd78a0ad3f09f67660f34b2fba421007f
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Turn on the IOMMU clocks in recovery as the recovered command
stream may have commands that read and write to IOMMU registers.
Change-Id: I6a8d9fa17a57b0c4f8564fc62d019b2e150b8399
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
In adreno_idle the default timeout value was always used,
even if the caller specified something different. The caller
can now specify a non default timeout value to be used
in adreno_idle.
Change-Id: Ibb6bcc094521d4a660d8385b32bf8c6ac68acbd9
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
The lowest non-zero GPU frequency is only meant to be set when
the device enters SLEEP and the clocks are off. Do not permit it
to be used as a thermal level.
Change-Id: I6eedf39849ec5211977f824148883e3694cae50b
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>
In recovery search for the timestamp of the command stream after
global eop instead of searching for the command stream of the
global eop and then moving the pointer forward till the start of
the command stream following global eop. This is because if the
ringbuffer is full then the global eop timestamp may have been
overwritten, so its more reliable to search for the timestamp
that occurs after global eop.
Change-Id: Ide272fd653bcba411f0c4b92b3dc2ac2fa8326a9
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Replay the all the commands that have not been executed by the GPU
in recovery with preamble turned on for the first command to see
if that succeeds. If it does then we have recovered all contexts.
If that fails then replay commands only from contexts that are
in a good state.
Change-Id: Ia2a5c9b0de5556c5182cfab3bd6ae576d15b53eb
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Expire the global timestamp and timestamps of contexts that have
cause at the end of recovery. This will prevent any users waiting
on timestamps of hung contexts to return without waiting.
Change-Id: I373de7211edd09646cdb0fb11d67b562a1ab2966
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Mark the context that has caused a hang in recovery before
calling the extraction function. This updates the context
state correctly early in recovery.
Change-Id: I68cbe5cc6f867c8d32dd0115eed6f1c65c54f796
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Write a separate function that sets the reset status of each
context after recovery.
Change-Id: I0fbd6578625637373beb9a565024043ece186e6d
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
We do not have to restore the per context timestamp states during
recovery as this data is irrelevant after recovery.
Change-Id: I3f0350afda4b4ecbdd60027720b7e4832adf04a3
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Turn on the preamble of the IB sequence in which hang is assumed
to have happened so that the bad sequence can be replayed.
Change-Id: I7123f6b5152a530c38999d6ef4aa5476d64960a7
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Make the code more readable by writing a separate function to
detect the end of last command that is assumed to be successfully
executed by the GPU
Change-Id: Icab1f4e563ea533c48aba9c57afb548501e19552
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Create a separate function that extracts valid commands from good
contexts and invalid commands from bad contexts during recovery
Change-Id: Icffe34821c1b70ca1a8c1a8e13a0b7e23d0ec193
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Use the data set in the recovery structure when extracting
valid commands from the ringbuffer instead of reading
it from the device/memory.
Change-Id: I9cbcc09b1ce95313c2d3cf59fbb649dad2cb9d99
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Create a new data structure that holds all the information required
for recovering from hangs. This data structure is initialized as soon
as hang is detected because if the GPU is under DOS attack then by
the time we start recovery the recovery data may have changed.
Change-Id: I862787267c582b9ca95bb191fb3503ceba1faa8c
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Add command sequence start identifier for commands submitted from
kgsl as well since this will be used during recovery. We will just
try recovery and not try to differentiate between commands from
kgsl or from user space.
Change-Id: I433e9f2786268c6c51a446e3c7cf9eb5fd429197
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
When performing postmortem, check for IOMMU pagetable change commands
which will allow us to find the IB from the correct pagetable.
Change-Id: I411f4c78632d1673d49d5557da435e2a9d31d420
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Start the sleep timer if recovery from hang is successful.
This prevents GPU from entering low power states during
recovery.
Change-Id: Icb021f1b1bfae7876394fa572acd156115372ac3
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Remove unneeded idle commands from the command stream when updating
pagetables and doing TLB flushes. Ths cuts down on the number of
commands in the ringbuffer during switches.
Change-Id: Ic0dedbad7a0d64c39518b2dd2bb6322f4ab74ba0
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Modify GMEM Restore procedure as per A3XX GPU spec to support
multiple GPU revisions.
Change-Id: I037cc32c5e259a2ff83cd579b586e9c1f585236f
Signed-off-by: Rajeev Kulkarni <krajeev@codeaurora.org>
Move almost all of the SOC based GPU identification code out
of the KGSL driver and into the board files. This cleans up
the GPU side and reduces our usage of the cpu_is and machine_is
macros.
Change-Id: Ic0dedbad34eb034316b7baabf05aa7f8e8aa9ced
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The pdata structure and associated structs have no business being
in msm_kgsl.h. Move them to include/mach/kgsl.h where they can
thrive in a non-userspace facing environment.
Change-Id: Ic0dedbadfe0d3d7d44f50c8a9a7f4cc59057d44b
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Modify GMEM Save procedure as per A3XX GPU spec to support
multiple GPU revisions. It also fixes A3XX GPU
hang caused by GMEM save procedure.
CRs-Fixed: 375034
Change-Id: I24a4d6c4fd8304b3f41a167c116a75f2e3680158
Signed-off-by: Rajeev Kulkarni <krajeev@codeaurora.org>
Before returning from recovery routine, wait for the GPU to
become completely idle to ensure that recovery passed
Change-Id: I381b2e62641b0eaf187b410e2d043a6d4f1f4e3b
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
The condition argument to the WARN call in ion_share_dma_buf
is missing. Add the argument to allow correct printing of
warning message.
Change-Id: I6ded3a78d3030f38bf29a1be063364690cbc0ddb
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Add an idle for synchronization after writing the SMMU ASID register.
Change-Id: I1e13c9725e52293b7d4b9034fd1bac045ca5f0d1
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
User memory needs to be zeroed out before it is sent to the user.
To do this, the kernel maps the page, memsets it to zero and then
unmaps it. By virtue of mapping it, this forces us to flush the
dcache to ensure cache coherency between kernel and user mappings.
Originally, the page_alloc loop was using GFP_ZERO (which does a
map, memset, and unmap for each individual page) and then we were
additionally calling flush_dcache_page() for each page killing us
on performance. It is far more efficient, especially for large
allocations (> 1MB), to allocate the pages without GFP_ZERO and
then to vmap the entire allocation, memset it to zero, flush the
cache and then unmap. This process is slightly slower for very
small allocations, but only by a few microseconds, and is well
within the margin of acceptability. In all, the new scheme is
faster than the default for all sizes greater than 16k, and is
almost 4X faster for 2MB and 4MB allocations which are common for
textures and very large buffer objects.
The downside is that if there isn't enough vmalloc room for the
allocation that we are forced to fallback to a slow page by
page memset/flush, but this should happen rarely (if at all) and
is only included for completeness.
CRs-Fixed: 372638
Change-Id: Ic0dedbadf3e27dcddf0f068594a40c00d64b495e
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
This reverts commit 5462907f72ddcc7d337ce9431d54428c8dc486eb.
Change-Id: Ibe6ec76096838b71e43cd9beac1f57ff179c5008
Signed-off-by: Sunil Joseph <sunilj@codeaurora.org>
Update the call sites of cpu_is_msm8930() to include checks
for the MSM8930AA() variant. Relevant drivers will be
updated for more driver-specific specific MSM8930AA checks
at a later time.
Change-Id: Iff1af7a5454ec56c40390682ce2b4b6d1d325c91
Signed-off-by: Stepan Moskovchenko <stepanm@codeaurora.org>
Add EXPORT_SYMBOL to specific functions that should be
allowed to be used from within kernel modules.
Change-Id: I666c1f1a18d5af58906cb2185c5cf29d8596013f
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Per revised design decisions, cpu_is_msm8930() shall only
return true on 8930, and not on the 8627 variant. Modify
the cpu_is_xxx functions to reflect this change, and update
call sites accordingly.
Change-Id: I50b943f80c731717e6cd5d7fffb13aeec0f85a40
Signed-off-by: Stepan Moskovchenko <stepanm@codeaurora.org>
last_expired_ctxt_id was used briefly for per context timestamps and now
can be removed as it is no longer used.
Change-Id: I51d3947fe6da366824d1b947ae3448bee3518477
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
There are two head switches for GPU, one is for the GPU core
itself, the other is for surrounding digital circuits. KGSL
needs to control them both.
The turn on sequence is:
1. Turn on GPU core
2. Turn on digital circuits
The turn off sequence is:
1. Turn off digital circuits
2. Turn off GPU core
Two regulator (actually head switch) names "vdd_dig" and "vdd"
are created in device tree file, representing digital
circuits and GPU core head switches respectively. In struct
kgsl_pwrctrl, adding a new regulator struct "gpu_dig"
which corresponds to vdd_dig. And the existing struct "gpu_reg"
corresponds to vdd.
Change-Id: Idb05caf547ad148885adb9cf498c47635067b70b
Signed-off-by: Pu Chen <puchen@codeaurora.org>
Check that the ops pointer is valid before using it, otherwise we
may fault.
Change-Id: I7f1420638eca53f6d85b8c3ff114033ce77cb408
CRs-fixed: 360810
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Remove unneeded idle commands from the command stream when updating
pagetables and doing TLB flushes. Ths cuts down on the number of
commands in the ringbuffer during switches.
Change-Id: I8f1fa31db33c465ecdbd0ca897f4651f2670c4e4
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
In case of IOMMU memory needs to only be flushed when it is
unmapped and not mappped. Also, when an application is running
the number of unmaps would in most cases be less than number of
maps, so this way the number of flushes wil be fewer.
Change-Id: Id4107631c343045edbc744125035202844405910
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Check global GPU status registers in intervals of 2 seconds,
if the registers did not change after 2 seconds trigger a hang.
When there is an actual GPU hang and GPU is stalled, this reduces
the current hang detection time from 10 seconds to 2 seconds.
Faster GPU hang detection reduces the overall time to recover
from GPU hang.
Change-Id: If432ccacc1b77b4fc7f08b756886bd3ae5edf04f
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
A busy loop is executed when we allocate space from the ringbuffer.
If the GPU is hung then we can wait indefinitely in the busy loop.
Add a wait time to this busy loop, and if the wait time elapses then
report a GPU hang instead of waiting indefinitely.
Change-Id: I035c39063cbfa25380702720f929df5319e73e61
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Pull in Ion patches for 3.4 upgrade. Inclues the following
patches from google:
commit 7191e9ba25
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Mon Jan 30 14:18:08 2012 -0800
ion: Switch map/unmap dma api to sg_tables
Switch these api's from scatterlists to sg_tables
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 6f9e56945d
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Tue Jan 31 09:40:30 2012 -0800
ion: Add reserve function to ion
Rather than requiring each platform call memblock_remove or reserve
from the board file, add this to ion
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 9ae7e01de1
Author: KyongHo Cho <pullip.cho@samsung.com>
Date: Wed Sep 7 11:27:07 2011 +0900
gpu: ion: several bugfixes and enhancements of ION
1. Verifying if the size of memory allocation in ion_alloc() is aligned
by PAGE_SIZE at least. If it is not, this change makes the size to be
aligned by PAGE_SIZE.
2. Unmaps all mappings to the kernel and DMA address spaces when
destroying ion_buffer in ion_buffer_destroy(). This prevents leaks in
those virtual address spaces.
3. Makes the return value of ion_alloc() to be explicit Linux error code
when it fails to allocate a buffer.
4. Makes ion_alloc() implementation simpler. Removes 'goto' statement and
relavant call to ion_buffer_put().
5. Checks if the task is valid before calling put_task_struct() due
to failure on creating a ion client in ion_client_create().
6. Returns error when buffer allocation requested by userspace is failed.
Signed-off-by: KyongHo Cho <pullip.cho@samsung.com>
commit 043a61468f
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Wed Feb 1 11:09:46 2012 -0800
ion: Switch ion to use dma-buf
Ion now uses dma-buf file descriptors to share
buffers with userspace. Ion becomes a dma-buf
exporter and any driver that can import dma-bufs
can now import ion file descriptors.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 0d1259b5f8
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Mon Apr 30 16:45:38 2012 -0700
gpu: ion: Use alloc_pages instead of vmalloc from the system heap
With this change the ion_system_heap will only use kernel address
space when the memory is mapped into the kernel (rare case).
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit be4a1ee79a
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Thu Apr 26 20:44:10 2012 -0700
gpu: ion: support begin/end and kmap/kunmap dma_buf ops
These ops were added in the 3.4 kernel. This patch adds support
for them to ion. Previous ion_map/unmap_kernel api is also
retained in addition to this new api.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 46d71337f9
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Mon May 7 16:06:32 2012 -0700
gpu: ion: Allocate the sg_table at creation time rather than dynamically
Rather than calling map_dma on the allocations dynamically, this patch
switches to creating the sg_table at the time the buffer is created.
This is necessary because in future updates the sg_table will be used
for cache maintenance.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 903f6c716d
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Wed May 23 12:55:55 2012 -0700
gpu: ion: Get an sg_table from an ion handle
This patch adds an interface to return and sg_table given a
valid ion handle.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
The commit also includes fixups needed for MSM specific code.
Change-Id: Idbcfa9d6af0febf06f56daaa6beaa59cc08e4351
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Add a function which adds an event to disable the iommu clocks
based on a timestamp. If timestamp elapses then the event function
will disable the iommu clocks. Remove the disabling of iommu clocks
from the power control code.
Change-Id: Ic1263771f8c532131059283dde05425a73554974
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Make sure that the busy times are calculated at the end of the
current sample and before the next sample starts. This prevents
false alarms of busy timer events being triggered. Applies only
to devices with idlestats power policy.
Change-Id: I1e28fff4128b72c25e8073d845d519867d67a9ca
Signed-off-by: Ranjhith Kalisamy <ranjhith@codeaurora.org>
Add TLB flush for each submit to prevent target freeze
in case of 8x25/7x27a/7x25a targets.
Change-Id: I350e72e4f71fef86c7cf9da0954ef65d5765b88f
Signed-off-by: Rajesh Kemisetti <rajeshk@codeaurora.org>
The pmem_kalloc APIs are now deprecated. A reference to
an old pmem memtype still exists in kgsl code even though
it isn't used. Remove it.
Change-Id: I9b8bd2037f1a49dc543fc766593395fc440b1318
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
SLEEP & SLUMBER should both wake at default power level on targets w/
GPU DCVS. Combine the calls to make this possible.
CRs-Fixed: 369927
Change-Id: I386a9ab04850b0beb249b47398ad623f91fbbfda
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>
We do not need to do a soft reset of the A3XX GPUs after power on
as all the interesting registers appear to be in a reasonable power
on state. This saves 20ms during suspend/resume.
CRs-Fixed: 369927
Change-Id: Ic0dedbad367e8b402aefbf6e7746ae5b17120ac8
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The IOMMU registers should only be accessed by priv bank. Create a separate
pagetable for priv bank and map the IOMMU registers to only this pagetable.
This is only supported for devices which can use the priv bank for
performing in stream pagetable switches.
Change-Id: If9edada5d3a05e9a660c126139e18091b1ab43cc
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Add support to allow ion_cp_heap.c to secure according to new
requirements. This includes adding versioning to the secure
calls to allow other types of securing to take place.
Change-Id: I65e07ebaeefa1d0572b6531753a707a28284aa0d
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
CL contexts turn off the VPC block while running to avoid a hang. The
VPC block needs to be turned back on before any GL calls. This includes
GMEM restore, so set the clock control value back to the original default
before doing any other operations.
CRs-fixed: 363922
Change-Id: Ic0dedbad6b031ccd9c0816013b75ba9effb18fcb
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
