Add an idle for synchronization after writing the SMMU ASID register.
Change-Id: I1e13c9725e52293b7d4b9034fd1bac045ca5f0d1
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
User memory needs to be zeroed out before it is sent to the user.
To do this, the kernel maps the page, memsets it to zero and then
unmaps it. By virtue of mapping it, this forces us to flush the
dcache to ensure cache coherency between kernel and user mappings.
Originally, the page_alloc loop was using GFP_ZERO (which does a
map, memset, and unmap for each individual page) and then we were
additionally calling flush_dcache_page() for each page killing us
on performance. It is far more efficient, especially for large
allocations (> 1MB), to allocate the pages without GFP_ZERO and
then to vmap the entire allocation, memset it to zero, flush the
cache and then unmap. This process is slightly slower for very
small allocations, but only by a few microseconds, and is well
within the margin of acceptability. In all, the new scheme is
faster than the default for all sizes greater than 16k, and is
almost 4X faster for 2MB and 4MB allocations which are common for
textures and very large buffer objects.
The downside is that if there isn't enough vmalloc room for the
allocation that we are forced to fallback to a slow page by
page memset/flush, but this should happen rarely (if at all) and
is only included for completeness.
CRs-Fixed: 372638
Change-Id: Ic0dedbadf3e27dcddf0f068594a40c00d64b495e
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
This reverts commit 5462907f72ddcc7d337ce9431d54428c8dc486eb.
Change-Id: Ibe6ec76096838b71e43cd9beac1f57ff179c5008
Signed-off-by: Sunil Joseph <sunilj@codeaurora.org>
Update the call sites of cpu_is_msm8930() to include checks
for the MSM8930AA() variant. Relevant drivers will be
updated for more driver-specific specific MSM8930AA checks
at a later time.
Change-Id: Iff1af7a5454ec56c40390682ce2b4b6d1d325c91
Signed-off-by: Stepan Moskovchenko <stepanm@codeaurora.org>
Add EXPORT_SYMBOL to specific functions that should be
allowed to be used from within kernel modules.
Change-Id: I666c1f1a18d5af58906cb2185c5cf29d8596013f
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Per revised design decisions, cpu_is_msm8930() shall only
return true on 8930, and not on the 8627 variant. Modify
the cpu_is_xxx functions to reflect this change, and update
call sites accordingly.
Change-Id: I50b943f80c731717e6cd5d7fffb13aeec0f85a40
Signed-off-by: Stepan Moskovchenko <stepanm@codeaurora.org>
last_expired_ctxt_id was used briefly for per context timestamps and now
can be removed as it is no longer used.
Change-Id: I51d3947fe6da366824d1b947ae3448bee3518477
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
There are two head switches for GPU, one is for the GPU core
itself, the other is for surrounding digital circuits. KGSL
needs to control them both.
The turn on sequence is:
1. Turn on GPU core
2. Turn on digital circuits
The turn off sequence is:
1. Turn off digital circuits
2. Turn off GPU core
Two regulator (actually head switch) names "vdd_dig" and "vdd"
are created in device tree file, representing digital
circuits and GPU core head switches respectively. In struct
kgsl_pwrctrl, adding a new regulator struct "gpu_dig"
which corresponds to vdd_dig. And the existing struct "gpu_reg"
corresponds to vdd.
Change-Id: Idb05caf547ad148885adb9cf498c47635067b70b
Signed-off-by: Pu Chen <puchen@codeaurora.org>
Check that the ops pointer is valid before using it, otherwise we
may fault.
Change-Id: I7f1420638eca53f6d85b8c3ff114033ce77cb408
CRs-fixed: 360810
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Remove unneeded idle commands from the command stream when updating
pagetables and doing TLB flushes. Ths cuts down on the number of
commands in the ringbuffer during switches.
Change-Id: I8f1fa31db33c465ecdbd0ca897f4651f2670c4e4
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
In case of IOMMU memory needs to only be flushed when it is
unmapped and not mappped. Also, when an application is running
the number of unmaps would in most cases be less than number of
maps, so this way the number of flushes wil be fewer.
Change-Id: Id4107631c343045edbc744125035202844405910
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Check global GPU status registers in intervals of 2 seconds,
if the registers did not change after 2 seconds trigger a hang.
When there is an actual GPU hang and GPU is stalled, this reduces
the current hang detection time from 10 seconds to 2 seconds.
Faster GPU hang detection reduces the overall time to recover
from GPU hang.
Change-Id: If432ccacc1b77b4fc7f08b756886bd3ae5edf04f
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
A busy loop is executed when we allocate space from the ringbuffer.
If the GPU is hung then we can wait indefinitely in the busy loop.
Add a wait time to this busy loop, and if the wait time elapses then
report a GPU hang instead of waiting indefinitely.
Change-Id: I035c39063cbfa25380702720f929df5319e73e61
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Pull in Ion patches for 3.4 upgrade. Inclues the following
patches from google:
commit 7191e9ba25
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Mon Jan 30 14:18:08 2012 -0800
ion: Switch map/unmap dma api to sg_tables
Switch these api's from scatterlists to sg_tables
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 6f9e56945d
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Tue Jan 31 09:40:30 2012 -0800
ion: Add reserve function to ion
Rather than requiring each platform call memblock_remove or reserve
from the board file, add this to ion
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 9ae7e01de1
Author: KyongHo Cho <pullip.cho@samsung.com>
Date: Wed Sep 7 11:27:07 2011 +0900
gpu: ion: several bugfixes and enhancements of ION
1. Verifying if the size of memory allocation in ion_alloc() is aligned
by PAGE_SIZE at least. If it is not, this change makes the size to be
aligned by PAGE_SIZE.
2. Unmaps all mappings to the kernel and DMA address spaces when
destroying ion_buffer in ion_buffer_destroy(). This prevents leaks in
those virtual address spaces.
3. Makes the return value of ion_alloc() to be explicit Linux error code
when it fails to allocate a buffer.
4. Makes ion_alloc() implementation simpler. Removes 'goto' statement and
relavant call to ion_buffer_put().
5. Checks if the task is valid before calling put_task_struct() due
to failure on creating a ion client in ion_client_create().
6. Returns error when buffer allocation requested by userspace is failed.
Signed-off-by: KyongHo Cho <pullip.cho@samsung.com>
commit 043a61468f
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Wed Feb 1 11:09:46 2012 -0800
ion: Switch ion to use dma-buf
Ion now uses dma-buf file descriptors to share
buffers with userspace. Ion becomes a dma-buf
exporter and any driver that can import dma-bufs
can now import ion file descriptors.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 0d1259b5f8
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Mon Apr 30 16:45:38 2012 -0700
gpu: ion: Use alloc_pages instead of vmalloc from the system heap
With this change the ion_system_heap will only use kernel address
space when the memory is mapped into the kernel (rare case).
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit be4a1ee79a
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Thu Apr 26 20:44:10 2012 -0700
gpu: ion: support begin/end and kmap/kunmap dma_buf ops
These ops were added in the 3.4 kernel. This patch adds support
for them to ion. Previous ion_map/unmap_kernel api is also
retained in addition to this new api.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 46d71337f9
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Mon May 7 16:06:32 2012 -0700
gpu: ion: Allocate the sg_table at creation time rather than dynamically
Rather than calling map_dma on the allocations dynamically, this patch
switches to creating the sg_table at the time the buffer is created.
This is necessary because in future updates the sg_table will be used
for cache maintenance.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
commit 903f6c716d
Author: Rebecca Schultz Zavin <rebecca@android.com>
Date: Wed May 23 12:55:55 2012 -0700
gpu: ion: Get an sg_table from an ion handle
This patch adds an interface to return and sg_table given a
valid ion handle.
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
The commit also includes fixups needed for MSM specific code.
Change-Id: Idbcfa9d6af0febf06f56daaa6beaa59cc08e4351
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Add a function which adds an event to disable the iommu clocks
based on a timestamp. If timestamp elapses then the event function
will disable the iommu clocks. Remove the disabling of iommu clocks
from the power control code.
Change-Id: Ic1263771f8c532131059283dde05425a73554974
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Make sure that the busy times are calculated at the end of the
current sample and before the next sample starts. This prevents
false alarms of busy timer events being triggered. Applies only
to devices with idlestats power policy.
Change-Id: I1e28fff4128b72c25e8073d845d519867d67a9ca
Signed-off-by: Ranjhith Kalisamy <ranjhith@codeaurora.org>
Add TLB flush for each submit to prevent target freeze
in case of 8x25/7x27a/7x25a targets.
Change-Id: I350e72e4f71fef86c7cf9da0954ef65d5765b88f
Signed-off-by: Rajesh Kemisetti <rajeshk@codeaurora.org>
The pmem_kalloc APIs are now deprecated. A reference to
an old pmem memtype still exists in kgsl code even though
it isn't used. Remove it.
Change-Id: I9b8bd2037f1a49dc543fc766593395fc440b1318
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
SLEEP & SLUMBER should both wake at default power level on targets w/
GPU DCVS. Combine the calls to make this possible.
CRs-Fixed: 369927
Change-Id: I386a9ab04850b0beb249b47398ad623f91fbbfda
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>
We do not need to do a soft reset of the A3XX GPUs after power on
as all the interesting registers appear to be in a reasonable power
on state. This saves 20ms during suspend/resume.
CRs-Fixed: 369927
Change-Id: Ic0dedbad367e8b402aefbf6e7746ae5b17120ac8
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The IOMMU registers should only be accessed by priv bank. Create a separate
pagetable for priv bank and map the IOMMU registers to only this pagetable.
This is only supported for devices which can use the priv bank for
performing in stream pagetable switches.
Change-Id: If9edada5d3a05e9a660c126139e18091b1ab43cc
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Add support to allow ion_cp_heap.c to secure according to new
requirements. This includes adding versioning to the secure
calls to allow other types of securing to take place.
Change-Id: I65e07ebaeefa1d0572b6531753a707a28284aa0d
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
CL contexts turn off the VPC block while running to avoid a hang. The
VPC block needs to be turned back on before any GL calls. This includes
GMEM restore, so set the clock control value back to the original default
before doing any other operations.
CRs-fixed: 363922
Change-Id: Ic0dedbad6b031ccd9c0816013b75ba9effb18fcb
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
When an axi error happens and the GPU is using IOMMU then a warning stack
trace was being reported. This is because during axi error interrupt
handling the IOMMU clocks were being turned on to read the current
pagetable base address. Switching the IOMMU clocks from interrupt context
is not allowed. Fix this by reporting the axi error and displaying 0
as the pagetable base address since we cannot read its value from
the axi error interrupt code.
Change-Id: If98538bc1b2df51a1b3632faf449d5d332bccc3e
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
In the IOMMU heap, the return value of msm_allocate_iova_address
needs to be checked instead of the value of the iova address.
Check the return value instead.
Change-Id: I4554515d17c926a7dd738824aff21eb65d2ba56a
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
When GMEM save is not performed, the events to reuse shader space
can get dropped. This make the CP block to wait indefinitely
causing a hang. To unblock this, issue a packet to set shader bases.
This applies only to A20x/220 GPU.
CRs-fixed: 365465
Change-Id: I3ea6a58821b238706f167845380370d5bd19c4b3
Signed-off-by: Ranjhith Kalisamy <ranjhith@codeaurora.org>
Reference count the calls to ion_{un}secure_heap. The secure SCM
call will only be made when the count goes from 0 -> 1 and the
unsecure call will only be made when the count goes from 1 -> 0.
Change-Id: Ia221b79782c9223d9853ef00c2848e1ab7a73091
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The next generation of content protection has separate SCM
calls into TZ. Add these functions in a dedicated file so they
can be used by any Ion heap.
Change-Id: I197130f6a5fc33efcd6871b0f7c362fa99f18d20
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
kgsl_core_exit() should not call kgsl_sharedmem_uninit_sysfs() and
device_unregister() if those resources have not been allocated.
We check this by seeing if kgsl_driver.virtdev has been
populated or not.
CRs-fixed: 359477
Change-Id: Ic040667b29bd4e006e6f29a50809d94a2bb4b469
Signed-off-by: Harsh Vardhan Dwivedi <hdwivedi@codeaurora.org>
Dump the shader memory (working memory and shadow memory) for A3XX GPUs
in the snapshot.
Change-Id: Ic0dedbadf28733b96323a680b014dbea10b1ceda
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Update the snapshot header to version 2 to return the raw chipid
in the dump.
Change-Id: Ic0dedbad82bbe61395924bf49cb0144e5e2e10e3
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
There are some subtle differences between revisions of GPUs
that are not reflected in the GPU ID value. Return the raw
chipid (derived from the hardware) along with the GPU ID
when queried.
Change-Id: Ic0dedbad035a80726e6fedc066ec11b78b79da2b
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The code seeking out the failing frame for a hang got confused if
the CP read and write pointers were equal resulting in missing
IBs.
Change-Id: Ic0dedbadb32e4a708d0f8ee9e8a853fdfaff10b0
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Use a iterator to cleanup how snapshot GPU objects are printed.
This removes a ton of ugly math, and allows us to do subtle
silly things like add padding at the end of a non dword
aligned object.
Change-Id: Ic0dedbadabeee50f42fe9be62ea0b7dfc1f333da
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Enhance the logic that checks to see if the CP_RPTR is stuck
mid packet. Ensure that when we do packet counting that we
use real type3 packets instead of regular data pretending to
be a type3 packet.
Change-Id: Ic0dedbadb502fa2790df0c6752309da300d6d1ae
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Currently we just check the most significant two bits of a dword
in the ringbuffer or IB are set to decide if the data is a type3
packet header. This has proven to be problematic, as it turns out
that the top two bits are often set in nominal data. Enhance
the type3 packet check to verify that various reserved bits in
the dword are 0 as expected.
Change-Id: Ic0dedbada5eff8353bc64da9eefd9919de127fa4
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
User side drivers are starting to use more indirect buffers
for CP_LOAD_STATE, so expand the snapshot to support grabbing
all of the possible combinations.
Change-Id: Ic0dedbadff743d03b5accb2a0299ab5dc0bc1a61
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
If there is a failure in drawctxt create, then the KGSL core will call
drawctxt destroy to free the memory. If the failure happened soon
enough, there might not a drawctxt to free.
CRs-Fixed: 371852
Change-Id: Ic0dedbad629090ae1827e354b9b94bbd3c5efcdd
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
This reverts commit 609467077ae6b794af47613e05949e8b2fb3a842.
Change-Id: I457634982603c957788b749198219d0974bd1de5
Signed-off-by: Zhoulu Luo <zluo@codeaurora.org>
Conditional interrupts were not getting fired at the correct time when
using per context timestamps. When a context was waiting on a timestamp,
the interrupt was being fired on the global timestamp rather than the per
context timestamp. Now the interrupt is fired on the per context
timestamp.
CRs-Fixed: 340620
Change-Id: I8393ca74aabfe4b711b61efb023d38d6ab852eec
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
Change prevents IOMMU pagefaults which may occur when
indexing reaches 4096/sizeof(kgsl_devmemstore) number
of contexts
CRs-Fixed: 370594
Change-Id: If726c2a1fcf7667b062e890453a71e95f5417c60
Signed-off-by: Richard Ruigrok <rruigrok@codeaurora.org>
Use one code path to dereference shared variables. Simplify code
for readability.
Change-Id: Ifcfe948b7c04a6594c3fa9eac4afea81f65e59af
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
Make a3xx ftrace always print timestamps in hex rather than
sometimes in hex and sometimes in decimal.
Change-Id: I6b1709458afd9361fd04bab06c1a4b22aaadd082
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
We need to add a draw workaround command sequence after every 50 context
switches if the gmem has not been saved in those 50 switches. This
workaround is required to prevent the GPU from deadlocking. Since gmem
save commands have draw calls in them we only need to execute this
workaround if the gmem has not been saved. The number 50 was chosen
empirically.
CRs-fixed: 362302
Change-Id: I1a659c85736c47fe6f45030b20ce7a12851b0815
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
In msm8930 V1.2, there are few HW fixes added for A305 GPU,
communicate new chipid to userspace driver to remove SW workarounds.
Change-Id: I51f272a39cb2cc60541607af9fa9be70cc838cb3
Signed-off-by: Sudhakara Rao Tentu <srtentu@codeaurora.org>
Get the current pagetable being used by the GPU by reading the
pagetable register during iommu pagefault and use that base
address to get the pid of the process corresponding to the
pagetable. Earlier we were using the pagetable address that the
IOMMU driver thinks is presently set for the GPU which may not
always be the case because we change the pagetable used by the
GPU without informing the IOMMU driver.
Change-Id: I7cf788f5d0f9454deb139ed3ad75bb4e750f4772
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Moves fixed area start address down, so that non secure
video firmware heap and secure video firmware area overlap.
Both use the region in a mutually exclusive manner.
ALso introduce a hole, that can be used by MM heap
lock down for content protection.
Change-Id: Ic240377da25e00423742ccb73dbb60a1ff037e37
Signed-off-by: Deepak Kotur <dkotur@codeaurora.org>
[sboyd: drop board file changes]
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Instead of using custom routines for cache operations use the
standard operations for cache flush, cache clean, and cache
invalidate. In addition only call outer cache routines if
outer cache is available.
Change-Id: Idb51bf7348fd526ebdea1e6ff2e366b8a3b4de33
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
