After all the current events are processed in the main loop, the next
pending event is marked to fire an interrupt to start the cycle anew.
When the event is marked, we check the timestamp again to make sure that
we are always caught up with the GPU. If the timestamp for the next
event happens to be retired when we get to this point, we silently leave
the function.
If there are no other subsequent events or waittimestamp requests, then
it is possible that another interrupt might never be fired for that context
and then the event gets missed. Return a 1 in this case all the way back
up the stack and make the event code keep processing events until we are
positive that none are past.
Change-Id: I5a770e7f06612e1aecbfe2e8ccbfeebdab3fc9fc
CRs-Fixed: 441191
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Instead of sending the currently expired timestamp to expired
events, send the timestamp that the event expected. This avoids
confusion for events that don't compare the actual timestamp
to the one they expected. Canceled events send the current
timestamp when they were canceled.
The kgsl_sync handler wasn't prepared for the second case so
add a check to prevent strange things from happening on the
timeline when a event is canceled.
Change-Id: Ic0dedbad3681db6222720884688dcddb9b06bda2
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Add reference counting to all KGSL events since they are dependent
on the context. This avoids a race condition where the context
could be destroyed while the events are being processed during cleanup.
Change-Id: Ifcd9655a6335d22c01f2606a0fbc5346d9a0314f
CRs-fixed: 438134
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
Move the event accounting from a master list on the device to individual
context lists. This makes the management code more simple and reduces
the amount of extra cycles taken trying to walk a single largish list.
The new added code makes the event code slightly bigger than kgsl.c can
bear. As events become more commonplace this code will be more important
and it will be easier to maintain in a dedicated file so move all the event
related code into kgsl_events.c
Change-Id: Ic0dedbadf00b96eee53aea1d26acb195f1a916ce
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Event based as well as interruptible based timestamp checks
use the same code, they differ on if the mutex is held or not.
Separate the common code out to avoid confusion.
Change-Id: Ie2ee01d7a07ff2ba7755ab8d327796345f9ca9ea
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
Signed-off-by: Rajeev Kulkarni <krajeev@codeaurora.org>
Stop parsing an indirect buffer if an error is encountered (such as
a missing buffer). This is a pretty good indication that the buffers
are not reliable and the further the parser goes with a unreliable
buffer the more likely it is to get confused.
Change-Id: Ic0dedbadf28ef374c9afe70613048d3c31078ec6
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The snapshot timestamp is currently based on wall time which makes
it very difficult to debug against a kernel log based on boot time.
It doesn't help matters that the snapshot timestamp is output in
human unfriendly hex. Use boot time instead of wall time for the
snapshot value and print it out in decimal.
Change-Id: Ic0dedbadf9b7d799a3cb4d5917787bccd350c90a
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
When taking a snapshot from a ramdump it is usually important that
we get the last IB2 that was executed so if we find it during
parsing put it in the static snapshot region as opposed to the
dynamic one.
Change-Id: Ic0dedbad878b961ebe29afe9145443e9bfd0bc6a
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
If there are a lot of sequential commands in the ringbuffer for the
same context there is a high probablity that some of the memory
for the older commands has been reused and so the contents may not
be reliable. Increase robustness by making sure we don't parse the
same IB over and over again and be more tolerant of errors and
strange sizes.
Change-Id: Ic0dedbade8504ea258ceea40593cb75e7d6211c8
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
When debugfs entity "ft_user_control" is set, it enables userspace to
control GPU fault tolerance policy. By default it is disabled.
Change-Id: Ia93ccf6d9595f6e7cda71863f9067819aaa0a68d
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
GPU pagefault policy can be set to below options:
a) enable/disable GPU HALT on pagefaults
b) log only one pagefault per page.
GPU pagefault policy can be controled using panel file
and debugfs.
Change-Id: I0613e0d087fc45eb5e519e1c834a33f32376672f
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
Log only one pagefault error per page. If we log all the
pagefault errors it causes lot of messages in kernel logs
and sometimes leads to watchdog timeouts.
Change-Id: If8ee4ee28f6289277eb98d4d8ba73dfda6fc1133
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
If userspace requests no fault tolerance using context flag
KGSL_CONTEXT_NO_FAULT_TOLERANCE, do not attempt denial of
service detection and do not attempt fault tolerance when
a hang is detected.
Change-Id: I5de0e4ae6e8407365725c30fcb0e00a601460ebc
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
Allow userspace to control fault tolerance policy,
this allows fault tolerance policy to be controlled
using panel file.
Change-Id: I991edf6f082384bc69454058fe5df3b5f535aa4c
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
Full postmortem logs take 600ms to 1sec and this affects
fault tolerance time. Reduce postmortem logs and print
only necessary information. Full postmortem logs can
be enabled using debugfs.
Change-Id: I9039746bbdcf67ceebc02f21e84ab873531035a5
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
GPU fault tolerance should take minimal time,
set the GPU clock to max freqency to acheive this.
When we set GPU clock to max freqency we vote for
high CPU clock and bus bandwidth.
Change-Id: I98a3f7c24f0939c08f9b12257817d17414e3f128
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
When expired global timestamp is not progressing,
check if an IB is talking more than 2 seconds
with no update in GPU read pointer, IB1 base
IB1 size, IB2 base, IB2 size and detect it as a
long running IB. Invalidate the context of the IB
and do not attempt fault tolerance on this context.
Change-Id: I7e9780f5aed73a5c8807069aa506bec16a1576b4
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
Store process id and name in context structure, this
information is used to print the process name when
when we detect a error in the driver, example
GPU fault tolerance, commands from bad context
in issueibcmds.
Change-Id: I6acd63933754fdec80d8274d3e1380ff2c23770e
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
Renaming recovery to fault tolerance and modifying
the functions and log messages accordingly.
Change-Id: I5f249806026ac514c4aff7da45c3a4e8cc2f8c34
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
Recovery algorithm is changed to
step 1: retry the same commands that hung the GPU
step 2: if step 1 fails, nop just the IB that hung the GPU
and retry
step 3: if step 2 fails, skip commands in current context
till end of frame and retry
step 4: if step 3 fails mark context as bad and execute
the remaining commands from good contexts.
Previously we used to return -EDEADLK when recovery succeeds,
this is the same error code if the context is not recoverable.
With new policy if recovery succeeds we return -EAGAIN so that
userspace treats recovered context differently from the ones
that are not recoverable. If recovery fails we mark the context
as bad and return -EDEADLK.
Change-Id: I9fa3c40801964186866b6002e62f19cf6aa41361
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
When a hang happens locate the index in ringbuffer from where replay
of recovery needs to be re-tried during initialization of recevery.
If the hang detection is false then hang reported by postmortem and
snapshot may be different, but recovery will always start from the
index obtained during initialization. Also, this allows us to capture
data into the snapshot that is in line with the point from where
recovery occurred.
Change-Id: Ie3877f0c81dad2ede888db7cb7d3ff13e2761763
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
In current design waittimestamp can trigger hang detection
in less than 50ms. Check if hang detection was called less than
50ms ago, if true do no attempt hang detection.
Change-Id: Iaeb2501316c4194957fbe211b6023eeb869bd0d5
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
Return error and do not increment ref count in case of failure to
protect the CP heap.
CRs-Fixed: 445400
Change-Id: Ia615773fe2c52386d9522de7a182edd633a66e1c
Signed-off-by: Praveen Chavan <pchava@codeaurora.org>
As a performance optimization, use higher-order page allocations if
possible. This can reduce TLB misses and the total number of page
structures.
CRs-Fixed: 449035
Change-Id: I6b76ec69599a100fd7209161c4286284ae347be0
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
As a performance optimization, omit the __GFP_ZERO flag when
allocating individual pages and, instead, zero out all of the pages in
one fell swoop.
CRs-Fixed: 449035
Change-Id: Ieb9a895d8792727a8a40b1e27cb1bbeae098f581
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
If the page_alloc loop fails, we changed the length of the scatterlist
for the memdesc to the last good page so we could have an efficient
loop through kgsl_sharedmem_free. Unfortunately, we were also using
the sglen to decide between using kmem or vmalloc memory for the sg
list in the first place. There is a chance that one could start out
with a large sg list from vmalloc, but if it failed early, then we
could truncate the size and attempt to kfree it.
Zero the stlist at init time and use a length == 0 as an indicator
that the list might be truncated.
CRs-Fixed: 443441
Change-Id: Ib48a08c69a7bbfc69b92f09e83fc80f9bdedffe0
Signed-off-by: Dhivya Subramanian <dthiru@codeaurora.org>
In a few spots the ADRENO_IDLE_TIMEOUT value (measured in milliseconds)
was used directly for jiffy calculations resulting in timeouts that were
1000/HZ times longer than they should have been. Use msecs_to_jiffies
for these situations.
CRs-Fixed: 448226
Change-Id: Ic0dedbad33630706d909931bf616bd69399f7ff1
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The kgsl_mmu_pagefault event is the only ftrace event with
it's format field being multiline. All other ftrace events
in the kernel have the format field in a single line.
Make the kgsl_mmu_pagefault event format consistent with
all other ftrace events.
Change-Id: I99ab00f6319ae13a6f99a31bc11511d05835b50c
Signed-off-by: Pushkar Joshi <pushkarj@codeaurora.org>
Add minimum amount of wait period in adreno_waittimestamp
to avoid instances of malformed wait period (such as msecs=0)
arguments from userspace and in turn avoid false hang detections.
CRs-fixed: 437738
Change-Id: I5cd40dc7589747e7fffeb6d5cc2001b73d6d845e
Signed-off-by: Harsh Vardhan Dwivedi <hdwivedi@codeaurora.org>
All userspace consumers should now be using IOCTL_KGSL_GPUMEM_ALLOC,
so we can safely remove this kludgy and deprecated code. The structs
and #defines are left in the header to avoid breaking userspace and
to serve as a reminder of the way we were, but any use of
IOCTL_KGSL_SHAREMEM_FROM_VMALLOC will return -ENOTTY.
Change-Id: Ic0dedbad1460acfe28116ee2dbd75faa52b76857
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The most important thing for the memory allocator is to not crash
or otherwise behave badly during low memory situations. Adding
GFP_NORETRY keeps the OOM killer from being invoked making the 32MB
padding on the allocation redundant. Remove the padding, add the flags
and go down the proper error path when alloc_pages() fails.
Change-Id: Ic0dedbadaf86e248b372b22cd3a41a5dabedb82e
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
We had a ugly mix of flags from usersapce mixed in with internal flags
and the flags were stored in priv and the internal flags were stored
in flags and general confusion all around.
So, external (user) flags moved to memdesc->flags, internal bits moved
to ->priv and now we are ready to handle the full onslaught of user
specified flags.
Change-Id: Ic0dedbad2236ae353b31e5dcaa6ff551713abaf1
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rajeev Kulkarni <krajeev@codeaurora.org>
Fast hang detection algorithm is improved to use additional
performance counters to monitor shader processor activity.
Shader processor's active alu cycles, icl0 misses
and fs cflow instructions are added to list of activities
monitored for fast hang detection.
Change-Id: Ie74b2ca2d8eb587dbdae40f8fafd901e71f50ddb
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>
Make sure the specified context_id returns a valid context
in adreno_gpummu_setstate and adreno_iommu_setstate and
silently return if it doesn't.
CRs-Fixed: 439367
Change-Id: Ic0dedbad1a91b53f19782b50380bd36842614351
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
This is Iee3db5401bdad1808a09d104e0daecfc3716cbb4 change ported
manually from the msm-3.4/msm-3.7 branch to the jb_2.5 branch.
The current code wakes up gpu on all kgsl_ioctl() calls that need
to hold device->mutex. However there are some kgsl_ioctl() calls that
don't need that. That fix introduces a bit of info to mark such
functions, and don't wake up gpu calling them to save some power.
Change-Id: I86224d2405c42e3803c7e361c02fe1d02d4cdff1
CRs-Fixed: 438932
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
Always mark GPU contexts that have cause a GPU hang as guilty.
If the GPU context recovered successfully, still mark it as guilty.
This will inform applications that something might have
gone wrong and to take actions to ensure everything is correct.
Also, swap out any active context following a hang. This will
ensure that any new submissions will force the context to be restored
before it is used.
CRs-fixed: 427603
CRs-fixed: 405434
Change-Id: I6b7cf2b2a680a4691f45535a147ca678a4d961f3
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
Previously, a parameter was added to secure heaps to only
allow secure allocations. Some clients still require
non-secure contiguous allocations even though the SMMU is
supported. Add support for those clients to force contiguous
allocations to go through.
Change-Id: I0d18d8a9ed3ab267b55c84c4ad160f8cec2c2b32
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Add reference counting to all KGSL events since they are dependent
on the context. This avoids a race condition where the context
could be destroyed while the events are being processed during cleanup.
CRs-fixed: 438134
Change-Id: Ic2cd484d441e4c392ab63b0a0131accaceafc711
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
When trying to add commands to the ringbuffer command stream, ensure
that the context point is not NULL before dereferencing it.
Change-Id: I6d8217ed4b180454eb43f2db161c10e86bc207da
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
Rather than ignore some categories of idle events,
bin the events with small timelines and process all
events with the GPU DCVS algorithm.
CRs-Fixed: 427559
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>
Change-Id: I1a5e8d8af6020a08cf95a9daa934e825bc5f2854
Due to some mismatch in applying patches, one patch was overriding
the changes added by commit bd5c9fcbf79d326817312138cf03298adf09df27.
This patch restores the changes.
Change-Id: I43223f01653c53df00682e8489b89e8978989944
Signed-off-by: Rajeev Kulkarni <krajeev@codeaurora.org>
We have always allowed the user to specify a timeout in waittimestamp
but we never obeyed it. Reorgaize the function to be less convoluted
and support a user specified timeout (or spin forever if 0 is passed).
Change-Id: Ic0dedbadbc368336d05b14d66829205631046515
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rajeev Kulkarni <krajeev@codeaurora.org>
Sometimes a write to CP_WPTR fails to latch on A20x GPUs. In the past
we worked around this by writing to WPTR numerous times in the various
loops that waited for GPU activity. Consolidate these disparate calls
by moving the poke into the hang detect function. If the core is idle
but the RPTR != WPTR, then rewrite the WPTR and go back through the
loop again.
Change-Id: Ic0dedbad474c7884c2467fe050d8afe673588add
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
If the userspace calls the waittimestamp ioctl with an invalid
timestamp for a context then set a flag indicating that the ioctl
was called with an invalid ts. If the flag is set then do not
print error message about this ioctl being called with an invalid
timestamp. This is required to prevent the kernel log from
spamming with error messages and causing a watchdog.
CRs-fixed: 374586
Change-Id: Iffa5c13d74ed90b78f88aba5c4c0e0f908eeaa19
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Signed-off-by: Rajeev Kulkarni <krajeev@codeaurora.org>
Use timestamp from userspace instead of incrementing context's last
issued timestamp. This allows userspace to defer submissions to KGSL.
Change-Id: I2a104f97420f73dac3bbd23464e92e6ea9d92869
Signed-off-by: Vijay Krishnamoorthy <adivarah@codeaurora.org>
Signed-off-by: Rajeev Kulkarni <krajeev@codeaurora.org>
Check to make sure that there is a valid context pointer before
dereferencing it when adding commands to the ringbuffer.
Change-Id: Icf9824de8f62a28cdeeef4378246d9fb334d39fa
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>
Use BIT to define all context related flags. This ensures
that these flags are unique and is easier to maintain. Also,
fix spelling of CTXT_FLAGS_BEING_DESTOYED to
CTXT_FLAGS_BEING_DESTROYED
Change-Id: I866c67b9c5d59d6117e31714756af3106018f9cb
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
When we map ion buffers as uncached we want to map them as
uncached but still be able to use buffering of transactions
since turning off buffering has a severe performance impact.
Change-Id: I5a09ac34bd3ce3f001d10d9efd71d140f632564d
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
When allocating pages for IOMMU heap it is advantageous
to try to allocate pages from highmem in addition to
lowmem since clients using the IOMMU heap can use
either highmem or lowmem.
CRs-fixed: 430457
Change-Id: Ifc794fc344c67dd53d8f7cebf86536ab9d44ed6c
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
