You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-06 - 00:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Jul 05, 2026 12:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
51251899812,4cyclictest131rcu_preempt08:30:032
512511997668,6cyclictest12-21ksoftirqd/007:45:030
512511997034,5cyclictest131rcu_preempt11:20:020
51251199701,51cyclictest131rcu_preempt12:30:420
51251199701,51cyclictest131rcu_preempt12:30:420
512511996959,7cyclictest12-21ksoftirqd/009:25:030
51251199686,49cyclictest0-21swapper/008:20:010
512511996731,16cyclictest539411-21unin-run09:55:010
512511996718,43cyclictest0-21swapper/012:00:010
512511996662,2cyclictest0-21swapper/011:40:030
51251199657,10cyclictest131rcu_preempt09:40:010
51251199646,27cyclictest131rcu_preempt10:39:230
51251199636,14cyclictest583528-21grep11:30:010
512511996343,15cyclictest131rcu_preempt08:50:010
512511996311,39cyclictest131rcu_preempt10:02:460
512518996242,9cyclictest518454-21kworker/2:109:59:592
512511996233,8cyclictest131rcu_preempt09:17:190
51251199623,20cyclictest3467602-21systemd-journal11:44:590
512511996226,11cyclictest12-21ksoftirqd/011:14:180
512511996212,41cyclictest12-21ksoftirqd/010:56:270
512511996212,41cyclictest12-21ksoftirqd/010:56:260
512511996026,4cyclictest131rcu_preempt12:20:030
512516995922,26cyclictest23-21ksoftirqd/107:10:011
512511995948,8cyclictest12-21ksoftirqd/011:47:140
512511995917,5cyclictest131rcu_preempt12:40:000
51251199591,5cyclictest131rcu_preempt10:07:250
51251199584,4cyclictest131rcu_preempt10:52:170
512511995827,14cyclictest551430-21ssh10:20:290
51251199582,48cyclictest606909-21cron12:20:000
51251199573,50cyclictest12-21ksoftirqd/008:30:020
512511995730,23cyclictest611131-21ssh12:28:400
512511995729,20cyclictest275-21dbus-daemon08:00:010
512511995713,7cyclictest131rcu_preempt07:55:010
512516995634,13cyclictest549677-21ssh10:17:001
512516995634,13cyclictest549677-21ssh10:16:591
51251699561,12cyclictest571941-21ssh11:05:011
512511995643,8cyclictest12-21ksoftirqd/011:01:030
512511995628,12cyclictest555629-21ssh10:30:010
512511995628,12cyclictest555629-21ssh10:30:000
51251199558,7cyclictest131rcu_preempt09:29:320
512511995528,5cyclictest131rcu_preempt10:11:200
512511995528,11cyclictest540206-21ssh09:56:270
512511995515,35cyclictest12-21ksoftirqd/010:35:010
512511995434,6cyclictest16850irq/134-800000011:20:180
512511995432,12cyclictest3467602-21systemd-journal07:19:590
51251199543,14cyclictest3467602-21systemd-journal10:50:000
512511995428,11cyclictest599548-21ssh12:04:010
512511995427,23cyclictest12-21ksoftirqd/012:05:200
512511995426,12cyclictest518712-21ssh09:10:020
512511995420,6cyclictest12-21ksoftirqd/009:41:530
512511995420,6cyclictest12-21ksoftirqd/009:41:520
512511995410,5cyclictest131rcu_preempt10:16:570
512511995410,5cyclictest131rcu_preempt10:16:560
51252199535,4cyclictest131rcu_preempt07:30:023
51252199535,4cyclictest131rcu_preempt07:30:013
512518995345,5cyclictest518454-21kworker/2:112:00:012
512518995344,5cyclictest581225-21kworker/2:311:34:502
512518995323,9cyclictest516367-21kworker/2:208:50:002
51251699535,3cyclictest131rcu_preempt07:30:011
51251699535,3cyclictest131rcu_preempt07:30:001
51251699531,9cyclictest275-21dbus-daemon08:45:011
51251199539,9cyclictest12-21ksoftirqd/011:33:130
512511995334,15cyclictest16850irq/134-800000011:06:500
512511995328,18cyclictest275-21dbus-daemon07:15:000
512511995328,17cyclictest3467602-21systemd-journal09:00:020
512511995328,17cyclictest3467602-21systemd-journal09:00:010
512511995326,17cyclictest3467602-21systemd-journal07:40:000
51252199522,5cyclictest532468-21munin-run09:40:003
512518995243,6cyclictest581225-21kworker/2:311:43:232
51251699529,6cyclictest131rcu_preempt11:15:001
51251699527,4cyclictest131rcu_preempt10:45:021
512511995243,7cyclictest12-21ksoftirqd/012:10:500
512511995238,10cyclictest12-21ksoftirqd/008:35:030
51252199512,3cyclictest131rcu_preempt12:02:073
512518995143,5cyclictest581225-21kworker/2:312:23:332
512518995143,5cyclictest516367-21kworker/2:209:36:312
512518995143,4cyclictest518454-21kworker/2:110:12:482
512518995141,7cyclictest581225-21kworker/2:312:05:502
512518995141,7cyclictest518454-21kworker/2:110:05:572
512518995141,6cyclictest581225-21kworker/2:312:10:592
512518995135,12cyclictest1-21systemd09:55:012
512516995135,12cyclictest1-21systemd11:40:001
512511995131,11cyclictest530120-21modprobe09:35:010
512511995131,11cyclictest530120-21modprobe09:35:000
512511995118,14cyclictest0-21swapper/007:35:020
512518995042,5cyclictest505360-21kworker/2:307:40:012
512518995040,6cyclictest518454-21kworker/2:111:25:012
512518995040,6cyclictest505360-21kworker/2:308:15:002
512516995031,15cyclictest1-21systemd12:30:011
512516995010,8cyclictest589796-21sshd11:43:171
51252199495,4cyclictest131rcu_preempt12:16:143
51252199494,3cyclictest131rcu_preempt09:33:433
51252199494,3cyclictest131rcu_preempt09:33:433
51252199491,3cyclictest131rcu_preempt09:10:053
512518994942,4cyclictest518454-21kworker/2:110:35:442
512518994942,4cyclictest516367-21kworker/2:211:03:392
512518994941,5cyclictest581225-21kworker/2:312:03:592
512518994941,5cyclictest518454-21kworker/2:111:17:312
512518994941,5cyclictest518454-21kworker/2:110:04:222
512518994941,5cyclictest518454-21kworker/2:109:30:382
512518994941,5cyclictest518454-21kworker/2:109:30:382
512518994941,5cyclictest516367-21kworker/2:209:11:152
512518994941,5cyclictest505360-21kworker/2:307:56:172
512518994940,6cyclictest518454-21kworker/2:109:47:302
512518994940,5cyclictest581225-21kworker/2:312:30:202
512518994940,5cyclictest581225-21kworker/2:312:30:192
512518994940,5cyclictest581225-21kworker/2:311:50:032
512518994940,5cyclictest499635-21kworker/2:207:25:012
512518994939,6cyclictest516367-21kworker/2:209:00:022
512518994939,6cyclictest516367-21kworker/2:209:00:012
51251699495,3cyclictest131rcu_preempt12:32:391
51251699495,3cyclictest131rcu_preempt12:32:381
51251699493,3cyclictest131rcu_preempt10:23:131
512516994932,13cyclictest1-21systemd09:00:021
512516994932,13cyclictest1-21systemd09:00:011
51251199494,8cyclictest131rcu_preempt09:05:010
51251199494,7cyclictest12-21ksoftirqd/011:51:580
512511994940,7cyclictest12-21ksoftirqd/010:45:010
512511994933,8cyclictest309-21in:imuxsock07:25:010
51252199484,3cyclictest131rcu_preempt11:02:053
51252199483,3cyclictest131rcu_preempt11:54:433
51252199483,3cyclictest131rcu_preempt10:15:003
51252199483,3cyclictest131rcu_preempt09:27:473
51252199482,5cyclictest131rcu_preempt09:43:583
51252199482,5cyclictest131rcu_preempt09:43:573
51252199482,3cyclictest131rcu_preempt12:24:583
512521994819,20cyclictest0-21swapper/310:42:543
51252199481,5cyclictest131rcu_preempt11:41:133
51252199481,4cyclictest131rcu_preempt08:30:013
512518994840,5cyclictest518454-21kworker/2:111:13:172
512518994840,5cyclictest518454-21kworker/2:110:25:372
512518994840,5cyclictest518454-21kworker/2:110:25:372
512518994840,5cyclictest518454-21kworker/2:110:21:542
512518994840,5cyclictest518454-21kworker/2:109:40:152
512518994840,5cyclictest518454-21kworker/2:109:40:152
512518994840,5cyclictest516367-21kworker/2:209:26:342
512518994840,5cyclictest516367-21kworker/2:209:18:572
512518994839,6cyclictest518454-21kworker/2:111:05:192
512518994817,23cyclictest0-21swapper/211:37:022
51251699485,5cyclictest131rcu_preempt09:55:001
51251699484,3cyclictest131rcu_preempt12:35:591
512516994828,16cyclictest275-21dbus-daemon10:15:011
51251699482,4cyclictest131rcu_preempt10:05:031
51251699481,4cyclictest131rcu_preempt10:52:351
512516994811,19cyclictest0-21swapper/112:03:361
512516994810,6cyclictest131rcu_preempt09:40:011
512511994813,7cyclictest131rcu_preempt07:10:020
51252199474,4cyclictest131rcu_preempt11:16:413
51252199474,3cyclictest131rcu_preempt12:27:103
51252199474,3cyclictest131rcu_preempt09:51:493
51252199473,4cyclictest131rcu_preempt12:39:593
51252199473,3cyclictest131rcu_preempt11:37:313
51252199473,3cyclictest131rcu_preempt10:03:213
51252199472,4cyclictest131rcu_preempt10:35:023
51252199472,4cyclictest131rcu_preempt09:17:243
51252199472,3cyclictest131rcu_preempt11:48:373
51252199472,3cyclictest131rcu_preempt11:33:393
512518994739,5cyclictest581225-21kworker/2:312:25:582
512518994739,5cyclictest581225-21kworker/2:311:48:572
512518994739,5cyclictest518454-21kworker/2:111:27:252
512518994739,5cyclictest518454-21kworker/2:110:49:082
512518994739,5cyclictest516367-21kworker/2:209:20:202
512518994738,6cyclictest581225-21kworker/2:312:37:442
512518994734,9cyclictest1-21systemd08:55:002
51251699475,3cyclictest131rcu_preempt12:07:521
51251699474,3cyclictest131rcu_preempt09:57:201
51251699474,3cyclictest131rcu_preempt09:15:031
51251699473,3cyclictest131rcu_preempt11:49:541
51251699472,4cyclictest131rcu_preempt11:19:551
51251699471,3cyclictest131rcu_preempt11:23:081
51251699471,3cyclictest131rcu_preempt10:27:261
51251699471,3cyclictest131rcu_preempt10:27:251
51251699471,3cyclictest131rcu_preempt09:27:241
51251199477,7cyclictest131rcu_preempt08:10:010
51251199477,6cyclictest131rcu_preempt08:14:590
512511994721,11cyclictest520385-21ssh09:13:580
512511994718,24cyclictest12-21ksoftirqd/008:39:220
51252199469,7cyclictest131rcu_preempt12:09:223
51252199465,8cyclictest37-21ksoftirqd/307:55:013
51252199465,7cyclictest37-21ksoftirqd/311:10:063
51252199465,3cyclictest131rcu_preempt10:55:423
51252199465,3cyclictest131rcu_preempt10:55:413
51252199464,4cyclictest131rcu_preempt09:45:133
51252199463,5cyclictest131rcu_preempt12:31:123
51252199463,5cyclictest131rcu_preempt12:31:123
51252199463,4cyclictest131rcu_preempt10:25:163
51252199463,4cyclictest131rcu_preempt10:25:153
51252199463,3cyclictest131rcu_preempt10:51:223
51252199463,3cyclictest131rcu_preempt10:24:443
51252199462,3cyclictest131rcu_preempt12:12:263
51252199462,3cyclictest131rcu_preempt11:05:493
51252199462,3cyclictest131rcu_preempt10:18:573
51252199462,3cyclictest131rcu_preempt10:18:573
51252199462,3cyclictest131rcu_preempt09:00:013
51252199462,3cyclictest131rcu_preempt09:00:003
512521994622,5cyclictest37-21ksoftirqd/311:56:003
51252199461,4cyclictest131rcu_preempt10:05:373
512521994610,6cyclictest131rcu_preempt07:46:163
512518994638,5cyclictest581225-21kworker/2:312:15:132
512518994638,5cyclictest518454-21kworker/2:110:41:172
512518994637,6cyclictest515827-21kworker/2:008:31:572
512518994637,5cyclictest516367-21kworker/2:210:32:142
512518994634,9cyclictest513850-21sshd07:35:012
51251699465,5cyclictest131rcu_preempt11:28:101
51251699465,3cyclictest131rcu_preempt09:44:271
51251699465,3cyclictest131rcu_preempt09:44:261
51251699464,3cyclictest131rcu_preempt10:35:101
51251699464,3cyclictest131rcu_preempt10:32:401
51251699463,3cyclictest131rcu_preempt09:49:211
51251699462,3cyclictest131rcu_preempt12:11:051
51251699462,3cyclictest131rcu_preempt09:18:451
51251699461,4cyclictest131rcu_preempt11:50:501
512511994616,28cyclictest12-21ksoftirqd/009:49:040
51252199455,4cyclictest131rcu_preempt08:35:043
51252199454,4cyclictest131rcu_preempt08:40:013
51252199453,4cyclictest131rcu_preempt10:49:383
51252199453,3cyclictest131rcu_preempt09:22:103
51252199452,3cyclictest131rcu_preempt11:26:023
51252199452,3cyclictest131rcu_preempt10:37:003
51252199451,4cyclictest131rcu_preempt11:21:323
512518994537,5cyclictest516367-21kworker/2:210:58:012
512518994537,5cyclictest516367-21kworker/2:210:58:002
512518994537,5cyclictest516367-21kworker/2:210:52:232
512518994537,5cyclictest505360-21kworker/2:307:52:472
512518994526,14cyclictest3467602-21systemd-journal10:17:002
512518994526,14cyclictest3467602-21systemd-journal10:16:592
51251699455,3cyclictest131rcu_preempt08:55:011
51251699454,3cyclictest131rcu_preempt12:20:181
51251699454,3cyclictest131rcu_preempt12:17:481
512516994535,7cyclictest23-21ksoftirqd/109:05:161
51251699453,3cyclictest131rcu_preempt10:07:531
51251699452,4cyclictest131rcu_preempt10:45:551
51251699452,3cyclictest131rcu_preempt11:56:071
51251699452,3cyclictest131rcu_preempt09:20:121
51251699451,3cyclictest131rcu_preempt10:58:431
51251699451,3cyclictest131rcu_preempt10:58:431
51251699451,3cyclictest131rcu_preempt09:30:301
51251699451,3cyclictest131rcu_preempt09:30:291
51251199452,5cyclictest131rcu_preempt08:50:030
51252199445,3cyclictest131rcu_preempt07:20:013
51252199444,7cyclictest512719-21latency_hist07:15:023
51252199444,3cyclictest131rcu_preempt08:20:023
51252199444,3cyclictest131rcu_preempt08:15:013
51252199441,4cyclictest131rcu_preempt09:55:523
512518994436,5cyclictest516367-21kworker/2:208:42:252
512518994435,5cyclictest505360-21kworker/2:308:10:002
51251699444,3cyclictest131rcu_preempt08:20:021
51251699443,4cyclictest131rcu_preempt11:31:001
51251699443,4cyclictest131rcu_preempt11:08:361
512516994429,12cyclictest1-21systemd07:50:011
51251199446,10cyclictest12-21ksoftirqd/008:24:590
512511994428,12cyclictest513468-21sshd07:29:180
512511994428,12cyclictest513468-21sshd07:29:170
51252199434,3cyclictest131rcu_preempt08:05:023
51252199433,3cyclictest131rcu_preempt08:00:023
51252199432,3cyclictest131rcu_preempt07:35:023
51252199431,3cyclictest131rcu_preempt08:10:033
512518994335,5cyclictest291rcuc/208:20:012
51251699435,4cyclictest131rcu_preempt07:40:031
51251699435,4cyclictest131rcu_preempt07:10:031
51251699434,3cyclictest131rcu_preempt08:35:011
51251699434,3cyclictest131rcu_preempt08:25:021
51251699433,3cyclictest131rcu_preempt07:55:031
51251699432,3cyclictest131rcu_preempt08:00:161
51251699431,3cyclictest131rcu_preempt08:05:021
512511994329,11cyclictest12-21ksoftirqd/008:05:010
512511994311,6cyclictest131rcu_preempt07:45:030
512511994311,6cyclictest131rcu_preempt07:45:030
51252199424,3cyclictest131rcu_preempt07:10:013
51252199423,3cyclictest131rcu_preempt08:25:023
51252199422,3cyclictest131rcu_preempt07:25:023
512521994222,16cyclictest275-21dbus-daemon09:10:013
512518994234,5cyclictest505360-21kworker/2:307:45:022
512518994234,5cyclictest505360-21kworker/2:307:45:012
512518994229,9cyclictest514558-21sed07:50:022
51251699427,4cyclictest131rcu_preempt07:55:011
51251699423,4cyclictest131rcu_preempt08:50:011
51251699423,4cyclictest131rcu_preempt07:25:011
51251699423,3cyclictest131rcu_preempt07:45:021
51251699423,3cyclictest131rcu_preempt07:45:011
51251699422,3cyclictest131rcu_preempt08:25:031
512516994217,4cyclictest23-21ksoftirqd/107:20:001
51251699421,3cyclictest131rcu_preempt07:35:021
512511994230,9cyclictest12-21ksoftirqd/008:40:180
51252199414,3cyclictest131rcu_preempt08:55:003
51252199412,3cyclictest131rcu_preempt09:00:023
51252199412,3cyclictest131rcu_preempt07:40:013
51252199411,4cyclictest131rcu_preempt08:50:023
512518994133,5cyclictest499635-21kworker/2:207:10:012
512518994128,10cyclictest515282-21sshd08:02:182
51252199403,3cyclictest131rcu_preempt08:45:023
512518994028,9cyclictest513465-21sshd07:28:182
512518994028,9cyclictest513465-21sshd07:28:172
51251699404,3cyclictest131rcu_preempt08:15:021
51251699402,3cyclictest131rcu_preempt09:02:181
51251699402,3cyclictest131rcu_preempt08:40:021
51252199391,4cyclictest131rcu_preempt07:45:013
51252199391,4cyclictest131rcu_preempt07:45:003
512518993928,8cyclictest513105-21date07:20:022
512518993919,14cyclictest275-21dbus-daemon08:40:002
51251899371,5cyclictest131rcu_preempt07:11:162
51251899371,4cyclictest131rcu_preempt09:10:012
51251899355,11cyclictest131rcu_preempt09:05:012
512518993526,7cyclictest30-21ksoftirqd/208:25:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional