You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 21:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Tue Apr 21, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3674210950,4sleep20-21swapper/216:26:172
4296998917,5cyclictest11721-21grep16:51:000
429699891,63cyclictest29814-21missed_timers19:26:070
429699891,63cyclictest29814-21missed_timers19:26:070
429699881,14cyclictest0-21swapper/017:40:470
429699876,62cyclictest0-21swapper/018:31:010
429699876,62cyclictest0-21swapper/018:31:010
239972840,2sleep00-21swapper/019:58:110
4296998361,3cyclictest0-21swapper/020:55:590
429699832,64cyclictest14598-21unixbench_singl16:56:150
429699821,22cyclictest24792-21ssh21:56:170
429699811,58cyclictest0-21swapper/020:41:030
429699811,24cyclictest0-21swapper/020:16:120
4296997819,4cyclictest17810-21tune2fs19:51:050
429699781,26cyclictest7820-21ssh21:36:080
4296997621,42cyclictest7850irq/56-dwc_otg_18:56:000
6842730,3sleep2686-21latency_hist19:30:482
6842730,3sleep2686-21latency_hist19:30:482
4296997315,46cyclictest0-21swapper/018:02:330
429699724,40cyclictest0-21swapper/017:51:010
4296997224,35cyclictest7950irq/56-dwc_otg_18:50:310
429699712,6cyclictest15131-21grep18:26:140
429699712,15cyclictest0-21swapper/020:11:020
4296997117,4cyclictest32671-21/usr/sbin/munin17:46:140
119302710,2sleep20-21swapper/220:22:412
119302710,2sleep20-21swapper/220:22:412
4296996915,8cyclictest0-21swapper/016:35:580
4296996815,4cyclictest29063-21runrttasks20:04:560
429699681,23cyclictest0-21swapper/019:01:280
4296996751,5cyclictest7750irq/56-dwc_otg18:40:470
4296996722,35cyclictest7950irq/56-dwc_otg_21:16:040
4296996717,39cyclictest0-21swapper/021:42:000
4296996716,5cyclictest7850irq/56-dwc_otg_19:37:140
4296996713,44cyclictest0-21swapper/020:53:260
429699671,15cyclictest14874-21/usr/sbin/munin20:26:130
429699661,16cyclictest11279-21ssh19:42:450
429699661,12cyclictest0-21swapper/021:54:360
429699661,12cyclictest0-21swapper/021:54:360
12032660,1sleep00-21swapper/020:10:430
4296996527,29cyclictest7750irq/56-dwc_otg20:50:120
4296996516,9cyclictest11470-21ssh20:21:510
4296996516,9cyclictest11470-21ssh20:21:510
4296996513,7cyclictest14681-21/usr/sbin/munin21:05:580
429699642,14cyclictest23245-21awk20:36:130
4296996413,7cyclictest0-21swapper/020:33:050
4296996413,11cyclictest0-21swapper/019:06:590
429699641,22cyclictest0-21swapper/017:09:160
429699641,22cyclictest0-21swapper/017:09:160
429699641,14cyclictest15598-21diskmemload19:45:530
429699639,13cyclictest5347-1kworker/u9:221:35:470
4296996315,10cyclictest0-21swapper/019:34:270
429699631,21cyclictest0-21swapper/021:26:040
429699631,17cyclictest0-21swapper/019:15:120
4296996212,4cyclictest7850irq/56-dwc_otg_21:01:030
429699621,16cyclictest2672-21ssh18:53:140
429699621,16cyclictest2672-21ssh18:53:140
429699621,14cyclictest0-21swapper/017:25:020
4296996210,5cyclictest21961-21ssh21:14:380
4300996113,13cyclictest7950irq/56-dwc_otg_21:41:012
4296996116,3cyclictest0-21swapper/019:21:010
429699611,12cyclictest0-21swapper/018:42:250
429699606,9cyclictest0-21swapper/017:27:420
429699603,15cyclictest0-21swapper/021:21:410
4296996018,9cyclictest0-21swapper/017:00:590
429699601,22cyclictest0-21swapper/017:40:540
429699601,15cyclictest0-21swapper/021:49:580
429699601,14cyclictest20904-21/usr/sbin/munin19:16:120
429699592,32cyclictest0-21swapper/016:30:580
429699592,22cyclictest0-21swapper/017:59:310
429699592,22cyclictest0-21swapper/017:59:310
4296995912,13cyclictest0-21swapper/018:12:540
429699591,14cyclictest10839-21/usr/sbin/munin16:46:140
26962596,18sleep02709-21/usr/sbin/munin16:26:030
429699589,11cyclictest0-21swapper/017:15:590
4296995816,4cyclictest8296-21/usr/sbin/munin16:41:040
191302580,3sleep3381ktimersoftd/317:11:043
4296995723,10cyclictest0-21swapper/018:05:580
429699571,11cyclictest0-21swapper/018:21:150
430199561,48cyclictest0-21swapper/318:11:003
429699551,16cyclictest908-21runrttasks17:35:370
429699551,14cyclictest0-21swapper/018:18:140
167122530,2sleep1221ktimersoftd/121:07:331
429699513,28cyclictest9-21ksoftirqd/017:11:160
429799506,8cyclictest0-21swapper/121:20:571
28942500,2sleep30-21swapper/316:26:063
4300994914,26cyclictest31-21ksoftirqd/218:15:582
430099487,27cyclictest0-21swapper/218:10:522
4297994822,3cyclictest0-21swapper/120:51:001
430099477,25cyclictest0-21swapper/217:30:532
430099457,23cyclictest0-21swapper/217:25:532
4300994518,5cyclictest20732-21aten_rbpower_en17:16:002
430099451,25cyclictest7850irq/56-dwc_otg_19:10:472
429799452,7cyclictest0-21swapper/121:31:001
73782440,4sleep17374-21ssh20:56:141
430199441,5cyclictest7750irq/56-dwc_otg20:20:593
430199441,5cyclictest7750irq/56-dwc_otg20:20:593
430099447,25cyclictest0-21swapper/217:36:002
429799442,11cyclictest0-21swapper/117:10:581
430099437,23cyclictest0-21swapper/218:01:012
4300994317,15cyclictest0-21swapper/217:01:012
429799435,7cyclictest0-21swapper/117:30:591
430099427,24cyclictest0-21swapper/218:05:512
430099424,17cyclictest0-21swapper/220:56:002
430199412,8cyclictest23894-21cut21:56:063
4301994119,4cyclictest15598-21diskmemload18:56:103
430099417,25cyclictest0-21swapper/220:46:012
251852410,2sleep10-21swapper/121:18:001
430199406,9cyclictest14686-21taskset21:05:583
430099407,4cyclictest7750irq/56-dwc_otg18:40:472
430099407,23cyclictest0-21swapper/216:45:522
430099407,22cyclictest0-21swapper/218:21:002
430099407,20cyclictest7950irq/56-dwc_otg_21:20:592
18152400,2sleep10-21swapper/120:50:341
4301993910,12cyclictest29723-21cat20:05:473
430099391,25cyclictest0-21swapper/219:01:002
429799392,3cyclictest0-21swapper/120:25:281
429799392,3cyclictest0-21swapper/120:25:281
4297993920,10cyclictest7950irq/56-dwc_otg_20:31:061
4297993911,14cyclictest0-21swapper/118:21:111
430199389,18cyclictest0-21swapper/319:30:163
430199389,18cyclictest0-21swapper/319:30:163
430199381,5cyclictest7950irq/56-dwc_otg_17:46:003
430099387,23cyclictest0-21swapper/217:06:012
430099387,23cyclictest0-21swapper/217:06:012
430099382,25cyclictest17176-21latency_hist19:50:472
429799386,6cyclictest0-21swapper/119:14:271
429799386,16cyclictest7750irq/56-dwc_otg18:45:561
4297993812,5cyclictest13332-21missed_timers19:06:081
429799381,23cyclictest15208-21ls17:00:481
36072385,20sleep13673-21/usr/sbin/munin16:26:171
266552380,8sleep1221ktimersoftd/121:59:271
430199377,22cyclictest0-21swapper/318:36:013
430199375,23cyclictest25069-21ssh19:20:593
4301993716,5cyclictest12487-21sh19:44:583
430199371,18cyclictest0-21swapper/320:26:023
430099377,21cyclictest0-21swapper/216:41:002
430099376,22cyclictest0-21swapper/220:26:012
430099375,24cyclictest24394-21ssh18:40:522
430099372,6cyclictest17517-21ssh19:50:592
4300993717,4cyclictest6059-21timerandwakeup19:36:132
4300993715,4cyclictest19083-21fschecks_count21:11:052
4300993712,6cyclictest7850irq/56-dwc_otg_21:05:582
430099371,22cyclictest20592-21latency_hist19:15:472
429799373,4cyclictest7750irq/56-dwc_otg21:50:591
429799373,4cyclictest7750irq/56-dwc_otg21:50:591
429799372,15cyclictest0-21swapper/117:42:041
4297993717,4cyclictest24376-21latency_hist17:25:481
4297993713,13cyclictest21-21rcuc/121:01:261
4297993712,4cyclictest0-21swapper/119:03:151
262852370,3sleep1221ktimersoftd/120:40:481
430199364,5cyclictest17164-21fschecks_time19:11:043
430199362,10cyclictest7950irq/56-dwc_otg_20:50:573
4301993616,7cyclictest13661-21ssh19:46:043
430199361,20cyclictest1102-21ssh19:31:023
430099368,6cyclictest7950irq/56-dwc_otg_21:35:472
430099368,6cyclictest7750irq/56-dwc_otg18:54:322
430099368,6cyclictest7750irq/56-dwc_otg18:54:322
430099368,17cyclictest0-21swapper/220:40:532
430099368,12cyclictest7950irq/56-dwc_otg_20:31:012
430099367,24cyclictest0-21swapper/219:20:592
430099365,5cyclictest30400-21/usr/sbin/munin17:41:042
430099364,16cyclictest7950irq/56-dwc_otg_17:21:012
4297993625,7cyclictest221ktimersoftd/118:55:591
4297993616,11cyclictest19652-21sh18:35:261
4297993616,11cyclictest19652-21sh18:35:261
4297993613,7cyclictest13193-21apt-config19:45:481
28512360,5sleep1429799cyclictest19:32:281
95372350,2sleep1429799cyclictest16:45:461
4301993525,4cyclictest7950irq/56-dwc_otg_19:17:593
430199352,4cyclictest22207-21apt-get20:36:003
4301993520,7cyclictest7950irq/56-dwc_otg_21:45:593
4301993519,9cyclictest908-21runrttasks20:11:443
4301993518,4cyclictest9377-21proc_pri19:01:113
430199351,7cyclictest22593-21munin-run21:15:473
430199351,23cyclictest0-21swapper/318:45:593
430099358,6cyclictest7950irq/56-dwc_otg_20:10:462
430099357,15cyclictest0-21swapper/219:31:002
430099351,5cyclictest7750irq/56-dwc_otg18:56:112
4300993514,16cyclictest0-21swapper/219:56:002
4300993510,14cyclictest15598-21diskmemload21:58:382
429799356,9cyclictest0-21swapper/118:10:521
429799354,4cyclictest0-21swapper/119:54:431
429799353,16cyclictest5120-21rm20:15:191
4297993516,5cyclictest8079-21sh19:39:271
429799351,23cyclictest11439-21cpuspeed21:41:021
4297993512,13cyclictest21205-21taskset21:13:151
4297993511,12cyclictest0-21swapper/120:10:251
4301993421,6cyclictest30650-21ssh21:25:283
430199341,7cyclictest7950irq/56-dwc_otg_21:40:583
4301993416,6cyclictest381ktimersoftd/321:52:523
4301993416,6cyclictest381ktimersoftd/321:52:523
4301993415,12cyclictest39-21ksoftirqd/320:55:593
4301993412,12cyclictest24080-21/usr/sbin/munin21:16:203
429799349,12cyclictest0-21swapper/121:46:121
429799348,4cyclictest0-21swapper/121:39:031
429799348,14cyclictest0-21swapper/119:15:571
429799346,13cyclictest7850irq/56-dwc_otg_19:47:301
4297993417,5cyclictest21034-21/usr/sbin/munin18:36:121
4297993415,11cyclictest22907-21head19:56:181
429799341,20cyclictest0-21swapper/121:27:301
4297993411,10cyclictest0-21swapper/120:02:011
430199336,12cyclictest7950irq/56-dwc_otg_20:10:043
4301993324,7cyclictest24381-21rm18:40:513
430199332,17cyclictest0-21swapper/319:07:453
4301993318,4cyclictest23736-21sh19:57:423
4301993315,11cyclictest0-21swapper/320:17:533
430199331,22cyclictest5490-21ntpq18:01:103
430199331,17cyclictest7950irq/56-dwc_otg_20:45:543
430199331,14cyclictest0-21swapper/316:56:003
430199331,11cyclictest0-21swapper/320:31:053
430099339,5cyclictest17173-21tail18:31:152
430099339,5cyclictest17173-21tail18:31:152
430099335,12cyclictest7950irq/56-dwc_otg_19:45:472
430099334,18cyclictest20629-21ls21:51:152
430099334,18cyclictest20629-21ls21:51:152
4300993318,6cyclictest17737-21ssh21:48:122
4300993317,6cyclictest1670-21aten_rbpower_po20:11:012
430099331,2cyclictest0-21swapper/220:03:582
4300993310,12cyclictest0-21swapper/221:35:512
429799334,15cyclictest7950irq/56-dwc_otg_20:41:041
429799333,13cyclictest0-21swapper/118:52:471
429799333,13cyclictest0-21swapper/118:52:471
429799332,8cyclictest25603-21cat18:41:141
429799331,6cyclictest0-21swapper/118:10:471
429799331,16cyclictest0-21swapper/117:16:061
4297993311,4cyclictest0-21swapper/120:26:161
430199324,21cyclictest6707-21ls16:36:083
430199322,17cyclictest0-21swapper/321:05:483
430199321,7cyclictest24199-21dpkg17:25:473
4301993216,10cyclictest20855-21sh19:55:013
4301993215,10cyclictest19629-21ssh18:35:243
4301993215,10cyclictest19629-21ssh18:35:243
430199321,23cyclictest0-21swapper/317:40:253
430199321,21cyclictest0-21swapper/318:21:003
430099327,4cyclictest0-21swapper/221:16:102
430099324,5cyclictest7950irq/56-dwc_otg_17:15:462
430099324,12cyclictest7850irq/56-dwc_otg_20:36:042
430099323,17cyclictest25577-21systemd-journal18:00:472
430099323,17cyclictest25577-21systemd-journal18:00:472
429799328,6cyclictest23-21ksoftirqd/120:16:011
429799326,12cyclictest7850irq/56-dwc_otg_16:31:151
429799325,5cyclictest7850irq/56-dwc_otg_17:51:121
4297993219,5cyclictest6602chrt17:47:331
429799321,22cyclictest11993-21latency_hist18:20:471
4297993210,11cyclictest0-21swapper/119:25:581
4297993210,11cyclictest0-21swapper/119:25:581
287682320,2sleep10-21swapper/119:25:401
430199316,13cyclictest0-21swapper/321:30:543
430199313,15cyclictest7850irq/56-dwc_otg_16:50:053
430199312,22cyclictest26028-21apt-get20:40:533
4301993116,4cyclictest8479-21latency_hist18:10:473
430199311,23cyclictest0-21swapper/318:51:383
430199311,23cyclictest0-21swapper/318:51:383
430199311,22cyclictest0-21swapper/317:31:053
430199311,20cyclictest0-21swapper/317:28:263
430099315,12cyclictest7950irq/56-dwc_otg_20:50:572
430099314,16cyclictest32428-21ls17:46:092
430099314,14cyclictest7850irq/56-dwc_otg_16:51:032
4300993114,6cyclictest4542-21sshd16:31:032
4300993111,5cyclictest31167-21sh21:25:582
430099311,14cyclictest7750irq/56-dwc_otg18:25:572
429799313,5cyclictest0-21swapper/116:35:581
429799312,17cyclictest0-21swapper/117:01:011
4297993112,5cyclictest11587-21latency_hist16:50:471
430199305,5cyclictest7950irq/56-dwc_otg_17:41:103
430199301,17cyclictest0-21swapper/319:37:143
430199301,16cyclictest0-21swapper/318:26:003
430199301,14cyclictest0-21swapper/321:35:573
430199301,12cyclictest7750irq/56-dwc_otg16:31:123
430099306,13cyclictest7750irq/56-dwc_otg18:46:032
430099303,13cyclictest7850irq/56-dwc_otg_16:56:142
4300993010,3cyclictest0-21swapper/219:16:562
429799307,20cyclictest0-21swapper/117:56:011
429799307,20cyclictest0-21swapper/117:56:011
4297993021,6cyclictest24687-21cpuspeed17:26:011
429799301,15cyclictest29319-21unixbench_multi17:36:151
430199299,10cyclictest3036-21grep17:56:013
430199299,10cyclictest3036-21grep17:56:013
430199296,10cyclictest21292-21missed_timers17:16:083
430199292,12cyclictest0-21swapper/318:15:483
4301992914,5cyclictest15269-21apt-get17:00:593
430199291,21cyclictest0-21swapper/321:26:023
430099296,13cyclictest7950irq/56-dwc_otg_17:51:112
430099296,10cyclictest7750irq/56-dwc_otg20:16:092
4300992923,4cyclictest6735-21ls16:36:082
429799292,7cyclictest25577-21systemd-journal16:55:471
429799291,5cyclictest17284-21sshd17:06:031
429799291,5cyclictest17284-21sshd17:06:031
430199287,5cyclictest7950irq/56-dwc_otg_17:51:153
430199281,20cyclictest0-21swapper/317:06:113
430199281,20cyclictest0-21swapper/317:06:113
430199281,19cyclictest0-21swapper/316:41:003
430199281,13cyclictest0-21swapper/316:50:593
430099286,4cyclictest0-21swapper/221:00:552
429799275,8cyclictest923-21sshd18:26:021
4297992617,6cyclictest5243-21if_wlan018:01:061
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional