You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-14 - 13:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Thu May 14, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
258230421410,3sleep23056-21PK-Backend20:55:142
257930021260,3sleep00-21swapper/020:45:220
262777121180,4sleep33056-21PK-Backend23:05:113
265289621140,3sleep00-21swapper/000:15:180
265289621140,3sleep00-21swapper/000:15:180
254621121080,6sleep02546212-21apt19:14:590
26117402980,4sleep234-21ksoftirqd/222:20:112
26117402980,4sleep234-21ksoftirqd/222:20:112
254424728543,14sleep00-21swapper/019:07:500
254294427545,21sleep20-21swapper/219:05:112
139099720,58rtkit-daemon0-21swapper/119:05:121
254406626748,13sleep30-21swapper/319:05:333
25713792620,4sleep33056-21PK-Backend20:25:133
25588362600,3sleep00-21swapper/019:50:000
26408202510,3sleep20-21swapper/223:40:192
25512382460,3sleep20-21swapper/219:25:202
254456199418,13cyclictest2655518-21latency_hist00:25:001
26402642400,3sleep33056-21PK-Backend23:40:113
254456199400,20cyclictest2567150-21seq20:10:411
254456199380,20cyclictest2545940-21taskset19:11:391
25454042370,6sleep3421ksoftirqd/319:10:173
254456199370,22cyclictest2635953-21taskset23:26:381
254456199370,20cyclictest2549881-21taskset19:21:261
254456199370,19cyclictest2618306-21seq22:39:421
254456199361,21cyclictest2551811-21runrttasks19:28:551
254456199361,19cyclictest2649791-21runrttasks00:05:551
254456199361,19cyclictest2577945-21runrttasks20:42:171
254456199361,17cyclictest2607336-21runrttasks22:05:541
254456199351,23cyclictest2646907-21acpi00:00:091
254456199350,20cyclictest2625202-21seq22:59:151
254456199350,20cyclictest2553361-21runrttasks19:31:361
254456199350,19cyclictest2637928-21runrttasks23:34:401
139099350,30rtkit-daemon3056-21PK-Backend20:54:483
254456199341,18cyclictest2570807-21taskset20:23:041
254456199340,23cyclictest2559085-21/usr/sbin/munin19:50:091
254456199340,21cyclictest2574640-21seq20:34:231
254456199340,20cyclictest2614615-21seq22:26:511
254456199340,20cyclictest2586476-21taskset21:05:391
254456199340,20cyclictest1095-21runrttasks21:12:321
254456199340,19cyclictest2605658-21taskset22:01:451
254456199340,19cyclictest2595173-21seq21:30:531
254456199340,19cyclictest2590038-21seq21:16:461
254456199340,19cyclictest2590038-21seq21:16:461
254456199340,19cyclictest1-21systemd00:34:551
254456199340,19cyclictest1095-21runrttasks21:01:501
254456199340,18cyclictest2643099-21taskset23:49:191
254456199340,18cyclictest2580241-21cat20:50:111
254456199331,19cyclictest2569136-21runrttasks20:18:511
254456199331,18cyclictest2633877-21taskset23:20:321
254456199331,18cyclictest2628778-21taskset23:07:051
254456199331,18cyclictest2603924-21seq21:56:431
254456199331,17cyclictest2156-21gnome-shell00:13:091
254456199330,20cyclictest2630381-21taskset23:10:251
254456199330,20cyclictest2598767-21seq21:42:321
254456199330,19cyclictest2612940-21runrttasks22:22:341
254456199330,19cyclictest2612940-21runrttasks22:22:341
254456199330,19cyclictest2562102-21taskset19:57:341
254456199330,18cyclictest2548102-21seq19:15:521
254456199323,15cyclictest2563940-21seq20:03:561
254456199321,17cyclictest2616579-21runrttasks22:34:451
254456199320,19cyclictest2590550-21taskset21:20:061
254456199320,19cyclictest2583265-21seq20:58:531
254456199320,18cyclictest2626772-21seq23:01:561
254456199320,18cyclictest2576261-21seq20:37:561
254456199320,18cyclictest2565400-21taskset20:05:271
254456199320,17cyclictest2597106-21taskset21:38:261
254456199320,17cyclictest2558795-21seq19:49:361
254456199311,17cyclictest2556854-21runrttasks19:41:521
254456199310,18cyclictest2602336-21seq21:53:351
254456199310,17cyclictest2639527-21seq23:37:561
254456199310,17cyclictest2619412-21ps22:40:201
254456199310,17cyclictest2609452-21seq22:12:271
254456199310,17cyclictest2572534-21seq20:28:011
254456199303,14cyclictest0-21swapper/123:56:571
254456199301,15cyclictest0-21swapper/123:43:101
254456199300,17cyclictest2644630-21taskset23:51:441
2544558993016,4cyclictest13-21ksoftirqd/019:37:380
254456199290,16cyclictest2653374-21taskset00:17:251
254456199290,16cyclictest2653374-21taskset00:17:251
254456199290,16cyclictest1895000-21snapd19:39:131
254456199280,17cyclictest1095-21runrttasks21:26:081
25909152260,3chrt3056-21PK-Backend21:20:133
254455899265,2cyclictest141rcu_preempt21:40:000
2544558992620,5cyclictest13-21ksoftirqd/022:13:240
2544558992612,4cyclictest13-21ksoftirqd/019:16:240
139099260,4rtkit-daemon1389-21rtkit-daemon00:23:502
2544561992511,12cyclictest2578415-21/usr/sbin/munin20:45:231
2544558992514,3cyclictest13-21ksoftirqd/023:07:090
2544558992514,3cyclictest13-21ksoftirqd/022:23:460
2544558992514,3cyclictest13-21ksoftirqd/022:23:460
2544558992514,2cyclictest141rcu_preempt00:27:300
2544558992513,2cyclictest141rcu_preempt20:31:220
254456199241,20cyclictest2620347-21apt22:45:111
2544558992420,3cyclictest13-21ksoftirqd/021:13:200
2544558992414,9cyclictest2646684-40updatedb.mlocat00:00:010
2544558992412,6cyclictest13-21ksoftirqd/020:06:430
2544558992412,4cyclictest13-21ksoftirqd/019:20:530
2544558992412,3cyclictest13-21ksoftirqd/000:09:030
2544558992411,4cyclictest13-21ksoftirqd/021:52:220
254457299230,21cyclictest2660137-21users00:35:213
2544561992312,9cyclictest2599230-21/usr/sbin/munin21:45:171
2544561992311,11cyclictest2659728-21needreboot00:35:171
254456199230,20cyclictest2631879-21ps23:15:201
254456199230,20cyclictest2631879-21ps23:15:201
254455899231,7cyclictest13-21ksoftirqd/020:35:210
2544558992316,5cyclictest2611399-21cron22:20:000
2544558992310,4cyclictest13-21ksoftirqd/023:49:190
254455899230,21cyclictest3056-21PK-Backend22:50:110
2544572992219,2cyclictest3056-21PK-Backend23:16:473
2544572992219,2cyclictest3056-21PK-Backend23:16:463
2544572992219,2cyclictest3056-21PK-Backend20:08:013
2544572992219,2cyclictest3056-21PK-Backend19:18:263
254457299220,20cyclictest3056-21PK-Backend00:13:383
2544568992220,1cyclictest0-21swapper/221:08:092
254456899220,21cyclictest0-21swapper/223:15:182
254456899220,21cyclictest0-21swapper/223:15:182
254456899220,21cyclictest0-21swapper/221:57:582
254456899220,21cyclictest0-21swapper/200:07:432
2544561992210,11cyclictest2609920-21/usr/sbin/munin22:15:121
254455899229,3cyclictest13-21ksoftirqd/020:18:590
254455899226,9cyclictest13-21ksoftirqd/020:55:190
2544558992220,1cyclictest0-21swapper/021:04:330
2544558992220,1cyclictest0-21swapper/019:54:500
2544558992220,1cyclictest0-21swapper/000:12:040
2544558992211,3cyclictest13-21ksoftirqd/019:29:470
2544558992210,2cyclictest141rcu_preempt20:50:110
2544572992119,1cyclictest3056-21PK-Backend23:39:113
2544572992119,1cyclictest3056-21PK-Backend21:37:103
2544572992119,1cyclictest3056-21PK-Backend20:13:353
2544572992119,1cyclictest3056-21PK-Backend20:03:413
2544572992119,1cyclictest0-21swapper/323:25:393
2544572992119,1cyclictest0-21swapper/323:22:013
2544572992119,1cyclictest0-21swapper/300:32:313
254457299210,19cyclictest3056-21PK-Backend23:50:203
254457299210,19cyclictest3056-21PK-Backend20:43:023
2544568992119,1cyclictest0-21swapper/223:55:402
2544568992119,1cyclictest0-21swapper/223:35:252
2544568992119,1cyclictest0-21swapper/223:04:432
2544568992119,1cyclictest0-21swapper/222:56:112
2544568992119,1cyclictest0-21swapper/221:54:132
2544568992119,1cyclictest0-21swapper/221:44:192
2544568992119,1cyclictest0-21swapper/221:21:212
2544568992119,1cyclictest0-21swapper/221:04:172
2544568992119,1cyclictest0-21swapper/220:45:182
2544568992119,1cyclictest0-21swapper/220:09:232
2544568992119,1cyclictest0-21swapper/219:56:172
2544568992119,1cyclictest0-21swapper/219:37:042
2544568992119,1cyclictest0-21swapper/219:32:312
2544568992119,1cyclictest0-21swapper/200:26:562
254456899210,20cyclictest3056-21PK-Backend20:54:582
254456899210,20cyclictest0-21swapper/223:31:202
254456899210,20cyclictest0-21swapper/221:27:432
254456899210,20cyclictest0-21swapper/200:11:492
254456899210,19cyclictest0-21swapper/221:16:342
254456899210,19cyclictest0-21swapper/221:16:342
254456199219,11cyclictest2156-21gnome-shell00:28:091
254455899218,5cyclictest13-21ksoftirqd/023:01:560
254455899218,4cyclictest13-21ksoftirqd/023:20:320
254455899218,4cyclictest13-21ksoftirqd/021:20:100
2544558992119,1cyclictest3056-21PK-Backend22:44:440
2544558992119,1cyclictest0-21swapper/023:43:170
2544558992119,1cyclictest0-21swapper/023:38:240
2544558992119,1cyclictest0-21swapper/023:33:310
2544558992119,1cyclictest0-21swapper/021:42:310
2544558992119,1cyclictest0-21swapper/021:05:340
2544558992119,1cyclictest0-21swapper/019:59:240
2544558992119,1cyclictest0-21swapper/019:41:520
2544558992119,1cyclictest0-21swapper/000:20:010
2544558992118,2cyclictest0-21swapper/021:59:550
2544558992116,3cyclictest2647324-21hddtemp_smartct00:00:140
2544558992113,6cyclictest2617275-21fw_packets22:35:150
254455899210,20cyclictest0-21swapper/023:27:490
254455899210,20cyclictest0-21swapper/022:30:170
139099210,15rtkit-daemon0-21swapper/222:12:552
2544572992018,1cyclictest3056-21PK-Backend21:26:303
2544572992018,1cyclictest3056-21PK-Backend21:00:403
2544572992018,1cyclictest3056-21PK-Backend20:19:093
2544572992018,1cyclictest3056-21PK-Backend19:27:253
2544572992017,2cyclictest0-21swapper/319:36:383
2544572992017,1cyclictest3056-21PK-Backend21:48:393
2544572992017,1cyclictest3056-21PK-Backend20:30:183
2544572992016,2cyclictest3056-21PK-Backend23:30:053
254457299200,19cyclictest3056-21PK-Backend21:54:543
254457299200,19cyclictest3056-21PK-Backend21:12:433
254457299200,19cyclictest0-21swapper/321:30:143
254457299200,19cyclictest0-21swapper/300:05:203
254457299200,18cyclictest3056-21PK-Backend21:57:043
2544568992018,1cyclictest0-21swapper/222:16:232
2544568992018,1cyclictest0-21swapper/222:08:392
2544568992018,1cyclictest0-21swapper/220:42:412
2544568992018,1cyclictest0-21swapper/219:51:442
2544568992018,1cyclictest0-21swapper/200:32:512
2544568992018,1cyclictest0-21swapper/200:16:562
2544568992018,1cyclictest0-21swapper/200:16:562
2544568992013,6cyclictest2629010-21cron23:10:002
2544568992013,6cyclictest1-21systemd19:20:002
2544568992012,6cyclictest436-20systemd-journal19:45:012
254456899200,19cyclictest3056-21PK-Backend19:24:412
254456899200,19cyclictest0-21swapper/223:45:262
254456899200,19cyclictest0-21swapper/222:51:322
254456899200,19cyclictest0-21swapper/222:40:232
254456899200,19cyclictest0-21swapper/221:48:182
254456899200,19cyclictest0-21swapper/220:34:502
254456899200,19cyclictest0-21swapper/220:27:472
254456899200,19cyclictest0-21swapper/220:14:222
254456899200,19cyclictest0-21swapper/200:37:442
254456899200,19cyclictest0-21swapper/200:04:392
254456199200,19cyclictest0-21swapper/122:50:211
254455899208,10cyclictest13-21ksoftirqd/022:00:190
254455899207,3cyclictest13-21ksoftirqd/020:21:450
2544558992018,1cyclictest3056-21PK-Backend21:46:440
2544558992018,1cyclictest3056-21PK-Backend00:32:240
2544558992018,1cyclictest0-21swapper/023:12:150
2544558992018,1cyclictest0-21swapper/020:44:590
2544558992017,2cyclictest0-21swapper/023:50:140
2544558992013,5cyclictest2572712-21munin-run20:30:000
254455899200,4cyclictest2156-21gnome-shell19:30:090
254455899200,19cyclictest3056-21PK-Backend00:39:550
254455899200,19cyclictest0-21swapper/023:19:310
254455899200,19cyclictest0-21swapper/023:19:310
254455899200,19cyclictest0-21swapper/022:08:130
254455899200,19cyclictest0-21swapper/021:31:430
2544572991917,1cyclictest3056-21PK-Backend22:12:463
2544572991917,1cyclictest3056-21PK-Backend20:20:453
2544572991917,1cyclictest3056-21PK-Backend19:20:293
2544572991916,2cyclictest3056-21PK-Backend21:15:403
2544572991916,2cyclictest3056-21PK-Backend21:15:403
254457299190,1cyclictest3056-21PK-Backend23:03:293
254457299190,1cyclictest3056-21PK-Backend22:33:553
254457299190,1cyclictest3056-21PK-Backend22:27:463
254457299190,1cyclictest3056-21PK-Backend22:02:453
254457299190,1cyclictest3056-21PK-Backend21:43:183
254457299190,1cyclictest3056-21PK-Backend20:59:383
254457299190,1cyclictest3056-21PK-Backend00:00:413
254457299190,17cyclictest3056-21PK-Backend22:08:263
2544568991917,1cyclictest0-21swapper/223:12:272
2544568991917,1cyclictest0-21swapper/220:35:532
2544568991917,1cyclictest0-21swapper/219:49:282
254456899190,1cyclictest0-21swapper/222:48:412
254456899190,1cyclictest0-21swapper/219:11:572
254456899190,18cyclictest0-21swapper/223:20:592
254456899190,18cyclictest0-21swapper/220:02:052
2544558991917,1cyclictest0-21swapper/021:18:590
2544558991917,1cyclictest0-21swapper/021:18:590
2544558991917,1cyclictest0-21swapper/020:00:380
2544558991913,5cyclictest2613790-21seq22:25:140
254455899190,1cyclictest3056-21PK-Backend22:56:260
254455899190,1cyclictest3056-21PK-Backend22:46:330
139099190,13rtkit-daemon3056-21PK-Backend22:24:183
139099190,13rtkit-daemon3056-21PK-Backend22:24:183
26114622180,3sleep3411rcuc/322:20:003
2544572991816,1cyclictest3056-21PK-Backend23:12:213
2544572991816,1cyclictest0-21swapper/322:49:093
2544572991816,1cyclictest0-21swapper/322:42:543
2544572991815,2cyclictest3056-21PK-Backend00:17:303
2544572991815,2cyclictest3056-21PK-Backend00:17:303
2544572991813,3cyclictest2645662-21sed23:55:163
2544572991811,6cyclictest2553673-21systemd-run19:35:013
254457299180,9cyclictest2655577-21/usr/sbin/munin00:25:143
254457299180,1cyclictest3056-21PK-Backend23:49:183
2544568991816,1cyclictest0-21swapper/223:28:292
2544568991816,1cyclictest0-21swapper/222:34:222
2544568991811,5cyclictest2605928-21cron22:05:002
2544568991810,7cyclictest2164-21gdbus21:13:202
254456899180,1cyclictest0-21swapper/221:35:062
254455899181,7cyclictest13-21ksoftirqd/021:25:190
2544558991811,5cyclictest2567046-21wc20:10:210
2544572991715,1cyclictest3056-21PK-Backend19:58:223
2544572991715,1cyclictest3056-21PK-Backend19:51:113
2544572991715,1cyclictest0-21swapper/322:52:473
2544572991712,3cyclictest2586014-21needreboot21:05:163
2544572991711,5cyclictest2623815-21apt22:55:123
254457299170,15cyclictest3056-21PK-Backend22:35:303
254456899174,11cyclictest2643575-21tr23:50:122
2544568991715,1cyclictest0-21swapper/222:38:342
2544568991715,1cyclictest0-21swapper/222:29:212
2544568991711,5cyclictest2595426-21seq21:34:062
2544572991614,1cyclictest3056-21PK-Backend19:48:073
2544572991614,1cyclictest0-21swapper/300:23:593
2544572991610,4cyclictest918-21rs:main3
2544568991614,1cyclictest0-21swapper/220:15:512
2544568991513,1cyclictest0-21swapper/220:24:232
2544572991412,1cyclictest3056-21PK-Backend20:36:543
254457299128,3cyclictest3056-21PK-Backend20:47:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional