You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 17:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Aug 20, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
311582245163,70sleep00-21swapper/007:06:450
310882233164,56sleep20-21swapper/207:05:502
294442205168,24sleep30-21swapper/307:05:023
310992201165,23sleep10-21swapper/107:05:571
1470121710,5sleep13149599cyclictest07:40:251
3150199103100,2cyclictest0-21swapper/211:21:302
31501999283,8cyclictest0-21swapper/209:23:282
132542620,3sleep03149199cyclictest07:40:110
253802530,1sleep30-21swapper/309:20:163
283652490,2sleep20-21swapper/209:25:242
31501993935,3cyclictest0-21swapper/212:18:082
31491993910,13cyclictest9223-21awk08:45:010
3150899361,26cyclictest0-21swapper/311:51:063
3149199369,13cyclictest20218-21munin-run09:10:010
3149199366,14cyclictest22410-21munin-run09:15:010
3149199348,12cyclictest9843-21perf10:00:020
3149199334,13cyclictest11407-21munin-run08:50:010
3149199326,13cyclictest3868-21awk11:00:010
3149199317,11cyclictest13618-21awk08:55:010
3149199307,11cyclictest6080-21perf11:05:010
3150899290,13cyclictest0-21swapper/307:26:333
3149199291,18cyclictest28711-21kworker/0:110:43:270
36192281,23sleep13737-21/usr/sbin/munin12:10:231
36192281,23sleep13737-21/usr/sbin/munin12:10:231
3150899281,17cyclictest27526-21grep09:25:163
3149599286,10cyclictest25414-21unixbench_singl11:45:251
3149599281,21cyclictest0-21swapper/109:48:061
3149599281,21cyclictest0-21swapper/109:48:061
31495992725,1cyclictest0-21swapper/112:15:211
3149599271,20cyclictest0-21swapper/110:21:141
3149599271,20cyclictest0-21swapper/110:07:571
3149199272,13cyclictest30582-21munin-run08:20:000
3149199271,13cyclictest18013-21munin-run09:05:010
3149199270,13cyclictest328-21perf08:25:010
3150899261,17cyclictest0-21swapper/310:24:573
3150899260,9cyclictest0-21swapper/312:28:073
3150899260,7cyclictest0-21swapper/311:36:393
31495992623,2cyclictest0-21swapper/112:08:381
31495992623,2cyclictest0-21swapper/111:44:211
31495992623,2cyclictest0-21swapper/110:37:281
31495992623,2cyclictest0-21swapper/108:52:211
31495992623,2cyclictest0-21swapper/108:49:321
31495992623,2cyclictest0-21swapper/108:32:151
3149599262,22cyclictest17897-21sendmail_mailst10:15:241
3149599262,11cyclictest0-21swapper/108:15:221
3149599261,19cyclictest0-21swapper/112:37:461
3149599261,12cyclictest21785-21munin-run08:00:021
3149199262,10cyclictest27869-21kworker/0:211:59:270
3149199261,13cyclictest15801-21munin-run09:00:010
31491992610,3cyclictest8375-21kworker/0:107:30:160
31491992610,2cyclictest21253-21kworker/0:311:40:160
3150899254,3cyclictest12050irq/123-ahci[0009:40:023
3150899251,18cyclictest0-21swapper/312:15:073
3150899251,17cyclictest29599-21kworker/u8:110:48:333
3150899251,16cyclictest0-21swapper/312:35:263
3150899251,15cyclictest87350irq/126-enp1s0-10:15:133
3150899250,6cyclictest0-21swapper/309:44:323
3150199250,17cyclictest0-21swapper/210:55:032
31495992522,2cyclictest0-21swapper/110:40:261
31495992522,2cyclictest0-21swapper/107:53:191
3149599252,11cyclictest1326-21munin-node07:15:011
3149599252,10cyclictest9550-21ntp_states11:10:211
3149599251,22cyclictest0-21swapper/109:15:231
3149599251,22cyclictest0-21swapper/108:40:011
3149599251,21cyclictest0-21swapper/110:00:011
3149599251,19cyclictest0-21swapper/110:02:371
3149599251,12cyclictest16442-21awk10:15:021
3149599251,11cyclictest6010-21diskstats09:50:141
3149599251,11cyclictest31149-21diskstats08:20:131
3149199252,17cyclictest26518-21kworker/0:109:28:350
3149199252,10cyclictest22954-21kworker/0:110:33:390
3149199251,12cyclictest8609-21awk07:30:020
3149199251,10cyclictest17856-21kworker/0:209:18:310
3150899249,6cyclictest0-21swapper/310:05:233
3150899246,10cyclictest0-21swapper/311:55:183
3150899242,7cyclictest9834-21munin-run10:00:013
31508992413,3cyclictest87350irq/126-enp1s0-10:35:163
3150899241,17cyclictest0-21swapper/312:22:593
3150899241,17cyclictest0-21swapper/309:50:243
3150899241,16cyclictest0-21swapper/310:00:063
3150899241,16cyclictest0-21swapper/309:17:493
3150899241,15cyclictest0-21swapper/312:02:263
3150899240,6cyclictest0-21swapper/311:45:093
3150899240,6cyclictest0-21swapper/310:51:293
3149599249,4cyclictest0-21swapper/112:28:351
3149599242,10cyclictest20218-21munin-run09:10:011
3149599241,17cyclictest0-21swapper/112:21:541
3149599241,11cyclictest32544-21cat12:05:021
3149599241,11cyclictest29544-21expr09:30:121
3149599241,11cyclictest28376-21latency_hist08:15:021
3149599241,11cyclictest23022-21latency_hist10:30:021
3149599241,11cyclictest22402-21latency_hist09:15:021
3149599241,11cyclictest20453-21needreboot11:35:191
3149599241,11cyclictest13292-21basename12:35:011
3149599241,10cyclictest30248-21basename12:00:021
3149599241,10cyclictest23968-21sshd08:04:571
3149599240,19cyclictest0-21swapper/109:41:341
3149199245,16cyclictest28384-21munin-run08:15:000
261312240,3sleep00-21swapper/009:20:240
3150899237,6cyclictest0-21swapper/311:07:193
3150899234,10cyclictest12477-21ps12:30:223
3150899231,15cyclictest0-21swapper/311:42:173
3150899231,10cyclictest0-21swapper/307:18:343
3150899230,6cyclictest0-21swapper/311:13:323
3149599231,20cyclictest0-21swapper/110:45:131
3149599231,20cyclictest0-21swapper/109:25:171
3149599231,20cyclictest0-21swapper/109:02:421
3149599231,20cyclictest0-21swapper/108:05:241
3149599231,11cyclictest16084-21hddtemp_smartct07:45:161
3149599231,11cyclictest15166-21taskset11:25:131
3149599231,10cyclictest5305-21gunzip11:00:231
3149599231,10cyclictest4193-21latency_hist07:20:011
3149599231,10cyclictest2742-21sed08:30:021
3149599231,10cyclictest0-21swapper/111:20:231
3149599231,10cyclictest0-21swapper/107:30:081
3149199236,15cyclictest4504-21perf12:15:010
3149199236,15cyclictest4504-21perf12:15:010
3149199232,10cyclictest16249-21kworker/0:210:19:190
3149199231,8cyclictest6463-21kworker/0:312:33:310
3149199231,5cyclictest0-21swapper/012:01:030
31491992310,3cyclictest5341-21kworker/0:110:01:310
3150899225,10cyclictest1994-21kworker/3:108:20:253
3150899223,11cyclictest87350irq/126-enp1s0-11:30:253
31508992217,3cyclictest16249-21ps11:25:243
3150899221,6cyclictest0-21swapper/310:26:423
31508992213,4cyclictest0-21swapper/310:59:243
3150899221,16cyclictest87350irq/126-enp1s0-09:13:063
3150899221,16cyclictest0-21swapper/312:07:173
3150899221,15cyclictest87350irq/126-enp1s0-10:12:543
3150899221,15cyclictest13898-21kworker/u8:210:41:293
3150899221,15cyclictest0-21swapper/309:34:353
3150899221,15cyclictest0-21swapper/308:43:333
3150899221,12cyclictest28605-21/usr/sbin/munin08:15:233
3150899221,11cyclictest0-21swapper/311:00:183
3150899220,16cyclictest0-21swapper/311:20:203
3150899220,15cyclictest0-21swapper/309:45:243
3150899220,15cyclictest0-21swapper/309:45:243
3149599229,2cyclictest20207-21diskmemload10:34:391
3149599228,4cyclictest0-21swapper/111:30:471
3149599221,20cyclictest0-21swapper/111:05:171
3149599221,19cyclictest0-21swapper/111:15:111
3149599221,19cyclictest0-21swapper/110:54:491
3149599221,19cyclictest0-21swapper/109:20:261
3149599221,19cyclictest0-21swapper/107:20:371
3149599221,19cyclictest0-21swapper/107:20:371
3149599221,18cyclictest6942-21expr07:25:121
3149599221,10cyclictest0-21swapper/107:35:121
3149199226,8cyclictest23701-21kworker/0:108:02:510
3149199225,15cyclictest19263-21perf11:35:020
3149199221,6cyclictest5151-21tune2fs12:15:120
3149199221,10cyclictest31638-21kworker/0:310:52:310
3150899218,2cyclictest25728-21sed08:05:273
3150899211,15cyclictest0-21swapper/311:18:593
3150899211,13cyclictest87350irq/126-enp1s0-08:35:183
3150899211,13cyclictest87350irq/126-enp1s0-08:35:183
3150899211,11cyclictest6224-21sshd07:22:563
3150899211,11cyclictest6224-21sshd07:22:563
3150899211,10cyclictest0-21swapper/310:30:363
3150199215,2cyclictest0-21swapper/208:41:322
3149599211,19cyclictest0-21swapper/108:40:011
3149599211,18cyclictest0-21swapper/110:55:091
3149599211,11cyclictest0-21swapper/109:35:251
3149599210,20cyclictest0-21swapper/111:54:251
3149199218,3cyclictest3121-21kworker/0:209:51:000
3149199218,2cyclictest3121-21kworker/0:209:49:080
3149199218,2cyclictest3121-21kworker/0:209:49:070
3149199217,4cyclictest931-21kworker/0:007:11:510
3149199217,3cyclictest3885-21kworker/0:207:22:520
3149199217,3cyclictest3885-21kworker/0:207:22:510
3149199216,3cyclictest19237-21kworker/0:307:55:550
3149199216,3cyclictest13115-21kworker/0:212:35:350
3149199215,2cyclictest5806-21kworker/0:311:11:270
3149199215,2cyclictest30203-21kworker/0:012:20:150
3149199215,2cyclictest27869-21kworker/0:211:54:030
3149199213,15cyclictest24882-21cut08:05:170
3149199213,15cyclictest23029-21munin-run10:30:010
3149199212,10cyclictest8026-21kworker/0:011:17:120
3149199211,11cyclictest12485-21kworker/0:211:21:160
31491992110,3cyclictest28756-21kworker/0:209:34:030
175532210,3sleep30-21swapper/309:00:253
3150899208,3cyclictest0-21swapper/307:40:253
3150899208,2cyclictest12379-21timerandwakeup07:35:243
3150899201,6cyclictest20564-21awk07:55:163
31508992016,3cyclictest0-21swapper/307:30:183
3150899201,10cyclictest15412-21/usr/sbin/munin07:45:223
3150899201,10cyclictest0-21swapper/309:10:013
3150199201,6cyclictest0-21swapper/211:14:252
31501992014,3cyclictest724-21grep07:10:242
3150199201,12cyclictest0-21swapper/209:35:552
3149599201,10cyclictest0-21swapper/108:35:591
3149599200,19cyclictest0-21swapper/109:00:011
3149199209,3cyclictest6463-21kworker/0:312:26:110
3149199209,2cyclictest8375-21kworker/0:107:35:110
3149199208,3cyclictest30991-21kworker/0:009:42:430
3149199208,2cyclictest30991-21kworker/0:009:36:110
3149199207,3cyclictest20745-21kworker/0:010:35:590
3149199206,8cyclictest9546-21kworker/0:010:07:390
3149199206,2cyclictest12485-21kworker/0:211:25:510
3149199204,14cyclictest4214-21perf07:20:010
3149199201,9cyclictest19237-21kworker/0:307:54:150
31491992013,4cyclictest20363-21taskset10:20:260
3149199201,11cyclictest29843-21/usr/sbin/munin10:45:230
3150899196,3cyclictest87650irq/129-enp1s0-08:00:183
3150899196,3cyclictest10601-21ps08:45:203
3150899191,7cyclictest3179-21runrttasks08:30:143
3150899191,6cyclictest13431-21sshd08:52:563
31508991914,3cyclictest0-21swapper/312:10:093
31508991914,3cyclictest0-21swapper/312:10:093
3150199198,3cyclictest5734-21switchtime07:20:232
3150199198,3cyclictest5734-21switchtime07:20:232
3150199196,8cyclictest24278-40dnf09:15:592
3150199196,4cyclictest8256-21grep09:55:152
3150199194,2cyclictest87650irq/129-enp1s0-08:00:252
3150199193,9cyclictest20243-21kworker/2:207:55:112
3150199192,9cyclictest0-21swapper/210:08:292
3150199192,15cyclictest0-21swapper/207:53:412
3150199192,15cyclictest0-21swapper/207:27:042
3150199191,8cyclictest0-21swapper/210:30:482
3150199191,5cyclictest0-21swapper/210:20:552
3150199191,4cyclictest0-21swapper/212:36:222
31501991913,3cyclictest9158-21df07:30:122
31501991912,4cyclictest15883-21grep07:45:142
3149199198,2cyclictest16249-21kworker/0:210:13:030
3149199194,4cyclictest5806-21kworker/0:311:06:590
3149199191,6cyclictest0-21swapper/011:37:440
3149199191,10cyclictest8375-21kworker/0:107:45:150
118562191,2sleep20-21swapper/210:02:532
3150899184,3cyclictest41-21ksoftirqd/308:25:233
3150899183,4cyclictest17630-21cpu07:50:093
3150899181,10cyclictest808-21snmpd08:56:563
3150899180,6cyclictest27011-21gltestperf08:10:153
3150199182,6cyclictest28605-21/usr/sbin/munin08:15:192
3150199181,7cyclictest12558-21cat08:50:212
3150199181,7cyclictest0-21swapper/212:31:332
3150199181,7cyclictest0-21swapper/212:29:372
3150199181,7cyclictest0-21swapper/209:52:342
3150199181,5cyclictest0-21swapper/212:06:532
3150199181,5cyclictest0-21swapper/209:32:542
3150199181,3cyclictest0-21swapper/212:02:462
3150199181,15cyclictest0-21swapper/210:10:112
31501991811,4cyclictest24216-21acpi08:05:132
3150199181,11cyclictest0-21swapper/211:50:102
3150199181,11cyclictest0-21swapper/211:35:232
3149199189,3cyclictest23701-21kworker/0:108:25:070
3149199183,4cyclictest4659-21kworker/0:008:35:310
3149199183,4cyclictest4659-21kworker/0:008:35:310
3149199182,9cyclictest32755-21kworker/0:112:05:550
3149199182,10cyclictest23701-21kworker/0:108:33:030
3149199181,10cyclictest23868-21/usr/sbin/munin11:45:130
96822171,2sleep20-21swapper/208:45:132
316942171,2sleep30-21swapper/307:10:093
3150199176,8cyclictest20577-21cpuspeed_turbos09:10:122
3150199174,3cyclictest32-21ksoftirqd/211:25:232
3150199173,4cyclictest16205-21cstates09:00:122
3150199172,6cyclictest26366-21memory10:35:202
3150199172,4cyclictest6930-21cpu12:20:132
3150199171,7cyclictest21149-21grep10:25:112
3150199171,7cyclictest17570-21memory10:15:192
3150199171,7cyclictest0-21swapper/210:40:212
3150199171,6cyclictest3898-21ntp_kernel_pll_08:30:222
3150199171,6cyclictest0-21swapper/210:50:492
3150199171,5cyclictest0-21swapper/211:59:342
31501991714,2cyclictest32-21ksoftirqd/207:45:012
3150199171,3cyclictest87350irq/126-enp1s0-11:47:352
31501991713,3cyclictest0-21swapper/212:11:372
31501991713,3cyclictest0-21swapper/212:11:372
31501991713,3cyclictest0-21swapper/208:40:002
31501991713,3cyclictest0-21swapper/208:39:592
31501991713,3cyclictest0-21swapper/207:36:582
31501991712,3cyclictest0-21swapper/211:34:382
31501991711,3cyclictest5125-21ntpq11:00:202
31501991711,3cyclictest14714-21cat08:55:192
3150199170,4cyclictest0-21swapper/209:45:172
3150199170,4cyclictest0-21swapper/209:45:172
3150199169,4cyclictest2671-21fschecks_count07:15:162
3150199162,4cyclictest11794-21sed11:15:222
3150199162,11cyclictest8265-21munin-run11:10:012
31501991612,3cyclictest0-21swapper/208:14:322
3150199161,14cyclictest0-21swapper/211:40:242
3150199161,14cyclictest0-21swapper/209:42:592
3150199161,12cyclictest30008-21tr10:45:132
31501991610,3cyclictest2735-21sed08:30:012
31501991610,3cyclictest19230-21uname09:05:202
3150199151,8cyclictest0-21swapper/208:20:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional