You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-17 - 04:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Wed Jun 17, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
276942238162,24sleep30-21swapper/319:07:183
277412235161,23sleep20-21swapper/219:07:532
276862232161,59sleep00-21swapper/019:07:110
278172203166,24sleep10-21swapper/119:08:531
2468121760,3sleep02798899cyclictest21:25:240
627721560,4sleep22799899cyclictest23:10:242
436021330,5sleep02798899cyclictest20:40:170
139712870,3sleep00-21swapper/021:00:240
162182680,1sleep10-21swapper/121:05:271
288212570,1sleep00-21swapper/021:35:190
282522550,4sleep30-21swapper/321:35:143
28005994835,5cyclictest0-21swapper/320:45:133
28005994832,3cyclictest0-21swapper/322:16:133
27998994832,7cyclictest6263-21df_inode20:45:132
28005994734,3cyclictest0-21swapper/323:08:053
28005994730,3cyclictest0-21swapper/322:06:453
28005994529,10cyclictest0-21swapper/322:31:453
27998994429,3cyclictest0-21swapper/222:19:332
2800599431,13cyclictest23833-21taskset22:39:223
27998994327,3cyclictest0-21swapper/219:12:412
28005994233,3cyclictest0-21swapper/323:47:093
2800599421,32cyclictest0-21swapper/323:17:533
119932420,1sleep10-21swapper/100:40:011
28005994130,3cyclictest0-21swapper/322:02:533
2800599411,31cyclictest0-21swapper/300:38:133
2800599401,29cyclictest0-21swapper/323:37:533
2800599401,28cyclictest0-21swapper/323:52:013
27998994029,3cyclictest0-21swapper/221:09:372
27998994027,8cyclictest0-21swapper/222:08:532
27998994026,3cyclictest0-21swapper/223:47:092
28005993925,9cyclictest0-21swapper/322:25:573
28005993925,7cyclictest0-21swapper/323:29:013
27998993927,3cyclictest0-21swapper/200:21:262
27998993926,3cyclictest0-21swapper/220:07:012
27998993925,3cyclictest0-21swapper/222:41:332
27998993925,3cyclictest0-21swapper/222:41:332
27998993924,3cyclictest0-21swapper/222:02:542
2798899399,14cyclictest9134-21munin-run23:20:020
28005993830,3cyclictest0-21swapper/323:58:573
28005993827,3cyclictest0-21swapper/321:49:533
28005993825,3cyclictest0-21swapper/323:14:053
2800599381,29cyclictest0-21swapper/322:44:263
2800599381,29cyclictest0-21swapper/322:44:263
2800599381,28cyclictest0-21swapper/321:55:453
27998993827,3cyclictest0-21swapper/223:08:052
27998993825,8cyclictest0-21swapper/220:23:332
27998993825,8cyclictest0-21swapper/220:23:332
27998993822,11cyclictest0-21swapper/220:58:052
27998993822,10cyclictest0-21swapper/222:30:572
28005993722,3cyclictest0-21swapper/320:43:533
2800599371,34cyclictest0-21swapper/300:34:253
2800599371,34cyclictest0-21swapper/300:13:093
27998993725,6cyclictest0-21swapper/223:17:532
27998993723,3cyclictest0-21swapper/223:58:572
27998993719,7cyclictest0-21swapper/223:35:332
27988993711,13cyclictest17930-21awk23:40:010
2800599361,28cyclictest0-21swapper/320:04:253
27998993625,3cyclictest0-21swapper/221:43:542
27998993624,3cyclictest0-21swapper/221:12:132
27998993624,3cyclictest0-21swapper/221:04:372
27998993621,10cyclictest0-21swapper/222:47:572
27998993620,10cyclictest0-21swapper/220:00:532
2798899367,13cyclictest7572-21munin-run00:30:010
27988993611,12cyclictest30498-21munin-run22:55:010
28005993521,9cyclictest0-21swapper/321:30:293
28005993521,9cyclictest0-21swapper/321:30:293
2800599351,32cyclictest0-21swapper/319:49:333
2800599351,32cyclictest0-21swapper/300:21:253
2800599351,27cyclictest0-21swapper/321:19:373
27998993526,3cyclictest0-21swapper/223:26:012
2800599341,26cyclictest0-21swapper/322:21:373
2800599341,21cyclictest0-21swapper/320:05:373
2800599341,21cyclictest0-21swapper/319:39:263
2800599341,14cyclictest30807-21cpuspeed_turbos22:55:133
27998993424,3cyclictest0-21swapper/200:10:452
27998993419,10cyclictest0-21swapper/219:19:342
27998993415,11cyclictest0-21swapper/200:15:142
28005993324,3cyclictest0-21swapper/320:23:253
28005993324,3cyclictest0-21swapper/320:23:253
2800599331,3cyclictest0-21swapper/321:09:373
2800599331,30cyclictest0-21swapper/322:10:093
2800599331,13cyclictest0-21swapper/323:32:573
27998993328,3cyclictest0-21swapper/200:34:262
27998993321,3cyclictest0-21swapper/222:59:372
2799899331,9cyclictest0-21swapper/221:30:132
2799899331,9cyclictest0-21swapper/221:30:132
27998993315,8cyclictest0-21swapper/221:23:012
28005993222,3cyclictest0-21swapper/319:40:493
28005993219,8cyclictest0-21swapper/321:14:173
28005993219,3cyclictest0-21swapper/323:01:053
28005993219,3cyclictest0-21swapper/321:22:253
2800599321,29cyclictest0-21swapper/321:54:413
2800599321,29cyclictest0-21swapper/321:43:533
2800599321,24cyclictest0-21swapper/320:50:293
28005993212,3cyclictest13713-21processes21:00:213
2800599321,22cyclictest0-21swapper/320:33:533
2800599321,16cyclictest0-21swapper/320:57:453
2800599321,13cyclictest0-21swapper/300:04:573
2799899329,3cyclictest0-21swapper/220:39:292
2799899326,18cyclictest827-21runrttasks21:53:382
27998993221,3cyclictest0-21swapper/223:04:132
27998993221,3cyclictest0-21swapper/222:28:492
27998993221,3cyclictest0-21swapper/220:41:412
2799899321,24cyclictest0-21swapper/220:52:252
28005993116,3cyclictest0-21swapper/319:28:253
2799899316,3cyclictest0-21swapper/223:43:132
27998993120,3cyclictest0-21swapper/219:45:372
2799899311,28cyclictest0-21swapper/223:53:572
2800599309,3cyclictest31146-21munin-run00:10:013
28005993016,3cyclictest0-21swapper/319:31:023
2800599301,14cyclictest0-21swapper/322:52:573
2800599301,13cyclictest0-21swapper/321:27:413
27998993018,3cyclictest0-21swapper/221:47:542
27998993012,8cyclictest0-21swapper/220:34:492
2799899300,15cyclictest0-21swapper/223:21:332
2799399301,21cyclictest0-21swapper/123:56:511
2800599296,4cyclictest0-21swapper/319:13:413
28005992919,3cyclictest0-21swapper/300:19:493
2800599291,21cyclictest0-21swapper/319:55:373
2800599291,16cyclictest0-21swapper/323:41:373
2800599291,15cyclictest0-21swapper/320:29:053
2800599290,20cyclictest0-21swapper/320:10:213
2800599290,15cyclictest0-21swapper/323:22:013
27998992918,3cyclictest0-21swapper/221:56:222
27998992915,3cyclictest0-21swapper/221:16:572
27998992914,6cyclictest0-21swapper/222:12:532
27998992912,3cyclictest0-21swapper/219:36:092
2799899291,10cyclictest0-21swapper/221:29:172
27998992910,3cyclictest0-21swapper/220:17:572
180412290,3sleep12799399cyclictest23:40:021
2800599288,8cyclictest0-21swapper/319:50:213
28005992814,4cyclictest0-21swapper/320:35:133
2799899280,16cyclictest0-21swapper/222:51:252
2799899280,11cyclictest0-21swapper/219:57:372
2799399281,18cyclictest28869-21df_inode22:50:151
2798899283,6cyclictest0-21swapper/023:50:120
2798899282,13cyclictest16695-21munin-run21:10:000
28005992717,3cyclictest0-21swapper/320:19:453
2800599271,13cyclictest0-21swapper/319:23:303
2800599271,11cyclictest0-21swapper/322:47:213
2799899270,11cyclictest0-21swapper/219:25:372
2798899277,17cyclictest6943-21munin-run23:15:020
27988992711,14cyclictest81ktimersoftd/000:05:010
2800599261,9cyclictest0-21swapper/300:25:333
28005992614,3cyclictest0-21swapper/319:15:053
27998992621,3cyclictest0-21swapper/222:35:092
27998992612,9cyclictest0-21swapper/221:36:142
2799899261,12cyclictest0-21swapper/200:06:012
2799899261,10cyclictest0-21swapper/219:40:292
2799399269,4cyclictest774-21avahi-daemon20:05:081
2799899259,3cyclictest0-21swapper/220:27:452
2799899256,9cyclictest0-21swapper/219:20:052
27998992514,3cyclictest0-21swapper/219:53:172
27998992512,3cyclictest0-21swapper/222:21:262
27998992414,3cyclictest0-21swapper/220:12:052
27998992411,3cyclictest0-21swapper/200:25:062
27998992410,3cyclictest0-21swapper/200:00:572
2799399241,8cyclictest0-21swapper/122:35:331
2799399241,16cyclictest0-21swapper/119:40:141
2798899246,15cyclictest28310-21basename22:50:010
27988992412,9cyclictest11-21rcuc/000:00:010
27998992310,8cyclictest0-21swapper/219:34:532
27993992317,3cyclictest32470-21cstates21:45:121
2799399231,5cyclictest808-21snmpd22:10:231
2799399231,3cyclictest23904-21netstat20:10:201
2799399231,3cyclictest13741-21sendmail_mailqu21:00:221
2799399231,16cyclictest0-21swapper/123:31:461
2799399231,16cyclictest0-21swapper/123:15:181
2799399231,12cyclictest28071-21grep21:35:131
2799399230,16cyclictest0-21swapper/121:20:181
2798899231,11cyclictest0-21swapper/000:32:370
27998992211,3cyclictest0-21swapper/223:30:372
27998992211,3cyclictest0-21swapper/200:35:052
2799399222,11cyclictest9769-40dnf22:05:261
2799399221,5cyclictest9578-21kworker/u8:223:25:211
2799399221,4cyclictest13132-21kworker/u8:222:23:441
2799399221,4cyclictest0-21swapper/123:23:531
27993992214,3cyclictest0-21swapper/122:15:431
2799399221,3cyclictest0-21swapper/121:31:341
2799399221,3cyclictest0-21swapper/121:31:341
2799399221,2cyclictest3393-21gltestperf23:05:171
2799399221,18cyclictest30550-21timerwakeupswit19:15:021
2799399221,16cyclictest0-21swapper/120:37:511
2799399221,15cyclictest0-21swapper/122:32:411
2799399221,14cyclictest0-21swapper/122:59:451
2799399221,11cyclictest6188-21tr00:25:161
2799399221,10cyclictest0-21swapper/123:41:351
2799399220,16cyclictest0-21swapper/100:08:371
2798899226,6cyclictest0-21swapper/021:57:440
2798899225,10cyclictest9813-21ntp_states22:05:240
2798899221,11cyclictest827-21runrttasks19:55:150
2798899221,11cyclictest808-21snmpd22:20:250
2798899220,11cyclictest0-21swapper/000:16:480
2799399211,8cyclictest9578-21kworker/u8:223:49:421
27993992113,2cyclictest0-21swapper/122:25:371
2799399211,2cyclictest17918-21taskset19:55:431
2799399211,15cyclictest0-21swapper/123:01:531
2799399211,15cyclictest0-21swapper/121:42:571
2799399211,15cyclictest0-21swapper/119:34:281
27993992110,3cyclictest5311-21diskstats23:10:141
2798899215,9cyclictest0-21swapper/023:40:210
2798899212,11cyclictest6539-21/usr/sbin/munin22:00:170
27988992116,3cyclictest3035-21grep20:35:260
2798899211,12cyclictest9703-21/usr/sbin/munin19:40:140
2799399209,3cyclictest14431-21df19:50:121
2799399206,3cyclictest125892hwlatdetect19:45:171
2799399201,4cyclictest0-21swapper/122:46:571
27993992013,2cyclictest0-21swapper/122:02:421
27993992013,2cyclictest0-21swapper/121:19:201
2799399201,14cyclictest0-21swapper/123:50:271
2799399201,14cyclictest0-21swapper/120:51:361
2799399201,14cyclictest0-21swapper/120:42:061
2799399201,14cyclictest0-21swapper/119:27:111
2799399201,14cyclictest0-21swapper/100:15:371
2799399201,11cyclictest2884-21latency_hist19:25:011
2799399201,10cyclictest5015-21hddtemp_smartct21:55:161
2799399201,10cyclictest28165-21ps00:00:221
27993992010,3cyclictest0-21swapper/122:40:211
27993992010,3cyclictest0-21swapper/122:40:211
2799399200,15cyclictest0-21swapper/120:49:281
2799399200,15cyclictest0-21swapper/100:14:411
2798899208,4cyclictest0-21swapper/021:30:490
2798899208,4cyclictest0-21swapper/021:30:490
2798899205,6cyclictest0-21swapper/023:34:310
27988992014,4cyclictest4108-21needreboot19:25:210
2798899201,11cyclictest7508-21/usr/sbin/munin19:35:140
2798899201,11cyclictest0-21swapper/023:04:470
2798899201,11cyclictest0-21swapper/022:35:100
2798899201,11cyclictest0-21swapper/000:24:180
2798899201,10cyclictest11903-21/usr/sbin/munin19:45:250
2798899200,11cyclictest16692-21diskmemload21:20:210
2799399199,2cyclictest2489-21diskstats21:50:141
27993991912,2cyclictest0-21swapper/120:02:171
27993991912,1cyclictest0-21swapper/121:25:201
2799399191,17cyclictest0-21swapper/119:11:411
2799399191,16cyclictest87350irq/126-enp1s0-00:30:111
2799399191,10cyclictest4746-21wc00:20:261
2799399191,10cyclictest18505-21users21:10:271
2799399191,10cyclictest11444-21ls20:55:241
2799399190,14cyclictest0-21swapper/120:26:301
2799399190,14cyclictest0-21swapper/120:24:471
2799399190,14cyclictest0-21swapper/120:24:471
2799399190,14cyclictest0-21swapper/120:17:031
2799399190,14cyclictest0-21swapper/119:36:421
2798899197,4cyclictest0-21swapper/021:43:250
2798899195,5cyclictest0-21swapper/000:35:340
2798899194,5cyclictest0-21swapper/021:52:100
2798899192,7cyclictest24125-21/usr/sbin/munin22:40:250
2798899192,7cyclictest24125-21/usr/sbin/munin22:40:250
2798899191,7cyclictest13200-21cpu22:15:140
2798899191,11cyclictest16692-21diskmemload23:45:150
2798899191,11cyclictest16692-21diskmemload23:27:020
2798899191,11cyclictest0-21swapper/022:25:240
2798899191,11cyclictest0-21swapper/022:10:290
2798899191,10cyclictest808-21snmpd19:34:470
2798899191,10cyclictest16692-21diskmemload22:30:020
2798899191,10cyclictest1429-21sed21:45:260
2798899191,10cyclictest0-21swapper/000:13:510
27993991812,2cyclictest0-21swapper/120:31:571
2798899187,4cyclictest0-21swapper/023:06:080
2798899187,3cyclictest29493-21/usr/sbin/munin20:25:180
2798899181,9cyclictest808-21snmpd19:14:420
2798899181,2cyclictest32531-21seq22:58:060
27988991812,4cyclictest0-21swapper/021:10:260
2798899181,16cyclictest81ktimersoftd/000:06:090
2798899181,10cyclictest0-21swapper/023:23:360
2798899181,10cyclictest0-21swapper/021:17:270
2798899180,9cyclictest0-21swapper/020:15:140
327042170,2sleep00-21swapper/019:15:420
2798899173,5cyclictest11-21rcuc/020:10:230
2798899172,8cyclictest0-21swapper/020:45:160
2798899172,6cyclictest879-21/usr/sbin/munin19:20:210
2798899171,7cyclictest10314-21/usr/sbin/munin20:55:160
27988991711,3cyclictest22107-21cat20:05:270
2798899170,14cyclictest15562-21grep19:50:240
204372171,2sleep00-21swapper/020:04:430
10702171,2sleep00-21swapper/020:32:270
2798899163,4cyclictest27989-21idleruntime20:20:160
2798899163,4cyclictest27989-21idleruntime20:20:160
27988991611,2cyclictest8116-21/usr/sbin/munin20:50:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional