You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-28 - 02:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sun Sep 27, 2026 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
984299281275,4cyclictest8787-21lspci11:16:241
95542175102,27sleep00-21swapper/007:09:480
94092164113,28sleep10-21swapper/107:07:571
93712163103,29sleep30-21swapper/307:07:273
95562159107,29sleep20-21swapper/207:09:512
217172780,1sleep00-21swapper/012:13:420
67402550,1sleep20-21swapper/212:25:272
256382550,2sleep20-21swapper/211:52:242
319472540,2sleep331945-21ssh11:10:163
236442540,2sleep20-21swapper/212:38:402
18992490,2sleep20-21swapper/209:13:172
260652480,1sleep30-21swapper/312:16:043
25452480,2sleep30-21swapper/310:25:133
245132480,1sleep30-21swapper/311:05:133
89612450,2sleep30-21swapper/311:16:253
322442420,1sleep00-21swapper/009:59:300
33092400,1sleep00-21swapper/011:35:330
184822390,2sleep118483-21test12:35:021
20582370,1sleep00-21swapper/010:00:210
984199365,16cyclictest0-21swapper/011:57:590
984199363,24cyclictest19223-21ssh11:24:330
984199355,15cyclictest0-21swapper/011:33:120
984199354,17cyclictest22653-21rm09:51:440
984199354,17cyclictest0-21swapper/012:03:400
984199346,11cyclictest10629-21ssh11:17:440
9841993428,4cyclictest2376-21ssh09:36:530
984199342,18cyclictest22523-21sh11:25:540
236242340,2chrt0-21swapper/111:04:361
984299331,29cyclictest27423-21ssh10:19:551
984199332,23cyclictest5890-21ssh10:27:120
34032330,2chrt0-21swapper/012:24:050
984199325,10cyclictest29249-21diskmemload11:10:210
984199324,8cyclictest0-21swapper/009:25:310
984199321,2cyclictest0-21swapper/009:16:450
984399312,4cyclictest28-21ksoftirqd/212:20:012
984399311,7cyclictest6234-21ssh09:15:502
984299310,28cyclictest18587-21ssh10:36:201
984299310,28cyclictest18587-21ssh10:36:201
984199317,9cyclictest16478-21ssh10:10:470
984199316,9cyclictest29249-21diskmemload12:06:460
984199316,8cyclictest29249-21diskmemload11:04:130
984199316,11cyclictest17698-21rm11:46:240
984199314,9cyclictest28333-21ssh12:18:220
984199314,15cyclictest0-21swapper/010:06:310
984199314,14cyclictest13884-21ssh10:33:220
984199314,14cyclictest0-21swapper/010:54:380
984199313,15cyclictest0-21swapper/009:40:190
984199313,14cyclictest18568-21ssh12:35:060
984199312,12cyclictest2150-21ssh10:48:300
984499301,26cyclictest12544-21/usr/sbin/munin08:30:163
984399301,26cyclictest4523-21ssh10:25:502
9842993022,5cyclictest29249-21diskmemload09:22:051
984299301,26cyclictest32025-21diskstats08:00:111
984199306,10cyclictest8139-21ssh12:26:520
984199304,9cyclictest29249-21diskmemload09:22:220
984199304,9cyclictest18164-21ssh09:49:020
984199304,9cyclictest18164-21ssh09:49:010
984199303,10cyclictest614-21ssh10:23:430
984199303,10cyclictest29249-21diskmemload11:09:270
9841993023,4cyclictest2966-21runrttasks08:05:490
984199301,27cyclictest0-21swapper/007:46:180
9844992923,4cyclictest3927-21ssh09:15:023
9843992922,5cyclictest52750irq/139-enp3s0-10:23:242
984399291,26cyclictest14822-21ssh10:34:192
984399291,25cyclictest26344-21/usr/sbin/munin07:45:222
984399290,27cyclictest8503-21ssh09:41:132
984399290,27cyclictest3174-21rm09:37:402
984199293,9cyclictest12904-21ssh10:55:400
9841992923,4cyclictest23120-21/usr/sbin/munin08:55:140
984199292,15cyclictest0-21swapper/010:18:250
984199291,3cyclictest18378-21runrttasks07:28:590
984499281,3cyclictest1818-21ssh09:36:193
984499281,25cyclictest0-21swapper/308:27:263
984499280,26cyclictest21198-21ssh10:38:563
984499280,26cyclictest21198-21ssh10:38:553
9843992824,3cyclictest271rcuc/212:03:332
9843992823,3cyclictest271rcuc/210:07:142
984399281,24cyclictest22630-21chrt08:52:512
984399281,24cyclictest15779-21ssh10:35:032
984399281,24cyclictest15779-21ssh10:35:022
9842992823,3cyclictest31970-21ssh10:45:561
9842992822,4cyclictest6001-21ssh10:27:191
984299281,3cyclictest32716-21ssh10:23:241
984299281,2cyclictest69642sleep109:16:331
984299281,25cyclictest18705-21irqstats09:25:171
984299281,25cyclictest0-21swapper/109:36:201
984299281,25cyclictest0-21swapper/108:22:071
984299281,25cyclictest0-21swapper/108:05:501
984299281,25cyclictest0-21swapper/107:42:441
984299281,25cyclictest0-21swapper/107:27:541
984199287,7cyclictest21856-21ssh11:50:080
984199284,9cyclictest16825-21ls10:35:190
984199284,9cyclictest16825-21ls10:35:190
984199283,9cyclictest835-21taskset09:12:150
984199282,8cyclictest13390-21ssh11:43:510
984199281,25cyclictest0-21swapper/008:50:430
984199281,25cyclictest0-21swapper/007:23:400
9844992722,3cyclictest26433-21switchtime07:45:233
9844992722,3cyclictest14272-21ssh10:33:443
9844992721,4cyclictest14947-21ssh09:22:393
984499271,24cyclictest0-21swapper/308:37:183
984499270,3cyclictest29675-21ssh09:33:323
984399275,9cyclictest28-21ksoftirqd/211:37:082
984399274,6cyclictest28-21ksoftirqd/211:15:132
9843992722,3cyclictest271rcuc/209:45:122
9843992722,3cyclictest271rcuc/209:45:112
984399272,23cyclictest0-21swapper/210:40:032
984399271,24cyclictest0-21swapper/208:10:112
984399271,24cyclictest0-21swapper/208:10:102
9843992711,2cyclictest141rcu_preempt11:40:262
984399270,6cyclictest28865-21apt-get11:55:122
9842992724,2cyclictest0-21swapper/112:26:051
9842992724,2cyclictest0-21swapper/110:14:381
9842992724,2cyclictest0-21swapper/107:23:061
984299271,2cyclictest0-21swapper/108:35:151
984299271,2cyclictest0-21swapper/107:55:341
984299271,25cyclictest0-21swapper/110:31:461
984299271,25cyclictest0-21swapper/108:40:231
984299271,24cyclictest0-21swapper/109:33:311
984299271,24cyclictest0-21swapper/109:05:481
984299271,24cyclictest0-21swapper/108:50:411
984299271,24cyclictest0-21swapper/108:15:531
984299271,24cyclictest0-21swapper/108:10:021
984299271,24cyclictest0-21swapper/108:10:011
984299271,24cyclictest0-21swapper/107:30:441
984199274,6cyclictest23247-21perl10:40:150
984199272,8cyclictest0-21swapper/012:30:030
9841992724,2cyclictest0-21swapper/009:33:400
984199271,2cyclictest0-21swapper/007:42:080
984199271,24cyclictest0-21swapper/008:35:140
984199271,24cyclictest0-21swapper/008:27:020
984199271,24cyclictest0-21swapper/008:23:150
984199271,24cyclictest0-21swapper/008:10:090
984199271,24cyclictest0-21swapper/008:10:080
984199271,24cyclictest0-21swapper/007:55:460
984199271,24cyclictest0-21swapper/007:52:540
984199271,24cyclictest0-21swapper/007:39:160
984199271,24cyclictest0-21swapper/007:30:290
984199270,25cyclictest0-21swapper/008:16:110
59132270,1sleep10-21swapper/109:40:081
296532270,2sleep10-21swapper/112:19:411
228212270,2sleep3560-21dbus-daemon08:55:003
9844992623,2cyclictest0-21swapper/309:41:233
9844992621,3cyclictest30289-21ssh10:20:593
9844992621,3cyclictest0-21swapper/307:20:003
984499261,23cyclictest0-21swapper/310:15:043
984499261,23cyclictest0-21swapper/309:05:373
984499261,23cyclictest0-21swapper/308:49:363
984499261,23cyclictest0-21swapper/308:40:343
984499261,23cyclictest0-21swapper/308:16:013
984499261,23cyclictest0-21swapper/308:11:393
984499261,23cyclictest0-21swapper/308:11:383
984499261,23cyclictest0-21swapper/308:06:243
984499261,23cyclictest0-21swapper/307:55:453
984499261,23cyclictest0-21swapper/307:52:313
984499261,23cyclictest0-21swapper/307:40:493
984499261,23cyclictest0-21swapper/307:30:323
984499261,23cyclictest0-21swapper/307:26:203
984499261,23cyclictest0-21swapper/307:23:083
984499260,3cyclictest19650-21sh12:11:393
984399266,8cyclictest28-21ksoftirqd/211:20:222
984399266,8cyclictest28-21ksoftirqd/211:09:142
984399266,10cyclictest28-21ksoftirqd/209:55:482
9843992621,3cyclictest271rcuc/210:51:062
984399261,9cyclictest28-21ksoftirqd/212:12:182
984399261,23cyclictest0-21swapper/210:15:032
984399261,23cyclictest0-21swapper/209:23:512
984399261,23cyclictest0-21swapper/208:28:112
984399261,23cyclictest0-21swapper/208:15:512
984399261,23cyclictest0-21swapper/208:05:302
984399261,23cyclictest0-21swapper/207:42:092
984399261,23cyclictest0-21swapper/207:25:372
984399260,9cyclictest0-21swapper/212:32:392
984399260,23cyclictest15981-21apt-get07:21:022
9842992624,1cyclictest0-21swapper/107:46:061
984299261,1cyclictest0-21swapper/108:30:271
984299261,1cyclictest0-21swapper/108:29:191
984299261,18cyclictest0-21swapper/108:55:281
984299260,24cyclictest0-21swapper/109:14:381
9841992623,2cyclictest0-21swapper/009:05:340
984199261,23cyclictest0-21swapper/008:40:320
984499251,20cyclictest53050irq/142-enp3s0-11:55:123
984499251,1cyclictest0-21swapper/311:28:273
984499251,1cyclictest0-21swapper/310:45:553
984499251,1cyclictest0-21swapper/310:12:503
984499251,1cyclictest0-21swapper/309:25:083
984499251,1cyclictest0-21swapper/308:21:453
984399257,3cyclictest141rcu_preempt08:30:122
984399256,7cyclictest28-21ksoftirqd/210:10:192
984399253,6cyclictest28-21ksoftirqd/212:21:052
984399253,6cyclictest28-21ksoftirqd/212:06:252
984399253,6cyclictest28-21ksoftirqd/211:01:462
9843992523,1cyclictest0-21swapper/209:26:262
984399251,23cyclictest0-21swapper/208:36:222
984399251,23cyclictest0-21swapper/208:21:352
984399251,23cyclictest0-21swapper/207:39:502
984399251,1cyclictest0-21swapper/209:34:202
984399251,1cyclictest0-21swapper/209:05:502
984399250,23cyclictest0-21swapper/207:30:072
9842992523,1cyclictest0-21swapper/112:34:331
9842992523,1cyclictest0-21swapper/111:28:271
9842992523,1cyclictest0-21swapper/107:52:461
9842992522,2cyclictest0-21swapper/110:40:041
984299250,2cyclictest0-21swapper/108:45:041
984299250,24cyclictest0-21swapper/112:11:391
9841992523,1cyclictest0-21swapper/008:49:430
9841992523,1cyclictest0-21swapper/008:30:250
24972250,2sleep20-21swapper/210:48:522
151582253,2sleep30-21swapper/310:57:563
9844992420,2cyclictest0-21swapper/308:55:143
984499241,22cyclictest0-21swapper/312:25:523
984499241,1cyclictest0-21swapper/312:34:333
984399246,10cyclictest28-21ksoftirqd/211:46:232
984399246,10cyclictest28-21ksoftirqd/211:26:022
984399244,10cyclictest28-21ksoftirqd/211:30:042
984399243,8cyclictest28-21ksoftirqd/209:50:152
9843992422,1cyclictest0-21swapper/208:55:192
9843992421,2cyclictest271rcuc/210:55:352
984399240,2cyclictest0-21swapper/208:49:142
984399240,22cyclictest0-21swapper/208:40:202
9842992421,2cyclictest0-21swapper/112:22:151
984299241,10cyclictest0-21swapper/111:30:121
9841992414,7cyclictest12354-21df_inode07:15:140
984499235,3cyclictest228182missed_timers11:50:183
984499231,9cyclictest15553-21gltestperf11:45:153
984499230,1cyclictest0-21swapper/310:40:043
984399232,7cyclictest28-21ksoftirqd/210:02:012
9843992319,2cyclictest271rcuc/207:15:012
984399231,7cyclictest28-21ksoftirqd/211:11:482
984399230,1cyclictest0-21swapper/208:00:192
984299238,13cyclictest0-21swapper/111:35:211
9842992315,6cyclictest11861-21strings10:55:191
984499221,8cyclictest3958-21taskset12:24:383
9844992212,8cyclictest560-21dbus-daemon07:15:003
984399222,8cyclictest28-21ksoftirqd/207:50:142
984399221,7cyclictest28-21ksoftirqd/207:11:182
984299222,14cyclictest806-21ssh10:00:131
9842992215,4cyclictest5811-21vmstat10:50:261
9842992210,9cyclictest27122-21grep09:55:181
984499217,3cyclictest141rcu_preempt09:55:003
9844992119,1cyclictest0-21swapper/307:39:593
984499211,6cyclictest1993-21fw_packets11:35:153
9844992114,4cyclictest29755-21sh11:31:203
9843992119,1cyclictest0-21swapper/207:55:462
9842992112,6cyclictest26146-21ntp_states09:00:201
9842992111,6cyclictest52950irq/141-enp3s0-11:45:111
9842992111,3cyclictest210352cstates07:35:131
984199210,8cyclictest0-21swapper/009:00:160
317492210,2sleep30-21swapper/309:10:453
984499209,4cyclictest53050irq/142-enp3s0-11:01:193
984499209,4cyclictest53050irq/142-enp3s0-10:54:093
984499209,4cyclictest53050irq/142-enp3s0-09:58:373
984499205,5cyclictest20572/usr/sbin/munin10:00:213
9844992011,7cyclictest14798-21cat11:45:013
9844992010,7cyclictest14203-21ssh09:45:233
9844992010,7cyclictest14203-21ssh09:45:223
9843992016,3cyclictest271rcuc/209:00:302
984299204,10cyclictest1393-21gpm09:45:111
984299204,10cyclictest1393-21gpm09:45:101
9842992012,6cyclictest13431-21unixbench_multi07:15:241
9842992011,2cyclictest22-21ksoftirqd/110:06:071
984199208,10cyclictest10092-21cat07:10:130
984199201,5cyclictest12-21ksoftirqd/008:01:180
9844991913,3cyclictest29249-21diskmemload12:35:153
9844991912,5cyclictest16319-21cat12:10:003
9844991911,6cyclictest9601-21cat12:05:003
9844991910,7cyclictest13575-21cstates11:20:153
984299195,2cyclictest141rcu_preempt12:00:001
984299191,9cyclictest0-21swapper/111:06:571
9842991914,3cyclictest52950irq/141-enp3s0-11:20:221
9842991913,4cyclictest27385-21sh11:54:101
9842991912,5cyclictest22962-21ssh09:52:041
9842991911,6cyclictest4679-21seq11:14:341
984499188,8cyclictest560-21dbus-daemon08:04:593
984499181,14cyclictest2703-21kworker/u8:010:05:263
9844991810,6cyclictest25531-21cat09:00:143
984299189,6cyclictest52950irq/141-enp3s0-12:05:131
9842991812,4cyclictest3154-21sh12:00:091
9842991811,5cyclictest13950-21seq11:44:241
9842991610,4cyclictest10069-21expr07:10:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional