You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 06:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Mar 15, 2026 00:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
126702239163,24sleep10-21swapper/119:08:251
124652235158,24sleep20-21swapper/219:05:442
125992230158,59sleep00-21swapper/019:07:280
126072215145,57sleep30-21swapper/319:07:353
12899999188,2cyclictest0-21swapper/319:52:103
12899999087,2cyclictest0-21swapper/320:51:133
12899996663,2cyclictest0-21swapper/323:30:303
12899995754,2cyclictest0-21swapper/321:41:073
109972570,3sleep09-21rcu_preempt21:30:140
12899994944,3cyclictest0-21swapper/320:00:303
1289399481,39cyclictest0-21swapper/219:40:102
1289399461,34cyclictest0-21swapper/222:18:192
1289399431,27cyclictest0-21swapper/221:06:472
1289399411,28cyclictest0-21swapper/220:52:512
1289399411,27cyclictest0-21swapper/220:57:112
1289399411,27cyclictest0-21swapper/220:31:592
1289399401,26cyclictest0-21swapper/221:59:302
1289399381,28cyclictest0-21swapper/223:11:272
1288899381,11cyclictest0-21swapper/121:44:551
1288899381,10cyclictest0-21swapper/119:26:431
1288399388,13cyclictest11963-21munin-run20:20:010
12883993810,13cyclictest14777-21munin-run21:40:010
12893993726,3cyclictest0-21swapper/221:20:312
1289399371,24cyclictest0-21swapper/222:46:352
1289399371,22cyclictest0-21swapper/220:05:512
1288899371,15cyclictest0-21swapper/120:56:391
1288899371,14cyclictest9388-21chrt23:54:591
1288899371,10cyclictest0-21swapper/119:50:391
12883993710,13cyclictest15393-21munin-run22:55:000
1289399361,24cyclictest0-21swapper/221:30:232
1289399361,24cyclictest0-21swapper/219:54:542
1289399361,24cyclictest0-21swapper/200:17:192
1289399361,23cyclictest0-21swapper/219:55:432
1289399361,21cyclictest0-21swapper/219:16:072
1289399361,20cyclictest0-21swapper/221:42:502
1289399361,12cyclictest0-21swapper/221:37:192
1288899361,14cyclictest0-21swapper/120:00:151
1288399361,12cyclictest0-21swapper/020:11:530
1289999350,25cyclictest6358-21kworker/3:020:24:133
12893993523,3cyclictest0-21swapper/222:32:592
1289399351,27cyclictest0-21swapper/222:44:152
1289399351,26cyclictest0-21swapper/219:37:102
1289399351,16cyclictest0-21swapper/200:10:112
1288899351,13cyclictest0-21swapper/123:07:071
1288899350,12cyclictest0-21swapper/121:50:271
1288899350,10cyclictest0-21swapper/120:09:551
1288399358,13cyclictest3785-21munin-run21:15:010
1289399341,31cyclictest0-21swapper/221:10:392
1289399341,22cyclictest0-21swapper/223:32:232
1289399341,21cyclictest0-21swapper/223:56:142
1289399341,21cyclictest0-21swapper/223:43:312
1289399341,21cyclictest0-21swapper/223:43:312
1289399341,20cyclictest0-21swapper/223:16:392
1289399341,19cyclictest0-21swapper/220:15:072
1289399341,12cyclictest0-21swapper/223:47:502
1288899341,13cyclictest0-21swapper/121:30:461
1288899341,13cyclictest0-21swapper/121:10:471
1288899341,11cyclictest0-21swapper/119:36:551
1288899340,14cyclictest0-21swapper/121:26:581
1288899340,12cyclictest0-21swapper/123:36:261
1289399334,20cyclictest0-21swapper/221:45:242
12893993323,3cyclictest0-21swapper/223:06:232
1289399331,25cyclictest0-21swapper/222:53:512
1289399331,24cyclictest0-21swapper/200:05:062
1289399331,24cyclictest0-21swapper/200:03:462
1289399331,22cyclictest0-21swapper/219:10:432
1289399331,21cyclictest0-21swapper/223:21:352
1289399331,21cyclictest0-21swapper/219:33:342
1289399331,20cyclictest0-21swapper/220:03:472
1289399331,20cyclictest0-21swapper/219:47:502
1289399331,18cyclictest0-21swapper/220:47:232
1289399331,18cyclictest0-21swapper/220:14:072
1289399331,17cyclictest0-21swapper/200:29:092
1289399331,12cyclictest0-21swapper/223:39:192
1288899331,9cyclictest0-21swapper/119:55:271
1288899331,13cyclictest0-21swapper/119:33:151
1288899331,12cyclictest0-21swapper/122:12:551
1288899331,10cyclictest0-21swapper/120:52:511
1288899330,13cyclictest0-21swapper/120:25:031
1288899330,10cyclictest0-21swapper/121:23:061
1288399335,13cyclictest29542-21munin-run21:00:010
1289999321,19cyclictest6358-21kworker/3:000:07:453
1289399321,23cyclictest0-21swapper/222:05:582
1289399321,19cyclictest0-21swapper/222:13:232
1289399321,13cyclictest0-21swapper/222:59:152
1289399321,10cyclictest0-21swapper/220:27:432
1288899321,22cyclictest5882-21smart_sda22:30:231
1288899321,21cyclictest0-21swapper/123:44:011
1288899321,21cyclictest0-21swapper/123:44:011
1288899321,15cyclictest0-21swapper/123:17:581
1288899321,12cyclictest10181-21perl23:55:151
1288899320,14cyclictest0-21swapper/120:38:351
1288899320,14cyclictest0-21swapper/100:33:431
1288899320,13cyclictest0-21swapper/122:17:191
1288399322,27cyclictest14175-21perf20:25:010
1289399311,22cyclictest0-21swapper/219:22:232
1289399311,19cyclictest0-21swapper/221:18:272
1289399311,19cyclictest0-21swapper/221:18:272
1289399311,17cyclictest0-21swapper/223:26:592
1289399311,14cyclictest0-21swapper/222:20:272
1289399311,13cyclictest0-21swapper/223:02:312
1289399311,13cyclictest0-21swapper/200:24:312
1289399311,10cyclictest0-21swapper/221:26:352
1289399310,11cyclictest0-21swapper/220:40:212
1288899311,12cyclictest0-21swapper/123:21:241
1288899311,11cyclictest0-21swapper/120:45:271
1288899311,10cyclictest0-21swapper/123:32:261
1288899310,9cyclictest0-21swapper/122:38:031
1288899310,13cyclictest0-21swapper/119:41:271
1288899310,12cyclictest0-21swapper/120:23:111
1288899310,10cyclictest0-21swapper/122:09:231
1288899310,10cyclictest0-21swapper/100:01:151
1288399311,9cyclictest0-21swapper/020:51:480
1288399311,10cyclictest0-21swapper/019:23:530
1288399310,9cyclictest0-21swapper/023:27:050
1288399310,11cyclictest0-21swapper/023:05:440
1288399310,10cyclictest0-21swapper/023:18:210
1289399301,11cyclictest0-21swapper/219:25:472
1288899306,3cyclictest0-21swapper/121:18:181
1288899306,3cyclictest0-21swapper/121:18:181
1288899301,13cyclictest0-21swapper/123:48:311
1288899301,10cyclictest0-21swapper/100:07:351
1288399303,12cyclictest22949-21munin-run20:45:010
1288399300,23cyclictest0-21swapper/021:21:490
1288399300,10cyclictest0-21swapper/020:02:130
1289999291,19cyclictest6358-21kworker/3:019:13:483
1289399297,8cyclictest0-21swapper/221:50:132
1289399291,25cyclictest0-21swapper/222:35:272
1289399291,20cyclictest0-21swapper/200:36:192
1289399291,18cyclictest0-21swapper/221:02:472
1289399291,16cyclictest0-21swapper/223:50:542
1289399291,16cyclictest0-21swapper/222:26:112
1289399291,12cyclictest0-21swapper/200:30:192
1289399291,10cyclictest0-21swapper/220:20:372
1288899297,3cyclictest0-21swapper/119:12:151
1288899295,3cyclictest0-21swapper/122:29:591
1288899291,9cyclictest0-21swapper/123:28:181
1288899291,9cyclictest0-21swapper/122:43:371
1288899291,14cyclictest8129-21df20:10:131
1288899290,8cyclictest0-21swapper/123:02:331
1288899290,12cyclictest0-21swapper/120:16:111
1288899290,10cyclictest17501-21/usr/sbin/munin20:30:191
1288399291,13cyclictest0-21swapper/000:21:410
1288399290,10cyclictest0-21swapper/020:09:010
1289999282,18cyclictest6358-21kworker/3:019:42:413
1288899281,9cyclictest0-21swapper/121:59:311
1288899281,12cyclictest0-21swapper/119:45:351
1288899280,19cyclictest0-21swapper/119:17:551
1288399287,4cyclictest0-21swapper/000:05:220
1288399282,12cyclictest11619-21idleruntime-cro00:00:010
1288399280,8cyclictest0-21swapper/022:29:370
1289999272,19cyclictest6358-21kworker/3:020:15:003
12899992710,8cyclictest6358-21kworker/3:019:22:493
1289399271,13cyclictest0-21swapper/222:04:302
1288899271,3cyclictest580-21systemd-journal19:25:001
1288899271,12cyclictest0-21swapper/122:21:231
1288899271,12cyclictest0-21swapper/121:02:071
1288899271,11cyclictest0-21swapper/100:38:191
1288899271,10cyclictest0-21swapper/120:44:191
12883992712,3cyclictest23524-21smartctl00:25:170
1288399271,11cyclictest0-21swapper/020:35:060
1288399271,10cyclictest0-21swapper/021:09:570
1288399271,10cyclictest0-21swapper/019:55:330
1289999262,11cyclictest6358-21kworker/3:019:49:133
1289999261,20cyclictest6358-21kworker/3:022:33:163
12899992610,8cyclictest6358-21kworker/3:021:16:093
12899992610,8cyclictest6358-21kworker/3:021:16:093
1289999260,19cyclictest6358-21kworker/3:022:08:133
1289399261,11cyclictest0-21swapper/220:38:072
1288899261,20cyclictest0-21swapper/100:13:081
1288899261,12cyclictest0-21swapper/100:15:031
1288899261,11cyclictest0-21swapper/122:04:071
1288899260,11cyclictest0-21swapper/122:47:351
1288399261,14cyclictest0-21swapper/022:40:490
1288399261,10cyclictest1577-21diskmemload22:48:080
1288399261,10cyclictest0-21swapper/020:31:590
1288399261,10cyclictest0-21swapper/019:25:450
1289999252,17cyclictest6358-21kworker/3:021:50:193
1289999252,10cyclictest6358-21kworker/3:023:54:393
1289999251,17cyclictest6358-21kworker/3:023:55:093
1289999251,12cyclictest6358-21kworker/3:023:11:493
1288899251,8cyclictest0-21swapper/121:45:271
1288899251,22cyclictest0-21swapper/100:25:161
1288899251,13cyclictest0-21swapper/121:09:591
1288899250,3cyclictest0-21swapper/122:53:511
1288399259,8cyclictest0-21swapper/023:36:250
1288399251,10cyclictest0-21swapper/019:40:090
1288399250,9cyclictest0-21swapper/021:02:450
1288399250,8cyclictest0-21swapper/019:18:250
1288399250,16cyclictest0-21swapper/019:45:250
1288399250,11cyclictest0-21swapper/023:14:090
1288399250,10cyclictest0-21swapper/021:43:410
1289999242,9cyclictest15953-21df_inode22:55:113
1289999242,8cyclictest6358-21kworker/3:023:21:213
1289999242,8cyclictest6358-21kworker/3:000:13:553
1289999241,17cyclictest6358-21kworker/3:022:12:333
12899992411,3cyclictest6358-21kworker/3:023:06:173
1289999241,13cyclictest6358-21kworker/3:022:49:053
1289999240,11cyclictest6358-21kworker/3:022:16:403
1288899241,21cyclictest0-21swapper/121:36:281
1288899240,8cyclictest0-21swapper/100:22:131
1288399249,3cyclictest22173-21perl21:55:160
1288399246,15cyclictest19785-21idleruntime-cro23:05:010
1288399241,9cyclictest0-21swapper/022:13:330
1288399241,3cyclictest4366-21kworker/u8:123:20:230
1288399241,12cyclictest0-21swapper/019:38:090
1288399241,10cyclictest0-21swapper/022:56:360
1288399240,10cyclictest0-21swapper/020:28:010
1289999236,3cyclictest6358-21kworker/3:022:04:373
1289999230,17cyclictest6358-21kworker/3:023:39:533
1288899231,11cyclictest16552-21sed22:55:161
1288899231,10cyclictest0-21swapper/123:14:231
1288399239,3cyclictest0-21swapper/019:10:290
1288399235,15cyclictest27971-21munin-run22:10:010
1288399234,3cyclictest0-21swapper/021:46:050
1288399231,9cyclictest0-21swapper/022:00:250
1288399231,9cyclictest0-21swapper/020:46:320
1288399231,11cyclictest18452-21/usr/sbin/munin00:15:250
1288399231,11cyclictest0-21swapper/023:40:580
1288399231,11cyclictest0-21swapper/023:40:580
1288399231,10cyclictest32063-21ntp_states23:30:220
1288399231,10cyclictest0-21swapper/023:50:540
1288399230,9cyclictest0-21swapper/021:53:250
1288399230,9cyclictest0-21swapper/000:37:210
1288399230,14cyclictest0-21swapper/000:03:130
1288399230,10cyclictest0-21swapper/019:30:130
1289999228,8cyclictest0-21swapper/321:20:113
12899992212,2cyclictest6358-21kworker/3:000:02:293
1289999221,10cyclictest6358-21kworker/3:021:30:353
1288399226,3cyclictest0-21swapper/022:18:150
1288399221,10cyclictest0-21swapper/023:48:040
1288399221,10cyclictest0-21swapper/021:26:310
12883992210,3cyclictest0-21swapper/022:35:050
1288399220,9cyclictest0-21swapper/000:13:290
260772211,3sleep326103-21ntp_states00:30:213
1289999215,4cyclictest1577-21diskmemload23:42:193
1289999215,4cyclictest1577-21diskmemload23:42:193
1289999215,2cyclictest6358-21kworker/3:022:36:363
1289999212,8cyclictest6358-21kworker/3:019:59:573
1289999212,8cyclictest6358-21kworker/3:000:29:523
1289999212,7cyclictest6358-21kworker/3:020:07:203
1289999212,10cyclictest6358-21kworker/3:020:41:553
1289999211,9cyclictest0-21swapper/321:10:483
1289999211,8cyclictest6358-21kworker/3:023:16:073
1289999211,7cyclictest833-21runrttasks21:26:243
1289999211,13cyclictest86550irq/126-enp1s0-20:15:123
1289999211,11cyclictest27564-21/usr/sbin/munin20:55:143
1289999211,11cyclictest14385-21/usr/sbin/munin20:25:133
1289999211,10cyclictest6358-21kworker/3:020:32:173
1288399217,3cyclictest4600-21grep21:15:160
1288399217,3cyclictest4600-21grep21:15:160
1289999206,2cyclictest6358-21kworker/3:019:37:033
1289999205,5cyclictest6358-21kworker/3:021:55:043
1289999201,7cyclictest6358-21kworker/3:021:45:493
1289999201,7cyclictest29762-21/usr/sbin/munin21:00:173
1289999201,6cyclictest27764-21fschecks_time00:35:153
1289999201,13cyclictest0-21swapper/300:18:573
1289999201,11cyclictest12795-21/usr/sbin/munin21:35:153
1289999201,10cyclictest5232-21/usr/sbin/munin23:45:163
1289999201,10cyclictest17948-21munin-run19:20:013
1288399207,3cyclictest0-21swapper/019:52:410
1288399204,5cyclictest0-21swapper/000:34:280
1288399201,4cyclictest5685-21ntp_states22:30:200
1289999191,6cyclictest17809-21/usr/sbin/munin23:00:153
12899991913,3cyclictest454-21df22:20:113
1289999191,12cyclictest0-21swapper/319:28:503
1289999191,12cyclictest0-21swapper/300:20:143
1289999190,9cyclictest6358-21kworker/3:019:33:273
1288399191,13cyclictest0-21swapper/022:21:370
1289999183,3cyclictest86850irq/129-enp1s0-20:35:253
1289999183,3cyclictest24460-21switchtime20:45:243
1289999181,7cyclictest9363-21diskstats22:40:123
1289999181,5cyclictest0-21swapper/323:25:183
1289999181,10cyclictest0-21swapper/322:25:123
1289999174,5cyclictest0-21swapper/321:05:253
1289999171,5cyclictest14544-21pmu-power22:50:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional