You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 02:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Fri Jun 12, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
176662238162,24sleep20-21swapper/219:07:392
177122225154,23sleep30-21swapper/319:08:163
176252204168,23sleep10-21swapper/119:07:081
176582203164,26sleep00-21swapper/019:07:320
2680321660,5sleep11793899cyclictest21:55:111
1135221180,2sleep21794299cyclictest23:45:212
17942991091,2cyclictest0-21swapper/221:11:372
202552690,3sleep230-21rcuc/200:05:232
17942995450,3cyclictest0-21swapper/220:04:492
295272510,2sleep00-21swapper/019:34:200
17932994312,3cyclictest6032-21awk19:55:010
1794999421,29cyclictest0-21swapper/322:40:243
17942994239,2cyclictest0-21swapper/219:54:112
1794299397,5cyclictest18224-21switchtime00:00:252
17942993721,8cyclictest0-21swapper/221:26:302
17932993611,3cyclictest21643-21/usr/sbin/munin20:30:210
1794999351,20cyclictest0-21swapper/323:34:563
1793299359,5cyclictest827-21runrttasks00:34:190
1794999340,20cyclictest0-21swapper/323:17:403
17942993431,2cyclictest0-21swapper/221:07:062
1794299340,9cyclictest0-21swapper/220:12:282
1793899341,10cyclictest2201-21diskstats19:45:141
1793299344,3cyclictest12050irq/123-ahci[0022:20:010
17932993411,3cyclictest8105-21/usr/sbin/munin23:40:260
17932993411,3cyclictest25950-21sort20:40:010
17932993411,3cyclictest14693-21/usr/sbin/munin23:55:180
1793299341,10cyclictest7223-21diskstats21:10:150
1794999331,16cyclictest0-21swapper/321:41:003
17942993318,9cyclictest0-21swapper/221:44:472
1794299331,16cyclictest0-21swapper/223:01:462
1793899331,13cyclictest0-21swapper/122:44:581
17932993311,3cyclictest31672-21/usr/sbin/munin23:20:230
1793299331,11cyclictest18478-21/usr/sbin/munin22:50:220
17932993310,3cyclictest827-21runrttasks20:05:480
17942993218,7cyclictest0-21swapper/222:08:592
17942993214,4cyclictest0-21swapper/221:23:542
1793899327,12cyclictest13605-21cpuspeed_turbos21:25:131
17938993216,8cyclictest0-21swapper/119:36:101
1793299327,5cyclictest802-21sssd_be19:15:010
1793299322,7cyclictest4670-21/usr/sbin/munin21:05:110
1793299321,9cyclictest7192-21sort23:35:240
1793299321,6cyclictest2741-21find23:25:220
1793299321,5cyclictest29380-21fschecks_time22:00:160
1793299321,5cyclictest29380-21fschecks_time22:00:160
17932993214,15cyclictest28832sleep022:15:010
17932993212,3cyclictest24137-21missed_timers19:20:180
17932993211,3cyclictest23764-21cat23:00:180
17932993210,3cyclictest827-21runrttasks21:18:290
17932993210,3cyclictest12185-21expr22:35:140
1794999311,14cyclictest0-21swapper/300:21:533
1794999311,14cyclictest0-21swapper/300:21:533
1794999311,13cyclictest0-21swapper/323:03:563
1794999311,13cyclictest0-21swapper/322:31:213
1794999311,12cyclictest28053-21cat21:55:243
1794999310,17cyclictest0-21swapper/300:09:453
1793899318,7cyclictest13312-21tune2fs20:10:141
17938993113,6cyclictest3495-21kworker/1:123:36:581
1793899311,22cyclictest0-21swapper/122:24:501
1793899311,12cyclictest0-21swapper/119:40:061
1793899311,10cyclictest0-21swapper/119:27:151
1793299318,7cyclictest19842-21if_enp1s000:05:170
1793299317,5cyclictest0-21swapper/021:58:050
1793299314,3cyclictest14822-21awk20:15:010
1793299312,13cyclictest10292-21head22:30:170
1793299311,6cyclictest32012-21/usr/sbin/munin19:40:240
1793299311,6cyclictest0-21swapper/000:10:210
1793299311,3cyclictest24455-21/usr/sbin/munin21:50:180
17932993111,3cyclictest23469-21cut21:45:210
17932993111,3cyclictest13984-21unixbench_multi23:50:250
1793299311,11cyclictest16062-21perf22:45:020
1794999301,22cyclictest0-21swapper/320:20:573
1794999301,15cyclictest0-21swapper/322:24:493
1794999301,15cyclictest0-21swapper/322:06:163
1794299308,8cyclictest0-21swapper/223:52:062
17942993013,3cyclictest0-21swapper/223:17:112
1793899308,9cyclictest1632-21timerandwakeup20:55:241
1793899308,9cyclictest1632-21timerandwakeup20:55:241
1793899301,12cyclictest21881-21ls19:15:181
1793299309,3cyclictest827-21runrttasks19:48:110
1793299302,3cyclictest28008-21munin-run20:45:010
1793299302,2cyclictest0-21swapper/021:30:050
1793299301,4cyclictest30904-21cat23:15:260
1793299301,4cyclictest15046-21/usr/sbin/munin20:15:270
1793299301,11cyclictest6248-21/usr/sbin/munin19:55:200
1793299301,10cyclictest6655-21diskmemload21:43:510
17932993010,3cyclictest827-21runrttasks19:19:140
17932993010,3cyclictest8196-21basename22:25:210
17932993010,3cyclictest21865-21processes22:55:240
17932993010,2cyclictest20028-21cat21:40:010
1794999291,16cyclictest0-21swapper/321:08:243
1794999291,13cyclictest0-21swapper/323:47:293
1794999291,12cyclictest0-21swapper/323:51:253
1794999291,12cyclictest0-21swapper/319:56:573
1794999290,14cyclictest0-21swapper/300:38:133
17942992917,3cyclictest0-21swapper/223:34:032
17942992913,3cyclictest11065-21ls22:30:262
1794299291,16cyclictest0-21swapper/221:33:222
1794299291,11cyclictest0-21swapper/223:37:472
1793899291,16cyclictest11066-21sort22:30:261
1793899291,10cyclictest16574-21memory21:30:191
1793899291,10cyclictest0-21swapper/120:27:221
1793299297,4cyclictest0-21swapper/021:28:510
1793299292,4cyclictest24195-21hddtemp_smartct00:15:170
17932992910,4cyclictest0-21swapper/021:20:180
17932992910,3cyclictest29470-21sendmail_mailqu20:45:220
17932992910,3cyclictest28383-21grep00:25:150
17949992813,3cyclictest0-21swapper/319:17:023
1794999281,14cyclictest0-21swapper/322:38:443
1794999281,13cyclictest0-21swapper/319:32:123
1794999280,25cyclictest0-21swapper/320:10:573
1794999280,17cyclictest0-21swapper/300:34:003
1794999280,14cyclictest0-21swapper/320:15:213
17942992811,3cyclictest0-21swapper/222:50:462
1793899285,1cyclictest0-21swapper/119:55:181
1793899282,12cyclictest4166-21ntp_states22:15:221
17938992811,14cyclictest0-21swapper/123:50:021
17938992810,4cyclictest9462-21munin-run22:30:011
1793299281,5cyclictest17343-21/usr/sbin/munin22:45:230
17932992811,3cyclictest25754-21sh19:25:110
17932992811,3cyclictest19443-21/usr/sbin/munin20:25:150
17932992810,3cyclictest10321-21grep23:45:110
1794999277,6cyclictest0-21swapper/320:35:473
1794999273,2cyclictest87450irq/127-enp1s0-19:44:423
1794999271,16cyclictest0-21swapper/321:37:123
1794999271,16cyclictest0-21swapper/320:31:113
1794999271,15cyclictest0-21swapper/300:04:373
1794999271,13cyclictest0-21swapper/322:02:043
1794999271,13cyclictest0-21swapper/322:02:043
1794999271,13cyclictest0-21swapper/319:53:523
1794999271,13cyclictest0-21swapper/319:24:163
1794999270,14cyclictest0-21swapper/322:18:213
1794999270,14cyclictest0-21swapper/322:13:293
1794999270,14cyclictest0-21swapper/321:27:363
1794999270,13cyclictest0-21swapper/321:20:203
17942992712,6cyclictest0-21swapper/221:57:592
17942992712,3cyclictest15829-21awk23:55:222
17942992712,3cyclictest0-21swapper/222:21:422
1793899278,3cyclictest0-21swapper/120:19:061
1793899276,11cyclictest0-21swapper/122:12:301
1793899272,11cyclictest27056-21munin-run23:10:021
17938992717,3cyclictest0-21swapper/121:54:221
17938992713,3cyclictest0-21swapper/120:30:341
1793899271,17cyclictest29828-21diskstats23:15:141
17938992711,2cyclictest0-21swapper/122:05:181
1793899271,11cyclictest30951-21fschecks_time20:50:141
1793899271,10cyclictest22670-21grep23:00:021
17938992710,4cyclictest0-21swapper/121:15:151
1793299275,3cyclictest0-21swapper/023:30:110
1793299271,9cyclictest0-21swapper/022:05:130
1793299271,5cyclictest4443-21latency_hist21:05:020
1793299271,3cyclictest31781-21sh19:40:000
1793299271,14cyclictest19216-21idleruntime-cro20:25:010
1794999262,2cyclictest87350irq/126-enp1s0-19:26:303
1794999261,15cyclictest0-21swapper/321:10:403
1794999261,14cyclictest0-21swapper/323:36:043
1794999261,14cyclictest0-21swapper/322:56:123
1794999261,13cyclictest0-21swapper/300:15:333
1794999261,13cyclictest0-21swapper/300:14:533
1794299261,9cyclictest0-21swapper/220:44:382
17942992613,3cyclictest0-21swapper/220:09:352
17942992611,6cyclictest0-21swapper/200:12:022
1794299261,13cyclictest0-21swapper/223:09:182
1794299260,11cyclictest0-21swapper/221:01:522
1793899263,3cyclictest87550irq/128-enp1s0-22:52:221
17938992616,3cyclictest25050-21processes20:35:221
1793899261,20cyclictest0-21swapper/122:02:231
1793899261,20cyclictest0-21swapper/122:02:231
1793899261,19cyclictest0-21swapper/121:41:121
1793899261,19cyclictest0-21swapper/100:20:521
1793899261,19cyclictest0-21swapper/100:20:521
1793899261,19cyclictest0-21swapper/100:19:191
1793899261,15cyclictest0-21swapper/121:05:121
1793899261,11cyclictest24862-21munin-run23:05:011
1793899260,20cyclictest0-21swapper/100:38:241
1793299261,10cyclictest0-21swapper/000:20:320
1793299261,10cyclictest0-21swapper/000:20:310
17932992610,2cyclictest17554-21grep00:00:150
1793299260,5cyclictest0-21swapper/000:39:210
1793299260,14cyclictest0-21swapper/020:53:140
1794999254,3cyclictest0-21swapper/320:06:473
1794999254,13cyclictest87350irq/126-enp1s0-23:26:263
1794999251,9cyclictest0-21swapper/320:50:133
1794999251,14cyclictest0-21swapper/321:51:243
1794999251,14cyclictest0-21swapper/321:32:403
1794999251,14cyclictest0-21swapper/320:01:493
1794999251,14cyclictest0-21swapper/300:25:083
1794999251,13cyclictest0-21swapper/323:21:003
1794999251,13cyclictest0-21swapper/319:36:283
1794999251,12cyclictest0-21swapper/321:00:203
17949992510,3cyclictest0-21swapper/322:45:243
1794999250,20cyclictest0-21swapper/323:55:093
1794299256,7cyclictest31456-21cat00:30:242
1794299251,15cyclictest0-21swapper/223:10:072
1794299251,15cyclictest0-21swapper/223:10:072
1794299251,14cyclictest0-21swapper/222:14:472
1793899258,6cyclictest0-21swapper/120:06:021
1793899251,8cyclictest0-21swapper/123:29:271
1793899251,12cyclictest28274-21expr19:30:151
1793899251,11cyclictest16662-21runrttasks00:00:001
1793899251,10cyclictest18998-21seq20:22:201
1793899250,3cyclictest0-21swapper/100:32:141
17932992512,2cyclictest0-21swapper/022:22:060
1793299250,10cyclictest0-21swapper/023:09:520
1794999244,13cyclictest87350irq/126-enp1s0-22:51:273
1794999243,3cyclictest0-21swapper/321:15:143
1794999241,9cyclictest0-21swapper/321:46:003
1794999241,13cyclictest0-21swapper/323:07:563
1794999241,11cyclictest8713-21ps22:25:293
1794299247,3cyclictest0-21swapper/219:28:232
17942992412,3cyclictest0-21swapper/223:29:072
1794299241,12cyclictest0-21swapper/223:24:072
1794299240,15cyclictest0-21swapper/221:45:232
1793899244,16cyclictest0-21swapper/120:00:421
1793899242,4cyclictest18253-21munin-run22:50:001
1793899241,15cyclictest0-21swapper/123:13:591
1793899241,15cyclictest0-21swapper/123:13:581
1793899241,14cyclictest0-21swapper/121:13:091
1793899241,12cyclictest0-21swapper/123:46:461
1793899241,11cyclictest5675-21latency_hist23:35:011
1793899241,11cyclictest20702-21users00:05:271
1793899241,10cyclictest28015-21idleruntime-cro20:45:001
1793899241,10cyclictest23833-21grep21:45:251
1793299241,6cyclictest27627-21diskstats23:10:130
1793299241,6cyclictest27627-21diskstats23:10:130
1794999231,16cyclictest0-21swapper/323:14:083
1794999231,16cyclictest0-21swapper/323:14:083
1794999231,14cyclictest0-21swapper/320:40:413
1794999230,13cyclictest0-21swapper/320:45:493
1794299237,4cyclictest21646-21cat22:55:202
1794299236,8cyclictest0-21swapper/219:38:552
1794299235,3cyclictest0-21swapper/220:54:502
1794299234,8cyclictest0-21swapper/221:35:062
1793899237,3cyclictest30367-21cat20:50:011
17938992314,1cyclictest0-21swapper/100:00:241
1793899231,3cyclictest87650irq/129-enp1s0-19:23:551
17938992312,6cyclictest0-21swapper/121:22:101
1793899231,14cyclictest0-21swapper/121:01:441
1793899231,11cyclictest0-21swapper/123:20:151
1793899230,14cyclictest0-21swapper/121:37:391
1794999226,10cyclictest0-21swapper/319:13:273
17949992212,7cyclictest3684-21sshd19:48:213
1794999221,13cyclictest0-21swapper/323:44:253
1794299228,3cyclictest87350irq/126-enp1s0-00:20:162
1794299228,3cyclictest87350irq/126-enp1s0-00:20:162
1794299228,3cyclictest0-21swapper/220:26:222
1794299227,3cyclictest0-21swapper/219:55:232
1794299227,3cyclictest0-21swapper/219:16:462
1794299224,10cyclictest0-21swapper/221:16:542
1794299221,8cyclictest0-21swapper/219:11:462
17942992214,6cyclictest0-21swapper/200:39:142
17942992214,4cyclictest23626-21cpuspeed_turbos00:15:122
1794299221,13cyclictest0-21swapper/220:32:422
1794299221,10cyclictest0-21swapper/222:28:382
1794299220,5cyclictest19662-21kworker/u8:221:54:222
1793899229,3cyclictest21638-21expr00:10:121
1793899229,3cyclictest0-21swapper/119:14:381
17938992212,3cyclictest0-21swapper/119:53:541
1793899221,10cyclictest28244-21cstates00:25:111
1794999211,11cyclictest0-21swapper/320:55:123
1794999211,11cyclictest0-21swapper/320:55:123
1794299216,7cyclictest0-21swapper/222:41:222
1794299216,3cyclictest0-21swapper/219:30:072
1794299215,3cyclictest0-21swapper/220:22:522
17942992111,3cyclictest0-21swapper/223:41:232
1794299211,12cyclictest0-21swapper/220:46:182
1794299211,11cyclictest0-21swapper/222:18:232
1794299211,10cyclictest0-21swapper/220:58:022
1794299211,10cyclictest0-21swapper/220:58:022
1794299211,10cyclictest0-21swapper/220:35:462
1794299209,3cyclictest0-21swapper/200:25:582
1794299206,5cyclictest34762sleep219:45:432
1794299206,3cyclictest0-21swapper/219:24:262
1794299201,6cyclictest17083-21irqstats22:45:182
1794299201,12cyclictest0-21swapper/222:04:182
1794299201,12cyclictest0-21swapper/222:04:182
1794299201,10cyclictest0-21swapper/220:17:382
1793899201,8cyclictest13144-21smartctl22:35:241
1793899200,10cyclictest0-21swapper/123:40:181
1793299207,2cyclictest87350irq/126-enp1s0-20:02:260
1794299190,11cyclictest0-21swapper/222:36:142
1794999171,9cyclictest0-21swapper/320:25:073
1794299173,2cyclictest87650irq/129-enp1s0-19:40:032
1793299175,3cyclictest32169-21kworker/0:220:55:450
1793299175,3cyclictest32169-21kworker/0:220:55:450
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional