You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 12:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri Apr 17, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
161299271,1cyclictest24834-21/usr/sbin/munin22:44:583
161299270,26cyclictest0-21swapper/322:30:513
161299270,1cyclictest31196-21apt-get23:54:463
161299260,25cyclictest0-21swapper/323:19:493
161299260,0cyclictest0-21swapper/322:38:163
160799260,25cyclictest20889-21latency_hist00:15:002
1605992624,1cyclictest0-21swapper/121:14:531
1612992522,3cyclictest0-21swapper/322:09:053
1612992521,3cyclictest16995-21ssh23:39:553
1612992520,4cyclictest7918-21ssh22:26:473
1612992520,4cyclictest28457-21diskmemload00:30:423
1612992520,4cyclictest0-21swapper/321:36:533
161299250,0cyclictest16498-21ssh23:07:573
160799250,24cyclictest28457-21diskmemload22:37:162
160799250,24cyclictest28064-21sh22:46:432
160299250,24cyclictest0-21swapper/021:55:080
1612992421,3cyclictest0-21swapper/323:41:283
1612992421,3cyclictest0-21swapper/322:49:093
1612992420,3cyclictest0-21swapper/322:13:463
1612992418,5cyclictest9175-21perf19:25:003
161299240,24cyclictest0-21swapper/300:16:523
161299240,23cyclictest32221-21ssh00:26:473
161299240,23cyclictest28457-21diskmemload23:11:533
161299240,23cyclictest28457-21diskmemload00:06:533
161299240,23cyclictest0-21swapper/322:02:573
161299240,21cyclictest28457-21diskmemload22:24:283
161299240,0cyclictest0-21swapper/323:49:233
161299240,0cyclictest0-21swapper/322:57:023
161299240,0cyclictest0-21swapper/300:12:093
1607992423,1cyclictest9971-21ssh00:36:522
160799240,2cyclictest605-21avahi-daemon19:15:002
160799240,23cyclictest3774-21ssh21:49:592
1605992424,0cyclictest0-21swapper/123:42:111
1605992423,1cyclictest28457-21diskmemload22:21:411
1605992422,1cyclictest0-21swapper/122:38:091
160599240,23cyclictest0-21swapper/123:03:571
160599240,1cyclictest0-21swapper/100:08:391
160599240,0cyclictest0-21swapper/121:38:511
1602992423,1cyclictest28457-21diskmemload23:10:150
1602992423,1cyclictest28457-21diskmemload22:29:210
1602992422,0cyclictest151rcu_preempt22:57:200
160299240,24cyclictest0-21swapper/021:33:180
160299240,24cyclictest0-21swapper/000:03:330
160299240,23cyclictest28457-21diskmemload22:17:390
160299240,23cyclictest0-21swapper/023:51:320
160299240,23cyclictest0-21swapper/000:36:470
160299240,23cyclictest0-21swapper/000:23:520
160299240,22cyclictest313-21systemd-journal00:00:010
160299240,1cyclictest0-21swapper/022:39:480
160299240,0cyclictest151rcu_preempt23:21:440
1612992323,0cyclictest0-21swapper/323:25:153
1612992323,0cyclictest0-21swapper/321:57:043
1612992321,2cyclictest0-21swapper/300:36:513
1612992320,3cyclictest0-21swapper/323:00:143
1612992320,3cyclictest0-21swapper/320:19:463
1612992319,3cyclictest28457-21diskmemload21:50:303
1612992319,3cyclictest28457-21diskmemload21:18:263
1612992319,3cyclictest28457-21diskmemload21:18:263
161299230,23cyclictest0-21swapper/323:33:023
161299230,23cyclictest0-21swapper/321:47:353
161299230,23cyclictest0-21swapper/300:22:013
161299230,22cyclictest805-21ssh22:52:003
161299230,22cyclictest606-21dbus-daemon21:10:003
161299230,22cyclictest29969-21ssh21:12:153
161299230,22cyclictest12211-21ssh21:26:273
161299230,22cyclictest0-21swapper/321:40:093
161299230,0cyclictest0-21swapper/323:20:123
161299230,0cyclictest0-21swapper/321:21:063
1607992323,0cyclictest28457-21diskmemload21:51:462
1607992323,0cyclictest0-21swapper/223:28:192
1607992323,0cyclictest0-21swapper/223:21:172
1607992323,0cyclictest0-21swapper/222:52:122
1607992323,0cyclictest0-21swapper/222:15:342
1607992323,0cyclictest0-21swapper/222:10:092
1607992323,0cyclictest0-21swapper/222:09:562
1607992323,0cyclictest0-21swapper/222:02:042
1607992321,1cyclictest0-21swapper/223:00:012
160799230,23cyclictest0-21swapper/222:24:402
160799230,23cyclictest0-21swapper/221:59:382
160799230,22cyclictest28457-21diskmemload23:35:322
160799230,22cyclictest22240-21ssh21:36:272
160799230,22cyclictest13724-21ssh21:28:592
160799230,20cyclictest0-21swapper/221:17:162
160799230,20cyclictest0-21swapper/221:17:162
160799230,0cyclictest0-21swapper/223:34:412
1605992323,0cyclictest28457-21diskmemload22:47:391
1605992323,0cyclictest28457-21diskmemload22:08:141
1605992323,0cyclictest0-21swapper/121:26:201
1605992323,0cyclictest0-21swapper/121:22:321
1605992323,0cyclictest0-21swapper/100:30:001
1605992321,1cyclictest20298-21sh23:10:151
1605992321,1cyclictest0-21swapper/123:34:511
1605992321,1cyclictest0-21swapper/122:55:401
1605992320,2cyclictest20387-21ntp_states21:34:531
160599230,23cyclictest0-21swapper/100:23:421
160599230,23cyclictest0-21swapper/100:16:271
160599230,22cyclictest32155-21diskmemload22:19:501
160599230,22cyclictest17049-21sh23:08:481
160599230,22cyclictest0-21swapper/122:11:431
160599230,22cyclictest0-21swapper/121:51:481
160599230,22cyclictest0-21swapper/120:39:451
160599230,0cyclictest28457-21diskmemload23:38:321
160599230,0cyclictest0-21swapper/122:01:241
160599230,0cyclictest0-21swapper/100:11:531
1602992323,0cyclictest28457-21diskmemload21:51:290
1602992323,0cyclictest0-21swapper/023:35:470
1602992322,1cyclictest141rcuc/000:26:040
1602992321,1cyclictest0-21swapper/023:16:190
1602992321,1cyclictest0-21swapper/000:32:150
160299231,21cyclictest13-21ksoftirqd/023:01:580
160299230,23cyclictest0-21swapper/022:40:550
160299230,22cyclictest4727-21apt-get22:24:490
160299230,22cyclictest28457-21diskmemload22:45:410
160299230,22cyclictest28457-21diskmemload00:16:040
160299230,22cyclictest0-21swapper/021:23:360
160299230,22cyclictest0-21swapper/000:14:470
160299230,1cyclictest0-21swapper/022:34:300
160299230,0cyclictest0-21swapper/022:50:450
160299230,0cyclictest0-21swapper/022:01:560
1612992220,2cyclictest0-21swapper/322:15:473
1612992220,1cyclictest606-21dbus-daemon20:40:013
1612992219,2cyclictest28457-21diskmemload23:57:003
1612992218,4cyclictest0-21swapper/320:59:493
161299220,21cyclictest7361-21kworker/u8:221:30:293
161299220,21cyclictest21149-21idleruntime-cro19:50:013
161299220,21cyclictest0-21swapper/321:04:473
161299220,1cyclictest619-21polkitd20:25:013
1607992222,0cyclictest28457-21diskmemload22:42:312
1607992222,0cyclictest0-21swapper/223:50:532
1607992222,0cyclictest0-21swapper/223:13:572
1607992222,0cyclictest0-21swapper/222:30:062
1607992222,0cyclictest0-21swapper/200:07:152
1607992221,1cyclictest28457-21diskmemload21:30:042
1607992220,0cyclictest0-21swapper/200:27:242
160799220,2cyclictest22669-21ssh00:17:442
160799220,22cyclictest0-21swapper/223:47:492
160799220,22cyclictest0-21swapper/221:21:322
160799220,22cyclictest0-21swapper/200:00:292
160799220,21cyclictest8032-21munin-run20:30:012
160799220,21cyclictest8032-21munin-run20:30:012
160799220,21cyclictest28457-21diskmemload23:05:492
160799220,21cyclictest28457-21diskmemload21:10:412
160799220,21cyclictest25606-21apt-get21:05:022
160799220,1cyclictest4187-21ssh00:30:412
160799220,1cyclictest14586-21apt-get19:39:492
160799220,0cyclictest27068-21ssh00:21:412
160799220,0cyclictest0-21swapper/223:17:512
1605992222,0cyclictest0-21swapper/123:58:471
1605992222,0cyclictest0-21swapper/121:42:521
1605992221,1cyclictest32039-21ssh23:23:091
1605992220,1cyclictest606-21dbus-daemon20:45:011
1605992220,1cyclictest1498-21ssh21:15:201
1605992220,1cyclictest1498-21ssh21:15:201
1605992220,1cyclictest0-21swapper/123:20:011
1605992220,1cyclictest0-21swapper/121:49:401
160599220,22cyclictest0-21swapper/122:31:061
160599220,21cyclictest0-21swapper/123:50:021
160599220,21cyclictest0-21swapper/123:45:331
160599220,21cyclictest0-21swapper/122:25:091
160599220,21cyclictest0-21swapper/121:55:301
160599220,21cyclictest0-21swapper/100:36:001
160599220,1cyclictest28412-21cron20:05:001
160599220,1cyclictest23213-21perf21:00:001
160599220,1cyclictest0-21swapper/100:03:531
1602992222,0cyclictest28457-21diskmemload21:16:530
1602992222,0cyclictest28457-21diskmemload21:16:530
1602992222,0cyclictest0-21swapper/023:43:120
1602992222,0cyclictest0-21swapper/021:36:220
1602992222,0cyclictest0-21swapper/021:26:510
1602992222,0cyclictest0-21swapper/000:06:000
1602992221,1cyclictest28457-21diskmemload21:40:300
1602992220,1cyclictest606-21dbus-daemon19:25:000
1602992220,1cyclictest0-21swapper/021:10:170
160299220,21cyclictest28457-21diskmemload22:05:090
160299220,21cyclictest0-21swapper/023:45:580
160299220,21cyclictest0-21swapper/021:47:140
160299220,1cyclictest6252-21apt-get20:29:490
160299220,1cyclictest6252-21apt-get20:29:490
160299220,1cyclictest24971-21sh22:11:260
160299220,1cyclictest15060-21ssh23:05:340
161299210,21cyclictest0-21swapper/319:34:473
161299210,20cyclictest21791-21apt-get19:54:473
161299210,20cyclictest0-21swapper/319:14:473
161299210,0cyclictest0-21swapper/320:04:453
1607992119,1cyclictest18406-21cron20:50:002
160799210,21cyclictest0-21swapper/223:40:352
160799210,21cyclictest0-21swapper/221:40:152
160799210,1cyclictest6886-21sh22:25:052
1605992119,1cyclictest26295-21dpkg21:05:021
160599210,21cyclictest0-21swapper/100:30:031
160599210,20cyclictest0-21swapper/121:04:471
160599210,0cyclictest0-21swapper/120:24:481
160299210,20cyclictest0-21swapper/021:04:480
1612992020,0cyclictest0-21swapper/320:09:473
1612992018,1cyclictest20813-21perf20:55:003
161299200,20cyclictest0-21swapper/320:49:473
161299200,0cyclictest0-21swapper/320:30:023
1607992020,0cyclictest0-21swapper/219:34:482
160799200,20cyclictest0-21swapper/220:19:472
160799200,0cyclictest0-21swapper/219:54:472
1605992020,0cyclictest0-21swapper/119:59:471
1602992020,0cyclictest0-21swapper/021:09:470
1602992020,0cyclictest0-21swapper/020:19:470
1602992018,1cyclictest313-21systemd-journal19:35:010
160299200,0cyclictest0-21swapper/020:49:470
1612991917,2cyclictest0-21swapper/319:45:013
1607991919,0cyclictest0-21swapper/220:39:462
1607991919,0cyclictest0-21swapper/220:24:492
1605991919,0cyclictest1414-21apt-get20:19:471
1605991919,0cyclictest0-21swapper/120:34:481
1605991919,0cyclictest0-21swapper/120:29:481
1605991919,0cyclictest0-21swapper/120:29:481
1602991919,0cyclictest0-21swapper/020:34:480
1602991919,0cyclictest0-21swapper/019:19:460
1602991918,1cyclictest21650irq/140-nvme0q119:55:010
1612991818,0cyclictest0-21swapper/320:29:493
1612991818,0cyclictest0-21swapper/320:29:493
1612991818,0cyclictest0-21swapper/319:19:463
1607991818,0cyclictest31443-21apt-get20:14:472
1607991818,0cyclictest24184-21apt-get19:59:472
1607991816,1cyclictest0-21swapper/219:45:012
1607991815,2cyclictest30796-21cron20:10:002
1602991816,2cyclictest4365-21basename19:15:000
1612991714,1cyclictest313-21systemd-journal19:40:013
1607991717,0cyclictest0-21swapper/220:54:452
1607991715,2cyclictest23189-21sh21:00:002
1605991717,0cyclictest0-21swapper/119:44:481
1605991717,0cyclictest0-21swapper/119:25:001
1605991716,1cyclictest0-21swapper/119:49:471
1602991717,0cyclictest0-21swapper/020:59:490
1602991717,0cyclictest0-21swapper/020:14:470
160299171,16cyclictest4582-21kworker/0:020:43:340
1612991616,0cyclictest0-21swapper/320:14:473
1607991616,0cyclictest0-21swapper/220:34:482
1605991616,0cyclictest0-21swapper/119:54:471
1605991615,1cyclictest0-21swapper/119:30:001
1605991615,0cyclictest19039-21apt-get20:54:451
1605991614,1cyclictest1496-21apt-get19:10:011
160599160,15cyclictest0-21swapper/120:49:491
160299163,0cyclictest151rcu_preempt20:39:490
1602991616,0cyclictest0-21swapper/020:24:490
1602991614,1cyclictest0-21swapper/019:59:390
1612991515,0cyclictest0-21swapper/319:29:473
1605991515,0cyclictest0-21swapper/120:09:461
1605991515,0cyclictest0-21swapper/119:34:481
1602991515,0cyclictest0-21swapper/019:49:470
1602991515,0cyclictest0-21swapper/019:29:470
160299150,1cyclictest0-21swapper/019:44:550
1612991414,0cyclictest0-21swapper/320:40:113
161299140,0cyclictest0-21swapper/319:55:103
1607991414,0cyclictest0-21swapper/220:00:372
1607991414,0cyclictest0-21swapper/219:20:012
160799140,14cyclictest0-21swapper/219:45:092
160799140,14cyclictest0-21swapper/219:26:122
160799140,14cyclictest0-21swapper/219:15:422
160799140,0cyclictest0-21swapper/221:00:432
160799140,0cyclictest0-21swapper/220:40:202
160599140,0cyclictest0-21swapper/120:10:031
160599140,0cyclictest0-21swapper/119:35:321
1602991414,0cyclictest0-21swapper/020:01:000
160299140,7cyclictest0-21swapper/020:50:260
160299140,14cyclictest0-21swapper/019:35:460
160299140,0cyclictest0-21swapper/020:05:370
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional