You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 20:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Jul 20, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
342828975,10sleep20-21swapper/207:06:442
340628875,9sleep00-21swapper/007:06:250
351228370,9sleep30-21swapper/307:07:493
335028366,13sleep10-21swapper/107:05:421
377399320,31cyclictest0-21swapper/109:30:221
377699281,27cyclictest0-21swapper/210:33:292
377399280,27cyclictest23290-21ntp_states09:25:211
377399272,3cyclictest12318-21apt-get08:35:131
3771992724,2cyclictest13-21ksoftirqd/007:30:000
377199270,24cyclictest0-21swapper/011:22:410
377399269,16cyclictest15996-21pmu-power09:40:231
377399262,21cyclictest0-21swapper/112:15:151
377399260,1cyclictest0-21swapper/110:00:221
377399260,10cyclictest11668-21kworker/u8:209:19:321
377399253,2cyclictest19356-21ntp_states10:40:231
377399250,4cyclictest21118-21sort11:40:001
377399242,2cyclictest28774-21diskmemload12:13:271
377399241,1cyclictest0-21swapper/111:15:151
377399240,3cyclictest0-21swapper/111:10:011
377399240,1cyclictest0-21swapper/111:50:221
377199240,7cyclictest0-21swapper/008:30:130
377399235,17cyclictest17471-21diskstats10:20:151
377399233,1cyclictest28774-21diskmemload10:26:361
377399230,5cyclictest28774-21diskmemload09:52:101
377399230,3cyclictest31142-21aten2_r7power_p12:05:151
377199230,1cyclictest0-21swapper/011:07:000
377199230,1cyclictest0-21swapper/009:39:260
377699220,22cyclictest0-21swapper/211:02:232
377699220,1cyclictest0-21swapper/211:57:422
377199220,2cyclictest0-21swapper/012:11:590
3780992120,1cyclictest0-21swapper/310:36:173
377699212,3cyclictest141rcu_preempt10:30:002
377699210,20cyclictest0-21swapper/212:11:592
377399215,2cyclictest28774-21diskmemload09:48:391
377399210,3cyclictest0-21swapper/111:23:531
173272210,1sleep339-21ksoftirqd/310:00:233
377699204,1cyclictest141rcu_preempt11:15:222
377699201,18cyclictest0-21swapper/209:12:162
377699200,3cyclictest0-21swapper/211:23:502
377699200,20cyclictest0-21swapper/212:18:382
377399201,3cyclictest28774-21diskmemload12:22:081
377399201,18cyclictest330-21ssh10:49:001
377399201,18cyclictest330-21ssh10:48:591
377399201,18cyclictest28774-21diskmemload09:22:501
378099192,13cyclictest0-21swapper/311:15:153
3780991918,1cyclictest39-21ksoftirqd/310:50:033
378099191,10cyclictest29911-21runrttasks11:05:523
377699192,3cyclictest141rcu_preempt10:40:592
377699190,3cyclictest0-21swapper/210:24:152
377699190,18cyclictest0-21swapper/211:28:512
377399195,6cyclictest12240-21ssh09:38:351
377399193,1cyclictest28774-21diskmemload11:57:561
377399192,1cyclictest0-21swapper/107:50:131
377199191,16cyclictest0-21swapper/010:26:300
377199190,2cyclictest0-21swapper/009:34:100
377199190,1cyclictest0-21swapper/010:19:470
377699188,3cyclictest225-21systemd-journal10:03:162
377699181,17cyclictest0-21swapper/210:09:142
377699181,10cyclictest537-21snmpd11:08:582
377399189,8cyclictest12015-21dpkg08:35:001
377399185,10cyclictest19691-21apache211:09:331
377399180,13cyclictest26932-21rm12:02:141
3780991716,1cyclictest0-21swapper/307:30:133
3776991716,1cyclictest7564-21ntp_states09:35:222
377699170,1cyclictest0-21swapper/211:33:022
377399179,7cyclictest15869-21cat07:35:111
377399173,2cyclictest27873-21apt-get08:00:131
377399170,2cyclictest0-21swapper/111:25:251
377399170,1cyclictest0-21swapper/110:39:461
377399170,17cyclictest0-21swapper/110:18:571
377399170,14cyclictest0-21swapper/109:13:031
377199174,9cyclictest28774-21diskmemload10:01:180
3771991716,1cyclictest0-21swapper/011:50:300
3771991714,3cyclictest13-21ksoftirqd/012:20:590
377199171,12cyclictest1137-21irqstats10:10:180
377199170,3cyclictest0-21swapper/010:07:150
377199170,17cyclictest0-21swapper/010:47:580
377199170,17cyclictest0-21swapper/010:47:570
377199170,17cyclictest0-21swapper/009:26:490
377199170,16cyclictest0-21swapper/010:57:170
377199170,0cyclictest0-21swapper/011:33:020
378099162,5cyclictest225-21systemd-journal09:15:003
3780991612,3cyclictest8736-21taskset07:20:043
377699169,4cyclictest0-21swapper/208:25:152
377699163,12cyclictest29886-21rm10:46:422
377699163,12cyclictest29886-21rm10:46:422
3776991615,1cyclictest0-21swapper/209:26:492
377699161,13cyclictest0-21swapper/209:00:142
377699160,2cyclictest0-21swapper/209:48:092
377399165,1cyclictest0-21swapper/112:35:261
377399161,5cyclictest24685-21kworker/1:111:00:261
377399160,2cyclictest0-21swapper/112:34:491
377199167,9cyclictest0-21swapper/012:26:350
3771991615,1cyclictest0-21swapper/012:03:050
378099150,0cyclictest0-21swapper/311:00:423
3776991514,1cyclictest0-21swapper/209:59:112
377699151,1cyclictest141rcu_preempt09:22:302
377699150,4cyclictest0-21swapper/210:19:462
377399158,6cyclictest3683-21sh10:11:401
377399158,6cyclictest30511-21tr08:05:141
377399158,6cyclictest27281-21ssh10:06:171
377399157,7cyclictest7180-21hostname07:15:181
377399157,7cyclictest6405-21ssh12:29:071
377399157,7cyclictest5771-21seq07:10:501
377399157,7cyclictest16527-21rm10:58:331
377399157,7cyclictest14683-21basename08:40:121
377399157,7cyclictest1335-21ssh11:46:561
377399154,2cyclictest225-21systemd-journal08:22:551
377399153,3cyclictest225-21systemd-journal10:55:011
377399152,2cyclictest225-21systemd-journal07:25:011
377399150,2cyclictest0-21swapper/108:25:151
377399150,14cyclictest9057-21rm11:32:191
377399150,14cyclictest27444-21taskset07:58:251
3771991514,1cyclictest0-21swapper/012:38:470
378099149,3cyclictest4070-21grep07:10:143
378099144,8cyclictest0-21swapper/307:55:143
3780991411,2cyclictest3074-21grep10:30:263
377699149,1cyclictest0-21swapper/210:10:272
377699144,6cyclictest2199-21apt-get10:50:132
377699143,10cyclictest30366-21systemd-cgroups12:04:542
377699142,1cyclictest0-21swapper/211:35:132
3776991413,1cyclictest0-21swapper/212:30:582
3776991411,1cyclictest643-21ssh11:46:252
377399147,6cyclictest4492-21chrt08:17:351
377399147,6cyclictest24736-21expr09:00:151
377399147,6cyclictest21087-21expr07:45:151
377399146,7cyclictest28110-21cat09:05:261
377399146,7cyclictest21460-21ssh11:40:131
377399146,1cyclictest927-21ssh10:30:041
377399145,2cyclictest7761-21tr09:55:141
377399141,2cyclictest225-21systemd-journal08:15:001
377399141,2cyclictest225-21systemd-journal07:25:001
377199144,3cyclictest14444-21awk08:39:590
3771991413,1cyclictest0-21swapper/009:53:480
377199140,4cyclictest0-21swapper/008:25:140
377199140,1cyclictest0-21swapper/011:27:260
377199140,14cyclictest0-21swapper/012:30:580
378099139,3cyclictest23831-21grep09:45:163
378099139,3cyclictest14992-21tr09:20:233
378099137,6cyclictest19691-21apache212:17:333
378099137,3cyclictest0-21swapper/311:35:273
378099131,5cyclictest22847-21perf07:50:003
3780991310,2cyclictest9474-21sed12:30:233
3780991310,2cyclictest6138-21grep11:30:243
3780991310,2cyclictest5220-21sed11:10:273
3780991310,2cyclictest330-21grep10:10:143
3780991310,2cyclictest30958-21grep08:05:173
3780991310,2cyclictest24613-21grep12:00:273
3780991310,2cyclictest24027-21grep12:20:143
3780991310,2cyclictest23216-21uname09:25:203
3780991310,2cyclictest12840-21grep07:25:233
3780991310,2cyclictest10010-21tr08:30:143
378099130,1cyclictest0-21swapper/311:22:473
377699139,1cyclictest22825-21cron07:50:002
377699131,6cyclictest31013-21apt12:05:132
3776991312,1cyclictest0-21swapper/211:50:432
3776991310,2cyclictest25199-21cron07:55:002
3776991310,2cyclictest20272-21head08:50:172
377699130,1cyclictest0-21swapper/207:55:142
377699130,13cyclictest0-21swapper/209:30:462
377699130,12cyclictest24132-21ls12:20:142
377699130,12cyclictest0-21swapper/212:28:432
377399136,1cyclictest21905-21grep08:55:141
377399131,2cyclictest225-21systemd-journal08:50:001
377399130,2cyclictest225-21systemd-journal07:35:001
377199135,3cyclictest28774-21diskmemload10:36:390
3771991311,1cyclictest13-21ksoftirqd/010:50:050
3771991310,2cyclictest3663-21tr11:10:160
377199130,1cyclictest0-21swapper/009:59:110
378099129,2cyclictest32704-21df12:25:163
378099129,2cyclictest32282-21cut09:50:163
378099129,2cyclictest31269-21tr12:05:153
378099129,2cyclictest15466-21tail08:40:183
378099129,2cyclictest10405-21sed10:15:253
378099128,3cyclictest26430-21tr10:45:153
378099128,3cyclictest26430-21tr10:45:153
378099128,3cyclictest18757-21sed08:45:283
378099128,2cyclictest6417-21cut07:15:153
378099128,2cyclictest18408-21rm11:00:013
378099126,3cyclictest28774-21diskmemload11:40:113
378099120,5cyclictest0-21swapper/309:30:113
377699129,2cyclictest7168-21head07:15:172
377699129,2cyclictest6101-21cron07:15:002
377699129,2cyclictest30510-21tr08:05:142
377699129,2cyclictest29991-21perf08:05:012
377699129,2cyclictest22918-21date09:45:002
377699129,2cyclictest15778-21cat07:35:012
377699121,3cyclictest0-21swapper/210:35:152
377699121,10cyclictest0-21swapper/209:54:382
377699120,12cyclictest17902-21processes12:35:212
377699120,11cyclictest23116-21grep08:55:222
377699120,11cyclictest0-21swapper/211:12:322
377399123,3cyclictest18919-21uniq07:40:171
377399123,2cyclictest20282-21grep08:50:171
377199129,2cyclictest24284-21chrt09:00:120
377199129,2cyclictest14916-21ssh11:16:150
377199129,2cyclictest14314-21sh11:55:160
377199129,2cyclictest10458-21grep07:20:270
377199125,1cyclictest28774-21diskmemload10:40:130
377199124,7cyclictest28774-21diskmemload11:39:020
377199124,7cyclictest22846-21munin-run07:50:000
377199122,9cyclictest28774-21diskmemload11:41:400
377199121,7cyclictest0-21swapper/008:15:140
3771991210,1cyclictest6067-21head09:15:190
3771991210,1cyclictest1913-21grep08:10:270
3771991210,1cyclictest13698-21ssh09:20:140
377199120,8cyclictest0-21swapper/012:07:180
304542120,2sleep30-21swapper/310:28:013
378099118,2cyclictest7366-21cut08:25:153
378099118,2cyclictest6966-21ssh12:10:133
378099118,2cyclictest6543-21grep09:35:153
378099118,2cyclictest5692-21ssh11:50:103
378099118,2cyclictest5243-21cut09:15:153
378099118,2cyclictest30775-21ssh11:25:283
378099118,2cyclictest30046-21ssh11:45:163
378099118,2cyclictest28456-21taskset09:06:093
378099118,2cyclictest27652-21ssh10:06:343
378099118,2cyclictest24035-21awk08:59:593
378099118,2cyclictest19159-21sed10:40:193
378099118,2cyclictest18998-21grep07:40:173
378099118,2cyclictest17954-21sed10:20:193
378099118,2cyclictest17832-21head12:35:213
378099118,2cyclictest17285-21ssh09:40:463
378099117,2cyclictest23819-21tail07:50:183
378099112,3cyclictest0-21swapper/311:55:153
377699119,2cyclictest15104-21expr08:40:162
377699119,2cyclictest14111-21chrt08:36:112
377699119,1cyclictest9829-21grep08:30:142
377699119,1cyclictest8113-21ssh09:16:102
377699119,1cyclictest70132sleep208:24:122
377699119,1cyclictest4643-21taskset08:19:332
377699119,1cyclictest27205-21df09:05:162
377699119,1cyclictest21231-21sh11:40:042
377699119,1cyclictest17315-21cut08:45:152
377699115,3cyclictest13271-21apt07:29:592
3776991110,1cyclictest10877-21cstates10:55:162
377199119,1cyclictest8444-21chrt07:19:350
377199119,1cyclictest5704-21grep08:20:170
377199119,1cyclictest30288-21aten2_r7power_e08:05:140
377199119,1cyclictest28316-21cat08:00:140
377199119,1cyclictest26195-21tail07:55:180
377199119,1cyclictest2618-21wc10:30:230
377199119,1cyclictest226642sleep010:23:230
377199119,1cyclictest20825-21wc11:00:260
377199119,1cyclictest18003-21sh07:40:000
377199119,1cyclictest15414-21chrt07:32:220
377199119,1cyclictest14884-21cut09:40:150
377199118,2cyclictest5690-21grep07:10:260
377199118,2cyclictest31772-21ssh09:10:500
377199118,2cyclictest26054-21ssh09:46:040
377199118,2cyclictest20625-21cut08:50:200
377199118,2cyclictest18816-21sed07:40:140
377199118,2cyclictest14729-21grep08:40:140
377199118,2cyclictest1027-21sh11:46:420
377199110,10cyclictest23033-21rm12:19:510
378099108,1cyclictest28344-21expr08:00:153
378099108,1cyclictest1822-21grep08:10:283
378099108,1cyclictest158572sleep307:35:053
378099108,1cyclictest12269-21chrt08:35:043
378099107,2cyclictest5223-21cut08:20:143
378099107,2cyclictest20727-21grep08:50:223
378099101,4cyclictest0-21swapper/308:15:193
378099100,9cyclictest8964-21ssh09:55:223
377699109,1cyclictest201992sleep207:41:502
377699108,2cyclictest16726-21basename07:35:172
377699108,1cyclictest970-21tail08:10:182
377699101,1cyclictest0-21swapper/207:20:132
377199109,1cyclictest188492sleep008:45:470
377199107,2cyclictest27757-21sed09:05:210
377199107,2cyclictest23355-21sed08:55:260
377199107,2cyclictest23202-21cut07:50:140
37809992,3cyclictest225-21systemd-journal09:00:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional