You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 06:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Apr 26, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
645529271,17sleep10-21swapper/119:10:011
546928943,40sleep00-21swapper/019:05:220
621128760,8sleep30-21swapper/319:06:533
522228568,12sleep20-21swapper/219:05:202
654499290,28cyclictest5565-21seq20:15:142
654499290,28cyclictest0-21swapper/221:47:352
654099290,1cyclictest141rcu_preempt23:25:241
6544992525,0cyclictest0-21swapper/221:21:102
654099222,19cyclictest0-21swapper/100:09:531
6546992119,1cyclictest0-21swapper/322:20:143
6540992119,1cyclictest0-21swapper/120:30:211
654699200,19cyclictest0-21swapper/321:25:143
654499200,17cyclictest0-21swapper/223:20:142
6540992019,1cyclictest0-21swapper/121:37:061
6540991917,1cyclictest6529-21cyclictest23:15:511
654099190,17cyclictest0-21swapper/122:45:581
653699190,1cyclictest0-21swapper/021:20:130
654699180,2cyclictest0-21swapper/321:53:233
654099180,16cyclictest0-21swapper/120:10:221
653699185,11cyclictest455-21dbus-daemon00:05:460
6536991816,2cyclictest13-21ksoftirqd/022:15:160
653699180,2cyclictest141rcu_preempt23:47:150
653699180,15cyclictest0-21swapper/000:04:250
6546991715,2cyclictest39-21ksoftirqd/300:37:543
654699171,15cyclictest0-21swapper/323:15:133
654499172,15cyclictest0-21swapper/221:14:542
6544991716,1cyclictest0-21swapper/200:12:392
6544991711,5cyclictest20848-21grep00:35:212
654099175,10cyclictest23234-21munin-run19:44:591
654099171,15cyclictest19767-21date22:20:001
6546991614,1cyclictest19786-21ntp_states00:15:223
6546991610,5cyclictest39-21ksoftirqd/300:22:123
654499162,11cyclictest0-21swapper/223:00:142
6544991614,1cyclictest0-21swapper/223:05:312
654499161,14cyclictest0-21swapper/223:45:062
653699162,7cyclictest31550-21diskmemload22:40:010
653699160,1cyclictest141rcu_preempt23:15:130
653699160,1cyclictest0-21swapper/022:09:280
654499153,5cyclictest477-21rs:main2
6544991513,1cyclictest311rcuc/219:45:202
654499151,14cyclictest0-21swapper/223:34:352
654499151,13cyclictest0-21swapper/221:25:342
654499150,14cyclictest0-21swapper/200:26:192
654499150,14cyclictest0-21swapper/200:26:182
654099159,4cyclictest25499-21ssh23:40:191
6540991514,1cyclictest0-21swapper/123:45:521
6540991511,3cyclictest25-21ksoftirqd/121:15:121
654099150,2cyclictest0-21swapper/123:08:321
6536991514,1cyclictest161rcuc/000:15:210
654699147,2cyclictest5562-21apt-get20:15:133
654699141,2cyclictest141rcu_preempt22:06:573
654699141,2cyclictest0-21swapper/323:44:373
6546991411,1cyclictest0-21swapper/300:00:353
6546991410,3cyclictest567-21grep21:10:183
654699140,1cyclictest0-21swapper/321:22:093
654499143,2cyclictest0-21swapper/223:38:562
6540991412,1cyclictest0-21swapper/119:15:161
6540991410,4cyclictest31550-21diskmemload21:31:391
654099140,13cyclictest31550-21diskmemload00:35:431
654099140,13cyclictest0-21swapper/100:11:301
653699146,5cyclictest13-21ksoftirqd/022:10:080
653699142,4cyclictest31550-21diskmemload23:20:140
653699142,4cyclictest225-21systemd-journal20:15:000
6536991412,2cyclictest13-21ksoftirqd/023:05:210
653699141,1cyclictest141rcu_preempt00:38:410
6536991410,2cyclictest13-21ksoftirqd/021:44:060
653699140,1cyclictest0-21swapper/020:15:140
654699139,3cyclictest9577-21ls21:15:213
654699139,3cyclictest30372-21grep22:45:183
654699139,2cyclictest9408-21tr19:15:163
6546991310,2cyclictest1819-21cat21:50:003
6546991310,2cyclictest11620-21grep21:55:153
6546991310,2cyclictest10963-21uniq20:25:173
654699130,1cyclictest0-21swapper/300:07:113
654499136,6cyclictest55650irq/127-enp2s0-20:06:282
654499135,7cyclictest26388-21irqstats00:00:192
654499133,3cyclictest477-21rs:main2
6544991312,1cyclictest0-21swapper/200:09:272
6544991311,2cyclictest30207-21ssh22:25:262
6544991310,2cyclictest19924-21grep19:35:212
6544991310,2cyclictest19233-21grep00:15:162
6544991310,2cyclictest16443-21grep23:15:212
6544991310,2cyclictest14536-21ls23:15:002
6544991310,2cyclictest11258-21cron19:20:012
654499130,11cyclictest7957-21sh22:12:212
654099139,3cyclictest25585-21uname20:55:181
654099139,3cyclictest25244-21tr21:25:151
654099139,2cyclictest28224-21wc19:54:591
654099137,1cyclictest23422-21apt-get23:20:141
654099136,7cyclictest94-21kswapd021:45:011
654099134,5cyclictest225-21systemd-journal23:52:061
654099133,6cyclictest225-21systemd-journal23:02:111
6540991310,2cyclictest7327-21grep22:50:261
6540991310,2cyclictest20351-21cut22:20:151
654099130,12cyclictest896-21cat00:05:011
654099130,12cyclictest19469-21tail19:35:181
653699137,5cyclictest28425-21/usr/sbin/munin19:55:250
653699136,5cyclictest449-21irqbalance23:32:310
653699135,4cyclictest31550-21diskmemload21:57:440
653699132,2cyclictest141rcu_preempt00:11:200
653699131,2cyclictest0-21swapper/021:46:430
6536991310,1cyclictest0-21swapper/022:45:150
653699130,3cyclictest0-21swapper/023:38:490
654699129,2cyclictest9639-21ssh22:52:113
654699129,2cyclictest30127-21grep19:55:253
654699129,2cyclictest27991-21perf19:54:593
654699129,2cyclictest25365-21sed23:20:273
654699129,2cyclictest24776-21cron00:00:003
654699129,2cyclictest22772-21grep20:50:163
654699129,2cyclictest22772-21grep20:50:153
654699129,2cyclictest22007-21grep22:40:193
654699129,2cyclictest20997-21sh22:00:283
654699129,2cyclictest20832-21sh19:39:593
654699129,2cyclictest19073-21cat00:35:013
654699129,2cyclictest10516-21seq23:31:043
6546991210,1cyclictest9669-21grep20:20:263
6546991210,1cyclictest1573-21grep20:05:193
654699120,9cyclictest0-21swapper/322:35:133
654499129,2cyclictest31118-21grep21:05:262
654499129,2cyclictest27726-21grep21:00:182
654499129,2cyclictest26217-21ssh22:42:482
654499129,2cyclictest24734-21ls23:40:142
654499129,2cyclictest24286-21ssh21:43:592
654499129,2cyclictest21613-21awk23:00:012
654499129,2cyclictest21226-21wget19:40:142
654499129,2cyclictest13964-21chrt22:15:292
654499129,2cyclictest11676-21basename19:20:142
654499129,2cyclictest10084-21grep21:35:172
654499127,4cyclictest26756-21cron21:00:012
654499126,2cyclictest225-21systemd-journal19:30:002
654499122,5cyclictest8567-21sshd19:11:062
654499121,4cyclictest225-21systemd-journal19:54:372
6544991210,1cyclictest4799-21tr22:30:152
6544991210,1cyclictest4799-21tr22:30:142
654499120,11cyclictest16841-21grep20:35:252
654099129,2cyclictest32537-21grep20:00:291
654099129,2cyclictest2737-21tr21:50:161
654099129,2cyclictest24253-21sed19:45:181
654099129,2cyclictest18093-21ssh23:35:261
654099129,2cyclictest17971-21sed19:30:271
654099129,2cyclictest15707-21grep22:55:271
654099127,4cyclictest7256-21grep20:15:261
654099127,4cyclictest12400-21rm00:30:201
6540991212,0cyclictest0-21swapper/100:15:151
654099121,1cyclictest141rcu_preempt00:25:161
654099121,1cyclictest141rcu_preempt00:25:151
654099120,12cyclictest0-21swapper/122:45:001
654099120,11cyclictest13109-21tr22:35:151
653699129,2cyclictest8003-21head19:10:230
653699129,2cyclictest7375-21grep22:50:260
653699129,2cyclictest30212-21sed21:05:170
653699129,2cyclictest28476-21chrt23:42:050
653699129,2cyclictest21588-21cron23:00:010
653699129,2cyclictest15064-21sort20:35:010
653699129,2cyclictest14661-21sort19:25:170
653699128,3cyclictest6117-21sh21:13:480
653699128,3cyclictest4322-21ssh22:30:070
653699128,3cyclictest4322-21ssh22:30:060
653699128,3cyclictest23267-21sed23:00:210
653699128,3cyclictest22155-21sh22:01:090
653699126,2cyclictest13-21ksoftirqd/023:27:450
653699121,7cyclictest23235-21dump-pmu-power19:44:590
653699120,11cyclictest8815-21grep21:15:160
654699119,1cyclictest28810-21ssh22:25:153
654699119,1cyclictest17079-21taskset20:37:583
654699119,1cyclictest15215-21grep22:55:223
654699119,1cyclictest13673-21ssh21:37:153
654699119,1cyclictest12807-21wc19:20:223
654699119,1cyclictest10963-21grep00:10:183
654699118,2cyclictest7645-21ssh22:31:203
654699118,2cyclictest7645-21ssh22:31:193
654699118,2cyclictest6334-21ssh23:10:023
654699118,2cyclictest5953-21ssh21:32:523
654699118,2cyclictest5163-21cat22:10:253
654699118,2cyclictest3465-21tr20:10:153
654699118,2cyclictest32356-21cat23:05:263
654699118,2cyclictest25363-21ssh21:44:503
654699118,2cyclictest24210-21grep19:45:173
654699118,2cyclictest2350-21sh23:45:303
654699118,2cyclictest21213-21ssh23:37:543
654699118,2cyclictest21095-21cat19:40:123
654699118,2cyclictest20616-21grep20:45:173
654699118,2cyclictest17861-21cat20:40:143
654699118,2cyclictest13426-21grep20:30:163
654699114,3cyclictest3568-21tail00:25:163
654699114,3cyclictest3568-21tail00:25:153
654699110,10cyclictest14584-21cut19:25:183
654499119,2cyclictest29808-21ssh22:05:382
654499119,2cyclictest14208-21ssh21:56:282
654499119,1cyclictest7969-21rm21:15:032
654499119,1cyclictest31492-21sed22:45:272
654499119,1cyclictest30888-21cut20:00:162
654499119,1cyclictest30207-21taskset19:55:322
654499119,1cyclictest26773-21sh00:20:112
654499119,1cyclictest20149-21ssh22:20:092
654499119,1cyclictest19108-21ssh22:00:082
654499119,1cyclictest16616-21grep19:30:152
654499119,1cyclictest12273-21uname00:30:202
654499118,2cyclictest5574-21rm21:32:352
654499118,2cyclictest4091-21sensors21:50:252
654499118,2cyclictest3571-21ls20:10:152
654499118,2cyclictest31468-21ssh23:25:002
654499118,2cyclictest22552-21grep20:50:152
654499118,2cyclictest22552-21grep20:50:152
654499118,2cyclictest18145-21wc23:55:192
654499118,2cyclictest12562-21rm22:35:022
654499118,2cyclictest10335-21apache_processe20:25:122
654499118,1cyclictest7967-21apt-get20:20:142
654499110,6cyclictest8680-21aten2_r7power_e23:50:132
654099119,1cyclictest17317-21tr23:55:151
654099118,2cyclictest8235-21tr20:20:141
654099118,2cyclictest7911-21taskset22:31:341
654099118,2cyclictest7911-21taskset22:31:341
654099118,2cyclictest30814-21ssh22:25:381
654099118,2cyclictest29404-21wc22:05:281
654099118,2cyclictest28674-21tr19:55:141
654099118,2cyclictest22709-21expr20:50:151
654099118,2cyclictest22709-21expr20:50:151
654099118,2cyclictest20539-21ssh22:00:221
654099118,2cyclictest17116-21taskset20:38:261
654099118,2cyclictest10167-21ssh23:30:461
654099117,3cyclictest7485-21ssh22:11:591
654099117,3cyclictest30456-21cat21:05:191
654099117,3cyclictest17493-21tail21:20:181
654099117,3cyclictest11654-21wc20:25:221
654099110,11cyclictest0-21swapper/121:55:511
654099110,11cyclictest0-21swapper/121:10:151
654099110,10cyclictest22535-21ssh21:42:361
653699118,2cyclictest9645-21grep19:15:160
653699118,2cyclictest610-21sh21:30:020
653699118,2cyclictest4179-21ssh21:50:270
653699118,2cyclictest27134-21grep00:20:160
653699118,2cyclictest2629-21sh00:25:070
653699118,2cyclictest2629-21sh00:25:060
653699118,2cyclictest24693-21chrt21:25:020
653699118,2cyclictest21519-21expr22:40:150
653699118,2cyclictest21390-21wc20:45:260
653699118,2cyclictest21158-21sort22:20:190
653699118,2cyclictest20590-21chrt19:36:590
653699118,2cyclictest17907-21sed19:30:260
653699118,2cyclictest16810-21ssh23:55:060
653699118,2cyclictest139002sleep000:30:590
653699118,2cyclictest12187-21ssh21:36:040
653699118,2cyclictest11905-21wc20:25:260
653699118,2cyclictest10528-21grep23:50:260
653699117,3cyclictest30856-21ssh22:25:390
653699116,1cyclictest141rcu_preempt20:00:280
653699111,1cyclictest141rcu_preempt19:45:190
653699110,10cyclictest6707-21cut23:10:140
653699110,10cyclictest10005-21chrt20:24:180
654699108,2cyclictest14428-21ssh22:15:513
654699108,1cyclictest29537-21awk21:05:153
654699108,1cyclictest2840-21ssh23:26:363
654699107,2cyclictest8908-21tr23:50:143
654699107,2cyclictest25372-21ssh23:01:243
654699107,2cyclictest18356-21taskset19:33:473
654499107,2cyclictest19215-21grep20:40:252
654099107,2cyclictest8090-21awk23:10:221
654099107,2cyclictest27293-21tr21:00:161
654099107,2cyclictest19614-21chrt20:44:481
654099107,2cyclictest15582-21ls19:25:261
654099107,2cyclictest1087-21tr20:05:151
654099104,3cyclictest452-21systemd-logind00:20:241
653699108,2cyclictest196182sleep020:44:510
653699108,1cyclictest22512-21tr20:50:150
653699108,1cyclictest22512-21tr20:50:150
653699107,2cyclictest28408-21which21:00:230
653699107,2cyclictest25964-21gzip20:55:210
653699107,2cyclictest25825-21chrt19:50:010
653699107,2cyclictest1410-21sort20:05:170
653699107,2cyclictest13043-21sed19:20:250
65469998,1cyclictest25007-21expr20:55:153
65469997,2cyclictest31662-21sed20:00:203
65469997,2cyclictest27450-21expr21:00:163
65469997,1cyclictest6726-21date19:10:003
65449998,1cyclictest14841-21taskset20:34:432
65409997,1cyclictest7567-21smartctl19:10:171
65409996,2cyclictest20370-21expr20:45:151
65409996,2cyclictest11870-21expr19:20:151
65369990,8cyclictest0-21swapper/020:35:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional