You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 05:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun May 17, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3252529565,10sleep30-21swapper/319:09:013
3234829583,8sleep00-21swapper/019:06:450
3225829178,9sleep20-21swapper/219:05:352
3239928467,13sleep10-21swapper/119:07:221
248492710,0sleep20-21swapper/223:43:372
248492710,0sleep20-21swapper/223:43:362
6572620,0sleep10-21swapper/121:33:311
80302520,0sleep10-21swapper/121:37:441
49542520,0sleep30-21swapper/321:16:143
32707993027,1cyclictest29793-21kworker/3:100:02:433
3270399290,28cyclictest0-21swapper/222:14:152
3270399280,28cyclictest0-21swapper/223:38:562
32701992410,13cyclictest1142-21runrttasks00:08:261
32703992322,1cyclictest0-21swapper/223:50:362
32703992320,2cyclictest13436-21smartctl23:55:262
3270399230,22cyclictest0-21swapper/221:50:442
3270399230,22cyclictest0-21swapper/200:05:252
3270199231,2cyclictest141rcu_preempt23:01:351
3270199231,2cyclictest0-21swapper/123:55:021
32697992323,0cyclictest0-21swapper/021:28:210
32707992220,1cyclictest7269-21smartctl22:15:253
32707992220,1cyclictest552-21smartctl22:50:183
32703992220,1cyclictest3626-21smartctl23:30:252
32703992220,1cyclictest27965-21smartctl21:30:172
3270399220,22cyclictest0-21swapper/223:46:282
3270199220,20cyclictest0-21swapper/123:46:581
3270199220,1cyclictest0-21swapper/123:35:561
3269799220,2cyclictest0-21swapper/023:34:280
3270799214,3cyclictest14007-21taskset23:55:373
32707992119,1cyclictest10905-21smartctl23:15:273
32703992120,1cyclictest55650irq/127-enp2s0-22:35:132
3270199210,1cyclictest0-21swapper/120:15:121
3270199210,17cyclictest0-21swapper/122:28:101
32697992119,1cyclictest4726-21smartctl21:35:250
3269799210,0cyclictest0-21swapper/021:21:260
32707992018,1cyclictest6947-21smartctl00:30:183
32703992017,3cyclictest0-21swapper/221:23:332
3270399200,1cyclictest0-21swapper/200:16:142
32701992019,1cyclictest0-21swapper/122:21:181
3270199200,20cyclictest0-21swapper/123:40:361
3270199200,20cyclictest0-21swapper/123:40:361
3270199200,1cyclictest0-21swapper/122:31:171
27342200,0sleep10-21swapper/122:13:191
3270799199,1cyclictest141rcu_preempt21:13:373
3270799190,1cyclictest29793-21kworker/3:123:52:233
32701991915,4cyclictest0-21swapper/121:10:041
3269799197,6cyclictest26502-21perf20:05:000
3269799180,15cyclictest0-21swapper/000:00:110
3270799172,1cyclictest0-21swapper/319:25:143
3270399170,1cyclictest25279-21diskmemload22:59:532
3270199170,1cyclictest0-21swapper/123:19:071
3270199170,0cyclictest0-21swapper/122:15:581
32697991713,3cyclictest13-21ksoftirqd/022:00:110
32707991614,2cyclictest0-21swapper/300:10:323
3270799161,3cyclictest55650irq/127-enp2s0-21:23:153
3270399160,2cyclictest0-21swapper/222:54:542
3270399160,16cyclictest0-21swapper/223:23:462
32701991614,1cyclictest0-21swapper/122:02:131
3270199161,1cyclictest141rcu_preempt21:53:241
3270199160,14cyclictest0-21swapper/120:00:131
3270199160,13cyclictest0-21swapper/119:35:241
3269799168,4cyclictest0-21swapper/021:15:140
3269799164,2cyclictest8636-21sed22:35:260
3269799160,16cyclictest0-21swapper/000:33:060
3269799160,0cyclictest0-21swapper/000:17:450
3270799153,10cyclictest24956-21aten2_r7power_v23:05:133
32707991512,2cyclictest31577-21uniq22:30:163
3270799150,14cyclictest0-21swapper/322:00:133
32703991513,2cyclictest25279-21diskmemload22:44:122
32703991513,2cyclictest0-21swapper/200:01:332
32701991514,1cyclictest25-21ksoftirqd/121:25:271
32701991513,2cyclictest0-21swapper/123:08:251
3270199150,4cyclictest0-21swapper/123:00:001
3270199150,3cyclictest0-21swapper/100:13:401
3270199150,2cyclictest0-21swapper/121:48:061
3269799153,6cyclictest16983-21idleruntime-cro19:45:000
3269799153,6cyclictest12215-21idleruntime-cro19:35:000
3269799152,11cyclictest0-21swapper/000:20:160
32697991511,2cyclictest0-21swapper/023:13:580
3269799150,1cyclictest0-21swapper/022:12:000
3270799141,2cyclictest25279-21diskmemload23:20:203
32707991411,2cyclictest15135-21ssh00:35:173
32707991410,3cyclictest20375-21grep19:50:173
32707991410,2cyclictest30200-21ls00:05:213
32703991411,2cyclictest11026-21ls21:40:002
32703991410,2cyclictest12039-21wc20:40:192
3270399140,14cyclictest0-21swapper/223:16:132
3270399140,14cyclictest0-21swapper/200:20:242
3270199141,12cyclictest27045-21tr23:25:181
32701991410,3cyclictest12299-21cat19:35:001
3269799143,7cyclictest0-21swapper/000:35:180
3269799141,6cyclictest27816-21awk21:50:000
3269799140,1cyclictest0-21swapper/000:26:520
3270799139,3cyclictest23002-21sed22:05:263
3270799139,3cyclictest12189-21sh19:35:003
3270799133,8cyclictest29441-21apache_accesses22:10:113
32707991312,1cyclictest0-21swapper/321:30:563
32707991312,1cyclictest0-21swapper/320:52:593
32707991310,2cyclictest9910-21apt23:35:003
32707991310,2cyclictest28187-21cat20:05:253
3270799130,12cyclictest14816-21basename22:40:003
3270399139,3cyclictest6989-21head20:30:172
3270399139,3cyclictest12769-21sleep00:15:002
3270399138,2cyclictest5845-21ssh23:12:482
3270399137,2cyclictest225-21systemd-journal23:05:572
3270399133,5cyclictest25279-21diskmemload22:34:042
32703991310,2cyclictest8455-21grep19:25:182
3270199139,3cyclictest32084-21head00:25:291
3270199139,3cyclictest13333-21cat21:40:261
3270199139,2cyclictest7484-21wc20:30:231
32701991310,2cyclictest8770-21wc19:25:201
32701991310,2cyclictest2304-21tr23:30:151
32701991310,2cyclictest2254-21cut20:20:171
32701991310,2cyclictest20363-21cut19:50:171
32701991310,2cyclictest10484-21tr21:20:141
3270199130,12cyclictest24175-21ssh22:06:171
3269799139,3cyclictest18922-21grep20:55:180
3269799136,5cyclictest16469-21apache_processe23:00:130
3269799135,1cyclictest0-21swapper/023:50:510
3269799131,6cyclictest5969-21munin-run20:30:000
32697991312,1cyclictest0-21swapper/022:48:510
32697991310,2cyclictest22959-21grep22:05:260
32697991310,2cyclictest2259-21grep20:20:170
32697991310,2cyclictest10702-21tr23:35:150
3269799130,8cyclictest22699-21awk21:05:000
3269799130,13cyclictest0-21swapper/022:27:240
3269799130,12cyclictest12643-21head23:55:180
3269799130,11cyclictest430-21python20:15:240
3269799130,11cyclictest11650-21grep20:40:160
3269799130,11cyclictest0-21swapper/022:21:040
113492131,1sleep325279-21diskmemload21:59:233
3270799129,2cyclictest3101-21grep20:20:263
3270799129,2cyclictest30965-21ssh00:25:203
3270799129,2cyclictest2712-21wc23:50:003
3270799129,2cyclictest24538-21head22:45:153
3270799129,2cyclictest22074-21cat21:00:263
3270799129,2cyclictest11008-21cron21:40:003
3270799128,3cyclictest23190-21grep19:55:213
3270799123,3cyclictest0-21swapper/323:05:003
32707991211,1cyclictest55650irq/127-enp2s0-22:25:073
32707991210,1cyclictest29014-21grep21:50:153
32707991210,1cyclictest25279-21diskmemload22:56:493
32707991210,1cyclictest21897-21grep21:45:273
3270799120,10cyclictest0-21swapper/300:23:313
3270399129,2cyclictest5016-21awk19:20:012
3270399129,2cyclictest27832-21grep20:05:202
3270399129,2cyclictest23623-21sed19:55:262
3270399129,2cyclictest2145-21cut20:20:172
3270399129,2cyclictest18321-21grep19:45:192
3270399129,2cyclictest17134-21sort23:00:172
3270399129,2cyclictest13875-21sh00:35:002
3270399129,2cyclictest12318-21ls22:00:002
3270399129,2cyclictest1185-21sh20:20:002
3270399129,2cyclictest10748-21sort19:30:292
3270399128,3cyclictest26473-21cron20:05:002
3270399122,9cyclictest27316-21ntp_states21:10:212
3270399121,1cyclictest0-21swapper/220:55:132
32703991210,1cyclictest20414-21smartctl21:25:272
3270399120,2cyclictest2655-21ssh00:27:462
3270399120,1cyclictest0-21swapper/222:09:362
3270399120,1cyclictest0-21swapper/221:00:132
3270399120,12cyclictest0-21swapper/221:42:112
3270199129,2cyclictest4552-21tail19:15:271
3270199129,2cyclictest3024-21cut21:15:211
3270199129,2cyclictest27471-21tail20:05:171
3270199129,2cyclictest23706-21grep21:05:161
3270199129,2cyclictest23196-21head19:55:211
3270199129,2cyclictest19475-21grep23:20:251
3270199129,2cyclictest18450-21head19:45:211
3270199129,2cyclictest15505-21wc00:35:191
3270199128,3cyclictest7258-21cat00:30:201
3270199128,3cyclictest1470-21wc22:50:261
3270199120,3cyclictest25279-21diskmemload23:11:001
3269799129,2cyclictest6808-21grep00:10:260
3269799129,2cyclictest3555-21grep19:15:170
3269799129,2cyclictest354-21tail22:30:250
3269799129,2cyclictest32547-21sh22:50:150
3269799129,2cyclictest29034-21head21:50:150
3269799129,2cyclictest21278-21taskset23:40:490
3269799129,2cyclictest21278-21taskset23:40:490
3269799129,2cyclictest1600-21grep19:10:200
3269799129,2cyclictest12183-21ls22:00:000
3269799128,3cyclictest28202-21ssh21:30:190
3269799126,2cyclictest225-21systemd-journal23:00:000
3269799124,4cyclictest225-21systemd-journal23:50:000
3269799124,4cyclictest225-21systemd-journal20:50:000
3269799124,4cyclictest225-21systemd-journal00:05:020
3269799123,6cyclictest25500-21cron23:25:000
3269799122,9cyclictest14582-21munin-run19:40:000
268292120,2sleep10-21swapper/100:22:551
3270799119,1cyclictest13813-21ls00:15:163
3270799118,2cyclictest5652-21expr19:20:143
3270799118,2cyclictest31172-21taskset20:14:293
3270799118,2cyclictest20808-21ssh21:25:313
3270799118,2cyclictest18978-21tr23:40:153
3270799118,2cyclictest18978-21tr23:40:153
3270799118,2cyclictest18418-21ls19:45:213
3270799118,2cyclictest16538-21ssh21:42:453
3270799118,2cyclictest15239-21ssh22:40:093
3270799118,2cyclictest14661-21sh23:37:123
3270799118,2cyclictest14197-21head20:45:173
3270799118,2cyclictest14039-21ssh22:20:093
3270799118,2cyclictest1119-21tr23:10:143
3270799112,6cyclictest27141-21memory23:25:193
3270399118,2cyclictest7370-21rm21:18:062
3270399118,2cyclictest7067-21taskset19:21:042
3270399118,2cyclictest26240-21ssh22:45:512
3270399118,2cyclictest20087-21ssh21:45:142
3270399118,2cyclictest19614-21ssh00:37:582
3270399118,2cyclictest17078-21ssh22:21:092
3270399118,2cyclictest16880-21ssh22:02:082
3270399118,2cyclictest14691-21wc19:40:002
3270399117,3cyclictest26181-21egrep23:25:142
3270399117,2cyclictest8726-21grep20:35:122
3270399115,5cyclictest0-21swapper/222:15:132
3270399113,5cyclictest23747-21sh22:25:222
3270399111,1cyclictest464-21apache_processe19:10:132
3270399110,2cyclictest22686-21apt-get21:05:012
3270199119,1cyclictest21303-21grep21:00:171
3270199118,2cyclictest8633-21cat22:35:261
3270199118,2cyclictest5562-21ntpq21:55:211
3270199118,2cyclictest26679-21ssh00:03:581
3270199118,2cyclictest17257-21grep22:40:271
3270199118,2cyclictest15541-21munin-run20:50:001
3270199117,3cyclictest23779-21ssh22:45:121
3270199117,3cyclictest15126-21wc00:15:261
3270199117,2cyclictest31373-21wc20:15:001
32701991110,1cyclictest0-21swapper/120:50:241
3269799119,1cyclictest7061-21if_enp0s31f620:30:180
3269799118,2cyclictest9931-21sort23:15:180
3269799118,2cyclictest5765-21wget22:15:150
3269799118,2cyclictest25874-21taskset21:10:080
3269799118,2cyclictest25813-21ssh23:25:050
3269799118,2cyclictest24929-21sh23:05:130
3269799118,2cyclictest23386-21grep21:05:150
3269799118,2cyclictest17932-21grep19:45:160
3269799118,2cyclictest15993-21ssh22:40:160
3269799118,2cyclictest15008-21ssh21:41:350
3269799117,3cyclictest18119-21date20:55:000
3269799110,1cyclictest141rcu_preempt20:05:210
3270799108,2cyclictest24080-21kernelversion21:05:203
3270799108,1cyclictest3423-21df_abs19:15:163
3270799108,1cyclictest14879-21hostname19:40:133
3270799108,1cyclictest13090-21cut19:35:163
3270799107,2cyclictest811-21cat19:10:153
3270799107,2cyclictest4083-21grep20:25:143
3270399108,1cyclictest5680-21taskset20:26:372
3270399108,1cyclictest195672sleep219:50:112
3270399108,1cyclictest17754-21taskset20:53:102
3270399108,1cyclictest14884-21wget19:40:132
3270399107,2cyclictest30064-21cut20:10:172
3270399107,2cyclictest14678-21grep20:45:212
3270199108,1cyclictest3218-21ssh23:50:081
3270199107,2cyclictest7079-21taskset19:21:141
3270199107,2cyclictest5543-21grep20:25:271
3270199107,2cyclictest15098-21tr19:40:141
3270199107,2cyclictest11681-21grep20:40:161
3270199107,2cyclictest10196-21wc20:35:241
3270199101,7cyclictest2646-21latency_hist19:15:001
3269799108,1cyclictest21617-21taskset19:54:000
3269799107,2cyclictest9421-21sed19:25:270
3269799107,2cyclictest6114-21if_enp0s31f619:20:170
3269799107,2cyclictest23068-21sed19:55:190
327079998,1cyclictest6603-21expr20:30:153
327079998,1cyclictest19303-21ntpq20:55:213
327079997,1cyclictest31516-21basename20:15:133
327079997,1cyclictest26653-21cat20:05:013
327079997,1cyclictest12626-21cat20:40:253
327079996,2cyclictest9331-21grep20:35:163
327019997,1cyclictest18655-21sh20:55:161
326979998,1cyclictest10010-21sendmail_mailtr20:35:210
326979996,2cyclictest29878-21tail20:10:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional