You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-02 - 11:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sat May 02, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2999529752,38sleep10-21swapper/119:05:211
3078029380,9sleep00-21swapper/019:07:460
3076229380,8sleep20-21swapper/219:07:312
3065329364,9sleep30-21swapper/319:06:083
3106899270,3cyclictest0-21swapper/300:13:443
31064992724,3cyclictest0-21swapper/221:37:542
3106899250,2cyclictest0-21swapper/323:33:593
3106899241,23cyclictest0-21swapper/300:36:363
3106499240,24cyclictest0-21swapper/223:04:102
31068992322,1cyclictest0-21swapper/300:32:323
31068992322,1cyclictest0-21swapper/300:08:353
3106899230,1cyclictest0-21swapper/322:05:153
3106499231,18cyclictest0-21swapper/223:12:032
3106499230,17cyclictest0-21swapper/223:26:442
3106899221,21cyclictest0-21swapper/300:22:243
3106499220,19cyclictest0-21swapper/200:00:582
31058992219,2cyclictest94-21kswapd021:38:390
3106899212,19cyclictest39-21ksoftirqd/323:10:103
31068992119,1cyclictest10546-21turbostat19:35:013
3106499212,2cyclictest0-21swapper/223:19:142
31060992118,1cyclictest0-21swapper/122:20:011
3105899210,1cyclictest0-21swapper/023:51:330
3105899210,1cyclictest0-21swapper/023:51:320
31068992018,1cyclictest23624-21diskmemload23:19:083
31068992018,1cyclictest23624-21diskmemload00:00:273
31068992016,3cyclictest23624-21diskmemload22:35:053
3106899200,3cyclictest0-21swapper/322:31:573
31064992017,3cyclictest0-21swapper/200:20:212
31068991918,1cyclictest39-21ksoftirqd/300:25:153
31068991918,1cyclictest0-21swapper/322:19:593
31068991917,2cyclictest0-21swapper/323:07:213
31068991912,6cyclictest30004-21aten2_r7power_v22:50:123
3106899190,1cyclictest0-21swapper/323:02:013
3106899190,1cyclictest0-21swapper/321:11:053
3105899191,9cyclictest13111-21apache_accesses22:40:110
3105899190,1cyclictest141rcu_preempt22:22:400
3105899190,19cyclictest0-21swapper/023:33:440
31068991816,2cyclictest23624-21diskmemload22:57:443
31068991815,2cyclictest0-21swapper/323:40:133
31068991815,0cyclictest23624-21diskmemload22:10:023
3106899180,2cyclictest0-21swapper/321:29:363
3106899180,18cyclictest0-21swapper/321:40:473
31064991817,1cyclictest0-21swapper/222:41:262
3106099182,15cyclictest0-21swapper/100:30:131
31060991816,1cyclictest0-21swapper/121:55:351
3106899176,4cyclictest624-21df_abs21:15:153
31068991715,1cyclictest0-21swapper/322:48:553
3106899170,17cyclictest0-21swapper/322:25:263
31060991716,1cyclictest0-21swapper/122:33:491
31060991715,1cyclictest477-21rs:main1
3106099170,16cyclictest0-21swapper/120:35:171
3105899175,6cyclictest20019-21awk00:39:590
3105899173,7cyclictest4374-21munin-run20:30:000
3105899171,2cyclictest141rcu_preempt22:07:490
3105899170,0cyclictest0-21swapper/023:02:430
31068991615,1cyclictest23624-21diskmemload23:29:323
31068991614,1cyclictest23624-21diskmemload22:01:073
31068991611,2cyclictest225-21systemd-journal19:14:593
3106899160,1cyclictest0-21swapper/323:49:133
3106499160,1cyclictest0-21swapper/222:10:152
3106499160,16cyclictest0-21swapper/222:00:462
3106499160,16cyclictest0-21swapper/221:32:312
3106499160,16cyclictest0-21swapper/200:28:002
31060991615,1cyclictest0-21swapper/120:25:121
3105899162,7cyclictest8166-21idleruntime-cro19:30:000
3105899161,8cyclictest24345-21awk23:45:000
3106899159,5cyclictest11125-21ssh23:55:263
3106899154,9cyclictest23624-21diskmemload23:37:363
31068991514,1cyclictest0-21swapper/323:50:123
31068991514,1cyclictest0-21swapper/323:50:123
3106899151,13cyclictest0-21swapper/321:35:343
3106499150,15cyclictest0-21swapper/221:58:412
3106499150,15cyclictest0-21swapper/200:31:522
3106099153,8cyclictest0-21swapper/121:25:141
31060991514,1cyclictest0-21swapper/121:19:311
3105899159,4cyclictest1370-21apache221:59:120
3105899157,7cyclictest15330-21awk19:45:000
3105899153,6cyclictest25535-21perf00:05:000
3105899151,8cyclictest1947-21munin-run20:25:000
3105899151,6cyclictest18877-21sendmail20:59:590
31058991512,3cyclictest0-21swapper/023:46:520
3105899150,2cyclictest0-21swapper/000:23:490
3105899150,15cyclictest0-21swapper/022:12:410
3105899150,15cyclictest0-21swapper/021:12:010
31068991412,2cyclictest39-21ksoftirqd/320:51:363
31064991411,2cyclictest22845-21ls22:45:202
31064991410,3cyclictest25861-21sed20:05:162
3106099148,5cyclictest25-21ksoftirqd/123:15:131
3106099147,6cyclictest5343-21head20:30:151
3106099143,2cyclictest20309-21sh22:44:281
31060991413,1cyclictest0-21swapper/123:34:331
3106099140,2cyclictest0-21swapper/123:50:201
3106099140,2cyclictest0-21swapper/123:50:201
3105899149,3cyclictest29566-21cron20:15:000
3105899143,7cyclictest946-21idleruntime-cro19:15:000
3105899141,1cyclictest23624-21diskmemload21:45:120
31058991411,2cyclictest1448-21ssh00:09:530
31058991411,2cyclictest10916-21wc20:40:320
31058991410,3cyclictest6665-21grep22:35:260
31058991410,3cyclictest4773-21ssh00:10:550
31058991410,2cyclictest9489-21wc19:30:190
3105899140,2cyclictest0-21swapper/000:18:470
3106899139,3cyclictest12603-21grep20:45:163
3106899139,1cyclictest21002-21munin-run22:44:593
3106899134,7cyclictest39-21ksoftirqd/322:21:183
3106899132,1cyclictest0-21swapper/321:50:123
31068991312,1cyclictest0-21swapper/300:16:453
31068991311,2cyclictest0-21swapper/321:21:363
31068991311,1cyclictest23624-21diskmemload21:32:333
31068991310,2cyclictest22944-21grep20:00:143
31068991310,2cyclictest16233-21grep19:45:253
31068991310,2cyclictest1485-21grep20:20:263
3106499139,3cyclictest1586-21cron00:10:012
31064991310,2cyclictest7337-21ls22:55:232
31064991310,2cyclictest30547-21head20:15:152
31064991310,2cyclictest18438-21df21:45:152
31064991310,2cyclictest17925-21ls20:55:222
31064991310,2cyclictest17422-21turbostat21:45:002
31064991310,2cyclictest16756-21gzip23:20:222
31064991310,2cyclictest13977-21cut19:40:152
31064991310,2cyclictest12448-21tr00:35:132
3106499130,1cyclictest0-21swapper/223:38:402
3106099139,3cyclictest2815-21cut00:10:161
3106099133,2cyclictest225-21systemd-journal21:40:001
3106099131,7cyclictest950-21idleruntime-cro19:15:001
3106099131,11cyclictest18994-21ssh22:05:121
3106099131,11cyclictest0-21swapper/121:07:241
31060991310,2cyclictest26454-21tr00:05:201
31060991310,2cyclictest26141-21tr23:45:201
31060991310,2cyclictest21729-21tr22:45:121
31060991310,2cyclictest13955-21sed19:40:151
31060991310,2cyclictest12039-21head22:00:211
31060991310,2cyclictest11020-21grep20:40:321
3105899139,3cyclictest31693-21grep23:10:150
3105899139,2cyclictest23342-21grep22:45:240
3105899134,6cyclictest9103-21cron20:40:010
3105899132,1cyclictest141rcu_preempt21:30:390
3105899131,6cyclictest17419-21munin-run21:45:000
31058991310,2cyclictest9886-21grep23:35:240
31058991310,2cyclictest31631-21cat22:50:250
31058991310,2cyclictest23904-21ssh23:05:190
31058991310,2cyclictest20320-21sed21:00:210
31058991310,2cyclictest17128-21grep23:20:260
31058991310,2cyclictest13600-21sed20:45:270
3105899130,12cyclictest9277-21grep21:20:160
3106899129,2cyclictest28684-21grep20:10:203
3106899129,2cyclictest23624-21diskmemload23:23:033
3106899129,2cyclictest14837-21sed19:40:243
3106899128,3cyclictest19175-21sh21:45:193
3106899128,3cyclictest18394-21grep20:55:273
31068991210,0cyclictest23624-21diskmemload21:56:303
3106499129,2cyclictest7754-21head20:35:152
3106499129,2cyclictest28139-21grep20:10:162
3106499129,2cyclictest2761-21forks00:10:162
3106499129,2cyclictest2548-21wc19:15:202
3106499129,2cyclictest24742-21seq21:10:142
3106499129,2cyclictest2450-21tr20:25:142
3106499129,2cyclictest21624-21grep22:25:192
3106499129,2cyclictest18188-21grep19:50:142
3106499129,2cyclictest17288-21grep23:40:162
3106499129,2cyclictest17241-21sed19:45:342
3106499129,2cyclictest17178-21grep21:25:142
3106499129,2cyclictest15864-21taskset22:21:522
3106499129,2cyclictest14849-21chrt21:23:512
3106499129,2cyclictest11541-21wc20:45:002
3106499124,4cyclictest5680-21hwlatdetect22:35:182
3106499122,2cyclictest0-21swapper/221:15:152
3106499120,11cyclictest0-21swapper/222:31:482
3106099129,2cyclictest7264-21ssh22:55:231
3106099129,2cyclictest4750-21ssh22:35:121
3106099129,2cyclictest28543-21ssh00:25:171
3106099129,2cyclictest26031-21grep21:30:161
3106099129,2cyclictest24398-21tr21:10:131
3106099129,2cyclictest16653-21cat20:55:001
3106099129,2cyclictest14684-21expr23:00:171
3106099129,2cyclictest10756-21grep00:15:151
3106099128,3cyclictest3397-21ssh22:15:121
3106099124,7cyclictest27731-21tr20:10:141
3106099121,1cyclictest141rcu_preempt21:54:181
31060991210,1cyclictest2011-21grep19:15:151
31060991210,1cyclictest17115-21cut23:40:161
31060991210,1cyclictest0-21swapper/120:55:141
31060991210,1cyclictest0-21swapper/100:20:041
3106099120,11cyclictest0-21swapper/120:00:141
3106099120,11cyclictest0-21swapper/119:50:131
3105899129,2cyclictest7021-21grep23:15:150
3105899129,2cyclictest30569-21grep20:15:150
3105899129,2cyclictest2832-21grep19:15:240
3105899129,2cyclictest1467-21tr21:15:210
3105899128,3cyclictest15023-21cut20:50:140
3105899126,4cyclictest13-21ksoftirqd/023:25:140
31058991211,1cyclictest0-21swapper/019:48:310
3105899120,7cyclictest20920-21cut19:55:150
3106899118,2cyclictest6681-21cut19:25:153
3106899118,2cyclictest6274-21sed20:30:253
3106899118,2cyclictest21072-21head19:55:173
3106899118,2cyclictest19206-21cut21:00:133
3106899117,3cyclictest10115-21grep20:40:233
3106899112,5cyclictest225-21systemd-journal20:40:003
3106499119,1cyclictest31369-21wc22:50:212
3106499119,1cyclictest10744-21aten2_r7power_e00:15:152
3106499118,2cyclictest956-21munin-run19:15:002
3106499118,2cyclictest5440-21sh23:52:252
3106499118,2cyclictest5440-21sh23:52:252
3106499118,2cyclictest4803-21tr19:20:172
3106499118,2cyclictest29658-21rm23:47:382
3106499118,2cyclictest24480-21sh23:05:242
3106499118,2cyclictest22642-21ntpq21:05:212
3106499118,2cyclictest20609-21grep19:55:132
3106499118,2cyclictest19474-21expr22:05:142
3106499118,2cyclictest12668-21rm23:56:312
3106499118,2cyclictest10371-21ssh22:19:182
3106499111,4cyclictest225-21systemd-journal20:01:562
3106499110,10cyclictest6451-21taskset20:31:292
3106099118,2cyclictest8984-21ssh23:55:021
3106099118,2cyclictest6068-21cut19:25:121
3106099118,2cyclictest32095-21grep23:10:181
3106099118,2cyclictest28199-21head22:10:181
3106099118,2cyclictest25874-21grep20:05:161
3106099118,2cyclictest24477-21ssh23:05:241
3106099118,2cyclictest23946-21tr23:25:151
3106099118,2cyclictest23116-21ssh22:26:001
3106099118,2cyclictest21694-21grep19:55:211
3106099118,2cyclictest19159-21grep21:45:201
3106099118,2cyclictest17573-21ssh23:20:411
3106099118,2cyclictest17480-21ssh00:00:051
3106099118,2cyclictest16227-21cut19:45:241
3106099118,2cyclictest11874-21ssh21:21:321
3106099117,3cyclictest29977-21grep20:15:131
3106099117,3cyclictest15788-21ssh00:36:421
3106099117,2cyclictest9331-21ntp_states23:35:201
3105899118,2cyclictest7792-21rm00:31:580
3105899118,2cyclictest6739-21idleruntime-cro20:35:000
3105899118,2cyclictest5716-21ssh22:55:050
3105899118,2cyclictest4623-21cut22:15:190
3105899118,2cyclictest4395-21tail19:20:130
3105899118,2cyclictest23004-21grep20:00:150
3105899118,2cyclictest22233-21grep21:05:170
3105899118,2cyclictest21169-21grep22:25:170
3105899118,2cyclictest20884-21chrt21:27:010
3105899118,2cyclictest1762-21ssh22:33:040
3105899118,2cyclictest13101-21ssh23:56:530
3106899108,1cyclictest4641-21expr19:20:163
3106899108,1cyclictest25427-21cat20:05:143
3106899108,1cyclictest1638-21expr19:15:133
3106899108,1cyclictest11018-21chrt19:35:133
3106899101,6cyclictest39-21ksoftirqd/320:25:113
3106899101,6cyclictest39-21ksoftirqd/319:50:143
3106499108,1cyclictest713-21cat23:30:182
3106499108,1cyclictest6457-21expr19:25:132
3106499108,1cyclictest16180-21taskset20:51:282
3106499108,1cyclictest12041-21aten2_r7power_e20:45:142
3106499108,1cyclictest11301-21sed19:35:142
3106499107,2cyclictest21227-21cron21:04:592
3106499107,2cyclictest10314-21taskset19:32:102
3106499100,1cyclictest0-21swapper/221:50:162
3106099108,2cyclictest9955-21expr21:40:131
3106099108,1cyclictest3888-21cat19:20:121
3106099108,1cyclictest32605-21expr20:20:151
3106099108,1cyclictest11729-21uname19:35:181
3106099107,2cyclictest20602-21wc21:00:251
3106099107,2cyclictest13086-21grep20:45:211
3106099102,4cyclictest225-21systemd-journal19:31:501
3105899108,1cyclictest26162-21sh21:50:090
3105899108,1cyclictest10446-21ssh22:00:040
3105899107,2cyclictest27455-21sh00:25:070
3105899107,2cyclictest12415-21grep19:35:260
3105899100,9cyclictest19973-21seq19:54:050
310689997,1cyclictest31101-21ps20:15:203
310689996,2cyclictest21991-21df21:05:163
310649996,2cyclictest32726-21df20:20:152
310589997,1cyclictest26840-21chrt20:05:310
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional