You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 20:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Apr 26, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2127029178,9sleep20-21swapper/207:05:422
2138228875,9sleep00-21swapper/007:07:090
2077628164,13sleep10-21swapper/107:05:251
2133828068,8sleep30-21swapper/307:06:363
7902700,0sleep0161rcuc/010:00:030
2169199280,0cyclictest0-21swapper/310:37:093
2168499240,2cyclictest0-21swapper/109:52:141
2168499240,1cyclictest0-21swapper/111:36:021
21691992315,2cyclictest39-21ksoftirqd/310:47:563
2169199230,1cyclictest0-21swapper/311:04:403
2168499231,21cyclictest0-21swapper/109:25:011
2168499230,22cyclictest0-21swapper/110:00:121
2168499230,1cyclictest0-21swapper/110:15:221
2168499230,1cyclictest0-21swapper/109:16:451
2169199222,17cyclictest39-21ksoftirqd/310:51:443
21684992212,1cyclictest0-21swapper/110:21:401
2168499220,2cyclictest0-21swapper/109:13:001
2168499220,1cyclictest0-21swapper/112:31:451
2168499220,1cyclictest0-21swapper/109:45:581
2168499220,1cyclictest0-21swapper/109:45:571
21679992216,5cyclictest13-21ksoftirqd/011:31:500
21691992119,2cyclictest39-21ksoftirqd/309:55:133
21691992118,2cyclictest39-21ksoftirqd/308:20:003
2168899210,21cyclictest0-21swapper/209:28:372
2168899210,19cyclictest0-21swapper/212:05:312
2168899210,19cyclictest0-21swapper/210:55:192
2168499211,20cyclictest0-21swapper/112:15:311
2168499210,2cyclictest0-21swapper/109:41:401
21691992018,1cyclictest381rcuc/310:34:043
21688992014,5cyclictest0-21swapper/211:00:232
2168899200,9cyclictest0-21swapper/211:39:162
2168899200,20cyclictest0-21swapper/210:34:172
2168899200,18cyclictest0-21swapper/212:01:142
2168499200,4cyclictest0-21swapper/110:46:171
2168499200,20cyclictest0-21swapper/109:32:011
251542190,0sleep20-21swapper/209:15:302
2169199192,3cyclictest39-21ksoftirqd/312:31:233
21691991917,2cyclictest39-21ksoftirqd/309:19:023
21688991915,3cyclictest0-21swapper/212:20:132
2168899190,16cyclictest0-21swapper/211:31:442
2168499191,15cyclictest14030-21sh11:25:181
2168499191,0cyclictest0-21swapper/111:07:371
2168499190,2cyclictest0-21swapper/110:38:541
2168499190,0cyclictest0-21swapper/111:55:061
21691991815,2cyclictest39-21ksoftirqd/311:45:073
21691991815,2cyclictest39-21ksoftirqd/309:43:253
21691991810,3cyclictest141rcu_preempt11:53:543
2169199180,2cyclictest0-21swapper/309:13:563
21688991817,1cyclictest0-21swapper/212:18:232
2168899181,0cyclictest0-21swapper/210:48:542
2168899180,18cyclictest0-21swapper/210:35:052
21684991817,1cyclictest0-21swapper/112:08:331
21684991816,1cyclictest0-21swapper/111:40:161
2168499180,1cyclictest0-21swapper/109:35:171
2168499180,0cyclictest0-21swapper/110:07:001
2167999184,14cyclictest0-21swapper/009:55:150
310182170,0sleep30-21swapper/311:55:153
2169199179,6cyclictest14279-21diskmemload12:24:593
2169199175,7cyclictest39-21ksoftirqd/311:20:073
21691991716,1cyclictest0-21swapper/312:29:143
2169199170,1cyclictest0-21swapper/310:09:293
2169199170,16cyclictest0-21swapper/311:34:233
2168899170,16cyclictest0-21swapper/211:44:042
21684991715,1cyclictest0-21swapper/108:20:151
2168499170,1cyclictest0-21swapper/112:02:061
2169199166,6cyclictest39-21ksoftirqd/309:36:013
2169199163,3cyclictest28889-21ssh12:13:213
2169199160,3cyclictest16399-21ssh12:05:223
2168899167,9cyclictest0-21swapper/211:50:242
21688991616,0cyclictest0-21swapper/209:39:322
21688991610,3cyclictest0-21swapper/208:45:142
2168899160,5cyclictest0-21swapper/209:22:582
21684991614,1cyclictest0-21swapper/110:55:161
2168499160,12cyclictest0-21swapper/111:34:041
2167999168,7cyclictest26425-21munin-run07:20:000
2167999163,7cyclictest10946-21perf07:55:010
2169199155,4cyclictest39-21ksoftirqd/309:22:423
2169199154,2cyclictest141rcu_preempt09:54:313
21691991513,1cyclictest381rcuc/310:19:223
2169199151,11cyclictest11631-21ssh09:27:153
2169199150,2cyclictest0-21swapper/310:10:473
21688991514,1cyclictest0-21swapper/211:19:422
21688991513,2cyclictest0-21swapper/209:48:322
21688991513,2cyclictest0-21swapper/209:48:312
2168899151,1cyclictest141rcu_preempt10:40:442
2168899150,14cyclictest0-21swapper/212:27:202
2168499150,8cyclictest19302-21grep10:30:181
2168499150,2cyclictest0-21swapper/111:21:541
2168499150,2cyclictest0-21swapper/109:59:241
2167999154,3cyclictest225-21systemd-journal10:15:070
2169199149,3cyclictest20442-21turbostat.cron08:15:013
2169199142,2cyclictest23531-21taskset11:11:153
2169199142,2cyclictest23531-21taskset11:11:143
21691991414,0cyclictest0-21swapper/311:25:133
2169199141,1cyclictest141rcu_preempt12:01:443
21691991411,2cyclictest23686-21dump-pmu-power09:35:013
21688991410,3cyclictest26498-21cat07:19:592
21688991410,3cyclictest14168-21grep11:05:252
21688991410,2cyclictest791-21grep12:35:142
2168499149,3cyclictest31206-21ls12:15:001
2168499143,1cyclictest0-21swapper/110:42:131
21684991414,0cyclictest0-21swapper/111:18:091
21684991412,1cyclictest0-21swapper/110:50:161
21684991410,3cyclictest7527-21grep08:55:151
21684991410,3cyclictest19959-21grep08:10:261
21679991413,1cyclictest0-21swapper/011:48:580
21679991411,2cyclictest13-21ksoftirqd/010:55:110
21679991410,3cyclictest20422-21grep10:30:260
21679991410,2cyclictest25625-21wc09:35:230
2167999140,13cyclictest0-21swapper/010:44:150
2169199139,4cyclictest5576-21sh12:37:543
2169199137,2cyclictest39-21ksoftirqd/311:07:093
2169199134,1cyclictest141rcu_preempt11:36:283
2169199132,1cyclictest141rcu_preempt11:40:533
21691991312,1cyclictest0-21swapper/310:40:543
21691991311,2cyclictest381rcuc/310:55:183
21691991311,1cyclictest8823-21apt-get07:50:153
21691991311,1cyclictest17989-21chrt10:29:593
21691991310,2cyclictest5754-21grep08:50:193
21691991310,2cyclictest39-21ksoftirqd/311:16:313
21691991310,2cyclictest3801-21ls08:45:223
21691991310,2cyclictest16215-21grep08:05:153
2169199130,1cyclictest0-21swapper/309:45:173
2169199130,1cyclictest0-21swapper/309:45:163
2168899139,3cyclictest26546-21wc08:25:212
2168899139,3cyclictest21939-21grep11:50:012
21688991313,0cyclictest0-21swapper/211:27:422
21688991312,1cyclictest0-21swapper/211:57:122
21688991312,0cyclictest6269-21ssh11:20:232
2168499139,3cyclictest27589-21cron08:30:011
21684991312,1cyclictest0-21swapper/111:10:321
21684991312,1cyclictest0-21swapper/111:10:321
2168499131,1cyclictest14279-21diskmemload11:00:031
2168499130,0cyclictest0-21swapper/112:27:521
2167999139,3cyclictest532-21sh12:35:130
2167999139,3cyclictest30872-21cron09:20:000
2167999139,3cyclictest20559-21grep11:10:140
2167999139,3cyclictest20559-21grep11:10:130
2167999139,3cyclictest17544-21sort09:50:180
2167999139,2cyclictest9705-21wc09:00:000
2167999139,2cyclictest13199-21grep11:25:140
2167999139,2cyclictest10025-21tr09:00:120
2167999138,5cyclictest4143-21apache209:49:590
2167999138,5cyclictest4143-21apache209:49:580
21679991310,2cyclictest6039-21sh11:59:590
21679991310,2cyclictest32473-21ls08:39:590
21679991310,2cyclictest29971-21tr11:35:150
21679991310,2cyclictest29789-21head07:25:190
2167999130,1cyclictest0-21swapper/012:28:310
2167999130,1cyclictest0-21swapper/009:13:040
2169199129,2cyclictest23818-21grep08:20:183
2169199129,2cyclictest2064-21cut10:00:183
2169199128,3cyclictest12934-21head09:05:173
2169199128,3cyclictest12380-21seq07:55:223
2169199128,3cyclictest12380-21seq07:55:213
2168899129,2cyclictest8893-21basename10:05:002
2168899129,2cyclictest7066-21grep07:45:152
2168899129,2cyclictest5695-21sort08:50:192
2168899129,2cyclictest20186-21uniq10:50:172
2168899128,3cyclictest638-21cat09:40:162
2168899128,3cyclictest3714-21cron07:40:002
2168899128,3cyclictest32379-21cron12:35:002
2168899128,3cyclictest18587-21grep08:10:152
2168899128,3cyclictest12678-21sort07:55:262
2168899128,3cyclictest12678-21sort07:55:252
2168899120,2cyclictest0-21swapper/210:23:452
2168899120,1cyclictest0-21swapper/209:53:492
2168499129,2cyclictest6157-21tr08:50:221
2168499129,2cyclictest4325-21tr07:40:171
2168499129,2cyclictest30584-21wc07:25:281
2168499129,2cyclictest25916-21grep07:15:261
2168499129,2cyclictest1419-21wc07:35:001
2168499129,2cyclictest10992-21cat07:55:011
2168499128,3cyclictest26732-21grep07:20:131
2168499128,3cyclictest23615-21grep07:10:261
2168499128,3cyclictest18114-21cat10:29:591
2168499127,3cyclictest21820-21grep08:15:211
2168499127,2cyclictest452-21systemd-logind08:03:561
2168499123,6cyclictest17646-21/usr/sbin/munin10:10:181
21684991211,1cyclictest38150irq/131-snd_hda12:38:241
21684991210,1cyclictest14279-21diskmemload11:45:071
2167999129,2cyclictest7499-21wc07:45:180
2167999129,2cyclictest5693-21wc11:00:270
2167999129,2cyclictest5332-21wc07:40:240
2167999129,2cyclictest32541-21grep12:15:170
2167999129,2cyclictest228722sleep009:34:230
2167999129,2cyclictest22163-21tr07:10:150
2167999129,2cyclictest20259-21tr10:50:190
2167999129,2cyclictest20023-21turbostat11:10:000
2167999129,2cyclictest13298-21sh08:00:010
2167999129,2cyclictest13298-21sh08:00:000
2167999128,3cyclictest12056-21sed10:25:260
2167999124,6cyclictest0-21swapper/011:20:310
2167999123,5cyclictest13-21ksoftirqd/009:43:580
2167999123,3cyclictest225-21systemd-journal12:05:000
21679991211,1cyclictest0-21swapper/012:33:230
2167999120,6cyclictest18079-21munin-run08:10:000
2169199119,2cyclictest39-21ksoftirqd/308:40:263
2169199119,2cyclictest0-21swapper/310:21:223
2169199118,2cyclictest7190-21sed07:45:173
2169199118,2cyclictest31455-21grep08:35:213
2169199118,2cyclictest27647-21cut07:20:203
2169199118,2cyclictest26748-21wc08:25:233
2169199118,2cyclictest22060-21tr07:10:143
2169199118,2cyclictest14303-21grep08:00:163
2169199118,2cyclictest1289-21vmstat12:15:273
2169199118,2cyclictest10561-21tail09:00:163
2168899118,2cyclictest4333-21cut07:40:162
2168899118,2cyclictest32508-21cat08:40:002
2168899118,2cyclictest32207-21df07:30:142
2168899118,2cyclictest29849-21grep07:25:192
2168899118,2cyclictest29255-21sensors08:30:242
2168899118,2cyclictest27404-21ssh09:55:392
2168899118,2cyclictest22920-21sh12:10:012
2168899118,2cyclictest20674-21cut11:10:152
2168899118,2cyclictest20674-21cut11:10:152
2168899118,2cyclictest19099-21ssh09:12:152
2168899118,2cyclictest16659-21perl08:05:172
2168899118,2cyclictest14443-21if_enp2s008:00:182
2168899118,2cyclictest12058-21grep10:25:262
2168899118,2cyclictest11445-21ssh10:05:302
2168899114,6cyclictest18469-21tr10:10:162
2168899110,10cyclictest21284-21/usr/sbin/munin08:15:162
2168499118,2cyclictest5753-21sh09:24:091
2168499118,2cyclictest2826-21grep08:45:151
2168499118,2cyclictest12081-21wc09:04:591
2168499118,2cyclictest1129-21cut08:40:201
2168499115,2cyclictest225-21systemd-journal07:39:121
2168499111,2cyclictest225-21systemd-journal08:10:001
2168499110,2cyclictest0-21swapper/112:21:361
2168499110,0cyclictest0-21swapper/111:52:441
2167999119,1cyclictest0-21swapper/011:15:130
2167999118,2cyclictest9415-21ssh10:05:150
2167999118,2cyclictest9088-21pmu-power12:20:220
2167999118,2cyclictest7501-21sed11:40:250
2167999118,2cyclictest6209-21pluginstate08:50:230
2167999118,2cyclictest3768-21lockfile-remove07:40:010
2167999118,2cyclictest303-21cut07:30:180
2167999118,2cyclictest2695-21basename10:20:160
2167999118,2cyclictest2339-21ssh09:21:310
2167999118,2cyclictest23387-21tr08:20:140
2167999118,2cyclictest22088-21ssh10:12:210
2167999118,2cyclictest22036-21rm11:50:040
2167999118,2cyclictest19853-21cat08:10:260
2167999118,2cyclictest18140-21ssh12:06:250
2167999118,2cyclictest13966-21ssh10:46:070
2167999118,2cyclictest13620-21cut08:00:140
2167999118,2cyclictest1318-21ssh10:39:200
2167999118,2cyclictest10559-21ssh09:26:230
2167999110,10cyclictest14015-21taskset09:06:540
2167999110,10cyclictest0-21swapper/012:13:060
2169199108,1cyclictest7404-21cut08:55:133
2169199108,1cyclictest3365-21chrt07:35:343
2169199107,2cyclictest5614-21expr07:40:273
2169199107,2cyclictest29604-21sed07:25:183
2169199106,3cyclictest25993-21grep07:15:273
2168899109,1cyclictest0-21swapper/207:50:152
2168899108,1cyclictest27091-21expr07:20:162
2168899108,1cyclictest23675-21taskset07:10:372
2168899108,1cyclictest23537-21expr08:20:152
2168899108,1cyclictest10705-21kernelversion09:00:172
2168899107,2cyclictest8102-21grep08:55:162
2168899107,2cyclictest25902-21ssh10:15:112
2168899107,2cyclictest15653-21sh09:30:052
2168899107,2cyclictest12441-21expr09:05:152
2168499108,1cyclictest82812sleep107:46:401
2168499108,1cyclictest31382-21ntp_offset08:35:201
2168499108,1cyclictest29718-21chrt08:31:561
2168499108,1cyclictest12306-21nfs4_client07:55:211
2168499108,1cyclictest12306-21nfs4_client07:55:201
2167999108,1cyclictest25461-21cat08:25:130
2167999107,2cyclictest4254-21sed08:45:260
2167999107,2cyclictest30058-21cat08:35:010
2167999107,2cyclictest27323-21cut07:20:160
2167999107,2cyclictest1849-21sed08:40:270
2167999102,7cyclictest22859-21perf08:20:010
216919997,1cyclictest27864-21apache_volume08:30:123
216919997,1cyclictest1214-21taskset07:33:273
216889997,1cyclictest32635-21apt08:40:142
216849996,2cyclictest12491-21expr09:05:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional