You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 07:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Fri Jul 10, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
257102238163,24sleep10-21swapper/119:06:421
257182237164,24sleep20-21swapper/219:06:462
256422206169,25sleep00-21swapper/019:05:490
239992202166,23sleep30-21swapper/319:05:033
1437921490,3sleep22605999cyclictest23:35:152
761421440,1sleep10-21swapper/119:40:001
1876321280,1sleep20-21swapper/223:45:152
26059996663,2cyclictest0-21swapper/222:34:262
306732580,2sleep00-21swapper/021:45:130
228962490,1sleep20-21swapper/223:55:142
26047993510,12cyclictest9748-21munin-run19:45:010
26047993510,12cyclictest4380-21munin-run22:00:010
26065993422,3cyclictest0-21swapper/323:55:053
26065993417,3cyclictest0-21swapper/323:38:293
2604799349,12cyclictest3151-21munin-run19:30:020
2604799345,14cyclictest29213-21munin-run00:10:020
2606599321,24cyclictest0-21swapper/323:14:523
2606599321,19cyclictest0-21swapper/321:36:373
26065993116,10cyclictest0-21swapper/300:13:173
26065993116,10cyclictest0-21swapper/300:04:133
26065993115,10cyclictest0-21swapper/322:32:013
2604799313,13cyclictest8169-21awk20:55:010
26065993014,10cyclictest0-21swapper/300:18:013
26065992916,3cyclictest0-21swapper/319:50:173
2606599291,12cyclictest0-21swapper/322:49:443
2606599291,10cyclictest0-21swapper/321:41:173
2604799291,13cyclictest11945-21munin-run19:50:010
2606599289,3cyclictest0-21swapper/322:05:173
26065992817,3cyclictest0-21swapper/320:13:293
26065992817,3cyclictest0-21swapper/319:45:413
26065992813,10cyclictest0-21swapper/300:30:053
26065992812,11cyclictest0-21swapper/300:07:133
2606599281,13cyclictest0-21swapper/322:25:293
2606599281,12cyclictest0-21swapper/323:01:363
2606599281,11cyclictest0-21swapper/319:41:533
2606599281,11cyclictest0-21swapper/300:28:133
2606599279,3cyclictest0-21swapper/319:27:533
26065992715,8cyclictest0-21swapper/320:18:053
26065992714,7cyclictest0-21swapper/322:03:493
26065992714,7cyclictest0-21swapper/321:02:243
26065992711,11cyclictest0-21swapper/321:27:293
2606599271,10cyclictest0-21swapper/320:40:123
26065992710,3cyclictest0-21swapper/321:49:013
2604799272,12cyclictest29545-21awk20:30:020
2604799270,13cyclictest10998-21idleruntime-cro22:15:010
2606599269,3cyclictest0-21swapper/322:44:573
2606599268,3cyclictest0-21swapper/322:16:293
2606599268,3cyclictest0-21swapper/322:16:293
26065992617,3cyclictest0-21swapper/323:47:373
26065992616,3cyclictest0-21swapper/323:40:373
26065992614,3cyclictest0-21swapper/319:13:443
26065992614,3cyclictest0-21swapper/300:20:493
2604799260,12cyclictest17588-21basename22:30:010
2606599259,3cyclictest0-21swapper/321:07:563
2606599259,3cyclictest0-21swapper/321:07:563
26065992515,3cyclictest0-21swapper/321:24:213
26065992513,3cyclictest0-21swapper/323:19:413
26065992513,2cyclictest0-21swapper/321:50:173
26065992511,8cyclictest0-21swapper/323:24:013
2606599251,17cyclictest0-21swapper/323:06:123
2606599251,17cyclictest0-21swapper/322:11:413
2606599251,15cyclictest0-21swapper/323:32:373
2606599251,15cyclictest0-21swapper/323:32:373
2605599252,13cyclictest18486-40dnf20:04:311
2605599251,18cyclictest0-21swapper/123:06:221
2605599251,16cyclictest0-21swapper/122:46:081
2604799257,15cyclictest19784-21awk22:35:010
2606599249,7cyclictest0-21swapper/321:55:293
2606599248,3cyclictest4940-21unixbench_singl19:30:253
2606599248,2cyclictest0-21swapper/300:35:413
2606599248,2cyclictest0-21swapper/300:35:413
2606599247,3cyclictest0-21swapper/321:15:133
26065992413,3cyclictest0-21swapper/322:50:283
2606599241,11cyclictest14477-21ntp_states22:20:233
2606599241,11cyclictest0-21swapper/322:56:363
2606599241,11cyclictest0-21swapper/321:32:133
2605599242,15cyclictest0-21swapper/122:54:361
2605599241,18cyclictest0-21swapper/122:24:051
2606599238,3cyclictest0-21swapper/320:39:213
2606599238,3cyclictest0-21swapper/320:07:173
2606599237,3cyclictest0-21swapper/323:29:293
2606599237,3cyclictest0-21swapper/319:57:413
2606599237,3cyclictest0-21swapper/319:57:413
2606599236,3cyclictest0-21swapper/320:49:403
26065992312,7cyclictest0-21swapper/322:36:333
2606599231,11cyclictest0-21swapper/321:10:293
26065992310,8cyclictest0-21swapper/320:22:493
2605999237,3cyclictest0-21swapper/223:16:432
2605599235,6cyclictest0-21swapper/122:25:211
2605599231,4cyclictest19775-21munin-run22:35:011
2605599231,20cyclictest0-21swapper/119:51:291
2605599231,19cyclictest8400-21df00:35:141
2605599231,19cyclictest8400-21df00:35:141
2605599231,17cyclictest0-21swapper/121:33:281
26055992311,11cyclictest20651-21kworker/u8:020:14:591
2604799230,22cyclictest29879-21kworker/0:222:07:430
303332220,5sleep32606599cyclictest20:30:153
2606599229,2cyclictest0-21swapper/323:50:093
2606599228,3cyclictest0-21swapper/320:03:453
2606599227,3cyclictest0-21swapper/319:17:453
26065992213,3cyclictest0-21swapper/320:54:563
26065992213,3cyclictest0-21swapper/320:29:053
26059992215,4cyclictest27827-21tail00:05:182
2605599223,16cyclictest20005-21grep21:20:161
26055992215,2cyclictest0-21swapper/123:22:541
2605599221,3cyclictest16968-21munin-run21:15:011
2605599221,16cyclictest0-21swapper/122:14:441
2605599221,15cyclictest0-21swapper/100:26:131
26055992211,2cyclictest165050irq/12-ttyS020:50:031
2605599221,11cyclictest0-21swapper/121:43:111
2604799225,7cyclictest0-21swapper/023:44:370
2604799222,9cyclictest28464-21kworker/0:123:06:490
2604799222,8cyclictest30698-21kworker/0:223:03:130
2604799222,8cyclictest29030-21kworker/0:200:18:470
2604799221,10cyclictest12446-21kworker/0:321:18:310
2605599219,5cyclictest28420-21kworker/u8:022:04:021
2605599217,2cyclictest23-21ksoftirqd/100:05:261
2605599215,9cyclictest0-21swapper/121:48:101
2605599212,5cyclictest0-21swapper/121:55:001
2605599211,9cyclictest0-21swapper/100:02:031
26055992117,3cyclictest0-21swapper/120:40:271
2605599211,5cyclictest0-21swapper/123:35:121
2605599211,5cyclictest0-21swapper/100:31:221
26055992115,4cyclictest13286-21grep23:30:251
26055992115,4cyclictest13286-21grep23:30:251
26055992113,3cyclictest0-21swapper/123:45:071
2605599211,18cyclictest0-21swapper/119:55:111
2605599211,18cyclictest0-21swapper/119:55:111
2605599211,18cyclictest0-21swapper/119:45:341
2605599211,18cyclictest0-21swapper/119:26:211
2605599211,17cyclictest7205-21munin-run23:20:011
2605599211,17cyclictest21771-21kworker/u8:123:00:021
2605599211,15cyclictest0-21swapper/123:43:181
2605599211,11cyclictest9976-21diskstats23:25:141
26055992110,3cyclictest28816-21ntp_kernel_err19:15:041
2605599210,15cyclictest0-21swapper/100:10:241
2605599210,10cyclictest0-21swapper/100:16:351
2604799214,2cyclictest0-21swapper/023:10:460
2604799211,9cyclictest20405-21kworker/0:123:55:030
2604799211,4cyclictest0-21swapper/023:46:100
2604799211,12cyclictest29767-21users21:40:280
2604799211,12cyclictest11823-21/usr/sbin/munin23:30:220
2604799211,12cyclictest11823-21/usr/sbin/munin23:30:220
2604799211,11cyclictest2509-21ntp_kernel_err00:20:220
2606599201,8cyclictest0-21swapper/319:24:293
2606599201,11cyclictest5570-21/usr/sbin/munin19:35:163
2605999203,7cyclictest30650-21unixbench_multi19:15:252
2605999201,7cyclictest0-21swapper/220:56:102
26059992015,3cyclictest32297-21head00:15:152
26059992015,3cyclictest14759-21sh21:10:012
26059992015,3cyclictest14759-21sh21:10:012
2605999201,10cyclictest0-21swapper/221:52:292
2605599209,3cyclictest3949-21unixbench_singl21:55:261
2605599202,16cyclictest0-21swapper/119:34:301
26055992016,3cyclictest0-21swapper/121:01:451
26055992016,3cyclictest0-21swapper/120:07:531
26055992016,3cyclictest0-21swapper/119:13:291
2605599201,3cyclictest13828-21awk21:05:221
2605599201,3cyclictest13828-21awk21:05:221
2605599201,17cyclictest0-21swapper/120:39:191
2605599201,11cyclictest28382-21kernelversion20:25:181
2605599201,11cyclictest13170-21crond22:20:011
2605599201,11cyclictest13170-21crond22:20:001
2605599201,10cyclictest5153-21sed23:15:011
2604799204,6cyclictest0-21swapper/021:13:470
2604799204,4cyclictest0-21swapper/022:50:270
2604799202,8cyclictest23275-21kworker/0:021:35:450
2604799201,5cyclictest0-21swapper/000:39:010
2604799201,5cyclictest0-21swapper/000:39:010
2604799201,11cyclictest0-21swapper/000:13:140
2604799201,10cyclictest4607-21/usr/sbin/munin22:00:120
152702201,2sleep20-21swapper/222:23:482
274802190,5sleep22605999cyclictest19:10:242
2606599191,12cyclictest0-21swapper/320:55:523
2605999194,6cyclictest30-21rcuc/222:25:202
2605999191,8cyclictest0-21swapper/220:05:212
2605999191,7cyclictest1-21systemd19:30:022
2605999191,12cyclictest0-21swapper/222:43:072
2605999191,12cyclictest0-21swapper/221:18:062
2605999191,12cyclictest0-21swapper/200:36:072
2605999191,12cyclictest0-21swapper/200:36:072
2605599198,3cyclictest10411-21cat21:00:011
2605599192,16cyclictest0-21swapper/121:36:201
2605599192,16cyclictest0-21swapper/121:15:151
2605599192,16cyclictest0-21swapper/120:45:051
2605599191,8cyclictest0-21swapper/122:40:451
2605599191,5cyclictest0-21swapper/123:57:581
26055991915,3cyclictest0-21swapper/120:16:121
2605599191,17cyclictest0-21swapper/123:01:371
2605599191,16cyclictest0-21swapper/120:30:471
2605599191,16cyclictest0-21swapper/119:42:561
2605599191,10cyclictest3435-21latency_hist00:25:011
2604799198,3cyclictest808-21snmpd23:50:410
2604799194,9cyclictest0-21swapper/022:17:410
2604799194,9cyclictest0-21swapper/022:17:410
2604799192,8cyclictest2460-21kworker/0:323:18:170
2604799191,9cyclictest9165-21kworker/0:123:25:360
2604799191,6cyclictest0-21swapper/023:23:490
2604799191,6cyclictest0-21swapper/021:31:030
2604799191,11cyclictest0-21swapper/022:40:290
2604799191,10cyclictest0-21swapper/022:21:440
2604799190,9cyclictest17533-21kworker/0:022:58:330
204882191,3sleep20-21swapper/221:20:242
136922191,2sleep20-21swapper/223:33:562
136922191,2sleep20-21swapper/223:33:562
322022181,2sleep20-21swapper/221:48:212
2605999183,6cyclictest30-21rcuc/223:25:162
2605999183,3cyclictest0-21swapper/221:10:252
2605999182,3cyclictest0-21swapper/220:02:502
2605999181,7cyclictest13308-21grep19:50:232
2605999181,7cyclictest0-21swapper/222:39:072
2605999181,5cyclictest29602-21irqstats22:55:202
26059991813,3cyclictest32482-21grep20:35:152
26059991812,4cyclictest10301-21df19:45:132
26059991811,4cyclictest11949-21iwlist22:15:222
26059991811,4cyclictest11949-21iwlist22:15:222
2605999180,3cyclictest87350irq/126-enp1s0-00:30:172
2605599181,2cyclictest0-21swapper/122:05:231
2605599181,16cyclictest0-21swapper/121:25:511
2605599181,15cyclictest0-21swapper/120:22:281
2605599181,10cyclictest0-21swapper/122:35:151
2605599180,16cyclictest0-21swapper/119:20:121
2604799187,3cyclictest87550irq/128-enp1s0-20:43:010
2604799187,3cyclictest31953-21/usr/sbin/munin20:35:220
2604799186,3cyclictest0-21swapper/021:29:200
2604799181,6cyclictest0-21swapper/021:23:510
2604799181,4cyclictest14203-21/usr/sbin/munin23:35:120
2604799181,11cyclictest0-21swapper/021:52:020
2604799181,10cyclictest0-21swapper/000:33:260
2604799181,10cyclictest0-21swapper/000:27:060
123512181,2sleep00-21swapper/021:02:280
33582170,2sleep20-21swapper/200:24:082
313962171,3sleep20-21swapper/220:30:562
267902171,2sleep20-21swapper/200:02:292
2605999177,3cyclictest32-21ksoftirqd/219:20:182
2605999173,5cyclictest3379-21df_inode23:10:132
2605999173,12cyclictest6118-21tail22:00:282
2605999171,9cyclictest0-21swapper/220:21:152
2605999171,9cyclictest0-21swapper/220:10:082
2605999171,7cyclictest29440-21/usr/sbin/munin00:10:142
2605999171,6cyclictest28882-21idleruntime21:40:172
2605999171,6cyclictest25202-21irqstats22:45:192
2605999171,6cyclictest12138-21head21:00:262
2605999171,6cyclictest0-21swapper/223:52:052
2605999171,6cyclictest0-21swapper/221:57:352
2605999171,6cyclictest0-21swapper/220:50:222
2605999171,4cyclictest27567-21users21:35:282
2605999171,3cyclictest0-21swapper/222:51:522
26059991712,3cyclictest10078-21ntp_states22:10:222
2605999171,15cyclictest0-21swapper/220:42:252
2605999171,15cyclictest0-21swapper/219:36:182
26059991711,3cyclictest5228-21sendmail_mailst20:45:222
26059991711,3cyclictest23886-21cpuspeed_turbos21:30:112
2605599171,7cyclictest20615-21/usr/sbin/munin23:50:241
2604799174,8cyclictest0-21swapper/020:20:180
2604799171,6cyclictest0-21swapper/019:15:130
2604799171,4cyclictest31726-21munin-run20:35:020
2604799171,4cyclictest14760-21diskmemload22:50:020
26047991711,3cyclictest8381-21/usr/sbin/munin20:55:260
2605999169,4cyclictest31010-21wc23:00:122
2605999165,3cyclictest87550irq/128-enp1s0-23:05:182
2605999165,3cyclictest0-21swapper/200:25:172
2605999161,7cyclictest21787-21cstates21:25:152
2605999161,6cyclictest7518-21cpuspeed_turbos23:20:122
2605999161,6cyclictest4850-21sed19:30:252
2605999161,6cyclictest17350-21sed23:40:232
2605999161,5cyclictest8758-21needreboot19:40:192
26059991610,3cyclictest14465-21cpuspeed_turbos19:55:102
26059991610,3cyclictest14465-21cpuspeed_turbos19:55:102
2605999160,6cyclictest0-21swapper/220:25:252
2604799166,7cyclictest13412-21grep21:05:180
2604799166,7cyclictest13412-21grep21:05:180
2604799165,3cyclictest23159-21/usr/sbin/munin20:15:250
2604799161,5cyclictest0-21swapper/020:05:130
2604799161,3cyclictest76122cat19:40:010
2604799161,3cyclictest0-21swapper/020:45:130
2604799161,2cyclictest0-21swapper/022:35:150
26047991612,3cyclictest0-21swapper/020:14:190
26047991612,3cyclictest0-21swapper/020:03:220
2604799161,13cyclictest31338-21/usr/sbin/munin19:20:150
26047991611,3cyclictest25086-21sort00:00:130
26047991611,3cyclictest15214-21sort19:55:170
26047991611,3cyclictest15214-21sort19:55:170
2605999151,6cyclictest0-21swapper/220:15:172
2605999151,12cyclictest7161-21diskstats22:05:142
26047991510,3cyclictest3642-21expr19:30:120
2604799150,9cyclictest0-21swapper/019:50:200
2604799141,5cyclictest0-21swapper/019:10:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional