You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 13:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Apr 14, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
232790724380,7sleep234-21ksoftirqd/219:05:222
241983721230,3sleep30-21swapper/323:25:133
240135621180,4sleep33056-21PK-Backend22:35:003
243731621090,4sleep20-21swapper/200:15:162
24137532980,1sleep00-21swapper/023:06:570
232839027339,13sleep10-21swapper/119:07:201
232776227040,24sleep30-21swapper/319:05:203
232850226243,14sleep00-21swapper/019:08:470
23689052430,4sleep33056-21PK-Backend21:00:133
23298842430,4sleep33056-21PK-Backend19:10:223
23782632400,7sleep22378264-21timerandwakeup21:25:232
24250522360,3sleep00-21swapper/023:40:130
232871899273,9cyclictest2363164-21apt20:45:131
232871599271,7cyclictest2379425-21diskstats21:30:150
2328718992619,1cyclictest0-21swapper/119:15:451
232871599261,13cyclictest1-21systemd21:20:000
2328718992518,5cyclictest0-21swapper/122:05:111
232871899251,22cyclictest2415768-21munin-run23:15:001
232871599250,8cyclictest0-21swapper/021:15:020
232871599250,15cyclictest0-21swapper/023:45:140
23614322240,3chrt0-21swapper/220:40:112
232871899249,10cyclictest436-20systemd-journal22:50:261
232871899249,10cyclictest436-20systemd-journal22:50:261
2328718992416,5cyclictest2375322-21turbostat.cron21:20:001
2328718992415,7cyclictest883-21dbus-daemon19:40:181
232871599241,13cyclictest0-21swapper/000:25:010
139099240,17rtkit-daemon3056-21PK-Backend20:42:543
2328725992320,2cyclictest3056-21PK-Backend20:32:563
2328725992313,6cyclictest2378778-21idleruntime-cro21:30:003
232872599230,21cyclictest3056-21PK-Backend00:07:583
2328718992321,1cyclictest0-21swapper/119:10:181
2328718992313,7cyclictest883-21dbus-daemon23:00:001
2328718992312,8cyclictest2366607-21cron20:55:001
2328718992312,8cyclictest2334375-21basename19:25:001
232871899230,21cyclictest2349393-21df20:05:131
232871599237,3cyclictest2405448-21diskstats22:45:140
232871599235,10cyclictest2392524-21chrt22:08:200
2328715992314,7cyclictest1-21systemd00:00:000
139099230,16rtkit-daemon3056-21PK-Backend23:35:133
24441882220,4sleep10-21swapper/100:35:161
23344212220,3sleep20-21swapper/219:25:002
2328725992220,1cyclictest3056-21PK-Backend22:23:013
2328725992220,1cyclictest3056-21PK-Backend22:11:183
2328725992219,2cyclictest3056-21PK-Backend21:41:373
2328725992219,2cyclictest3056-21PK-Backend21:24:283
232872599220,20cyclictest3056-21PK-Backend23:08:233
232872599220,20cyclictest3056-21PK-Backend22:36:463
232872599220,20cyclictest2360841-21chrt20:36:073
2328723992220,1cyclictest0-21swapper/223:17:122
2328723992220,1cyclictest0-21swapper/223:11:172
2328723992220,1cyclictest0-21swapper/220:58:342
2328723992220,1cyclictest0-21swapper/220:00:012
2328723992220,1cyclictest0-21swapper/219:27:572
2328718992220,1cyclictest0-21swapper/122:16:541
2328718992220,1cyclictest0-21swapper/121:34:221
2328718992220,1cyclictest0-21swapper/121:05:441
2328718992220,1cyclictest0-21swapper/120:32:171
2328718992215,6cyclictest2368391-21systemd-run21:00:001
2328718992214,6cyclictest2441801-21sleep00:30:001
232871599222,7cyclictest2394010-21users22:10:220
2328715992220,1cyclictest0-21swapper/023:24:270
2328715992220,1cyclictest0-21swapper/023:10:470
2328715992220,1cyclictest0-21swapper/022:22:550
2328715992220,1cyclictest0-21swapper/021:46:050
2328715992220,1cyclictest0-21swapper/021:46:040
2328715992220,1cyclictest0-21swapper/020:02:350
2328715992220,1cyclictest0-21swapper/019:54:180
2328715992220,1cyclictest0-21swapper/019:23:560
2328715992220,1cyclictest0-21swapper/000:32:120
2328715992220,1cyclictest0-21swapper/000:10:150
2328715992220,1cyclictest0-21swapper/000:04:210
232871599221,4cyclictest2434860-21awk00:10:000
232871599220,20cyclictest436-20systemd-journal00:30:000
2328725992119,1cyclictest3056-21PK-Backend23:42:573
2328725992119,1cyclictest3056-21PK-Backend23:22:503
2328725992119,1cyclictest3056-21PK-Backend23:16:073
2328725992119,1cyclictest3056-21PK-Backend22:51:203
2328725992119,1cyclictest3056-21PK-Backend22:51:203
2328725992119,1cyclictest3056-21PK-Backend21:54:013
2328725992119,1cyclictest3056-21PK-Backend20:29:463
2328725992119,1cyclictest3056-21PK-Backend19:41:053
2328725992115,4cyclictest42-21ksoftirqd/321:55:223
232872599210,20cyclictest3056-21PK-Backend21:18:063
232872599210,20cyclictest3056-21PK-Backend20:24:123
232872599210,20cyclictest3056-21PK-Backend20:18:023
232872599210,20cyclictest3056-21PK-Backend00:01:163
232872599210,19cyclictest3056-21PK-Backend19:20:313
2328723992119,1cyclictest3056-21PK-Backend23:49:232
2328723992119,1cyclictest0-21swapper/223:32:532
2328723992119,1cyclictest0-21swapper/223:25:092
2328723992119,1cyclictest0-21swapper/221:24:032
2328723992119,1cyclictest0-21swapper/220:36:252
2328723992119,1cyclictest0-21swapper/220:26:372
2328723992119,1cyclictest0-21swapper/220:21:512
2328723992119,1cyclictest0-21swapper/220:18:402
2328723992119,1cyclictest0-21swapper/219:51:432
2328723992119,1cyclictest0-21swapper/219:39:132
2328723992119,1cyclictest0-21swapper/219:30:342
2328723992119,1cyclictest0-21swapper/200:04:162
232872399210,20cyclictest0-21swapper/223:06:232
232872399210,20cyclictest0-21swapper/221:56:412
232872399210,20cyclictest0-21swapper/221:49:522
232872399210,20cyclictest0-21swapper/221:49:522
232872399210,20cyclictest0-21swapper/221:00:102
232872399210,20cyclictest0-21swapper/219:41:012
232872399210,20cyclictest0-21swapper/219:11:002
232872399210,1cyclictest0-21swapper/200:26:402
2328718992119,1cyclictest0-21swapper/123:47:181
2328718992119,1cyclictest0-21swapper/123:25:351
2328718992119,1cyclictest0-21swapper/123:18:521
2328718992119,1cyclictest0-21swapper/123:06:291
2328718992119,1cyclictest0-21swapper/121:20:361
2328718992119,1cyclictest0-21swapper/119:56:071
2328718992119,1cyclictest0-21swapper/119:29:101
2328718992113,6cyclictest2410809-21expr23:00:131
232871899210,20cyclictest0-21swapper/122:24:111
232871899210,20cyclictest0-21swapper/121:59:571
232871899210,20cyclictest0-21swapper/121:47:341
232871899210,20cyclictest0-21swapper/121:47:341
232871899210,20cyclictest0-21swapper/121:41:461
232871899210,20cyclictest0-21swapper/100:09:291
232871899210,19cyclictest0-21swapper/120:25:551
232871599218,10cyclictest2354233-21apt20:20:130
2328715992119,1cyclictest0-21swapper/023:54:340
2328715992119,1cyclictest0-21swapper/022:35:520
2328715992119,1cyclictest0-21swapper/022:25:450
2328715992119,1cyclictest0-21swapper/021:51:040
2328715992119,1cyclictest0-21swapper/020:47:310
2328715992119,1cyclictest0-21swapper/020:43:180
2328715992119,1cyclictest0-21swapper/019:48:020
2328715992119,1cyclictest0-21swapper/019:40:110
2328715992119,1cyclictest0-21swapper/019:18:490
2328715992118,2cyclictest0-21swapper/021:29:280
232871599211,7cyclictest2336366-21apt19:30:000
2328715992113,4cyclictest2382515-21apt21:40:120
232871599210,8cyclictest0-21swapper/021:55:120
232871599210,20cyclictest0-21swapper/019:33:080
139099210,14rtkit-daemon0-21swapper/222:20:082
23713632200,5sleep23056-21PK-Backend21:05:212
2328725992018,1cyclictest3056-21PK-Backend23:48:383
2328725992018,1cyclictest3056-21PK-Backend21:47:253
2328725992018,1cyclictest3056-21PK-Backend21:47:253
2328725992018,1cyclictest3056-21PK-Backend21:07:573
2328725992018,1cyclictest3056-21PK-Backend00:24:143
2328725992017,2cyclictest3056-21PK-Backend20:12:353
232872599200,1cyclictest3056-21PK-Backend20:05:043
232872599200,1cyclictest3056-21PK-Backend00:15:423
232872599200,1cyclictest3056-21PK-Backend00:11:503
232872599200,19cyclictest3056-21PK-Backend23:57:243
232872599200,19cyclictest3056-21PK-Backend23:52:303
232872599200,19cyclictest3056-21PK-Backend23:01:403
232872599200,19cyclictest3056-21PK-Backend22:57:493
232872599200,19cyclictest3056-21PK-Backend20:59:263
232872599200,19cyclictest3056-21PK-Backend19:58:363
232872599200,19cyclictest3056-21PK-Backend19:35:513
232872599200,19cyclictest0-21swapper/300:30:433
2328723992018,1cyclictest3056-21PK-Backend21:35:392
2328723992018,1cyclictest0-21swapper/223:41:392
2328723992018,1cyclictest0-21swapper/222:37:582
2328723992018,1cyclictest0-21swapper/222:06:212
2328723992018,1cyclictest0-21swapper/219:57:102
2328723992018,1cyclictest0-21swapper/200:36:142
2328723992018,1cyclictest0-21swapper/200:11:132
232872399200,1cyclictest0-21swapper/221:40:392
232872399200,1cyclictest0-21swapper/220:13:122
232872399200,19cyclictest0-21swapper/222:03:302
232872399200,19cyclictest0-21swapper/221:31:262
232872399200,19cyclictest0-21swapper/220:45:372
232872399200,19cyclictest0-21swapper/220:09:212
232872399200,19cyclictest0-21swapper/219:48:042
232871899202,13cyclictest2338360-21apt19:35:121
2328718992018,1cyclictest0-21swapper/123:36:581
2328718992018,1cyclictest0-21swapper/122:25:121
2328718992018,1cyclictest0-21swapper/122:10:391
2328718992018,1cyclictest0-21swapper/121:53:211
2328718992018,1cyclictest0-21swapper/120:12:431
2328718992013,5cyclictest2438221-21sshd00:18:571
2328718992012,5cyclictest436-20systemd-journal00:23:571
232871899200,19cyclictest0-21swapper/122:46:011
232871899200,19cyclictest0-21swapper/122:40:331
232871899200,19cyclictest0-21swapper/120:37:511
232871899200,19cyclictest0-21swapper/120:23:311
232871899200,19cyclictest0-21swapper/119:47:151
232871899200,19cyclictest0-21swapper/100:13:211
232871599209,9cyclictest2421689-21/usr/sbin/munin23:30:160
2328715992018,1cyclictest0-21swapper/023:02:020
2328715992018,1cyclictest0-21swapper/022:57:080
2328715992018,1cyclictest0-21swapper/022:03:410
2328715992018,1cyclictest0-21swapper/021:07:100
2328715992018,1cyclictest0-21swapper/020:05:260
2328715992018,1cyclictest0-21swapper/019:38:220
2328715992018,1cyclictest0-21swapper/000:17:590
232871599201,7cyclictest0-21swapper/020:55:210
2328715992014,5cyclictest2400936-21sed22:30:240
232871599200,19cyclictest0-21swapper/023:25:270
232871599200,19cyclictest0-21swapper/023:15:400
232871599200,19cyclictest0-21swapper/022:53:160
232871599200,19cyclictest0-21swapper/022:53:160
232871599200,19cyclictest0-21swapper/020:51:430
232871599200,19cyclictest0-21swapper/020:35:540
232871599200,19cyclictest0-21swapper/019:59:450
232871599200,19cyclictest0-21swapper/000:36:520
139099200,14rtkit-daemon0-21swapper/222:53:222
139099200,14rtkit-daemon0-21swapper/222:53:222
139099200,14rtkit-daemon0-21swapper/219:17:412
2328725991917,1cyclictest3056-21PK-Backend23:12:153
2328725991917,1cyclictest3056-21PK-Backend19:47:063
2328725991917,1cyclictest3056-21PK-Backend19:29:023
2328725991917,1cyclictest3056-21PK-Backend19:16:583
2328725991917,1cyclictest0-21swapper/320:01:263
2328725991916,2cyclictest3056-21PK-Backend22:49:303
232872599190,1cyclictest3056-21PK-Backend22:07:403
232872599190,1cyclictest3056-21PK-Backend21:35:013
232872599190,1cyclictest3056-21PK-Backend00:37:123
232872599190,18cyclictest0-21swapper/322:40:253
2328723991917,1cyclictest0-21swapper/220:52:262
2328723991917,1cyclictest0-21swapper/200:23:502
2328723991917,1cyclictest0-21swapper/200:06:192
2328723991912,5cyclictest2423717-21awk23:35:202
232872399190,1cyclictest0-21swapper/223:57:212
232872399190,1cyclictest0-21swapper/222:13:382
232872399190,18cyclictest0-21swapper/222:25:482
2328718991917,1cyclictest0-21swapper/123:58:541
2328718991917,1cyclictest0-21swapper/122:32:291
2328718991917,1cyclictest0-21swapper/120:18:111
2328718991917,1cyclictest0-21swapper/119:32:351
2328718991913,4cyclictest2425358-21strings23:40:181
2328718991912,5cyclictest2428678-21ls23:50:171
2328718991912,5cyclictest2344778-21sshd19:51:571
2328718991911,6cyclictest2391064-21cut22:05:001
232871899190,1cyclictest0-21swapper/122:36:551
232871899190,1cyclictest0-21swapper/121:27:461
232871899190,1cyclictest0-21swapper/121:03:191
232871899190,18cyclictest0-21swapper/123:31:161
2328715991917,1cyclictest0-21swapper/022:15:380
2328715991917,1cyclictest0-21swapper/020:18:230
2328715991916,2cyclictest2369010-21diskstats21:00:140
2328715991911,5cyclictest2355966-21apt20:25:120
232871599190,1cyclictest0-21swapper/023:37:030
232871599190,1cyclictest0-21swapper/022:43:090
139099190,13rtkit-daemon3056-21PK-Backend21:32:333
2328725991816,1cyclictest3056-21PK-Backend20:49:183
2328725991815,2cyclictest3056-21PK-Backend19:32:263
2328723991816,1cyclictest3056-21PK-Backend21:52:302
2328723991816,1cyclictest0-21swapper/223:52:272
2328723991816,1cyclictest0-21swapper/223:20:162
2328723991816,1cyclictest0-21swapper/222:44:282
2328723991816,1cyclictest0-21swapper/222:33:182
232872399180,1cyclictest0-21swapper/223:01:302
232871899182,11cyclictest2380781-21apt21:35:121
2328718991816,1cyclictest0-21swapper/100:30:241
2328718991810,6cyclictest2372419-21ls21:10:131
232872599170,15cyclictest2422306-21nm-dispatcher23:30:273
232871899171,11cyclictest2361430-21apt20:40:131
2328718991710,5cyclictest2418097-21expr23:20:131
2328718991710,5cyclictest2347495-21ls20:00:131
2328715991715,1cyclictest0-21swapper/020:11:070
2328715991711,3cyclictest2328939-21apt19:10:110
2328715991710,5cyclictest2382400-21wc21:40:000
2328715991710,5cyclictest2376108-21cat21:20:170
2328725991614,1cyclictest3056-21PK-Backend22:19:223
2328725991614,1cyclictest3056-21PK-Backend21:10:213
2328725991614,1cyclictest3056-21PK-Backend20:51:563
2328725991614,1cyclictest3056-21PK-Backend00:27:043
2328725991612,2cyclictest42-21ksoftirqd/319:55:003
232872399169,5cyclictest2373103-21sed21:10:212
2328723991614,1cyclictest3056-21PK-Backend00:33:232
2328723991614,1cyclictest0-21swapper/222:57:382
2328723991613,2cyclictest3056-21PK-Backend22:45:282
2328723991611,3cyclictest34-21ksoftirqd/220:35:022
2328718991612,3cyclictest2432385-21ntp_offset00:00:181
2328715991610,5cyclictest13-21ksoftirqd/020:30:130
232872599154,10cyclictest42-21ksoftirqd/322:00:223
2328725991513,1cyclictest0-21swapper/322:25:513
2328723991410,3cyclictest2394843-21cpuspeed_turbos22:15:142
2328723991410,3cyclictest2373820-21/usr/sbin/munin21:15:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional