You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-19 - 22:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Aug 19, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
324885921270,3sleep23248801-21latency_hist12:10:012
315098521240,3sleep33056-21PK-Backend07:30:123
314534621120,3sleep10-21swapper/107:15:021
321224721030,4sleep03056-21PK-Backend10:25:130
32018382990,2sleep10-21swapper/109:55:141
32246872970,4sleep33056-21PK-Backend11:00:163
139099870,70rtkit-daemon0-21swapper/307:07:053
314285026939,25sleep00-21swapper/007:09:530
314270326545,14sleep10-21swapper/107:08:021
314277526443,13sleep20-21swapper/207:08:552
31529382410,2sleep3391irq_work/307:35:173
31856642360,3sleep234-21ksoftirqd/209:10:122
31717432360,2sleep00-21swapper/008:30:130
32393232310,4sleep03056-21PK-Backend11:40:200
32518762300,4sleep03056-21PK-Backend12:15:250
32206062300,4chrt3056-21PK-Backend10:50:120
31506972290,4chrt0-21swapper/207:30:122
314301899261,23cyclictest3251910-21taskset12:15:413
32030642250,3sleep2331rcuc/210:00:002
3143002992516,6cyclictest3160803-21cron07:59:590
139099250,4rtkit-daemon1389-21rtkit-daemon07:33:131
314301899240,23cyclictest0-21swapper/310:01:303
3143002992415,6cyclictest2175-21JS10:27:480
31678612230,2sleep33056-21PK-Backend08:20:003
314301899230,21cyclictest3238145-21seq11:36:543
139099230,4rtkit-daemon1389-21rtkit-daemon07:32:150
139099230,15rtkit-daemon3056-21PK-Backend11:47:180
314301899221,8cyclictest3239070-21hddtemp_smartct11:40:173
3143015992215,5cyclictest3236657-21cron11:35:002
3143002992220,1cyclictest0-21swapper/010:13:110
3143002992220,1cyclictest0-21swapper/008:20:290
3143002992220,1cyclictest0-21swapper/008:06:160
3143002992220,1cyclictest0-21swapper/007:36:230
3143002992220,1cyclictest0-21swapper/007:18:110
3143002992219,2cyclictest13-21ksoftirqd/011:29:090
314300299221,17cyclictest3248808-21turbostat12:10:000
139099220,3rtkit-daemon0-21swapper/010:44:370
139099220,18rtkit-daemon13-21ksoftirqd/007:10:110
139099220,16rtkit-daemon3056-21PK-Backend10:23:161
139099220,14rtkit-daemon0-21swapper/112:00:291
32298972210,3sleep30-21swapper/311:15:163
32226952210,4sleep13056-21PK-Backend10:55:171
3143018992114,5cyclictest0-21swapper/310:15:133
314301899210,20cyclictest0-21swapper/309:35:353
314301899210,20cyclictest0-21swapper/309:35:343
3143015992115,4cyclictest3195396-21sshd09:35:452
3143015992115,4cyclictest3195396-21sshd09:35:442
314301599210,20cyclictest1020-21ntpd07:18:462
314301599210,20cyclictest0-21swapper/209:29:592
314301599210,1cyclictest0-21swapper/207:51:452
314301599210,1cyclictest0-21swapper/207:51:452
3143007992116,3cyclictest3213235-21runrttasks10:28:521
3143007992115,4cyclictest26-21ksoftirqd/108:40:111
314300799211,19cyclictest0-21swapper/109:45:201
314300799210,19cyclictest0-21swapper/107:55:161
314300299212,17cyclictest3236660-21cron11:34:590
3143002992119,1cyclictest0-21swapper/012:36:480
3143002992119,1cyclictest0-21swapper/012:36:480
3143002992119,1cyclictest0-21swapper/010:03:580
3143002992119,1cyclictest0-21swapper/008:41:380
3143002992119,1cyclictest0-21swapper/008:10:020
3143002992119,1cyclictest0-21swapper/007:50:080
3143002992119,1cyclictest0-21swapper/007:50:080
3143002992119,1cyclictest0-21swapper/007:49:210
3143002992119,1cyclictest0-21swapper/007:40:420
3143002992118,2cyclictest3056-21PK-Backend12:20:260
3143002992118,2cyclictest3056-21PK-Backend10:06:080
314300299210,20cyclictest0-21swapper/010:36:570
314300299210,20cyclictest0-21swapper/009:29:380
314300299210,20cyclictest0-21swapper/009:12:210
314300299210,20cyclictest0-21swapper/009:03:290
314300299210,19cyclictest3056-21PK-Backend10:55:150
139099210,14rtkit-daemon0-21swapper/112:37:361
139099210,14rtkit-daemon0-21swapper/112:37:361
139099210,14rtkit-daemon0-21swapper/111:31:151
314301899208,10cyclictest3212245-21diskstats10:25:143
3143018992018,1cyclictest3056-21PK-Backend07:51:243
3143018992018,1cyclictest3056-21PK-Backend07:51:243
3143018992017,2cyclictest3056-21PK-Backend08:41:073
3143018992017,2cyclictest3056-21PK-Backend07:17:253
3143018992014,4cyclictest2156-21gnome-shell11:34:093
3143018992012,6cyclictest3241859-21cron11:50:003
314301899200,19cyclictest3056-21PK-Backend12:39:473
314301899200,19cyclictest3056-21PK-Backend12:39:473
314301899200,19cyclictest3056-21PK-Backend09:28:453
314301899200,19cyclictest0-21swapper/312:22:453
314301899200,19cyclictest0-21swapper/311:58:103
314301899200,19cyclictest0-21swapper/311:10:383
314301899200,19cyclictest0-21swapper/309:10:143
314301899200,18cyclictest3056-21PK-Backend08:38:573
3143015992018,1cyclictest0-21swapper/211:56:282
3143015992018,1cyclictest0-21swapper/210:52:192
3143015992018,1cyclictest0-21swapper/210:49:222
3143015992018,1cyclictest0-21swapper/208:58:292
3143015992018,1cyclictest0-21swapper/208:44:312
3143015992018,1cyclictest0-21swapper/208:38:222
3143015992018,1cyclictest0-21swapper/208:33:012
3143015992018,1cyclictest0-21swapper/208:23:072
3143015992012,6cyclictest436-20systemd-journal07:13:442
314301599200,1cyclictest3056-21PK-Backend12:16:012
314301599200,19cyclictest3056-21PK-Backend08:49:382
314301599200,19cyclictest0-21swapper/211:18:562
314301599200,19cyclictest0-21swapper/210:43:272
314301599200,19cyclictest0-21swapper/210:14:202
314301599200,19cyclictest0-21swapper/209:32:092
314301599200,19cyclictest0-21swapper/209:19:122
314301599200,19cyclictest0-21swapper/209:01:272
314301599200,19cyclictest0-21swapper/208:50:262
314301599200,19cyclictest0-21swapper/207:45:492
3143007992012,6cyclictest3257953-21sed12:35:001
3143002992018,1cyclictest3056-21PK-Backend11:56:330
3143002992018,1cyclictest0-21swapper/012:12:560
3143002992018,1cyclictest0-21swapper/011:52:480
3143002992018,1cyclictest0-21swapper/010:17:310
3143002992018,1cyclictest0-21swapper/009:52:370
3143002992018,1cyclictest0-21swapper/009:36:080
3143002992018,1cyclictest0-21swapper/009:36:070
3143002992018,1cyclictest0-21swapper/009:30:260
3143002992018,1cyclictest0-21swapper/008:56:590
3143002992016,3cyclictest0-21swapper/009:40:260
3143002992016,3cyclictest0-21swapper/009:40:260
3143002992014,4cyclictest2156-21gnome-shell12:27:090
3143002992013,6cyclictest3173751-21sed08:35:170
314300299200,1cyclictest0-21swapper/010:45:020
314300299200,1cyclictest0-21swapper/008:27:130
314300299200,1cyclictest0-21swapper/007:25:010
314300299200,19cyclictest3056-21PK-Backend11:20:170
314300299200,19cyclictest0-21swapper/012:32:150
314300299200,19cyclictest0-21swapper/011:13:340
314300299200,19cyclictest0-21swapper/011:00:220
314300299200,19cyclictest0-21swapper/010:23:120
314300299200,19cyclictest0-21swapper/007:28:310
139099200,14rtkit-daemon3056-21PK-Backend10:13:261
139099200,14rtkit-daemon0-21swapper/111:47:531
139099200,14rtkit-daemon0-21swapper/110:43:001
139099200,13rtkit-daemon3056-21PK-Backend11:58:221
3143018991917,1cyclictest3056-21PK-Backend10:47:333
3143018991917,1cyclictest0-21swapper/312:06:153
3143018991917,1cyclictest0-21swapper/308:30:323
3143018991916,2cyclictest3056-21PK-Backend11:24:113
3143018991916,2cyclictest3056-21PK-Backend08:00:243
3143018991912,5cyclictest3246525-21cat12:00:243
3143018991912,5cyclictest3200854-21sshd09:50:443
3143018991912,5cyclictest3199465-21cron09:49:593
3143018991912,4cyclictest1095-21runrttasks09:55:133
314301899191,16cyclictest3249806-21ntp_states12:10:193
314301899191,16cyclictest3188211-21python309:15:213
314301899191,16cyclictest3170134-21diskstats08:25:143
314301899190,1cyclictest3056-21PK-Backend10:57:203
314301899190,1cyclictest3056-21PK-Backend09:05:203
314301899190,1cyclictest3056-21PK-Backend08:58:163
314301899190,1cyclictest3056-21PK-Backend08:21:533
314301899190,1cyclictest3056-21PK-Backend07:42:263
314301899190,1cyclictest3056-21PK-Backend07:29:003
314301899190,1cyclictest0-21swapper/310:21:393
314301899190,18cyclictest3056-21PK-Backend10:31:113
314301899190,18cyclictest0-21swapper/311:06:193
314301899190,17cyclictest3056-21PK-Backend08:50:053
3143015991917,1cyclictest0-21swapper/212:28:052
3143015991917,1cyclictest0-21swapper/211:45:122
3143015991917,1cyclictest0-21swapper/210:30:162
3143015991917,1cyclictest0-21swapper/210:29:282
3143015991917,1cyclictest0-21swapper/210:07:182
3143015991917,1cyclictest0-21swapper/209:51:362
3143015991917,1cyclictest0-21swapper/209:20:482
3143015991917,1cyclictest0-21swapper/209:07:222
3143015991917,1cyclictest0-21swapper/208:18:352
3143015991915,3cyclictest978-21snmpd12:12:432
3143015991914,3cyclictest3222024-21latency_hist10:55:012
3143015991912,6cyclictest3150363-21cron07:30:012
3143015991910,7cyclictest3253397-21python312:20:202
314301599190,1cyclictest0-21swapper/212:32:382
314301599190,1cyclictest0-21swapper/211:54:522
314301599190,1cyclictest0-21swapper/211:36:542
314301599190,1cyclictest0-21swapper/211:26:272
314301599190,1cyclictest0-21swapper/209:48:382
314301599190,1cyclictest0-21swapper/208:09:292
314301599190,1cyclictest0-21swapper/207:42:522
314301599190,18cyclictest0-21swapper/212:00:132
314301599190,18cyclictest0-21swapper/211:40:392
3143007991917,1cyclictest0-21swapper/110:35:151
3143007991912,5cyclictest1-21systemd07:40:011
3143007991911,6cyclictest3255755-21munin-run12:29:591
3143007991910,7cyclictest3149469-21memory07:25:161
314300299192,12cyclictest3213610-21apt10:30:110
3143002991917,1cyclictest3056-21PK-Backend12:00:180
3143002991917,1cyclictest3056-21PK-Backend11:35:510
3143002991917,1cyclictest0-21swapper/011:08:270
3143002991917,1cyclictest0-21swapper/009:21:130
3143002991917,1cyclictest0-21swapper/008:50:310
3143002991914,3cyclictest3162023-21threads08:00:230
314300299190,1cyclictest0-21swapper/011:16:320
314300299190,1cyclictest0-21swapper/009:46:080
314300299190,1cyclictest0-21swapper/008:47:330
314300299190,18cyclictest0-21swapper/009:15:180
31448142180,3sleep30-21swapper/307:15:003
314301899186,10cyclictest3056-21PK-Backend12:34:003
3143018991812,4cyclictest3181874-21find09:00:103
3143018991811,5cyclictest3221674-21sshd10:50:463
314301899181,15cyclictest3207384-21/usr/sbin/munin10:10:183
314301599189,7cyclictest883-21dbus-daemon08:12:382
3143015991816,1cyclictest0-21swapper/211:04:102
3143015991816,1cyclictest0-21swapper/210:22:592
3143015991816,1cyclictest0-21swapper/208:27:402
3143015991816,1cyclictest0-21swapper/208:01:242
3143015991816,1cyclictest0-21swapper/207:55:292
3143015991816,1cyclictest0-21swapper/207:36:232
3143015991811,5cyclictest3210065-21date10:20:012
3143015991811,5cyclictest3204861-21cut10:05:022
3143007991813,3cyclictest3167757-21cron08:20:011
3143007991812,5cyclictest3228751-21sed11:10:231
3143007991811,5cyclictest436-20systemd-journal09:20:001
3143007991811,5cyclictest3234179-21ntpq11:25:181
3143007991811,5cyclictest3224857-21cat11:00:181
3143007991811,5cyclictest3220889-21expr10:50:121
3143007991811,5cyclictest3213964-21cat10:30:131
3143007991811,5cyclictest3147160-21apt07:20:121
3143007991810,6cyclictest3237701-21awk11:35:221
3143007991810,6cyclictest3232184-21ntpq11:20:171
314300299182,10cyclictest3183346-21apt-get09:05:040
3143002991816,1cyclictest0-21swapper/008:19:420
3143018991715,1cyclictest0-21swapper/310:05:013
314301899171,12cyclictest3242136-21apt11:50:113
3143018991710,5cyclictest3254744-21awk12:25:163
314301899170,15cyclictest3056-21PK-Backend09:22:093
314301899170,15cyclictest3056-21PK-Backend07:55:303
3143015991715,1cyclictest0-21swapper/211:10:512
3143015991711,5cyclictest3196624-21expr09:40:132
3143015991711,5cyclictest3196624-21expr09:40:122
314301599171,10cyclictest3230941-21apt-get11:20:042
314300799179,6cyclictest3219407-21memory10:45:191
314300799172,13cyclictest3194430-21cpuspeed_turbos09:35:121
314300799172,13cyclictest3194430-21cpuspeed_turbos09:35:121
314300799172,11cyclictest3190816-21apt09:25:131
3143007991715,1cyclictest0-21swapper/108:54:101
3143007991713,3cyclictest3177631-21threads08:45:221
3143007991713,3cyclictest3168151-21tr08:20:131
3143007991712,3cyclictest3169908-21cpuspeed_turbos08:25:141
3143007991712,3cyclictest3161693-21missed_timers08:00:181
3143007991712,3cyclictest3155180-21timerwakeupswit07:40:221
3143007991711,5cyclictest3252073-21sshd12:17:451
3143007991711,5cyclictest3193517-21sed09:30:221
3143007991711,4cyclictest3164989-21cat08:10:151
3143007991710,6cyclictest1-21systemd09:45:011
3143007991710,6cyclictest1-21systemd09:45:001
3143007991710,5cyclictest3239177-21cut11:40:181
3143007991710,5cyclictest3156548-21uname07:45:171
314301899163,12cyclictest3163760-21timerandwakeup08:05:233
3143018991614,1cyclictest3056-21PK-Backend10:40:443
3143018991614,1cyclictest0-21swapper/309:31:303
314301899160,15cyclictest3056-21PK-Backend08:47:213
3143015991614,1cyclictest0-21swapper/212:38:452
3143015991614,1cyclictest0-21swapper/212:38:452
3143015991610,5cyclictest436-20systemd-journal10:39:552
314300799169,6cyclictest3226334-21expr11:05:131
3143007991614,1cyclictest0-21swapper/109:50:191
3143007991613,2cyclictest3056-21PK-Backend11:19:221
3143007991611,4cyclictest436-20systemd-journal09:00:011
3143007991611,4cyclictest3185762-21grep09:10:121
314300799161,13cyclictest3189940-21sensors09:20:211
3143007991610,5cyclictest3173599-21seq08:35:131
3143007991610,4cyclictest3205980-21grep10:05:221
314300799160,6cyclictest3164289-21idleruntime-cro08:10:001
314300799160,14cyclictest3171703-21cat08:30:121
3143018991513,1cyclictest3056-21PK-Backend11:29:043
3143018991512,2cyclictest3144904-21munin-node07:20:123
3143018991512,2cyclictest3056-21PK-Backend10:38:003
314301899150,14cyclictest3056-21PK-Backend09:42:583
314301899150,14cyclictest3056-21PK-Backend09:42:583
3143015991513,1cyclictest0-21swapper/207:22:182
314301599151,11cyclictest3226753-21ntp_states11:05:182
314300799159,5cyclictest3184629-21uname09:05:201
3143007991513,1cyclictest3056-21PK-Backend11:52:201
3143007991513,1cyclictest0-21swapper/107:50:151
3143007991513,1cyclictest0-21swapper/107:50:141
3143007991512,2cyclictest3056-21PK-Backend12:24:591
314300799151,12cyclictest3144663-21sh07:15:011
314300799150,14cyclictest0-21swapper/110:00:061
314301899140,12cyclictest3155859-21/usr/sbin/munin07:45:163
314300799147,5cyclictest3208853-21cat10:15:181
3143007991412,1cyclictest0-21swapper/109:01:051
314300799140,1cyclictest3056-21PK-Backend12:06:281
314300799140,13cyclictest0-21swapper/112:10:191
3143018991310,2cyclictest3056-21PK-Backend08:15:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional