You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-27 - 18:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Aug 27, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
345598728040,14sleep30-21swapper/307:08:223
345581926944,16sleep20-21swapper/207:06:122
139099680,58rtkit-daemon0-21swapper/107:06:281
345595626242,14sleep00-21swapper/007:08:000
35673532590,2sleep30-21swapper/312:25:003
35405102460,4sleep00-21swapper/011:05:180
34751632440,4sleep13056-21PK-Backend08:00:131
35592422290,4sleep00-21swapper/012:00:150
3456255992815,11cyclictest3532598-21unin-run10:45:003
34594602270,5sleep342-21ksoftirqd/307:15:193
35190702260,5sleep33518683-21/usr/sbin/munin10:05:143
34897672250,3chrt3056-21PK-Backend08:40:202
35508052240,10sleep013-21ksoftirqd/011:35:180
3456252992415,7cyclictest3460138-21cron07:20:002
3456255992320,2cyclictest3056-21PK-Backend09:46:223
3456252992312,9cyclictest3529132-21unin-run10:35:002
3456248992320,2cyclictest3056-21PK-Backend07:54:371
3456244992314,6cyclictest3497397-21perf09:05:000
3456255992220,1cyclictest0-21swapper/309:19:253
3456255992220,1cyclictest0-21swapper/309:10:403
345625599220,21cyclictest0-21swapper/308:40:253
345625599220,21cyclictest0-21swapper/307:53:483
3456252992220,1cyclictest0-21swapper/212:09:352
3456252992217,3cyclictest1095-21runrttasks10:20:192
3456252992217,3cyclictest1095-21runrttasks10:20:182
3456252992213,7cyclictest436-20systemd-journal09:00:002
345625299220,21cyclictest0-21swapper/211:15:292
345625299220,20cyclictest0-21swapper/211:45:092
345625299220,20cyclictest0-21swapper/209:07:262
345625299220,20cyclictest0-21swapper/208:54:342
345625299220,20cyclictest0-21swapper/208:20:272
3456248992220,1cyclictest3056-21PK-Backend10:22:121
3456248992220,1cyclictest3056-21PK-Backend10:22:121
3456248992220,1cyclictest0-21swapper/110:37:261
3456248992213,7cyclictest3572726-21perf12:40:001
345624899220,20cyclictest3056-21PK-Backend08:51:001
3456255992119,1cyclictest3056-21PK-Backend10:15:363
3456255992119,1cyclictest0-21swapper/311:36:083
3456255992119,1cyclictest0-21swapper/311:04:433
3456255992119,1cyclictest0-21swapper/307:45:303
3456255992119,1cyclictest0-21swapper/307:29:483
3456255992116,3cyclictest3564614-21memory12:15:163
345625599210,20cyclictest0-21swapper/311:08:303
3456252992119,1cyclictest0-21swapper/212:38:422
3456252992119,1cyclictest0-21swapper/211:41:102
3456252992119,1cyclictest0-21swapper/211:11:422
3456252992119,1cyclictest0-21swapper/211:07:092
3456252992119,1cyclictest0-21swapper/210:54:462
3456252992119,1cyclictest0-21swapper/210:38:302
3456252992119,1cyclictest0-21swapper/210:25:462
3456252992119,1cyclictest0-21swapper/209:56:272
3456252992119,1cyclictest0-21swapper/209:45:102
3456252992119,1cyclictest0-21swapper/208:46:502
3456252992119,1cyclictest0-21swapper/208:17:312
3456252992119,1cyclictest0-21swapper/207:58:512
3456252992119,1cyclictest0-21swapper/207:12:002
3456252992114,5cyclictest28650irq/123-ahci[0010:55:192
3456252992113,6cyclictest3476375-21sed08:05:012
3456252992112,7cyclictest883-21dbus-daemon08:40:012
345625299210,20cyclictest0-21swapper/212:11:122
345625299210,20cyclictest0-21swapper/207:30:452
3456248992119,1cyclictest3056-21PK-Backend11:46:551
3456248992119,1cyclictest3056-21PK-Backend11:24:111
3456248992119,1cyclictest3056-21PK-Backend11:03:291
3456248992119,1cyclictest3056-21PK-Backend10:26:191
3456248992119,1cyclictest3056-21PK-Backend09:57:111
3456248992119,1cyclictest3056-21PK-Backend09:11:021
3456248992119,1cyclictest3056-21PK-Backend07:13:051
3456248992119,1cyclictest0-21swapper/109:32:441
3456248992119,1cyclictest0-21swapper/108:37:011
3456248992119,1cyclictest0-21swapper/108:31:401
3456248992118,2cyclictest3056-21PK-Backend10:31:111
3456248992113,6cyclictest3523135-21sed10:15:221
345624899210,20cyclictest3056-21PK-Backend11:43:441
345624899210,20cyclictest0-21swapper/108:27:071
345624899210,20cyclictest0-21swapper/108:27:071
345624899210,20cyclictest0-21swapper/108:16:531
345624899210,19cyclictest3056-21PK-Backend11:12:211
345624899210,19cyclictest3056-21PK-Backend10:09:561
3456244992111,8cyclictest883-21dbus-daemon07:20:000
3456244992111,7cyclictest436-20systemd-journal11:50:000
139099210,3rtkit-daemon1389-21rtkit-daemon09:42:300
3456255992018,1cyclictest3056-21PK-Backend09:41:433
3456255992018,1cyclictest0-21swapper/312:07:173
3456255992018,1cyclictest0-21swapper/310:59:363
3456255992018,1cyclictest0-21swapper/310:52:333
3456255992018,1cyclictest0-21swapper/310:31:243
3456255992018,1cyclictest0-21swapper/309:03:513
3456255992018,1cyclictest0-21swapper/308:48:433
3456255992017,2cyclictest3056-21PK-Backend08:20:043
3456255992015,3cyclictest3476596-21apt08:05:113
3456255992014,5cyclictest3526412-21cut10:25:163
345625599200,1cyclictest3056-21PK-Backend11:57:573
345625599200,1cyclictest3056-21PK-Backend08:12:133
345625599200,1cyclictest0-21swapper/307:34:483
345625599200,19cyclictest3056-21PK-Backend12:00:203
345625599200,19cyclictest3056-21PK-Backend11:51:483
345625599200,19cyclictest3056-21PK-Backend09:23:173
345625599200,19cyclictest3056-21PK-Backend08:38:023
345625599200,19cyclictest3056-21PK-Backend08:19:163
345625599200,19cyclictest0-21swapper/310:46:523
345625599200,19cyclictest0-21swapper/310:36:193
345625599200,19cyclictest0-21swapper/308:01:323
345625599200,19cyclictest0-21swapper/307:37:263
3456252992018,1cyclictest0-21swapper/211:50:432
3456252992018,1cyclictest0-21swapper/211:36:232
3456252992018,1cyclictest0-21swapper/209:30:582
3456252992018,1cyclictest0-21swapper/209:24:562
3456252992018,1cyclictest0-21swapper/209:17:462
3456252992018,1cyclictest0-21swapper/209:12:182
3456252992018,1cyclictest0-21swapper/208:11:022
3456252992016,3cyclictest3517692-21ntpq10:00:182
3456252992015,4cyclictest978-21snmpd07:49:382
3456252992013,5cyclictest3568468-21ntpq12:25:172
345625299200,1cyclictest0-21swapper/209:35:372
345625299200,19cyclictest0-21swapper/212:32:202
345625299200,19cyclictest0-21swapper/210:49:062
345625299200,19cyclictest0-21swapper/209:01:582
345625299200,19cyclictest0-21swapper/208:33:182
345625299200,19cyclictest0-21swapper/208:27:102
345625299200,19cyclictest0-21swapper/208:27:102
345625299200,19cyclictest0-21swapper/207:37:352
345625299200,18cyclictest0-21swapper/207:52:082
3456248992018,1cyclictest3056-21PK-Backend10:57:481
3456248992018,1cyclictest3056-21PK-Backend10:54:171
3456248992018,1cyclictest3056-21PK-Backend09:00:541
3456248992018,1cyclictest3056-21PK-Backend08:42:091
3456248992018,1cyclictest3056-21PK-Backend07:40:161
3456248992018,1cyclictest0-21swapper/109:06:551
3456248992018,1cyclictest0-21swapper/108:22:011
345624899200,1cyclictest3056-21PK-Backend07:35:021
345624899200,19cyclictest0-21swapper/111:57:031
345624899200,19cyclictest0-21swapper/110:12:391
345624899200,19cyclictest0-21swapper/108:56:141
345624899200,18cyclictest3056-21PK-Backend11:16:061
3456244992014,4cyclictest3529092-21cron10:35:010
35169642190,4sleep13056-21PK-Backend10:00:101
3456255991918,1cyclictest0-21swapper/311:16:473
3456255991917,1cyclictest3056-21PK-Backend09:05:133
3456255991917,1cyclictest0-21swapper/312:30:363
3456255991917,1cyclictest0-21swapper/311:33:093
3456255991917,1cyclictest0-21swapper/311:27:143
3456255991917,1cyclictest0-21swapper/311:23:293
3456255991917,1cyclictest0-21swapper/311:13:023
3456255991917,1cyclictest0-21swapper/309:55:283
3456255991917,1cyclictest0-21swapper/308:52:363
3456255991917,1cyclictest0-21swapper/307:57:403
3456255991915,3cyclictest3562778-21irqstats12:10:143
3456255991911,6cyclictest3571600-21df12:35:113
3456255991911,6cyclictest3456846-21fw_packets07:10:153
345625599190,1cyclictest3056-21PK-Backend09:53:333
345625599190,1cyclictest3056-21PK-Backend09:27:103
345625599190,1cyclictest3056-21PK-Backend09:27:093
345625599190,1cyclictest0-21swapper/311:49:253
345625599190,1cyclictest0-21swapper/310:22:593
345625599190,1cyclictest0-21swapper/310:22:593
345625599190,1cyclictest0-21swapper/309:34:193
345625599190,18cyclictest3056-21PK-Backend08:55:543
345625599190,18cyclictest3056-21PK-Backend08:27:213
345625599190,18cyclictest3056-21PK-Backend08:27:203
345625599190,18cyclictest0-21swapper/310:02:253
345625599190,18cyclictest0-21swapper/307:41:523
3456252991917,1cyclictest0-21swapper/207:43:502
3456252991917,1cyclictest0-21swapper/207:23:092
3456252991911,6cyclictest3567368-21sed12:25:002
3456252991911,6cyclictest3560310-21cron12:05:002
345625299190,1cyclictest0-21swapper/211:25:552
345625299190,1cyclictest0-21swapper/211:00:272
345625299190,1cyclictest0-21swapper/210:06:402
345625299190,1cyclictest0-21swapper/209:50:382
345625299190,18cyclictest0-21swapper/209:28:142
345625299190,18cyclictest0-21swapper/209:28:132
3456248991917,1cyclictest3056-21PK-Backend12:07:571
3456248991917,1cyclictest3056-21PK-Backend07:47:191
3456248991917,1cyclictest0-21swapper/109:49:271
3456248991917,1cyclictest0-21swapper/109:37:251
3456248991917,1cyclictest0-21swapper/108:14:441
3456248991916,2cyclictest3056-21PK-Backend12:00:131
3456248991911,6cyclictest3504330-21perf09:24:591
345624899190,1cyclictest3056-21PK-Backend12:34:481
345624899190,1cyclictest3056-21PK-Backend11:06:261
345624899190,1cyclictest3056-21PK-Backend10:45:511
345624899190,1cyclictest3056-21PK-Backend08:05:441
345624899190,1cyclictest3056-21PK-Backend07:30:031
345624899190,1cyclictest0-21swapper/109:26:081
345624899190,1cyclictest0-21swapper/109:26:081
345624899190,18cyclictest0-21swapper/111:54:391
3456244991912,5cyclictest3530117-21ntp_states10:35:200
3456244991910,7cyclictest883-21dbus-daemon10:59:430
3456255991814,3cyclictest978-21snmpd09:36:023
3456255991812,5cyclictest42-21ksoftirqd/307:20:133
3456255991812,5cyclictest3552283-21diskstats11:40:133
345625299189,7cyclictest3523637-21apt-get10:20:002
3456252991816,1cyclictest0-21swapper/211:55:292
3456248991816,1cyclictest3056-21PK-Backend09:18:251
3456248991816,1cyclictest3056-21PK-Backend07:55:381
3456248991816,1cyclictest3056-21PK-Backend07:15:291
3456248991811,6cyclictest3514972-21cron09:55:001
345624499189,7cyclictest436-20systemd-journal10:50:000
345624499185,3cyclictest35522162expr11:40:120
345624499183,13cyclictest3471131-21munin-plugin-st07:50:000
3456244991813,3cyclictest35142342sleep009:50:200
3456244991812,5cyclictest3489159-21expr08:40:120
3456244991812,5cyclictest3466864-21wc07:35:220
3456244991811,5cyclictest436-20systemd-journal08:31:120
3456244991811,5cyclictest3484978-21cron08:30:000
3456244991811,5cyclictest3484978-21cron08:29:590
3456244991811,5cyclictest3474604-21latency_hist08:00:000
3456252991715,1cyclictest0-21swapper/211:30:282
3456252991715,1cyclictest0-21swapper/211:20:002
3456252991715,1cyclictest0-21swapper/209:41:042
3456252991712,3cyclictest3462933-21latency07:25:172
3456252991711,5cyclictest3565578-21wc12:20:002
3456252991710,5cyclictest3477358-21awk08:05:182
3456248991715,1cyclictest3056-21PK-Backend10:44:291
345624499179,5cyclictest3561369-21processes12:05:190
345624499172,13cyclictest3526058-21tr10:25:120
345624499172,11cyclictest3494188-21apt08:55:120
3456244991712,3cyclictest3471774-21sh07:50:130
3456244991712,3cyclictest3461635-21timerandwakeup07:20:210
3456244991711,5cyclictest3547271-21sed11:25:160
3456244991711,5cyclictest3542956-21cron11:14:590
3456244991711,5cyclictest3480617-21cat08:15:170
3456244991711,5cyclictest13-21ksoftirqd/009:05:190
3456244991710,6cyclictest3475361-21sed08:00:170
3456244991710,5cyclictest3556100-21ntpq11:50:180
3456244991710,5cyclictest3512670-21ls09:45:230
3456244991710,5cyclictest3488805-21cut08:40:000
3456244991710,5cyclictest3457205-21awk07:10:190
345625599169,6cyclictest3568000-21expr12:25:113
3456255991614,1cyclictest3056-21PK-Backend08:32:413
345625599160,14cyclictest3521040-21missed_timers10:10:193
345625299169,5cyclictest3531875-21pluginstate10:40:182
3456248991614,1cyclictest3056-21PK-Backend11:27:571
3456248991614,1cyclictest3056-21PK-Backend08:49:051
3456248991614,1cyclictest0-21swapper/109:40:091
345624499169,5cyclictest436-20systemd-journal12:19:550
345624499169,5cyclictest3522258-21ls10:15:120
345624499169,5cyclictest3507066-21ntp_states09:30:190
345624499169,5cyclictest3504740-21grep09:25:130
345624499169,5cyclictest3504740-21grep09:25:130
345624499169,5cyclictest3503985-21ls09:21:280
345624499169,5cyclictest3464644-21cut07:30:150
345624499168,6cyclictest436-20systemd-journal09:40:000
3456244991614,1cyclictest0-21swapper/010:24:080
3456244991614,1cyclictest0-21swapper/010:24:080
3456244991612,3cyclictest3478299-21/usr/sbin/munin08:10:170
3456244991611,4cyclictest13-21ksoftirqd/009:15:160
3456244991611,4cyclictest13-21ksoftirqd/007:45:000
345624499161,13cyclictest3538165-21grep11:00:120
3456244991611,3cyclictest3482379-21missed_timers08:20:170
3456244991610,5cyclictest883-21dbus-daemon10:45:000
3456244991610,4cyclictest2156-21gnome-shell11:24:090
3456244991610,4cyclictest2156-21gnome-shell08:48:090
345624499160,15cyclictest0-21swapper/010:02:390
139099160,13rtkit-daemon1389-21rtkit-daemon11:55:280
3456252991510,3cyclictest3521902-21latency_hist10:15:012
3456248991513,1cyclictest3056-21PK-Backend11:36:481
3456248991512,2cyclictest3056-21PK-Backend12:25:281
345624499159,4cyclictest3549029-21grep11:30:170
345624499158,5cyclictest3569227-21cron12:34:550
345624499158,5cyclictest3535412-21cut10:50:210
345624499157,6cyclictest2013-21gdbus12:39:430
345624499152,11cyclictest3544708-21perf11:20:000
3456244991511,3cyclictest3563208-21switchtime12:10:200
3456244991511,3cyclictest1095-21runrttasks09:55:160
3456244991510,3cyclictest3567270-21apt-key12:25:000
3456244991510,3cyclictest3519249-21latency10:05:170
3456244991510,3cyclictest3500057-21grep09:10:190
3456244991510,3cyclictest3493397-21xargs08:50:220
3456248991412,1cyclictest3056-21PK-Backend12:20:421
3456248991412,1cyclictest3056-21PK-Backend11:33:511
345624499149,3cyclictest3568768-21timerwakeupswit12:25:210
345624499149,3cyclictest3463065-21needreboot07:25:180
3456244991412,1cyclictest0-21swapper/010:10:030
345624899138,4cyclictest34606462chrt07:20:031
345624899137,5cyclictest926-21udisksd12:18:031
345624499130,1cyclictest0-21swapper/008:05:110
345624899129,2cyclictest3056-21PK-Backend12:10:211
3456248991210,1cyclictest3056-21PK-Backend07:29:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional