You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-08 - 20:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri May 08, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
366410621130,2sleep10-21swapper/109:10:011
368859821030,4sleep13056-21PK-Backend10:15:211
361862228553,22sleep30-21swapper/307:05:123
139099790,69rtkit-daemon0-21swapper/107:08:261
361948427647,20sleep00-21swapper/007:05:250
361898527344,21sleep20-21swapper/207:05:152
36278432560,5chrt3627820-21cron07:29:591
37152692330,4sleep13056-21PK-Backend11:35:011
362031299309,19cyclictest3625021-21which07:20:140
3620312992916,11cyclictest3638292-21cron08:00:010
362031299291,11cyclictest3682327-21apt10:00:110
362031299280,3cyclictest3622137-21ls07:14:590
36410712270,3sleep13056-21PK-Backend08:05:131
362031299278,17cyclictest3692027-21wc10:25:190
362031299278,17cyclictest3632527-21sed07:40:210
362031299278,16cyclictest3719789-21proc_pri11:45:210
362031299272,13cyclictest3720733-21apt11:50:120
362031299268,16cyclictest3644277-21apt-get08:14:590
362031299266,5cyclictest3630785-21timerandwakeup07:35:210
362031299262,22cyclictest3690882-21latency_hist10:25:010
139099260,4rtkit-daemon1389-21rtkit-daemon12:26:561
36446862250,3sleep30-21swapper/308:15:003
362031299259,14cyclictest3641475-21tr08:05:170
362031299252,4cyclictest0-21swapper/012:35:100
3620312992510,13cyclictest3735348-21sh12:30:170
139099250,3rtkit-daemon1389-21rtkit-daemon09:26:211
3620326992414,7cyclictest436-20systemd-journal08:00:002
362031299244,5cyclictest436-20systemd-journal09:05:000
362031299241,7cyclictest3628499-21hddtemp_smartct07:30:150
3620312992410,13cyclictest3670845-21munin-run09:30:010
3620312992410,12cyclictest3680962-21df_inode09:55:120
3620312992410,12cyclictest3676430-21timerwakeupswit09:40:200
139099240,4rtkit-daemon1389-21rtkit-daemon10:49:361
36232632230,3sleep10-21swapper/107:15:131
36232632230,3sleep10-21swapper/107:15:131
3620326992314,6cyclictest3732578-21cron12:25:002
362032699231,2cyclictest3721095-21df11:50:132
362031899230,21cyclictest3056-21PK-Backend11:00:401
362031899230,21cyclictest3056-21PK-Backend11:00:391
362031299238,13cyclictest3627014-21needreboot07:25:180
362031299237,14cyclictest0-21swapper/011:55:100
362031299236,5cyclictest436-20systemd-journal12:20:010
362031299236,15cyclictest0-21swapper/011:20:150
362031299235,4cyclictest3733169-21expr12:25:110
362031299235,4cyclictest3694840-21entropy10:35:130
362031299234,5cyclictest3696520-21expr10:40:120
362031299231,6cyclictest3623262-21/usr/sbin/munin07:15:130
362031299231,6cyclictest3623262-21/usr/sbin/munin07:15:120
362031299231,3cyclictest0-21swapper/010:55:120
3620312992310,12cyclictest3688557-21timerwakeupswit10:15:210
362031299230,8cyclictest0-21swapper/008:45:000
362031299230,6cyclictest3692881-21cpuspeed_turbos10:30:120
362031299230,21cyclictest1095-21runrttasks09:09:470
3620335992220,1cyclictest3056-21PK-Backend11:49:223
3620335992220,1cyclictest0-21swapper/311:42:253
3620335992220,1cyclictest0-21swapper/309:42:133
362033599220,21cyclictest3056-21PK-Backend09:14:143
362033599220,21cyclictest0-21swapper/311:07:443
362033599220,20cyclictest3056-21PK-Backend10:33:163
3620326992220,1cyclictest0-21swapper/212:36:022
3620326992220,1cyclictest0-21swapper/211:56:222
3620326992219,2cyclictest0-21swapper/207:33:342
3620326992213,7cyclictest3632703-21users07:40:232
3620318992220,1cyclictest3056-21PK-Backend10:03:131
3620318992220,1cyclictest3056-21PK-Backend09:08:111
3620318992220,1cyclictest0-21swapper/110:37:211
3620318992220,1cyclictest0-21swapper/110:21:111
3620318992219,2cyclictest3056-21PK-Backend12:01:231
3620318992219,1cyclictest3056-21PK-Backend07:45:591
362031299226,4cyclictest13-21ksoftirqd/008:18:220
362031299226,3cyclictest3633946-21missed_timers07:45:170
362031299224,4cyclictest3685846-21sort10:10:110
362031299222,6cyclictest3668479-21python309:20:210
362031299222,5cyclictest3697989-21cpuspeed10:45:120
362031299222,4cyclictest3718758-21package-data-do11:45:010
362031299221,6cyclictest1788610-21gdbus08:46:220
362031299221,5cyclictest3665878-21apt09:15:110
3620312992213,7cyclictest3709458-21python311:15:200
362031299220,8cyclictest3684436-21/usr/sbin/munin10:05:130
362031299220,8cyclictest3652229-21grep08:35:160
362031299220,7cyclictest3679055-21cat09:50:120
362031299220,11cyclictest436-20systemd-journal12:15:010
362031299220,11cyclictest3651476-21munin-run08:35:010
139099220,18rtkit-daemon3669108-21munin-run09:25:011
3620335992119,1cyclictest3056-21PK-Backend09:20:443
3620335992119,1cyclictest0-21swapper/312:20:383
3620335992119,1cyclictest0-21swapper/312:09:223
3620335992119,1cyclictest0-21swapper/311:56:173
3620335992119,1cyclictest0-21swapper/311:16:303
3620335992119,1cyclictest0-21swapper/311:00:013
3620335992119,1cyclictest0-21swapper/310:19:113
3620335992119,1cyclictest0-21swapper/310:01:463
3620335992119,1cyclictest0-21swapper/309:00:023
3620335992119,1cyclictest0-21swapper/308:31:033
3620335992119,1cyclictest0-21swapper/308:13:313
3620335992113,6cyclictest3689392-21expr10:20:123
362033599210,20cyclictest3056-21PK-Backend08:43:393
362033599210,20cyclictest0-21swapper/312:16:333
362033599210,20cyclictest0-21swapper/312:03:133
362033599210,20cyclictest0-21swapper/311:37:183
362033599210,20cyclictest0-21swapper/307:50:273
362033599210,1cyclictest0-21swapper/308:30:013
362033599210,19cyclictest3056-21PK-Backend10:35:053
362033599210,19cyclictest3056-21PK-Backend07:25:333
362033599210,19cyclictest3056-21PK-Backend07:23:163
3620326992119,1cyclictest0-21swapper/212:25:492
3620326992119,1cyclictest0-21swapper/211:22:012
3620326992119,1cyclictest0-21swapper/210:31:252
3620326992119,1cyclictest0-21swapper/210:27:472
3620326992119,1cyclictest0-21swapper/210:06:582
3620326992119,1cyclictest0-21swapper/208:53:162
3620326992119,1cyclictest0-21swapper/208:22:272
3620326992119,1cyclictest0-21swapper/208:15:462
3620326992119,1cyclictest0-21swapper/208:04:102
3620326992117,3cyclictest0-21swapper/208:40:132
3620326992114,5cyclictest3633743-21forks07:45:132
362032699210,20cyclictest0-21swapper/212:17:372
362032699210,20cyclictest0-21swapper/212:12:302
362032699210,20cyclictest0-21swapper/211:26:552
362032699210,20cyclictest0-21swapper/211:17:072
362032699210,20cyclictest0-21swapper/208:27:212
362032699210,20cyclictest0-21swapper/208:11:542
3620318992119,1cyclictest3056-21PK-Backend11:11:141
3620318992119,1cyclictest3056-21PK-Backend08:23:581
3620318992119,1cyclictest3056-21PK-Backend08:12:351
3620318992119,1cyclictest0-21swapper/111:46:281
3620318992119,1cyclictest0-21swapper/111:27:291
3620318992119,1cyclictest0-21swapper/109:32:171
3620318992119,1cyclictest0-21swapper/107:55:441
3620318992119,1cyclictest0-21swapper/107:31:381
3620318992118,2cyclictest3056-21PK-Backend11:20:461
362031899210,20cyclictest0-21swapper/112:10:221
362031899210,20cyclictest0-21swapper/110:25:371
362031899210,20cyclictest0-21swapper/109:52:181
362031899210,1cyclictest3056-21PK-Backend10:05:031
362031899210,19cyclictest3056-21PK-Backend12:08:191
362031899210,19cyclictest3056-21PK-Backend11:16:541
362031899210,19cyclictest3056-21PK-Backend09:47:531
362031299218,11cyclictest436-20systemd-journal07:55:010
362031299217,13cyclictest0-21swapper/011:00:180
362031299217,13cyclictest0-21swapper/011:00:180
362031299215,4cyclictest3724923-21awk12:00:180
362031299215,4cyclictest3700070-21idleruntime10:50:150
362031299215,3cyclictest3191-21update-notifier09:30:020
362031299213,4cyclictest0-21swapper/011:07:290
3620312992119,1cyclictest0-21swapper/009:47:540
362031299211,7cyclictest3646798-21ls08:20:120
362031299211,6cyclictest3664234-21sshd09:10:110
362031299211,5cyclictest3727405-21awk12:09:590
362031299210,7cyclictest3713547-21awk11:30:000
362031299210,20cyclictest3056-21PK-Backend08:00:190
3620335992018,1cyclictest3056-21PK-Backend10:52:173
3620335992018,1cyclictest3056-21PK-Backend10:43:103
3620335992018,1cyclictest3056-21PK-Backend10:25:263
3620335992018,1cyclictest3056-21PK-Backend09:59:583
3620335992018,1cyclictest3056-21PK-Backend07:19:573
3620335992018,1cyclictest3056-21PK-Backend07:19:573
3620335992018,1cyclictest0-21swapper/312:14:293
3620335992018,1cyclictest0-21swapper/311:12:373
3620335992018,1cyclictest0-21swapper/310:46:483
3620335992014,4cyclictest2156-21gnome-shell12:32:083
3620335992013,5cyclictest3652465-21ntp_kernel_pll_08:35:183
362033599200,19cyclictest3056-21PK-Backend11:50:233
362033599200,19cyclictest0-21swapper/311:29:063
362033599200,19cyclictest0-21swapper/309:51:063
362033599200,19cyclictest0-21swapper/309:46:393
362033599200,19cyclictest0-21swapper/307:39:243
362032699202,13cyclictest3707989-21sshd11:14:142
3620326992018,1cyclictest3056-21PK-Backend12:30:562
3620326992018,1cyclictest0-21swapper/211:35:052
3620326992018,1cyclictest0-21swapper/211:31:012
3620326992018,1cyclictest0-21swapper/210:58:362
3620326992018,1cyclictest0-21swapper/210:38:422
3620326992018,1cyclictest0-21swapper/210:21:312
3620326992018,1cyclictest0-21swapper/209:42:322
3620326992018,1cyclictest0-21swapper/209:29:052
3620326992018,1cyclictest0-21swapper/209:17:432
3620326992018,1cyclictest0-21swapper/208:35:192
3620326992011,7cyclictest436-20systemd-journal11:09:592
362032699200,8cyclictest3682046-21cron10:00:002
362032699200,1cyclictest0-21swapper/210:45:102
362032699200,19cyclictest0-21swapper/212:08:242
362032699200,19cyclictest0-21swapper/211:48:102
362032699200,19cyclictest0-21swapper/209:39:412
362032699200,19cyclictest0-21swapper/209:07:372
362032699200,19cyclictest0-21swapper/208:47:092
362032699200,19cyclictest0-21swapper/208:05:112
3620318992018,1cyclictest3056-21PK-Backend10:12:201
3620318992018,1cyclictest3056-21PK-Backend09:36:091
3620318992018,1cyclictest3056-21PK-Backend08:51:151
3620318992018,1cyclictest3056-21PK-Backend08:38:231
3620318992018,1cyclictest3056-21PK-Backend07:38:421
3620318992018,1cyclictest0-21swapper/111:55:271
3620318992018,1cyclictest0-21swapper/108:49:121
362031899200,19cyclictest3056-21PK-Backend12:24:131
362031899200,19cyclictest0-21swapper/112:35:161
362031899200,19cyclictest0-21swapper/110:50:041
362031899200,19cyclictest0-21swapper/109:55:561
362031899200,19cyclictest0-21swapper/109:17:181
362031899200,19cyclictest0-21swapper/107:12:521
362031299201,8cyclictest3707181-21memory11:10:170
362031299200,8cyclictest3715504-21/usr/sbin/munin11:35:220
362031299200,7cyclictest436-20systemd-journal11:35:000
362031299200,7cyclictest3660406-21latency_hist09:00:010
139099200,17rtkit-daemon1997-21gvfs-afc-volume08:28:540
36355362190,3chrt3635066-21/usr/sbin/munin07:50:132
3620335991917,1cyclictest3056-21PK-Backend08:45:423
362033599191,4cyclictest3686652-21python310:10:193
3620335991911,6cyclictest3663541-21taskset09:05:533
362033599190,1cyclictest3056-21PK-Backend12:38:353
362033599190,1cyclictest3056-21PK-Backend09:19:423
362033599190,1cyclictest0-21swapper/309:36:323
362033599190,1cyclictest0-21swapper/308:54:423
362033599190,1cyclictest0-21swapper/308:22:033
362033599190,1cyclictest0-21swapper/308:08:383
362033599190,18cyclictest3056-21PK-Backend12:29:363
362033599190,18cyclictest0-21swapper/311:34:133
362033599190,18cyclictest0-21swapper/307:40:393
3620326991917,1cyclictest0-21swapper/209:23:242
3620326991917,1cyclictest0-21swapper/209:10:272
3620326991917,1cyclictest0-21swapper/209:04:462
3620326991917,1cyclictest0-21swapper/207:21:572
3620326991913,4cyclictest3703511-21sshd11:00:142
3620326991913,4cyclictest3703511-21sshd11:00:142
3620326991911,6cyclictest3672801-21apt-get09:35:012
362032699190,1cyclictest0-21swapper/210:17:522
362032699190,1cyclictest0-21swapper/209:49:342
362032699190,18cyclictest0-21swapper/212:04:202
3620318991917,1cyclictest3056-21PK-Backend11:38:321
3620318991917,1cyclictest3056-21PK-Backend10:56:471
3620318991917,1cyclictest3056-21PK-Backend09:40:021
3620318991916,2cyclictest3056-21PK-Backend07:53:151
362031899190,1cyclictest3056-21PK-Backend12:17:171
362031899190,1cyclictest3056-21PK-Backend08:17:151
362031899190,1cyclictest0-21swapper/107:21:181
362031899190,18cyclictest3056-21PK-Backend07:43:291
362031899190,18cyclictest0-21swapper/110:40:591
362031299190,7cyclictest3657607-21fw_packets08:50:140
362031299190,4cyclictest3732604-21/usr/sbin/munin12:25:010
36205372180,4sleep33620083-21apt-get07:10:013
3620335991816,1cyclictest3056-21PK-Backend09:33:423
3620335991816,1cyclictest3056-21PK-Backend07:31:203
3620335991816,1cyclictest0-21swapper/310:06:263
3620335991811,5cyclictest3711044-21ntpq11:20:183
362033599181,15cyclictest3638830-21apt08:00:103
362033599180,16cyclictest3703237-21grep11:00:123
362033599180,16cyclictest3703237-21grep11:00:113
362033599180,16cyclictest3633941-21memory07:45:163
3620326991816,1cyclictest3056-21PK-Backend09:50:352
3620326991816,1cyclictest0-21swapper/210:41:332
3620326991816,1cyclictest0-21swapper/208:30:252
3620326991816,1cyclictest0-21swapper/207:10:162
3620326991812,4cyclictest3623452-21sed07:15:152
3620326991812,4cyclictest3623452-21sed07:15:142
3620326991812,4cyclictest2176-21JS10:27:482
3620326991810,6cyclictest3687285-21systemd-run10:15:002
3620318991816,1cyclictest3056-21PK-Backend11:40:341
3620318991816,1cyclictest0-21swapper/112:34:151
362031299180,8cyclictest3674810-21munin-run09:40:000
3620335991715,1cyclictest3056-21PK-Backend09:29:023
3620335991712,3cyclictest3636799-21acpi07:55:103
362033599170,15cyclictest3659426-21irqstats08:55:163
3620326991715,1cyclictest0-21swapper/207:36:042
3620326991715,1cyclictest0-21swapper/207:25:302
3620318991715,1cyclictest3056-21PK-Backend08:32:421
3620318991715,1cyclictest3056-21PK-Backend08:04:031
3620318991715,1cyclictest0-21swapper/111:53:241
3620318991715,1cyclictest0-21swapper/108:43:161
3620318991711,4cyclictest3692913-21expr10:30:121
3620326991614,1cyclictest0-21swapper/210:53:422
3620318991614,1cyclictest3056-21PK-Backend08:28:511
362032699151,8cyclictest3717265-21apt11:40:112
3620326991510,3cyclictest3659688-21sh08:55:172
3620318991513,1cyclictest3056-21PK-Backend11:05:331
3620318991513,1cyclictest3056-21PK-Backend08:55:201
3620318991511,3cyclictest3661637-21sort09:00:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional