You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 00:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Apr 01, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
365178321230,3sleep00-21swapper/007:55:200
365473721010,1sleep00-21swapper/008:05:140
139099890,76rtkit-daemon0-21swapper/207:06:172
363240427640,9sleep30-21swapper/307:05:183
363276727145,16sleep10-21swapper/107:05:231
363321426850,12sleep00-21swapper/007:06:440
37143472470,3sleep20-21swapper/210:55:142
36599532450,2sleep00-21swapper/008:20:140
363365299329,12cyclictest3717471-21apt11:05:130
3633652993210,9cyclictest3701671-21cat10:20:010
363365299318,9cyclictest3744484-21apt-get12:20:030
363365299314,16cyclictest0-21swapper/010:05:030
36434922280,12sleep23643493-21cron07:35:012
363365299286,20cyclictest1788620-21WebKitWebProces07:13:200
363365299284,15cyclictest0-21swapper/008:00:120
363365299284,14cyclictest0-21swapper/009:45:140
363365299284,14cyclictest0-21swapper/008:45:120
363365299277,8cyclictest3710276-21systemd-run10:45:000
363365299274,14cyclictest0-21swapper/011:35:180
363365299274,14cyclictest0-21swapper/007:50:030
363365299274,14cyclictest0-21swapper/007:50:030
3633652992712,13cyclictest3639799-21unin-run07:25:010
363365299264,14cyclictest0-21swapper/009:40:130
3633652992613,10cyclictest3664886-21apt08:35:120
363365299256,6cyclictest3715737-21apt11:00:110
363365299254,8cyclictest0-21swapper/007:45:000
3633652992518,5cyclictest3751468-21awk12:40:000
3633652992518,5cyclictest3751468-21awk12:40:000
363365299243,9cyclictest0-21swapper/011:20:020
363365299242,10cyclictest0-21swapper/012:33:590
3633652992414,7cyclictest3643770-21apt07:35:130
3633652992413,9cyclictest3720714-21apt-config11:15:010
3633652992410,2cyclictest3683373-21chrt09:26:060
139099240,17rtkit-daemon0-21swapper/210:19:532
3633664992319,2cyclictest3056-21PK-Backend08:10:083
363366499230,21cyclictest3056-21PK-Backend07:26:413
3633661992313,8cyclictest2330-21gdbus12:09:012
3633657992318,3cyclictest3742254-21apt12:15:011
363365299236,6cyclictest1095-21runrttasks11:33:040
363365299233,8cyclictest0-21swapper/011:25:190
363365299232,20cyclictest0-21swapper/009:54:150
363365299232,20cyclictest0-21swapper/009:54:150
3633652992319,3cyclictest0-21swapper/012:29:200
3633652992315,5cyclictest3722057-21python311:15:200
3633652992315,5cyclictest3707041-21apt10:35:130
3633652992314,6cyclictest3677766-21ntp_states09:10:180
3633652992310,11cyclictest3637776-21latency_hist07:20:000
3633664992220,1cyclictest3056-21PK-Backend11:30:493
3633664992219,2cyclictest3056-21PK-Backend12:08:293
3633664992219,2cyclictest3056-21PK-Backend09:42:063
3633664992219,2cyclictest3056-21PK-Backend09:39:033
363366499220,20cyclictest3056-21PK-Backend11:25:563
363366499220,20cyclictest3056-21PK-Backend10:31:343
363366499220,20cyclictest3056-21PK-Backend10:15:273
3633661992220,1cyclictest0-21swapper/212:38:342
3633661992220,1cyclictest0-21swapper/212:38:342
3633661992220,1cyclictest0-21swapper/211:40:272
3633661992220,1cyclictest0-21swapper/211:06:412
3633661992220,1cyclictest0-21swapper/208:31:352
3633661992214,6cyclictest3683091-21python309:25:202
363366199220,21cyclictest0-21swapper/207:42:212
3633657992220,1cyclictest0-21swapper/112:06:111
363365299225,4cyclictest0-21swapper/007:30:130
363365299224,5cyclictest0-21swapper/010:45:130
363365299222,19cyclictest0-21swapper/009:58:200
3633652992220,1cyclictest0-21swapper/011:47:030
3633652992220,1cyclictest0-21swapper/010:54:100
3633652992220,1cyclictest0-21swapper/010:28:480
3633652992220,1cyclictest0-21swapper/008:26:140
3633652992220,1cyclictest0-21swapper/008:14:030
3633652992218,3cyclictest0-21swapper/010:10:370
3633652992213,7cyclictest3668097-21awk08:44:590
363365299221,2cyclictest0-21swapper/010:55:110
3633652992212,8cyclictest3730993-21python311:40:210
3633652992212,8cyclictest3681959-21idleruntime-cro09:25:010
3633652992212,8cyclictest3678882-21grep09:15:120
3633652992212,8cyclictest3648815-21apt-config07:49:590
139099220,15rtkit-daemon3056-21PK-Backend10:59:553
139099220,14rtkit-daemon3056-21PK-Backend09:53:133
139099220,14rtkit-daemon3056-21PK-Backend09:53:133
3633664992119,1cyclictest3056-21PK-Backend12:04:363
3633664992119,1cyclictest3056-21PK-Backend11:52:133
3633664992119,1cyclictest3056-21PK-Backend11:39:353
3633664992119,1cyclictest3056-21PK-Backend11:13:543
3633664992119,1cyclictest3056-21PK-Backend09:03:343
3633664992118,2cyclictest3056-21PK-Backend11:48:213
3633664992113,6cyclictest3690886-21cron09:50:003
363366499210,20cyclictest3056-21PK-Backend08:59:293
363366499210,19cyclictest3056-21PK-Backend09:55:123
363366499210,19cyclictest3056-21PK-Backend09:31:063
3633661992119,1cyclictest3056-21PK-Backend09:05:492
3633661992119,1cyclictest0-21swapper/212:18:002
3633661992119,1cyclictest0-21swapper/211:37:232
3633661992119,1cyclictest0-21swapper/211:21:152
3633661992119,1cyclictest0-21swapper/210:47:002
3633661992119,1cyclictest0-21swapper/210:22:002
3633661992119,1cyclictest0-21swapper/209:46:302
3633661992119,1cyclictest0-21swapper/208:43:522
3633661992119,1cyclictest0-21swapper/208:36:422
3633661992119,1cyclictest0-21swapper/207:39:512
3633661992112,7cyclictest883-21dbus-daemon07:50:002
363366199210,20cyclictest0-21swapper/211:59:142
363366199210,20cyclictest0-21swapper/209:22:442
363366199210,20cyclictest0-21swapper/207:54:382
363366199210,20cyclictest0-21swapper/207:54:372
363366199210,20cyclictest0-21swapper/207:11:112
363366199210,1cyclictest0-21swapper/212:11:022
363365799219,10cyclictest3681968-21munin-run09:25:011
363365799217,12cyclictest3724685-21unin-run11:25:001
3633657992119,1cyclictest0-21swapper/111:51:431
3633657992119,1cyclictest0-21swapper/111:40:081
3633657992119,1cyclictest0-21swapper/111:27:171
3633657992119,1cyclictest0-21swapper/111:17:161
3633657992119,1cyclictest0-21swapper/111:05:141
3633657992119,1cyclictest0-21swapper/111:01:081
3633657992119,1cyclictest0-21swapper/110:47:011
3633657992119,1cyclictest0-21swapper/110:42:561
3633657992119,1cyclictest0-21swapper/110:15:321
3633657992119,1cyclictest0-21swapper/109:44:021
3633657992119,1cyclictest0-21swapper/109:30:561
3633657992119,1cyclictest0-21swapper/109:29:551
3633657992119,1cyclictest0-21swapper/108:45:011
3633657992119,1cyclictest0-21swapper/108:11:351
3633657992119,1cyclictest0-21swapper/107:44:311
3633657992119,1cyclictest0-21swapper/107:36:121
3633657992119,1cyclictest0-21swapper/107:23:081
3633657992119,1cyclictest0-21swapper/107:12:201
3633657992113,6cyclictest3729886-21systemd-run11:40:001
3633657992113,6cyclictest3720662-21apt-get11:15:011
3633657992112,7cyclictest3636887-21memory07:15:171
363365799210,20cyclictest0-21swapper/112:20:361
363365299214,5cyclictest0-21swapper/007:25:120
3633652992119,1cyclictest0-21swapper/011:58:380
3633652992113,6cyclictest3743260-21ntp_states12:15:190
3633652992113,6cyclictest13-21ksoftirqd/011:50:130
3633652992112,7cyclictest3737360-21ls12:00:110
3633652992112,7cyclictest3705607-21ls10:30:110
3633652992112,7cyclictest3686010-21expr09:35:120
3633652992112,7cyclictest3673330-21wc09:00:010
3633652992111,8cyclictest3740677-21latency_hist12:10:010
3633652992111,8cyclictest3673682-21cpuspeed_turbos09:00:130
3633652992111,8cyclictest3657953-21runrttasks08:15:120
3633652992110,9cyclictest3670184-21grep08:50:120
3633652992110,9cyclictest3670184-21grep08:50:120
139099210,14rtkit-daemon3056-21PK-Backend07:38:223
139099210,14rtkit-daemon0-21swapper/212:02:012
3633664992018,1cyclictest3056-21PK-Backend12:18:023
3633664992018,1cyclictest3056-21PK-Backend11:16:583
3633664992018,1cyclictest3056-21PK-Backend11:01:493
3633664992018,1cyclictest3056-21PK-Backend09:08:283
3633664992018,1cyclictest3056-21PK-Backend08:47:253
3633664992018,1cyclictest3056-21PK-Backend08:33:333
3633664992018,1cyclictest3056-21PK-Backend07:56:083
3633664992018,1cyclictest3056-21PK-Backend07:33:443
3633664992018,1cyclictest3056-21PK-Backend07:23:103
3633664992018,1cyclictest0-21swapper/308:00:283
3633664992017,2cyclictest3056-21PK-Backend11:05:553
3633664992017,1cyclictest3056-21PK-Backend10:35:403
3633664992016,3cyclictest42-21ksoftirqd/308:40:293
363366499200,1cyclictest3056-21PK-Backend12:20:523
363366499200,1cyclictest3056-21PK-Backend10:43:393
363366499200,19cyclictest3056-21PK-Backend12:29:233
363366499200,19cyclictest3056-21PK-Backend12:14:103
363366499200,19cyclictest3056-21PK-Backend10:12:223
363366499200,19cyclictest3056-21PK-Backend09:12:193
363366499200,19cyclictest3056-21PK-Backend08:28:263
363366499200,19cyclictest3056-21PK-Backend07:45:333
363366499200,19cyclictest3056-21PK-Backend07:12:223
363366499200,18cyclictest3056-21PK-Backend08:15:273
3633661992018,1cyclictest0-21swapper/212:22:532
3633661992018,1cyclictest0-21swapper/210:30:252
3633661992018,1cyclictest0-21swapper/209:12:452
3633661992018,1cyclictest0-21swapper/208:12:362
3633661992018,1cyclictest0-21swapper/207:21:322
363366199200,1cyclictest978-21snmpd08:08:032
363366199200,19cyclictest0-21swapper/210:53:102
363366199200,19cyclictest0-21swapper/210:11:312
363366199200,19cyclictest0-21swapper/209:52:402
363366199200,19cyclictest0-21swapper/209:52:392
363366199200,19cyclictest0-21swapper/209:19:412
363366199200,19cyclictest0-21swapper/208:28:162
363366199200,19cyclictest0-21swapper/207:27:332
363366199200,18cyclictest3056-21PK-Backend08:55:212
363365799209,9cyclictest3706794-21unin-run10:35:011
3633657992018,1cyclictest0-21swapper/111:56:381
3633657992018,1cyclictest0-21swapper/110:07:211
3633657992018,1cyclictest0-21swapper/109:19:561
3633657992018,1cyclictest0-21swapper/108:35:001
3633657992018,1cyclictest0-21swapper/107:54:041
3633657992018,1cyclictest0-21swapper/107:54:041
3633657992018,1cyclictest0-21swapper/107:34:581
3633657992012,6cyclictest3655951-21/usr/sbin/munin08:09:591
3633657992012,6cyclictest3647718-21cat07:45:151
363365799201,17cyclictest3651122-21sort07:55:131
363365799200,19cyclictest0-21swapper/112:31:121
363365799200,19cyclictest0-21swapper/112:19:351
363365799200,19cyclictest0-21swapper/110:37:491
363365799200,19cyclictest0-21swapper/109:06:171
363365799200,19cyclictest0-21swapper/108:36:011
363365799200,19cyclictest0-21swapper/108:27:511
363365799200,19cyclictest0-21swapper/108:20:141
363365799200,19cyclictest0-21swapper/108:15:541
3633652992011,7cyclictest3694818-21/usr/sbin/munin10:00:120
3633652992011,7cyclictest3675292-21/usr/sbin/munin09:05:100
3633652992011,7cyclictest3663319-21tr08:30:140
3633652992010,8cyclictest3740882-21cpuspeed_turbos12:10:130
3633652992010,8cyclictest3684016-21grep09:30:120
36709472190,3chrt3056-21PK-Backend08:50:203
36709472190,3chrt3056-21PK-Backend08:50:203
3633664991917,1cyclictest3056-21PK-Backend11:56:043
3633664991917,1cyclictest3056-21PK-Backend11:20:023
3633664991917,1cyclictest3056-21PK-Backend07:17:293
3633664991916,2cyclictest3056-21PK-Backend10:51:503
3633664991916,2cyclictest3056-21PK-Backend09:26:123
363366499190,1cyclictest3056-21PK-Backend12:37:083
363366499190,1cyclictest3056-21PK-Backend12:37:083
363366499190,1cyclictest3056-21PK-Backend12:33:153
363366499190,1cyclictest3056-21PK-Backend10:08:163
363366499190,1cyclictest3056-21PK-Backend07:44:193
3633661991917,1cyclictest0-21swapper/212:27:462
3633661991917,1cyclictest0-21swapper/211:53:192
3633661991917,1cyclictest0-21swapper/211:26:222
3633661991917,1cyclictest0-21swapper/208:04:452
3633661991912,6cyclictest3660395-21awk08:20:192
363366199190,1cyclictest0-21swapper/211:13:502
363366199190,1cyclictest0-21swapper/209:34:012
363366199190,1cyclictest0-21swapper/207:16:182
3633657991917,1cyclictest0-21swapper/110:59:051
3633657991917,1cyclictest0-21swapper/110:20:381
3633657991917,1cyclictest0-21swapper/110:11:261
3633657991917,1cyclictest0-21swapper/110:03:151
3633657991917,1cyclictest0-21swapper/109:50:111
3633657991917,1cyclictest0-21swapper/109:50:101
3633657991917,1cyclictest0-21swapper/109:48:081
3633657991912,6cyclictest3674498-21sed09:00:221
3633657991912,5cyclictest3746245-21cat12:25:111
3633657991912,5cyclictest3732880-21cat11:45:231
363365799190,1cyclictest0-21swapper/110:53:571
363365799190,1cyclictest0-21swapper/109:38:551
363365799190,1cyclictest0-21swapper/109:14:011
363365799190,1cyclictest0-21swapper/108:59:061
363365799190,1cyclictest0-21swapper/107:26:401
363365799190,18cyclictest0-21swapper/111:32:111
3633652991910,8cyclictest3701912-21/usr/sbin/munin10:20:160
3633664991816,1cyclictest3056-21PK-Backend10:20:343
3633664991816,1cyclictest3056-21PK-Backend09:24:093
3633664991816,1cyclictest3056-21PK-Backend08:39:283
3633664991816,1cyclictest3056-21PK-Backend08:23:053
3633664991816,1cyclictest3056-21PK-Backend07:50:203
3633664991816,1cyclictest3056-21PK-Backend07:50:193
3633664991815,2cyclictest3056-21PK-Backend10:00:193
3633661991816,1cyclictest0-21swapper/211:47:232
3633661991812,5cyclictest3710010-21sshd10:42:062
3633661991811,5cyclictest3708698-21grep10:40:002
3633657991816,1cyclictest0-21swapper/108:42:581
3633657991811,5cyclictest3737776-21ntpq12:00:171
3633657991811,5cyclictest3670792-21ntpq08:50:171
3633657991811,5cyclictest3670792-21ntpq08:50:171
3633657991810,6cyclictest883-21dbus-daemon08:04:591
363365799180,1cyclictest0-21swapper/109:58:071
3633664991715,1cyclictest3056-21PK-Backend10:49:473
3633664991715,1cyclictest3056-21PK-Backend09:18:143
3633661991715,1cyclictest0-21swapper/212:33:412
3633661991715,1cyclictest0-21swapper/208:17:092
3633661991715,1cyclictest0-21swapper/207:58:082
3633661991713,3cyclictest3696763-21tr10:05:132
3633661991713,3cyclictest34-21ksoftirqd/210:30:012
3633661991712,4cyclictest978-21snmpd09:55:562
363366199164,11cyclictest3687197-21systemd-run09:40:012
3633661991614,1cyclictest0-21swapper/209:41:242
3633661991611,4cyclictest3716437-21awk11:00:182
3633661991610,4cyclictest3670756-21seq08:50:172
3633661991610,4cyclictest3670756-21seq08:50:162
3633657991614,1cyclictest0-21swapper/110:25:461
3633657991611,4cyclictest26-21ksoftirqd/112:39:281
3633657991611,4cyclictest26-21ksoftirqd/112:39:271
363366499159,4cyclictest42-21ksoftirqd/311:40:133
3633661991513,1cyclictest0-21swapper/211:15:072
3633661991511,3cyclictest978-21snmpd09:02:402
3633661991510,3cyclictest3696084-21latency_hist10:05:012
363366499148,5cyclictest3655464-21timerandwakeup08:05:233
3633664991412,1cyclictest3056-21PK-Backend10:25:413
363366199139,3cyclictest3726833-21tr11:30:142
139099130,3rtkit-daemon0-21swapper/208:49:562
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional