You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 00:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sat May 23, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
374089222640,5sleep00-21swapper/007:05:220
384861721410,3sleep20-21swapper/212:10:162
378299321300,4sleep00-21swapper/009:05:000
381462021130,3sleep00-21swapper/010:35:100
374146629980,13sleep10-21swapper/107:08:211
139099960,77rtkit-daemon0-21swapper/307:06:493
374124927959,14sleep20-21swapper/207:05:332
37598482570,4sleep00-21swapper/008:00:010
38114342490,2sleep10-21swapper/110:25:131
38357892420,4sleep33056-21PK-Backend11:35:133
38357892420,4sleep33056-21PK-Backend11:35:123
38460432390,2sleep00-21swapper/012:05:000
37804052370,1sleep13780407-21awk08:55:191
139099370,3rtkit-daemon0-21swapper/307:11:273
38396582330,3sleep00-21swapper/011:45:150
3741744992714,9cyclictest436-20systemd-journal09:27:002
139099270,5rtkit-daemon3848254-21df12:10:101
38296362260,3sleep00-21swapper/011:15:210
38007222240,2sleep13056-21PK-Backend09:55:131
37770062240,10sleep013-21ksoftirqd/008:45:200
3741744992413,8cyclictest883-21dbus-daemon09:06:592
37871132230,4sleep13056-21PK-Backend09:15:161
3741748992320,2cyclictest0-21swapper/309:53:213
3741748992319,2cyclictest3056-21PK-Backend11:57:033
139099230,4rtkit-daemon1389-21rtkit-daemon09:18:490
38469572220,4sleep33056-21PK-Backend12:05:203
37887232220,3sleep13056-21PK-Backend09:20:131
3741748992220,1cyclictest0-21swapper/310:58:443
3741748992220,1cyclictest0-21swapper/310:53:103
3741748992220,1cyclictest0-21swapper/309:02:043
3741748992220,1cyclictest0-21swapper/307:18:133
3741748992218,3cyclictest3778340-21diskstats08:50:143
374174899220,20cyclictest3056-21PK-Backend10:31:053
3741744992220,1cyclictest0-21swapper/212:36:412
3741744992220,1cyclictest0-21swapper/211:24:562
3741744992220,1cyclictest0-21swapper/208:36:192
374174499220,21cyclictest0-21swapper/209:58:322
3741739992220,1cyclictest3056-21PK-Backend09:30:151
3741738992220,1cyclictest0-21swapper/011:50:330
3741738992214,6cyclictest3822438-21python310:55:210
139099220,19rtkit-daemon3194-21gmain09:21:580
374174899212,16cyclictest3772710-21cnf-update-db08:35:013
3741748992119,1cyclictest3056-21PK-Backend08:21:483
3741748992119,1cyclictest3056-21PK-Backend08:04:243
3741748992119,1cyclictest3056-21PK-Backend07:50:393
3741748992119,1cyclictest0-21swapper/310:47:353
3741748992119,1cyclictest0-21swapper/310:44:233
3741748992119,1cyclictest0-21swapper/309:35:423
3741748992119,1cyclictest0-21swapper/309:31:093
3741748992119,1cyclictest0-21swapper/308:31:143
3741748992119,1cyclictest0-21swapper/307:39:093
3741748992115,4cyclictest2156-21gnome-shell09:58:093
374174899210,20cyclictest3056-21PK-Backend11:51:153
374174899210,20cyclictest3056-21PK-Backend11:01:543
374174899210,20cyclictest0-21swapper/309:22:183
374174899210,20cyclictest0-21swapper/308:40:203
374174899210,19cyclictest3056-21PK-Backend11:45:283
374174899210,19cyclictest3056-21PK-Backend11:05:053
374174499213,13cyclictest436-20systemd-journal07:21:542
3741744992119,1cyclictest3056-21PK-Backend08:25:032
3741744992119,1cyclictest0-21swapper/211:53:432
3741744992119,1cyclictest0-21swapper/210:50:012
3741744992119,1cyclictest0-21swapper/210:41:162
3741744992119,1cyclictest0-21swapper/210:22:232
3741744992119,1cyclictest0-21swapper/209:14:162
3741744992119,1cyclictest0-21swapper/209:04:102
3741744992119,1cyclictest0-21swapper/208:56:132
3741744992119,1cyclictest0-21swapper/208:49:362
3741744992119,1cyclictest0-21swapper/208:08:202
3741744992119,1cyclictest0-21swapper/207:19:262
3741744992118,2cyclictest3754466-21latency_hist07:45:012
3741744992114,5cyclictest3771443-21strings08:30:182
3741744992112,7cyclictest1788610-21gdbus07:36:572
374174499210,20cyclictest0-21swapper/212:32:152
374174499210,20cyclictest0-21swapper/210:37:182
374174499210,20cyclictest0-21swapper/207:54:542
3741739992119,1cyclictest3056-21PK-Backend08:50:191
3741739992119,1cyclictest0-21swapper/112:02:511
3741739992119,1cyclictest0-21swapper/111:44:321
3741739992119,1cyclictest0-21swapper/111:38:241
3741739992119,1cyclictest0-21swapper/111:38:241
3741739992119,1cyclictest0-21swapper/111:27:421
3741739992119,1cyclictest0-21swapper/107:59:221
3741739992118,2cyclictest3056-21PK-Backend10:48:151
3741739992113,6cyclictest3831251-21ntp_states11:20:181
3741739992110,8cyclictest3830234-21cron11:20:001
374173999210,20cyclictest0-21swapper/111:07:561
374173999210,20cyclictest0-21swapper/108:11:331
374173999210,19cyclictest3056-21PK-Backend07:39:361
3741738992119,1cyclictest0-21swapper/012:35:290
3741738992119,1cyclictest0-21swapper/012:28:250
3741738992119,1cyclictest0-21swapper/012:18:450
3741738992119,1cyclictest0-21swapper/012:12:100
3741738992119,1cyclictest0-21swapper/012:06:220
3741738992119,1cyclictest0-21swapper/010:10:430
3741738992119,1cyclictest0-21swapper/010:07:310
3741738992119,1cyclictest0-21swapper/009:54:140
3741738992119,1cyclictest0-21swapper/009:30:400
3741738992119,1cyclictest0-21swapper/008:34:090
3741738992119,1cyclictest0-21swapper/008:03:540
3741738992119,1cyclictest0-21swapper/007:53:340
3741738992119,1cyclictest0-21swapper/007:42:530
3741738992119,1cyclictest0-21swapper/007:28:330
3741738992115,4cyclictest1095-21runrttasks12:20:200
374173899210,20cyclictest0-21swapper/007:23:200
374174899203,5cyclictest3752932-21cpu07:40:123
3741748992018,1cyclictest3056-21PK-Backend12:23:263
3741748992018,1cyclictest3056-21PK-Backend11:12:153
3741748992018,1cyclictest0-21swapper/308:57:243
3741748992018,1cyclictest0-21swapper/308:26:083
3741748992018,1cyclictest0-21swapper/307:48:503
3741748992017,2cyclictest3056-21PK-Backend08:18:173
374174899200,1cyclictest3056-21PK-Backend07:56:543
374174899200,19cyclictest3056-21PK-Backend12:15:013
374174899200,19cyclictest0-21swapper/310:21:473
374174899200,19cyclictest0-21swapper/309:46:253
374174899200,19cyclictest0-21swapper/309:44:003
374174899200,19cyclictest0-21swapper/307:24:023
3741744992018,1cyclictest0-21swapper/211:57:082
3741744992018,1cyclictest0-21swapper/210:34:552
3741744992018,1cyclictest0-21swapper/209:37:292
3741744992018,1cyclictest0-21swapper/209:19:442
3741744992018,1cyclictest0-21swapper/208:24:152
3741744992018,1cyclictest0-21swapper/208:17:452
3741744992018,1cyclictest0-21swapper/207:55:562
3741744992018,1cyclictest0-21swapper/207:25:272
3741744992017,2cyclictest0-21swapper/210:45:152
3741744992015,3cyclictest3743390-21grep07:15:002
374174499201,1cyclictest0-21swapper/207:33:192
3741744992011,7cyclictest3845450-21users12:00:222
374174499200,1cyclictest3056-21PK-Backend08:13:122
374174499200,19cyclictest0-21swapper/211:39:292
374174499200,19cyclictest0-21swapper/211:39:292
374174499200,19cyclictest0-21swapper/211:03:422
374174499200,19cyclictest0-21swapper/210:09:402
374174499200,19cyclictest0-21swapper/210:04:062
374174499200,19cyclictest0-21swapper/209:42:492
374174499200,19cyclictest0-21swapper/208:44:492
3741739992018,1cyclictest0-21swapper/111:04:091
3741739992018,1cyclictest0-21swapper/108:43:511
374173999200,1cyclictest0-21swapper/111:53:171
374173999200,19cyclictest0-21swapper/112:05:141
374173999200,19cyclictest0-21swapper/108:17:421
374173899204,14cyclictest3837178-21turbostat11:40:000
374173899204,14cyclictest3837178-21turbostat11:39:590
3741738992018,1cyclictest0-21swapper/012:30:150
3741738992018,1cyclictest0-21swapper/011:25:250
3741738992018,1cyclictest0-21swapper/009:37:230
374173899201,18cyclictest0-21swapper/007:33:190
374173899200,1cyclictest0-21swapper/009:56:070
374173899200,1cyclictest0-21swapper/008:29:370
374173899200,19cyclictest0-21swapper/011:55:330
374173899200,19cyclictest0-21swapper/011:13:160
374173899200,19cyclictest0-21swapper/011:02:540
374173899200,19cyclictest0-21swapper/009:41:090
374173899200,19cyclictest0-21swapper/009:26:560
374173899200,19cyclictest0-21swapper/009:07:150
374173899200,19cyclictest0-21swapper/008:52:080
374173899200,19cyclictest0-21swapper/008:22:210
374173899200,19cyclictest0-21swapper/008:16:400
374173899200,19cyclictest0-21swapper/007:36:370
374173899200,19cyclictest0-21swapper/007:11:560
37816572190,2sleep13056-21PK-Backend09:00:131
3741748991917,1cyclictest3056-21PK-Backend08:08:153
3741748991912,5cyclictest3811729-21sed10:25:183
3741748991911,6cyclictest3814806-21expr10:35:123
3741748991911,6cyclictest3748011-21df07:25:143
374174899190,1cyclictest3056-21PK-Backend11:34:193
374174899190,1cyclictest3056-21PK-Backend08:45:413
374174899190,1cyclictest0-21swapper/309:29:353
374174499192,13cyclictest3797054-21apt09:45:122
3741744991917,1cyclictest0-21swapper/212:20:472
3741744991917,1cyclictest0-21swapper/208:54:502
3741744991917,1cyclictest0-21swapper/208:01:382
3741744991917,1cyclictest0-21swapper/207:49:262
3741744991916,2cyclictest1095-21runrttasks09:31:272
3741744991912,5cyclictest3806169-21cut10:10:122
374174499190,18cyclictest0-21swapper/210:19:122
374173999192,13cyclictest3851339-21/usr/sbin/munin12:20:241
3741739991917,1cyclictest3056-21PK-Backend10:58:151
3741739991917,1cyclictest3056-21PK-Backend10:33:341
3741739991917,1cyclictest3056-21PK-Backend10:07:331
3741739991917,1cyclictest3056-21PK-Backend07:31:371
3741739991916,2cyclictest3056-21PK-Backend09:46:371
3741739991912,5cyclictest436-20systemd-journal12:39:591
3741739991912,5cyclictest3768052-21uname08:20:171
374173999190,1cyclictest0-21swapper/112:31:431
374173999190,1cyclictest0-21swapper/111:47:431
374173999190,18cyclictest3056-21PK-Backend10:24:551
374173999190,18cyclictest3056-21PK-Backend07:24:281
374173999190,18cyclictest0-21swapper/108:03:011
3741738991917,1cyclictest0-21swapper/008:10:520
3741738991917,1cyclictest0-21swapper/008:08:480
3741738991914,3cyclictest3797230-21cpuspeed_turbos09:45:130
374173899190,1cyclictest0-21swapper/011:32:340
374173899190,1cyclictest0-21swapper/011:07:400
374173899190,1cyclictest0-21swapper/010:53:220
374173899190,1cyclictest0-21swapper/010:18:270
374173899190,1cyclictest0-21swapper/010:03:320
374173899190,1cyclictest0-21swapper/008:57:560
374173899190,1cyclictest0-21swapper/008:44:520
374173899190,1cyclictest0-21swapper/008:35:300
374173899190,1cyclictest0-21swapper/007:15:560
374173899190,18cyclictest3056-21PK-Backend10:32:180
374173899190,18cyclictest0-21swapper/010:24:350
38033182180,4chrt3056-21PK-Backend10:00:241
3741748991816,1cyclictest0-21swapper/310:00:303
3741748991816,1cyclictest0-21swapper/309:18:133
3741748991812,5cyclictest3808625-21ls10:15:233
3741748991811,5cyclictest3853648-21sed12:25:163
3741748991810,6cyclictest3750875-21pluginstate07:35:013
3741744991815,2cyclictest1095-21runrttasks09:50:392
3741744991813,3cyclictest3789840-21perf09:25:002
3741744991810,6cyclictest883-21dbus-daemon11:35:002
3741744991810,6cyclictest883-21dbus-daemon11:30:002
374173999182,12cyclictest3853096-21apt12:25:111
3741739991815,2cyclictest3056-21PK-Backend10:35:451
3741739991815,2cyclictest3056-21PK-Backend09:54:131
3741739991813,3cyclictest3850473-21awk12:15:191
374173999181,13cyclictest3769042-21apt08:25:111
3741738991816,1cyclictest0-21swapper/010:45:380
3741738991816,1cyclictest0-21swapper/010:43:140
3741738991816,1cyclictest0-21swapper/009:13:510
3741738991816,1cyclictest0-21swapper/007:47:190
374173899180,1cyclictest0-21swapper/010:26:110
3741748991715,1cyclictest3056-21PK-Backend11:41:303
3741748991715,1cyclictest0-21swapper/312:37:203
3741748991715,1cyclictest0-21swapper/309:06:083
3741744991712,4cyclictest3849795-21cpuspeed_turbos12:15:132
3741744991712,3cyclictest978-21snmpd12:05:052
3741744991712,3cyclictest38539552sleep212:25:202
3741739991715,1cyclictest3056-21PK-Backend08:38:101
3741739991715,1cyclictest0-21swapper/108:47:361
3741739991712,3cyclictest3827085-21cpu11:10:131
3741739991711,4cyclictest3753305-21forks07:40:161
374173999171,11cyclictest3833958-21/usr/sbin/munin11:30:241
3741739991710,5cyclictest3844313-21sed12:00:011
374173999170,16cyclictest3056-21PK-Backend09:15:001
139099170,3rtkit-daemon0-21swapper/011:42:050
374174899169,5cyclictest3807502-21date10:15:003
3741748991614,1cyclictest3056-21PK-Backend11:15:273
3741748991613,2cyclictest3056-21PK-Backend12:14:153
374174899160,4cyclictest3056-21PK-Backend12:34:083
374174899160,3cyclictest3785281-21diskstats09:10:143
374174899160,14cyclictest3804357-21diskstats10:05:133
374174499168,6cyclictest3812036-21python310:25:222
3741744991614,1cyclictest0-21swapper/211:49:312
3741744991614,1cyclictest0-21swapper/211:14:212
3741744991611,3cyclictest3822087-21cut10:55:172
3741739991614,1cyclictest3056-21PK-Backend10:43:221
3741739991614,1cyclictest3056-21PK-Backend10:18:071
3741739991614,1cyclictest3056-21PK-Backend08:32:421
374173999160,14cyclictest3056-21PK-Backend10:50:241
374173899168,6cyclictest3830713-21expr11:20:130
374174899159,5cyclictest3833723-21basename11:30:003
374174899150,13cyclictest37648742sleep308:13:023
374174499158,5cyclictest3829818-21grep11:15:222
374173999159,5cyclictest3757233-21gzip07:50:211
3741739991513,1cyclictest3056-21PK-Backend09:35:421
3741739991510,3cyclictest3762490-21needreboot08:05:191
374174499149,4cyclictest3838219-21awk11:40:192
374174499148,5cyclictest3825506-21cat11:05:132
374173999140,1cyclictest3056-21PK-Backend09:44:271
374173999140,1cyclictest3056-21PK-Backend07:26:521
3741748991311,1cyclictest3056-21PK-Backend12:04:273
374173999139,3cyclictest3056-21PK-Backend07:14:151
3741739991311,1cyclictest3056-21PK-Backend09:28:521
374173999130,1cyclictest3056-21PK-Backend10:11:041
139099130,10rtkit-daemon3056-21PK-Backend09:06:001
374174899127,4cyclictest3056-21PK-Backend11:21:583
374173999128,3cyclictest3056-21PK-Backend07:47:351
3741739991210,1cyclictest3056-21PK-Backend07:17:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional