You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-27 - 01:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Wed Aug 26, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29987202970,2sleep00-21swapper/012:09:090
139099880,53rtkit-daemon0-21swapper/007:08:450
139099880,53rtkit-daemon0-21swapper/007:08:450
29505742860,2sleep20-21swapper/209:50:172
289279026747,14sleep20-21swapper/207:08:572
289279026747,14sleep20-21swapper/207:08:572
289268526544,14sleep30-21swapper/307:07:383
289268526544,14sleep30-21swapper/307:07:373
289265326143,13sleep10-21swapper/107:07:111
289265326143,13sleep10-21swapper/107:07:111
29380182530,4sleep13056-21PK-Backend09:15:141
29714932510,4sleep10-21swapper/110:50:181
28976202400,2sleep20-21swapper/207:20:152
139099390,3rtkit-daemon3056-21PK-Backend11:52:491
139099380,3rtkit-daemon3056-21PK-Backend08:03:341
289303699287,10cyclictest2951167-21taskset09:51:083
289303699270,13cyclictest0-21swapper/312:00:093
289303699270,12cyclictest2928484-21cut08:50:013
289303699266,8cyclictest2954664-21taskset10:01:353
289303699262,14cyclictest2926814-21date08:45:003
289303699260,15cyclictest2977750-21apt11:10:113
2893036992516,7cyclictest2949755-21/usr/sbin/munin09:50:003
2893036992510,13cyclictest2940112-21ntpq09:20:193
2893036992510,12cyclictest883-21dbus-daemon08:05:013
289303699250,14cyclictest0-21swapper/312:30:333
2893025992519,4cyclictest2902091-21cron07:35:000
139099250,17rtkit-daemon3056-21PK-Backend07:40:001
139099250,16rtkit-daemon3056-21PK-Backend08:25:221
289303699248,8cyclictest0-21swapper/307:57:253
2893036992414,7cyclictest436-20systemd-journal07:55:003
289303699241,14cyclictest2941956-21python309:25:213
289303699240,8cyclictest0-21swapper/311:50:183
289303699240,15cyclictest2974286-21apt11:00:143
289303699240,15cyclictest2893747-21latency07:10:163
289303699240,14cyclictest0-21swapper/310:24:533
2893033992413,3cyclictest34-21ksoftirqd/210:08:522
289303699232,16cyclictest2964745-21python310:30:223
2893036992319,2cyclictest2955218-21apt10:05:123
289303699230,9cyclictest2995176-21taskset11:58:123
289303699230,9cyclictest0-21swapper/307:25:193
289303699230,14cyclictest0-21swapper/310:13:073
289303699230,13cyclictest0-21swapper/312:07:543
289303699230,13cyclictest0-21swapper/311:23:393
289303699230,13cyclictest0-21swapper/310:40:533
289303699230,13cyclictest0-21swapper/307:17:543
289303699230,12cyclictest2919725-21taskset08:24:403
289303699230,10cyclictest2904059-21cat07:40:013
2893033992319,3cyclictest331rcuc/207:52:182
2893033992319,3cyclictest331rcuc/207:35:542
2893033992315,6cyclictest2975148-21python311:00:222
289303399230,21cyclictest2985057-21diskstats11:30:122
289302799230,21cyclictest3056-21PK-Backend12:00:561
2893025992321,1cyclictest0-21swapper/009:20:180
139099230,16rtkit-daemon3056-21PK-Backend08:53:171
139099230,15rtkit-daemon3056-21PK-Backend11:29:001
139099230,15rtkit-daemon3056-21PK-Backend09:11:261
139099230,14rtkit-daemon0-21swapper/008:45:130
28947912220,3sleep13056-21PK-Backend07:15:011
289303699222,10cyclictest0-21swapper/311:42:323
2893036992220,1cyclictest0-21swapper/309:02:253
2893036992220,1cyclictest0-21swapper/308:35:013
2893036992220,1cyclictest0-21swapper/308:18:253
2893036992219,2cyclictest0-21swapper/308:10:483
289303699221,6cyclictest2923244-21perf08:34:593
289303699221,6cyclictest2923244-21perf08:34:593
289303699220,9cyclictest3004119-21taskset12:23:223
289303699220,8cyclictest2973854-21runrttasks10:58:113
289303699220,8cyclictest2962730-21memory10:25:183
289303699220,8cyclictest2962730-21memory10:25:183
289303699220,14cyclictest0-21swapper/312:37:063
289303699220,14cyclictest0-21swapper/311:06:273
289303699220,13cyclictest0-21swapper/311:34:063
289303699220,13cyclictest0-21swapper/310:52:083
289303699220,13cyclictest0-21swapper/310:46:463
289303699220,13cyclictest0-21swapper/310:18:073
289303699220,13cyclictest0-21swapper/309:18:493
289303699220,13cyclictest0-21swapper/309:10:073
289303699220,13cyclictest0-21swapper/308:06:583
289303699220,13cyclictest0-21swapper/307:40:153
289303699220,12cyclictest2945835-21taskset09:37:493
289303699220,12cyclictest0-21swapper/312:27:313
289303699220,12cyclictest0-21swapper/312:14:123
289303699220,12cyclictest0-21swapper/309:57:343
289303699220,11cyclictest2921454-21taskset08:29:373
289303699220,10cyclictest0-21swapper/309:40:513
2893033992218,3cyclictest331rcuc/211:55:262
289303399220,21cyclictest0-21swapper/211:25:122
289303399220,20cyclictest2966083-21cat10:35:182
2893027992220,1cyclictest3056-21PK-Backend12:25:301
2893027992220,1cyclictest3056-21PK-Backend10:25:121
2893027992220,1cyclictest3056-21PK-Backend10:25:111
2893027992220,1cyclictest3056-21PK-Backend08:37:101
2893027992219,2cyclictest3056-21PK-Backend11:41:221
289302799220,20cyclictest3056-21PK-Backend11:14:591
2893025992220,1cyclictest0-21swapper/008:57:200
2893025992214,6cyclictest2941674-21memory09:25:170
2893025992213,7cyclictest2921109-21users08:25:210
139099220,3rtkit-daemon0-21swapper/012:22:330
139099220,15rtkit-daemon3056-21PK-Backend09:22:161
139099220,14rtkit-daemon0-21swapper/009:50:140
289303699212,13cyclictest2990148-21apt11:45:123
2893036992119,1cyclictest0-21swapper/307:32:223
2893036992114,5cyclictest2930160-21apt-get08:55:013
2893036992113,5cyclictest2175-21JS10:27:483
289303699210,9cyclictest2966844-21taskset10:37:083
289303699210,8cyclictest2980893-21runrttasks11:19:303
289303699210,13cyclictest0-21swapper/307:45:123
289303699210,12cyclictest0-21swapper/312:19:173
289303699210,12cyclictest0-21swapper/309:31:073
289303699210,11cyclictest0-21swapper/311:36:393
2893033992119,1cyclictest0-21swapper/208:46:072
2893033992118,2cyclictest331rcuc/210:33:502
2893033992117,3cyclictest331rcuc/212:27:302
2893033992117,3cyclictest331rcuc/212:19:552
2893033992117,3cyclictest331rcuc/212:05:292
2893033992117,3cyclictest331rcuc/211:44:532
2893033992117,3cyclictest331rcuc/210:03:382
2893033992117,3cyclictest331rcuc/209:24:102
2893033992117,3cyclictest331rcuc/208:56:072
2893033992117,3cyclictest331rcuc/208:55:002
2893033992117,3cyclictest331rcuc/207:28:522
2893033992117,3cyclictest331rcuc/207:17:262
2893033992116,4cyclictest34-21ksoftirqd/212:10:132
2893033992115,5cyclictest34-21ksoftirqd/208:10:012
2893033992115,5cyclictest331rcuc/207:49:192
2893033992114,5cyclictest34-21ksoftirqd/210:15:012
2893027992119,1cyclictest3056-21PK-Backend11:03:231
2893027992119,1cyclictest3056-21PK-Backend10:35:591
2893027992119,1cyclictest3056-21PK-Backend07:21:331
2893027992119,1cyclictest0-21swapper/108:05:251
2893027992118,2cyclictest3056-21PK-Backend11:08:241
2893027992118,2cyclictest3056-21PK-Backend10:46:011
2893027992118,2cyclictest3056-21PK-Backend10:40:591
289302799210,20cyclictest3056-21PK-Backend10:31:471
289302799210,20cyclictest3056-21PK-Backend08:21:141
289302799210,20cyclictest3056-21PK-Backend07:46:071
289302799210,19cyclictest3056-21PK-Backend08:45:271
289302799210,19cyclictest3056-21PK-Backend07:40:111
2893025992119,1cyclictest0-21swapper/011:16:580
2893025992119,1cyclictest0-21swapper/010:59:220
2893025992119,1cyclictest0-21swapper/010:42:310
2893025992119,1cyclictest0-21swapper/010:30:420
2893025992119,1cyclictest0-21swapper/010:27:180
2893025992119,1cyclictest0-21swapper/010:27:170
2893025992119,1cyclictest0-21swapper/010:23:520
2893025992119,1cyclictest0-21swapper/010:00:060
2893025992119,1cyclictest0-21swapper/009:40:540
2893025992119,1cyclictest0-21swapper/008:37:050
2893025992119,1cyclictest0-21swapper/008:16:440
2893025992119,1cyclictest0-21swapper/008:06:440
2893025992119,1cyclictest0-21swapper/007:15:470
289302599210,20cyclictest0-21swapper/012:25:270
289302599210,20cyclictest0-21swapper/011:13:340
289302599210,20cyclictest0-21swapper/010:18:520
289302599210,19cyclictest3056-21PK-Backend10:10:410
139099210,15rtkit-daemon3056-21PK-Backend10:57:031
139099210,14rtkit-daemon3056-21PK-Backend09:08:160
2893036992016,3cyclictest0-21swapper/308:50:303
289303399209,2cyclictest141rcu_preempt09:30:222
289303399206,5cyclictest34-21ksoftirqd/208:45:002
289303399206,12cyclictest34-21ksoftirqd/209:19:092
2893033992018,1cyclictest0-21swapper/210:54:432
2893033992018,1cyclictest0-21swapper/209:41:022
2893033992016,3cyclictest331rcuc/212:32:032
2893033992016,3cyclictest331rcuc/212:04:412
2893033992016,3cyclictest331rcuc/211:54:462
2893033992016,3cyclictest331rcuc/209:56:372
2893033992016,3cyclictest331rcuc/209:46:272
2893033992016,3cyclictest331rcuc/209:28:502
2893033992016,3cyclictest331rcuc/209:11:462
2893033992016,3cyclictest331rcuc/209:03:362
2893033992016,3cyclictest331rcuc/208:34:252
2893033992016,3cyclictest331rcuc/208:34:252
2893033992016,3cyclictest331rcuc/208:26:142
2893033992016,3cyclictest331rcuc/208:23:542
2893033992016,3cyclictest331rcuc/208:19:172
2893033992016,3cyclictest331rcuc/208:12:272
2893033992014,4cyclictest2912861-21systemd-run08:05:012
2893033992013,6cyclictest2961550-21sshd10:20:582
289303399200,1cyclictest0-21swapper/207:44:092
289303399200,19cyclictest0-21swapper/211:21:472
289303399200,19cyclictest0-21swapper/210:46:182
289303399200,19cyclictest0-21swapper/210:15:032
289303399200,19cyclictest0-21swapper/209:36:022
289303399200,19cyclictest0-21swapper/209:06:142
289303399200,19cyclictest0-21swapper/207:56:112
289303399200,19cyclictest0-21swapper/207:32:182
289303399200,19cyclictest0-21swapper/207:10:092
2893027992018,1cyclictest3056-21PK-Backend12:19:421
2893027992018,1cyclictest3056-21PK-Backend09:53:141
2893027992018,1cyclictest3056-21PK-Backend08:44:391
2893027992018,1cyclictest3056-21PK-Backend08:33:241
2893027992018,1cyclictest3056-21PK-Backend08:33:241
2893027992018,1cyclictest3056-21PK-Backend07:18:071
289302799200,1cyclictest3056-21PK-Backend12:39:151
289302799200,1cyclictest0-21swapper/108:55:341
289302799200,19cyclictest3056-21PK-Backend09:45:021
2893025992018,1cyclictest3056-21PK-Backend07:22:360
2893025992018,1cyclictest0-21swapper/011:54:370
2893025992018,1cyclictest0-21swapper/011:48:490
2893025992018,1cyclictest0-21swapper/011:43:010
2893025992018,1cyclictest0-21swapper/011:39:360
2893025992018,1cyclictest0-21swapper/011:20:230
2893025992018,1cyclictest0-21swapper/011:04:210
2893025992018,1cyclictest0-21swapper/010:39:060
2893025992018,1cyclictest0-21swapper/009:45:530
2893025992018,1cyclictest0-21swapper/009:15:180
2893025992018,1cyclictest0-21swapper/008:53:210
2893025992018,1cyclictest0-21swapper/008:13:190
2893025992018,1cyclictest0-21swapper/007:37:370
2893025992012,6cyclictest2907368-21apt-get07:50:000
289302599200,19cyclictest0-21swapper/009:35:060
289302599200,19cyclictest0-21swapper/009:03:420
289302599200,19cyclictest0-21swapper/008:41:390
139099200,14rtkit-daemon3056-21PK-Backend12:32:521
139099200,14rtkit-daemon3056-21PK-Backend11:16:421
139099200,14rtkit-daemon3056-21PK-Backend09:27:331
139099200,14rtkit-daemon3056-21PK-Backend07:28:171
139099200,14rtkit-daemon0-21swapper/011:05:460
2893036991917,1cyclictest0-21swapper/307:24:513
289303399197,4cyclictest34-21ksoftirqd/211:15:202
289303399193,4cyclictest0-21swapper/210:55:242
2893033991916,2cyclictest331rcuc/212:24:302
2893033991916,2cyclictest331rcuc/211:37:342
2893033991915,3cyclictest331rcuc/212:35:372
2893033991915,3cyclictest331rcuc/210:26:162
2893033991915,3cyclictest331rcuc/210:26:162
2893033991913,5cyclictest2991598-21cron11:50:002
2893033991913,4cyclictest34-21ksoftirqd/211:10:122
289303399190,1cyclictest0-21swapper/211:06:342
289302799198,10cyclictest883-21dbus-daemon07:30:581
2893027991917,1cyclictest3056-21PK-Backend11:25:011
2893027991917,1cyclictest3056-21PK-Backend08:13:381
2893027991917,1cyclictest3056-21PK-Backend07:55:521
289302799190,1cyclictest3056-21PK-Backend10:20:581
289302799190,1cyclictest0-21swapper/110:09:231
289302799190,18cyclictest3056-21PK-Backend12:11:301
2893025991917,1cyclictest0-21swapper/008:30:230
2893025991917,1cyclictest0-21swapper/008:30:230
289302599190,1cyclictest3056-21PK-Backend08:20:100
289302599190,1cyclictest0-21swapper/007:28:050
139099190,13rtkit-daemon0-21swapper/012:10:080
139099190,12rtkit-daemon0-21swapper/009:11:390
289303699181,12cyclictest2934163-21apt09:05:113
2893033991816,1cyclictest0-21swapper/208:36:472
2893027991816,1cyclictest3056-21PK-Backend11:34:471
2893027991816,1cyclictest3056-21PK-Backend10:00:371
2893027991816,1cyclictest3056-21PK-Backend08:17:021
289302599187,10cyclictest2019-21gmain07:55:010
2893027991715,1cyclictest3056-21PK-Backend09:32:531
2893025991715,1cyclictest3056-21PK-Backend08:00:220
2893025991715,1cyclictest0-21swapper/011:34:370
2893025991715,1cyclictest0-21swapper/011:29:370
2893025991715,1cyclictest0-21swapper/010:54:210
2893025991715,1cyclictest0-21swapper/010:45:080
2893025991712,3cyclictest2995351-21basename12:00:010
2893033991612,3cyclictest331rcuc/210:40:522
2893027991610,5cyclictest2957724-21which10:10:191
289302599169,6cyclictest3001258-21cstates12:15:120
2893025991614,1cyclictest0-21swapper/012:34:390
2893025991614,1cyclictest0-21swapper/007:55:360
289302799159,4cyclictest2935625-21turbostat.cron09:10:001
289302799155,7cyclictest2946024-21sh09:40:001
2893027991513,1cyclictest3056-21PK-Backend11:59:201
2893027991513,1cyclictest3056-21PK-Backend09:44:151
2893027991512,2cyclictest3056-21PK-Backend11:48:471
289302799151,13cyclictest2932572-21cpuspeed_turbos09:00:131
2893027991510,4cyclictest2959537-21sensors_temp10:15:191
2893027991510,3cyclictest3003709-21switchtime12:20:231
289302599153,11cyclictest2951924-21cpuspeed_turbos09:55:130
2893025991513,1cyclictest3056-21PK-Backend10:09:540
2893025991513,1cyclictest0-21swapper/007:43:320
2893025991512,2cyclictest2995602-21apt12:00:100
2893025991511,3cyclictest2893691-21irqstats07:10:160
2893025991510,4cyclictest3008671-21tr12:35:180
2893027991412,1cyclictest3056-21PK-Backend09:57:271
2893025991410,3cyclictest2942940-21cpuspeed_turbos09:30:110
289302799139,3cyclictest3056-21PK-Backend12:06:451
2893027991310,2cyclictest3056-21PK-Backend07:50:111
29866722120,5sleep13056-21PK-Backend11:35:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional