You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-13 - 14:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Aug 13, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
370853521400,4sleep23056-21PK-Backend21:25:002
372350721360,4sleep13056-21PK-Backend22:05:131
375126221090,3sleep13056-21PK-Backend23:25:001
139099860,73rtkit-daemon0-21swapper/319:08:243
366041428439,16sleep20-21swapper/219:07:262
366046026444,14sleep10-21swapper/119:08:011
366048026239,18sleep00-21swapper/019:08:170
36901602500,2sleep13056-21PK-Backend20:30:211
37287182460,4sleep13056-21PK-Backend22:20:131
3660757992616,6cyclictest1095-21runrttasks22:05:042
139099260,18rtkit-daemon3056-21PK-Backend23:56:183
366076199250,23cyclictest3697483-21taskset20:50:333
3660756992511,12cyclictest3705896-21memory21:15:171
3660761992410,12cyclictest3728127-21unin-run22:20:003
3660751992417,5cyclictest883-21dbus-daemon00:05:010
3660757992310,11cyclictest3774020-21unin-run00:30:002
3660756992315,6cyclictest436-20systemd-journal23:55:001
366075199233,18cyclictest3666355-21nm-dispatcher19:24:540
3660751992315,6cyclictest3711987-21cron21:35:000
37629772220,13sleep20-21swapper/223:56:152
36690552220,5chrt0-21swapper/019:30:180
3660761992220,1cyclictest0-21swapper/323:10:253
3660761992219,2cyclictest3056-21PK-Backend19:32:343
366076199221,17cyclictest0-21swapper/320:10:103
366076199220,21cyclictest3056-21PK-Backend19:15:093
3660757992220,1cyclictest0-21swapper/223:37:522
3660757992220,1cyclictest0-21swapper/223:13:312
3660757992220,1cyclictest0-21swapper/222:39:252
3660757992220,1cyclictest0-21swapper/222:18:092
3660757992220,1cyclictest0-21swapper/222:11:272
3660757992220,1cyclictest0-21swapper/221:16:372
3660757992220,1cyclictest0-21swapper/220:04:382
3660757992220,1cyclictest0-21swapper/200:19:152
3660751992210,9cyclictest3729827-21apt-get22:25:000
366075199220,21cyclictest0-21swapper/000:38:460
139099220,15rtkit-daemon0-21swapper/219:11:402
139099220,14rtkit-daemon0-21swapper/320:58:313
37197802210,2sleep00-21swapper/021:55:140
366076199218,12cyclictest3694621-21/usr/sbin/munin20:45:143
366076199214,15cyclictest3734186-21expr22:35:113
3660761992119,1cyclictest3056-21PK-Backend23:53:593
3660761992119,1cyclictest3056-21PK-Backend19:10:033
3660761992119,1cyclictest0-21swapper/322:46:533
3660761992119,1cyclictest0-21swapper/322:30:383
3660761992119,1cyclictest0-21swapper/300:12:443
3660761992115,4cyclictest42-21ksoftirqd/321:04:563
3660761992114,6cyclictest3691577-21fw_packets20:35:163
366076199210,2cyclictest0-21swapper/323:00:113
366076199210,20cyclictest0-21swapper/323:32:023
366076199210,20cyclictest0-21swapper/322:44:233
366076199210,20cyclictest0-21swapper/322:26:593
366076199210,20cyclictest0-21swapper/321:36:573
366076199210,20cyclictest0-21swapper/319:27:393
366076199210,20cyclictest0-21swapper/319:27:393
366076199210,19cyclictest3056-21PK-Backend00:22:243
366076199210,19cyclictest3056-21PK-Backend00:22:243
366076199210,19cyclictest3056-21PK-Backend00:05:343
3660757992119,1cyclictest0-21swapper/223:17:232
3660757992119,1cyclictest0-21swapper/223:17:232
3660757992119,1cyclictest0-21swapper/222:50:332
3660757992119,1cyclictest0-21swapper/221:08:532
3660757992119,1cyclictest0-21swapper/220:57:042
3660757992119,1cyclictest0-21swapper/220:51:302
3660757992119,1cyclictest0-21swapper/220:37:232
3660757992119,1cyclictest0-21swapper/220:32:232
3660757992119,1cyclictest0-21swapper/220:13:522
3660757992119,1cyclictest0-21swapper/220:08:582
3660757992119,1cyclictest0-21swapper/219:36:472
3660757992119,1cyclictest0-21swapper/219:22:082
3660757992119,1cyclictest0-21swapper/219:18:572
3660757992119,1cyclictest0-21swapper/200:36:052
3660757992113,6cyclictest3715434-21cron21:45:012
366075799210,20cyclictest0-21swapper/223:46:102
366075799210,20cyclictest0-21swapper/222:00:242
366075799210,20cyclictest0-21swapper/221:02:182
366075799210,20cyclictest0-21swapper/220:19:202
366075799210,20cyclictest0-21swapper/219:52:282
366075699210,20cyclictest3056-21PK-Backend22:53:501
366075699210,1cyclictest0-21swapper/121:34:141
366075699210,19cyclictest3056-21PK-Backend19:53:091
139099210,4rtkit-daemon0-21swapper/220:20:362
139099210,15rtkit-daemon3056-21PK-Backend21:32:262
139099210,15rtkit-daemon0-21swapper/323:19:303
139099210,15rtkit-daemon0-21swapper/323:19:303
139099210,15rtkit-daemon0-21swapper/220:48:102
139099210,14rtkit-daemon0-21swapper/219:47:162
139099210,14rtkit-daemon0-21swapper/200:09:552
3660761992018,1cyclictest3056-21PK-Backend00:18:323
3660761992018,1cyclictest0-21swapper/323:23:443
3660761992018,1cyclictest0-21swapper/322:22:123
3660761992018,1cyclictest0-21swapper/321:28:053
3660761992018,1cyclictest0-21swapper/320:30:323
3660761992017,2cyclictest3056-21PK-Backend23:43:243
3660761992012,6cyclictest3774261-21apt00:30:123
3660761992012,6cyclictest3688163-21cat20:25:163
366076199200,1cyclictest3056-21PK-Backend21:16:013
366076199200,19cyclictest3056-21PK-Backend21:34:073
366076199200,19cyclictest0-21swapper/322:58:153
366076199200,19cyclictest0-21swapper/321:10:343
366076199200,19cyclictest0-21swapper/320:43:303
366076199200,19cyclictest0-21swapper/320:16:193
366076199200,19cyclictest0-21swapper/319:37:273
366076199200,18cyclictest436-20systemd-journal19:45:433
3660757992018,1cyclictest0-21swapper/223:44:142
3660757992018,1cyclictest0-21swapper/222:43:032
3660757992018,1cyclictest0-21swapper/219:55:392
3660757992018,1cyclictest0-21swapper/200:02:122
3660757992013,6cyclictest3716102-21awk21:45:142
3660757992012,6cyclictest436-20systemd-journal19:45:012
366075799201,17cyclictest3729836-21perf22:25:002
366075799200,1cyclictest0-21swapper/221:26:042
366075799200,19cyclictest0-21swapper/223:50:362
366075799200,19cyclictest0-21swapper/220:28:522
3660756992018,1cyclictest3056-21PK-Backend22:56:541
3660756992017,2cyclictest3056-21PK-Backend23:06:411
3660756992012,6cyclictest3772512-21cat00:25:121
366075699200,1cyclictest3056-21PK-Backend21:57:531
366075699200,19cyclictest3056-21PK-Backend22:46:001
366075699200,19cyclictest3056-21PK-Backend00:12:311
366075699200,19cyclictest3056-21PK-Backend00:10:011
366075699200,19cyclictest0-21swapper/123:45:551
366075699200,19cyclictest0-21swapper/122:38:221
366075699200,19cyclictest0-21swapper/121:23:541
366075699200,19cyclictest0-21swapper/119:33:341
366075699200,19cyclictest0-21swapper/100:21:231
366075699200,19cyclictest0-21swapper/100:21:231
3660751992018,1cyclictest0-21swapper/020:34:440
3660751992014,4cyclictest1095-21runrttasks23:44:290
3660751992013,5cyclictest13-21ksoftirqd/019:15:150
3660751992012,6cyclictest3706089-21awk21:15:200
366075199200,1cyclictest0-21swapper/022:00:210
366075199200,19cyclictest0-21swapper/023:55:530
366075199200,19cyclictest0-21swapper/023:50:190
366075199200,19cyclictest0-21swapper/023:22:000
366075199200,19cyclictest0-21swapper/023:17:340
366075199200,19cyclictest0-21swapper/023:17:340
366075199200,19cyclictest0-21swapper/022:59:510
366075199200,19cyclictest0-21swapper/022:08:530
366075199200,19cyclictest0-21swapper/021:06:200
366075199200,19cyclictest0-21swapper/021:01:270
366075199200,19cyclictest0-21swapper/020:25:510
366075199200,19cyclictest0-21swapper/020:04:160
366075199200,19cyclictest0-21swapper/019:38:190
366075199200,19cyclictest0-21swapper/000:07:290
139099200,14rtkit-daemon3056-21PK-Backend21:20:333
139099200,14rtkit-daemon0-21swapper/223:20:532
139099200,14rtkit-daemon0-21swapper/221:36:572
139099200,14rtkit-daemon0-21swapper/200:33:252
37094292190,3sleep10-21swapper/121:25:191
37092622190,4sleep03708756-21/usr/sbin/munin21:25:190
366076199192,15cyclictest3721581-21sort22:00:183
3660761991917,1cyclictest0-21swapper/322:10:093
3660761991917,1cyclictest0-21swapper/321:05:063
3660761991917,1cyclictest0-21swapper/320:01:203
3660761991917,1cyclictest0-21swapper/319:51:593
3660761991916,2cyclictest3056-21PK-Backend23:48:243
3660761991912,5cyclictest42-21ksoftirqd/321:50:103
3660761991912,4cyclictest3752292-21python323:25:223
366076199190,1cyclictest3056-21PK-Backend23:36:283
366076199190,18cyclictest3056-21PK-Backend00:26:503
3660757991917,1cyclictest3056-21PK-Backend19:30:112
3660757991917,1cyclictest0-21swapper/223:29:332
3660757991917,1cyclictest0-21swapper/222:31:002
3660757991913,5cyclictest3767261-21wc00:10:122
3660757991912,5cyclictest3745206-21ntpq23:05:192
3660757991911,6cyclictest3737300-21awk22:45:112
366075799190,1cyclictest0-21swapper/222:28:292
366075799190,18cyclictest0-21swapper/223:01:552
366075799190,18cyclictest0-21swapper/222:59:592
3660756991917,1cyclictest3056-21PK-Backend19:29:161
3660756991917,1cyclictest3056-21PK-Backend19:29:161
3660756991917,1cyclictest0-21swapper/119:10:091
3660756991916,2cyclictest3056-21PK-Backend22:16:041
3660756991916,2cyclictest3056-21PK-Backend00:35:161
3660756991912,5cyclictest3757163-21cat23:40:161
3660756991912,5cyclictest3664605-21sshd19:19:441
366075699190,1cyclictest3056-21PK-Backend20:17:211
366075699190,1cyclictest3056-21PK-Backend19:40:101
366075699190,1cyclictest0-21swapper/119:24:011
366075699190,1cyclictest0-21swapper/100:17:311
366075699190,18cyclictest3056-21PK-Backend23:04:111
366075699190,18cyclictest3056-21PK-Backend22:30:111
366075699190,18cyclictest3056-21PK-Backend20:23:441
366075699190,18cyclictest3056-21PK-Backend19:39:021
366075699190,18cyclictest0-21swapper/123:56:291
366075699190,18cyclictest0-21swapper/123:16:081
366075699190,18cyclictest0-21swapper/123:16:071
366075699190,18cyclictest0-21swapper/100:04:131
366075699190,17cyclictest3056-21PK-Backend20:27:151
366075699190,17cyclictest3056-21PK-Backend20:06:541
3660751991917,1cyclictest0-21swapper/023:47:500
3660751991917,1cyclictest0-21swapper/023:30:180
3660751991917,1cyclictest0-21swapper/023:08:420
3660751991917,1cyclictest0-21swapper/022:17:520
3660751991917,1cyclictest0-21swapper/021:22:090
3660751991917,1cyclictest0-21swapper/020:53:160
3660751991917,1cyclictest0-21swapper/020:49:440
3660751991917,1cyclictest0-21swapper/020:44:440
3660751991917,1cyclictest0-21swapper/020:11:450
3660751991917,1cyclictest0-21swapper/019:45:090
3660751991917,1cyclictest0-21swapper/000:19:050
3660751991912,6cyclictest3684958-21awk20:15:180
366075199190,1cyclictest3056-21PK-Backend19:40:360
366075199190,1cyclictest0-21swapper/023:04:500
366075199190,1cyclictest0-21swapper/022:40:020
366075199190,1cyclictest0-21swapper/022:30:290
366075199190,18cyclictest0-21swapper/023:37:480
366075199190,18cyclictest0-21swapper/023:10:380
366075199190,18cyclictest0-21swapper/022:47:530
366075199190,18cyclictest0-21swapper/020:21:320
366075199190,18cyclictest0-21swapper/000:31:490
366075199190,18cyclictest0-21swapper/000:21:010
366075199190,18cyclictest0-21swapper/000:21:010
139099190,13rtkit-daemon0-21swapper/321:58:383
139099190,13rtkit-daemon0-21swapper/319:40:403
139099190,13rtkit-daemon0-21swapper/223:32:032
366076199182,4cyclictest3680901-21cpuspeed_turbos20:05:133
3660761991816,1cyclictest0-21swapper/322:50:453
3660761991816,1cyclictest0-21swapper/320:22:553
366076199181,4cyclictest3776734-21ntp_states00:35:203
3660761991811,5cyclictest3677652-21sed19:55:163
366076199180,4cyclictest3764320-21ntp_states00:00:193
366076199180,4cyclictest3716455-21ntp_states21:45:183
3660757991816,1cyclictest0-21swapper/221:10:372
3660757991816,1cyclictest0-21swapper/200:20:372
3660757991816,1cyclictest0-21swapper/200:20:372
3660757991812,5cyclictest3667285-21cat19:25:172
3660757991812,5cyclictest3667285-21cat19:25:172
3660756991816,1cyclictest3056-21PK-Backend22:00:431
3660756991816,1cyclictest3056-21PK-Backend21:14:061
3660756991816,1cyclictest3056-21PK-Backend21:04:331
3660756991816,1cyclictest3056-21PK-Backend20:04:171
3660756991816,1cyclictest3056-21PK-Backend19:47:541
3660756991816,1cyclictest3056-21PK-Backend00:32:461
3660756991812,4cyclictest3697099-21awk20:50:181
366075699180,1cyclictest3056-21PK-Backend19:58:081
366075699180,1cyclictest0-21swapper/123:36:281
366075699180,17cyclictest3056-21PK-Backend21:35:561
3660751991816,1cyclictest3056-21PK-Backend21:45:340
3660751991816,1cyclictest3056-21PK-Backend21:35:200
3660751991816,1cyclictest0-21swapper/022:54:490
3660751991816,1cyclictest0-21swapper/021:51:150
3660751991816,1cyclictest0-21swapper/021:40:270
3660751991816,1cyclictest0-21swapper/020:39:440
3660751991816,1cyclictest0-21swapper/020:08:000
3660751991816,1cyclictest0-21swapper/019:59:020
3660751991816,1cyclictest0-21swapper/000:27:230
366075199180,1cyclictest3056-21PK-Backend21:11:140
139099180,12rtkit-daemon0-21swapper/220:43:102
3660761991715,1cyclictest3056-21PK-Backend19:21:513
3660761991713,3cyclictest3723142-21sort22:05:113
3660761991711,5cyclictest42-21ksoftirqd/321:40:183
366076199170,14cyclictest3744469-21/usr/sbin/munin23:05:143
3660757991713,3cyclictest34-21ksoftirqd/221:55:002
3660757991711,4cyclictest3720690-21sshd21:57:282
3660756991715,1cyclictest3056-21PK-Backend20:40:201
3660756991715,1cyclictest3056-21PK-Backend20:10:051
3660756991711,5cyclictest3747402-21sshd23:11:291
3660756991711,5cyclictest3716755-21cat21:45:221
3660756991710,5cyclictest3731827-21cron22:30:001
366075199178,7cyclictest3698975-21python320:55:210
3660751991715,1cyclictest0-21swapper/023:28:220
3660751991715,1cyclictest0-21swapper/019:14:540
3660751991715,1cyclictest0-21swapper/000:10:480
3660751991713,3cyclictest3668113-21cron19:29:590
3660751991713,3cyclictest3668113-21cron19:29:590
3660751991711,5cyclictest3675855-21sed19:50:160
366075199170,4cyclictest3733815-21sort22:35:090
3660756991614,1cyclictest3056-21PK-Backend21:05:071
3660756991614,1cyclictest3056-21PK-Backend20:59:531
3660756991614,1cyclictest3056-21PK-Backend20:37:221
3660756991611,4cyclictest3719116-21cat21:55:011
3660751991614,1cyclictest0-21swapper/022:13:390
366075699159,5cyclictest3725601-21grep22:10:171
3660756991511,3cyclictest1095-21runrttasks21:40:181
366075699150,1cyclictest3056-21PK-Backend20:46:351
3660751991513,1cyclictest3056-21PK-Backend22:25:440
366075699138,3cyclictest3752369-21xargs23:25:221
3660756991310,2cyclictest3056-21PK-Backend23:33:241
366075699105,4cyclictest3056-21PK-Backend22:42:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional