You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 00:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sat Sep 19, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
386926721290,3sleep10-21swapper/107:10:201
387966621160,3sleep33056-21PK-Backend07:40:133
386741629140,29sleep20-21swapper/207:05:232
386805927840,14sleep30-21swapper/307:08:263
139099740,61rtkit-daemon0-21swapper/007:08:400
386813427240,9sleep10-21swapper/107:09:241
39774542620,4chrt0-21swapper/112:20:011
39316852490,2sleep2331rcuc/210:10:012
39195302470,3sleep20-21swapper/209:35:012
39195302470,3sleep20-21swapper/209:35:002
139099410,4rtkit-daemon3056-21PK-Backend08:15:010
139099390,4rtkit-daemon3056-21PK-Backend10:11:410
139099370,3rtkit-daemon3056-21PK-Backend09:33:320
139099370,3rtkit-daemon3056-21PK-Backend09:33:320
139099360,3rtkit-daemon3056-21PK-Backend08:57:030
139099360,3rtkit-daemon3056-21PK-Backend08:06:570
139099360,3rtkit-daemon0-21swapper/008:45:380
386830299350,22cyclictest0-21swapper/110:47:331
386830299350,18cyclictest3927998-21taskset09:59:571
386830299350,16cyclictest3877657-21/usr/sbin/munin07:35:111
139099350,3rtkit-daemon3056-21PK-Backend09:25:400
139099350,3rtkit-daemon0-21swapper/007:55:460
386830299346,14cyclictest0-21swapper/111:17:011
386830299340,21cyclictest3892750-21chrt08:16:121
386830299340,21cyclictest0-21swapper/108:32:031
386830299336,14cyclictest0-21swapper/110:13:011
386830299330,20cyclictest0-21swapper/111:41:141
386830299330,18cyclictest3899364-21cat08:35:171
386830299320,21cyclictest0-21swapper/107:19:371
386830299320,18cyclictest3973503-21runrttasks12:08:571
386830299320,18cyclictest3910564-21taskset09:08:481
386830299320,18cyclictest3880811-21taskset07:43:381
386830299320,16cyclictest3920960-21taskset09:38:441
386830299310,21cyclictest0-21swapper/110:28:281
386830299310,21cyclictest0-21swapper/110:28:281
386830299310,20cyclictest0-21swapper/108:53:481
386830299310,17cyclictest3913925-21taskset09:17:251
386830299310,16cyclictest3943558-21seq10:41:031
386830299300,18cyclictest3982326-21chrt12:32:241
386830299300,18cyclictest3980583-21taskset12:27:251
386830299300,17cyclictest3922672-21seq09:43:281
386830299300,15cyclictest3984007-21runrttasks12:36:441
3868302992912,15cyclictest3931065-21sed10:05:211
386830299290,17cyclictest3887755-21chrt08:03:491
386830299290,16cyclictest3947334-21seq10:51:421
386830299290,16cyclictest0-21swapper/112:11:211
386830299290,15cyclictest3940124-21seq10:34:491
3868305992812,14cyclictest3905498-21unin-run08:55:002
3868302992813,13cyclictest1-21systemd11:29:591
386830299280,16cyclictest3907176-21chrt08:59:551
386830299280,15cyclictest0-21swapper/111:01:481
386830299280,14cyclictest0-21swapper/108:25:201
386830299270,16cyclictest3890033-21cpuspeed_turbos08:10:131
386830299270,15cyclictest3908771-21taskset09:03:031
386830299270,15cyclictest0-21swapper/107:27:021
386830299260,15cyclictest0-21swapper/111:58:251
386830299260,15cyclictest0-21swapper/110:22:101
386830299260,14cyclictest0-21swapper/111:39:581
386830299260,14cyclictest0-21swapper/109:49:501
3868310992512,11cyclictest3961500-21unin-run11:35:003
3868305992515,1cyclictest141rcu_preempt07:10:222
386830299250,14cyclictest0-21swapper/108:49:261
386830299241,21cyclictest3971271-21/usr/sbin/munin12:00:241
3868302992412,10cyclictest3941184-21open_files10:35:221
3868302992410,13cyclictest3888679-21grep08:05:171
386830299240,22cyclictest3872457-21/usr/sbin/munin07:20:141
38744892230,3chrt3056-21PK-Backend07:25:133
3868310992320,2cyclictest3056-21PK-Backend09:15:173
3868310992315,5cyclictest436-20systemd-journal10:15:193
386830299236,15cyclictest3876472-21cat07:30:161
386830299235,16cyclictest3953352-21cat11:10:151
386830299230,5cyclictest0-21swapper/111:46:441
386830299230,21cyclictest3915129-21ntpq09:20:201
386829899231,5cyclictest3872227-21basename07:20:010
139099230,15rtkit-daemon3056-21PK-Backend12:30:090
3868310992220,1cyclictest3056-21PK-Backend09:39:513
3868310992220,1cyclictest3056-21PK-Backend08:25:493
3868310992219,2cyclictest3056-21PK-Backend12:35:533
3868310992219,2cyclictest3056-21PK-Backend11:57:133
3868310992217,4cyclictest3941913-21http10:40:003
3868305992220,1cyclictest3056-21PK-Backend07:55:302
3868305992220,1cyclictest0-21swapper/212:01:012
3868305992220,1cyclictest0-21swapper/211:01:052
3868305992220,1cyclictest0-21swapper/210:57:412
3868305992220,1cyclictest0-21swapper/210:52:392
3868305992220,1cyclictest0-21swapper/210:47:402
3868305992220,1cyclictest0-21swapper/208:41:192
3868305992220,1cyclictest0-21swapper/208:20:592
3868305992220,1cyclictest0-21swapper/207:21:302
3868305992214,6cyclictest883-21dbus-daemon11:30:002
386830599220,21cyclictest0-21swapper/208:30:182
386830299226,14cyclictest3884780-21cat07:55:121
386830299221,18cyclictest3910943-21apt09:10:111
386830299220,5cyclictest0-21swapper/112:20:371
386830299220,5cyclictest0-21swapper/112:20:361
386830299220,20cyclictest3956955-21ls11:20:161
3868298992220,1cyclictest0-21swapper/010:24:470
3868298992220,1cyclictest0-21swapper/009:22:340
3868298992217,3cyclictest3979309-21/usr/sbin/munin12:25:130
3868298992213,7cyclictest3895821-21ntp_states08:25:180
139099220,14rtkit-daemon0-21swapper/010:46:490
139099220,14rtkit-daemon0-21swapper/008:23:020
39203572210,7sleep2341ksoftirqd/209:35:212
38974392210,2sleep33056-21PK-Backend08:30:173
386831099214,15cyclictest3958334-21cat11:25:143
3868310992119,1cyclictest3056-21PK-Backend10:40:473
3868310992119,1cyclictest0-21swapper/310:07:153
3868310992119,1cyclictest0-21swapper/308:59:483
3868310992119,1cyclictest0-21swapper/308:44:013
3868310992118,2cyclictest3056-21PK-Backend09:48:023
386831099211,17cyclictest3887263-21python308:00:203
386831099210,20cyclictest0-21swapper/307:49:193
386831099210,19cyclictest3056-21PK-Backend12:22:073
386831099210,19cyclictest3056-21PK-Backend12:22:073
386831099210,19cyclictest3056-21PK-Backend12:00:243
386831099210,19cyclictest3056-21PK-Backend09:06:043
386831099210,19cyclictest3056-21PK-Backend07:10:393
386830599213,15cyclictest3887240-21sshd08:00:192
3868305992119,1cyclictest0-21swapper/212:17:232
3868305992119,1cyclictest0-21swapper/211:43:032
3868305992119,1cyclictest0-21swapper/211:35:402
3868305992119,1cyclictest0-21swapper/210:28:062
3868305992119,1cyclictest0-21swapper/210:28:062
3868305992119,1cyclictest0-21swapper/210:23:202
3868305992119,1cyclictest0-21swapper/210:19:212
3868305992119,1cyclictest0-21swapper/210:02:512
3868305992119,1cyclictest0-21swapper/209:56:022
3868305992119,1cyclictest0-21swapper/209:44:132
3868305992119,1cyclictest0-21swapper/208:56:002
3868305992119,1cyclictest0-21swapper/208:48:232
3868305992119,1cyclictest0-21swapper/208:39:102
3868305992119,1cyclictest0-21swapper/208:27:282
3868305992119,1cyclictest0-21swapper/208:12:052
3868305992119,1cyclictest0-21swapper/207:51:182
3868305992113,6cyclictest3953652-21memory11:10:202
3868305992113,6cyclictest28650irq/123-ahci[0008:15:212
386830599210,20cyclictest0-21swapper/209:53:262
386830299216,13cyclictest3916352-21cstates09:25:131
386830299216,13cyclictest3893893-21cat08:20:181
3868302992119,1cyclictest0-21swapper/108:42:311
386830299211,17cyclictest0-21swapper/110:00:141
386830299210,20cyclictest3882417-21chrt07:47:011
386830299210,20cyclictest0-21swapper/111:30:351
386830299210,20cyclictest0-21swapper/110:17:481
386830299210,19cyclictest3967101-21expr11:50:131
386830299210,19cyclictest3883343-21chrt07:50:181
3868298992119,1cyclictest3056-21PK-Backend07:51:020
3868298992119,1cyclictest0-21swapper/012:38:510
3868298992119,1cyclictest0-21swapper/012:19:310
3868298992119,1cyclictest0-21swapper/011:12:050
3868298992119,1cyclictest0-21swapper/011:04:270
3868298992119,1cyclictest0-21swapper/010:53:400
3868298992119,1cyclictest0-21swapper/008:32:330
3868298992119,1cyclictest0-21swapper/007:30:340
3868298992118,2cyclictest3056-21PK-Backend10:09:260
3868298992118,2cyclictest3056-21PK-Backend08:36:380
386829899210,5cyclictest3875691-21apt07:30:000
386829899210,20cyclictest3056-21PK-Backend11:46:530
386829899210,20cyclictest0-21swapper/011:59:240
386829899210,20cyclictest0-21swapper/011:08:400
386829899210,20cyclictest0-21swapper/010:27:100
386829899210,20cyclictest0-21swapper/010:27:100
386829899210,20cyclictest0-21swapper/009:10:590
386829899210,20cyclictest0-21swapper/009:03:080
386829899210,20cyclictest0-21swapper/007:12:290
386829899210,19cyclictest3056-21PK-Backend11:23:000
139099210,14rtkit-daemon0-21swapper/010:00:300
386831099202,13cyclictest3902273-21apt08:45:143
3868310992018,1cyclictest0-21swapper/309:25:303
386831099201,7cyclictest3921318-21/usr/sbin/munin09:40:113
3868310992017,2cyclictest3056-21PK-Backend09:24:433
3868310992013,5cyclictest3981608-21sed12:30:173
3868310992012,6cyclictest3892813-21cron08:17:003
3868310992011,7cyclictest1-21systemd11:50:013
386831099200,19cyclictest3056-21PK-Backend11:09:203
386831099200,19cyclictest3056-21PK-Backend11:02:453
386831099200,19cyclictest3056-21PK-Backend10:21:413
386831099200,19cyclictest0-21swapper/311:20:433
3868305992018,1cyclictest0-21swapper/209:49:132
3868305992018,1cyclictest0-21swapper/208:09:302
3868305992013,5cyclictest3942809-21seq10:40:192
386830599200,1cyclictest0-21swapper/211:50:262
386830599200,1cyclictest0-21swapper/211:09:302
386830599200,19cyclictest0-21swapper/209:21:422
386830599200,19cyclictest0-21swapper/209:11:282
386830599200,19cyclictest0-21swapper/207:34:212
386830599200,19cyclictest0-21swapper/207:27:252
386830299201,17cyclictest3952083-21ntp_states11:05:201
386830299201,17cyclictest3947852-21acpi10:55:121
386830299201,17cyclictest3925549-21ntp_states09:50:181
386830299200,18cyclictest3918466-21grep09:30:191
386830299200,18cyclictest3918466-21grep09:30:191
3868298992018,1cyclictest0-21swapper/010:59:280
3868298992018,1cyclictest0-21swapper/009:39:380
3868298992018,1cyclictest0-21swapper/007:43:520
3868298992017,2cyclictest3056-21PK-Backend10:18:380
3868298992011,6cyclictest3910661-21cron09:10:000
386829899200,19cyclictest3924781-21/usr/sbin/munin09:50:130
386829899200,19cyclictest0-21swapper/010:35:070
386829899200,19cyclictest0-21swapper/010:31:090
386829899200,18cyclictest3056-21PK-Backend07:46:160
139099200,14rtkit-daemon3056-21PK-Backend08:16:580
139099200,14rtkit-daemon0-21swapper/008:54:570
139099200,13rtkit-daemon0-21swapper/012:22:490
139099200,13rtkit-daemon0-21swapper/012:22:490
38845152190,4sleep3411rcuc/307:55:013
3868310991917,1cyclictest3056-21PK-Backend10:54:343
3868310991917,1cyclictest3056-21PK-Backend09:13:073
3868310991917,1cyclictest3056-21PK-Backend08:22:113
3868310991916,2cyclictest3056-21PK-Backend09:53:493
386831099191,5cyclictest3962481-21ntp_states11:35:203
3868310991912,5cyclictest3919387-21sed09:35:003
3868310991912,5cyclictest3919387-21sed09:35:003
386831099190,1cyclictest3056-21PK-Backend12:28:293
386831099190,1cyclictest3056-21PK-Backend12:05:103
386831099190,1cyclictest3056-21PK-Backend08:51:043
386831099190,1cyclictest0-21swapper/308:06:293
386831099190,18cyclictest0-21swapper/308:39:073
3868305991917,1cyclictest0-21swapper/212:24:472
3868305991917,1cyclictest0-21swapper/212:24:462
3868305991917,1cyclictest0-21swapper/210:10:432
3868305991917,1cyclictest0-21swapper/209:26:152
3868305991917,1cyclictest0-21swapper/207:15:492
3868305991913,4cyclictest28650irq/123-ahci[0010:35:232
3868305991912,6cyclictest3984266-21cron12:39:592
3868305991912,5cyclictest3956263-21cron11:20:002
3868305991911,6cyclictest3961500-21munin-run11:35:002
3868305991911,6cyclictest1-21systemd12:33:192
3868305991910,7cyclictest4176944-21snapd07:49:132
386829899192,12cyclictest3970407-21apt12:00:120
3868298991917,1cyclictest0-21swapper/009:17:280
3868298991911,6cyclictest3972528-21fw_packets12:05:140
386829899190,1cyclictest0-21swapper/008:00:000
139099190,13rtkit-daemon0-21swapper/009:49:000
386831099185,12cyclictest0-21swapper/311:17:033
3868310991816,1cyclictest3056-21PK-Backend10:03:513
3868310991816,1cyclictest3056-21PK-Backend07:33:373
3868310991816,1cyclictest0-21swapper/308:13:193
3868310991815,2cyclictest3056-21PK-Backend11:41:513
3868310991813,3cyclictest3884859-21cpuspeed_turbos07:55:133
386831099181,15cyclictest0-21swapper/307:20:183
3868305991816,1cyclictest0-21swapper/209:05:462
3868305991811,6cyclictest3965881-21cut11:45:172
3868305991810,6cyclictest3939387-21awk10:30:192
3868298991816,1cyclictest0-21swapper/011:50:040
3868298991812,4cyclictest3963207-21munin-plugin-st11:39:590
3868298991811,5cyclictest3943871-21cron10:44:590
386831099179,6cyclictest3931873-21sort10:10:133
3868310991715,1cyclictest3056-21PK-Backend11:50:033
3868310991715,1cyclictest3056-21PK-Backend09:59:383
3868310991715,1cyclictest3056-21PK-Backend07:35:133
3868310991715,1cyclictest3056-21PK-Backend07:18:503
3868310991715,1cyclictest0-21swapper/310:25:413
3868310991715,1cyclictest0-21swapper/310:25:413
3868305991715,1cyclictest0-21swapper/212:10:002
3868305991715,1cyclictest0-21swapper/209:16:352
3868305991715,1cyclictest0-21swapper/209:03:372
3868305991715,1cyclictest0-21swapper/207:40:092
3868305991712,4cyclictest3974285-21ls12:10:172
3868305991712,3cyclictest3979494-21cpuspeed_turbos12:25:142
3868298991712,3cyclictest3963564-21cpuspeed_turbos11:40:130
3868298991710,5cyclictest3974316-21cat12:10:170
3868298991710,5cyclictest3877676-21cat07:35:100
3868310991611,3cyclictest3908444-21timerandwakeup09:00:213
386830599161,11cyclictest3968675-21apt11:55:142
386829899169,5cyclictest3922252-21sed09:40:210
386829899169,5cyclictest3874082-21cat07:25:000
3868298991614,1cyclictest3056-21PK-Backend11:16:520
3868298991612,3cyclictest3927386-21processes09:55:230
3868310991513,1cyclictest3056-21PK-Backend12:18:563
3868310991513,1cyclictest3056-21PK-Backend11:11:563
3868310991510,4cyclictest42-21ksoftirqd/310:45:143
386830599158,5cyclictest3956626-21ls11:20:132
3868298991513,1cyclictest3056-21PK-Backend11:26:110
3868298991513,1cyclictest3056-21PK-Backend08:42:400
3868298991510,3cyclictest3961481-21turbostat.cron11:34:590
386831099149,4cyclictest3056-21PK-Backend12:11:523
386830599146,6cyclictest3877684-21apt07:35:122
3868310991310,2cyclictest3056-21PK-Backend10:55:153
386831099110,10cyclictest3056-21PK-Backend10:30:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional