You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 04:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon Jun 01, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139099790,69rtkit-daemon0-21swapper/019:07:020
14132227645,22sleep30-21swapper/319:05:153
14232226243,13sleep10-21swapper/119:08:561
14210226242,14sleep20-21swapper/219:06:082
1458782320,4sleep33056-21PK-Backend19:15:233
142555993114,4cyclictest242878-21grep23:55:012
139099310,4rtkit-daemon1389-21rtkit-daemon00:06:270
14255099302,16cyclictest0-21swapper/122:26:471
139099280,3rtkit-daemon1389-21rtkit-daemon21:51:130
14255099273,22cyclictest214806-21latency_hist22:35:011
1877022260,3chrt0-21swapper/221:15:202
14255099267,17cyclictest247729-21irqstats00:10:111
14255099267,17cyclictest225410-21expr23:05:131
14255099266,18cyclictest255307-21seq00:30:261
142550992612,12cyclictest178557-21diskstats20:50:151
14255099258,15cyclictest196020-21ls21:40:131
14255099258,15cyclictest0-21swapper/121:35:011
14255099257,17cyclictest243529-21fw_packets23:55:161
14255099257,16cyclictest239977-21ls23:45:121
14255099257,16cyclictest235983-21sed23:35:011
14255099257,16cyclictest230570-21latency_hist23:20:011
14255099257,16cyclictest217785-21sshd22:41:331
14255099257,16cyclictest156043-21strings19:45:181
14255099256,17cyclictest189385-21sort21:20:171
14255099256,16cyclictest201903-21sed21:55:221
14255099254,19cyclictest154818-21fwupdmgr19:41:341
14255099253,20cyclictest171223-21acpi20:30:111
14255099253,19cyclictest253825-21python300:28:491
14255099251,4cyclictest0-21swapper/120:45:131
14255099251,3cyclictest0-21swapper/120:15:111
142550992512,11cyclictest26-21ksoftirqd/121:00:131
142550992510,13cyclictest184149-21ntpq21:05:201
142550992510,13cyclictest170174-21tail20:25:181
142550992510,13cyclictest170174-21tail20:25:181
14255099250,3cyclictest187491-21df21:15:161
14255099250,23cyclictest917-21in:imuxsock22:25:001
14255099250,23cyclictest0-21swapper/120:09:231
139099250,4rtkit-daemon1389-21rtkit-daemon23:41:551
14255599241,3cyclictest146818-21/usr/sbin/munin19:20:142
14255099248,14cyclictest168688-21switchtime20:20:231
14255099248,14cyclictest1095-21runrttasks20:13:141
14255099247,15cyclictest220285-21cstates22:50:141
14255099247,15cyclictest207659-21cron22:15:001
14255099247,15cyclictest0-21swapper/122:35:131
14255099242,10cyclictest158842-21apt19:55:111
142550992419,4cyclictest0-21swapper/120:02:271
139099240,3rtkit-daemon1389-21rtkit-daemon19:16:240
139099240,3rtkit-daemon1389-21rtkit-daemon00:37:230
14255099238,13cyclictest231271-21latency23:20:171
14255099238,13cyclictest231271-21latency23:20:171
14255099238,13cyclictest1095-21runrttasks22:47:041
14255099237,13cyclictest224036-21kernelversion23:00:151
14255099236,16cyclictest158135-21cat19:50:231
14255099235,16cyclictest249853-21sensors_temp00:15:221
14255099234,18cyclictest175673-21sort20:40:231
14255099234,17cyclictest180462-21ls20:55:171
14255099233,3cyclictest0-21swapper/119:35:061
14255099231,9cyclictest237680-21/usr/sbin/munin23:40:011
14255099231,7cyclictest173345-21df_inode20:35:121
14255099231,5cyclictest256103-21cpuspeed_turbos00:35:121
14255099231,4cyclictest227230-21apt-get23:10:041
14255099231,4cyclictest0-21swapper/122:00:151
14255099231,3cyclictest186862-21idleruntime-cro21:14:591
14255099231,19cyclictest0-21swapper/122:55:121
14255099230,21cyclictest143654-21switchtime19:10:191
142547992314,7cyclictest144199-21cron19:15:000
142547992314,6cyclictest436-20systemd-journal00:00:000
142558992217,3cyclictest978-21snmpd22:46:273
14255899220,3cyclictest143110-21df_inode19:10:113
142555992220,1cyclictest0-21swapper/223:19:292
142555992220,1cyclictest0-21swapper/222:37:522
142555992220,1cyclictest0-21swapper/222:17:312
142555992220,1cyclictest0-21swapper/220:04:012
142555992216,5cyclictest2164-21gdbus23:28:472
14255599220,21cyclictest0-21swapper/222:08:522
14255099227,13cyclictest194047-21/usr/sbin/munin21:35:151
14255099225,15cyclictest145399-21if_enp2s019:15:171
14255099224,2cyclictest204684-21cstates22:05:131
14255099224,15cyclictest241956-21latency23:50:171
14255099223,5cyclictest190888-21expr21:25:121
14255099221,5cyclictest246488-21latency_hist00:05:011
14255099221,4cyclictest197520-21/usr/sbin/munin21:45:191
14255099220,6cyclictest0-21swapper/100:20:151
14255099220,3cyclictest208518-21memory22:15:171
142547992214,6cyclictest436-20systemd-journal23:28:470
142547992214,6cyclictest191667-21users21:25:210
14254799221,20cyclictest226741-21apt-get23:10:030
14254799220,20cyclictest244895-21apt00:00:120
139099220,3rtkit-daemon1389-21rtkit-daemon23:34:550
1605262210,4sleep20-21swapper/220:00:012
142558992112,6cyclictest195542-21sh21:40:003
14255899210,1cyclictest3056-21PK-Backend00:15:373
14255899210,19cyclictest3056-21PK-Backend23:32:453
14255899210,19cyclictest3056-21PK-Backend21:04:143
14255899210,19cyclictest3056-21PK-Backend20:52:103
14255899210,19cyclictest3056-21PK-Backend20:40:423
142555992119,1cyclictest3056-21PK-Backend20:11:582
142555992119,1cyclictest0-21swapper/223:58:162
142555992119,1cyclictest0-21swapper/223:12:402
142555992119,1cyclictest0-21swapper/223:06:382
142555992119,1cyclictest0-21swapper/222:52:452
142555992119,1cyclictest0-21swapper/221:52:502
142555992119,1cyclictest0-21swapper/221:34:102
142555992119,1cyclictest0-21swapper/221:24:592
142555992119,1cyclictest0-21swapper/221:12:352
142555992119,1cyclictest0-21swapper/220:32:592
142555992119,1cyclictest0-21swapper/219:52:182
142555992119,1cyclictest0-21swapper/200:32:012
142555992119,1cyclictest0-21swapper/200:21:352
142555992119,1cyclictest0-21swapper/200:15:332
142555992119,1cyclictest0-21swapper/200:12:562
142555992111,6cyclictest34-21ksoftirqd/223:33:342
14255599210,20cyclictest0-21swapper/222:43:062
14255599210,20cyclictest0-21swapper/222:12:172
14255599210,20cyclictest0-21swapper/221:56:142
14255599210,20cyclictest0-21swapper/221:49:262
14255599210,20cyclictest0-21swapper/221:41:482
14255599210,20cyclictest0-21swapper/221:29:582
14255599210,20cyclictest0-21swapper/220:29:492
14255599210,20cyclictest0-21swapper/220:29:492
14255599210,20cyclictest0-21swapper/219:37:512
14255599210,20cyclictest0-21swapper/200:39:532
14255599210,1cyclictest0-21swapper/220:35:242
142550992119,1cyclictest0-21swapper/121:50:351
142550992119,1cyclictest0-21swapper/100:07:101
14255099211,6cyclictest233329-21python323:25:211
142547992119,1cyclictest0-21swapper/021:58:310
142547992119,1cyclictest0-21swapper/020:39:570
142547992118,2cyclictest0-21swapper/023:45:460
142547992113,6cyclictest166908-21python320:15:210
142547992110,5cyclictest164698-21cat20:10:170
14254799210,20cyclictest0-21swapper/021:18:290
14254799210,20cyclictest0-21swapper/021:07:550
14254799210,20cyclictest0-21swapper/000:20:390
14254799210,20cyclictest0-21swapper/000:10:260
139099210,3rtkit-daemon1389-21rtkit-daemon21:38:020
1962112200,4sleep0196302-21tr21:40:170
14255899204,5cyclictest42-21ksoftirqd/319:55:003
142558992018,1cyclictest3056-21PK-Backend23:36:093
142558992018,1cyclictest0-21swapper/319:21:203
142558992017,2cyclictest3056-21PK-Backend21:16:243
142558992013,5cyclictest177172-21ntpq20:45:203
14255899201,17cyclictest150647-21irqstats19:30:153
14255899200,19cyclictest3056-21PK-Backend23:46:103
14255899200,19cyclictest3056-21PK-Backend23:41:103
14255899200,19cyclictest3056-21PK-Backend23:00:553
14255899200,19cyclictest3056-21PK-Backend21:30:103
14255899200,19cyclictest3056-21PK-Backend20:38:193
14255899200,19cyclictest3056-21PK-Backend20:28:593
14255899200,19cyclictest3056-21PK-Backend20:28:583
14255899200,19cyclictest3056-21PK-Backend20:23:393
14255899200,19cyclictest3056-21PK-Backend00:24:013
14255899200,19cyclictest0-21swapper/319:28:093
14255899200,18cyclictest3056-21PK-Backend23:18:313
14255899200,18cyclictest3056-21PK-Backend22:25:463
14255899200,18cyclictest3056-21PK-Backend20:02:163
14255899200,18cyclictest0-21swapper/320:30:353
142555992018,1cyclictest0-21swapper/222:30:012
142555992018,1cyclictest0-21swapper/222:25:342
142555992018,1cyclictest0-21swapper/222:03:522
142555992018,1cyclictest0-21swapper/221:37:362
142555992018,1cyclictest0-21swapper/220:48:082
142555992018,1cyclictest0-21swapper/220:20:302
142555992018,1cyclictest0-21swapper/219:34:542
142555992018,1cyclictest0-21swapper/219:27:512
14255599200,1cyclictest0-21swapper/222:48:202
14255599200,1cyclictest0-21swapper/219:44:332
14255599200,19cyclictest0-21swapper/223:40:102
14255599200,19cyclictest0-21swapper/220:08:462
14255599200,19cyclictest0-21swapper/219:49:552
14255599200,19cyclictest0-21swapper/219:15:202
14255099203,15cyclictest0-21swapper/119:27:321
142550992015,4cyclictest0-21swapper/119:31:511
14255099200,19cyclictest0-21swapper/119:21:091
142547992018,1cyclictest0-21swapper/023:06:370
142547992018,1cyclictest0-21swapper/022:06:560
142547992018,1cyclictest0-21swapper/019:58:390
142547992013,5cyclictest883-21dbus-daemon00:20:010
142547992012,7cyclictest179143-21sed20:50:230
14254799200,19cyclictest0-21swapper/023:04:490
14254799200,19cyclictest0-21swapper/022:20:350
14254799200,19cyclictest0-21swapper/022:18:450
14254799200,19cyclictest0-21swapper/020:46:180
14254799200,19cyclictest0-21swapper/020:41:320
14254799200,19cyclictest0-21swapper/020:24:140
14254799200,19cyclictest0-21swapper/019:50:560
14254799200,19cyclictest0-21swapper/019:40:280
14255899198,10cyclictest3056-21PK-Backend21:14:163
142558991917,1cyclictest3056-21PK-Backend23:26:573
142558991917,1cyclictest3056-21PK-Backend23:13:313
142558991917,1cyclictest3056-21PK-Backend22:54:533
142558991917,1cyclictest3056-21PK-Backend21:59:443
142558991917,1cyclictest3056-21PK-Backend00:03:473
142558991917,1cyclictest0-21swapper/322:44:393
142558991917,1cyclictest0-21swapper/319:36:003
142558991916,2cyclictest3056-21PK-Backend23:58:473
142558991916,2cyclictest3056-21PK-Backend22:19:443
142558991912,6cyclictest190971-21entropy21:25:143
142558991912,6cyclictest162052-21apt-get20:05:033
14255899190,1cyclictest3056-21PK-Backend23:50:233
14255899190,1cyclictest3056-21PK-Backend22:23:093
14255899190,1cyclictest3056-21PK-Backend21:53:553
14255899190,1cyclictest3056-21PK-Backend21:46:333
14255899190,1cyclictest3056-21PK-Backend21:20:223
14255899190,1cyclictest3056-21PK-Backend20:15:213
14255899190,1cyclictest3056-21PK-Backend20:14:333
14255899190,1cyclictest3056-21PK-Backend19:49:593
14255899190,1cyclictest3056-21PK-Backend00:37:413
14255899190,1cyclictest0-21swapper/322:05:323
14255899190,1cyclictest0-21swapper/300:11:253
14255899190,18cyclictest3056-21PK-Backend22:12:553
14255899190,18cyclictest3056-21PK-Backend19:43:163
14255899190,18cyclictest3056-21PK-Backend00:30:523
14255899190,18cyclictest0-21swapper/323:22:453
14255899190,18cyclictest0-21swapper/323:22:453
142555991917,1cyclictest0-21swapper/223:00:372
142555991915,3cyclictest183126-21sh21:04:592
14255599190,1cyclictest0-21swapper/221:05:592
14255599190,1cyclictest0-21swapper/220:41:462
14255599190,1cyclictest0-21swapper/219:14:332
142547991917,1cyclictest3056-21PK-Backend19:35:210
142547991917,1cyclictest0-21swapper/023:43:090
142547991917,1cyclictest0-21swapper/023:39:440
142547991917,1cyclictest0-21swapper/022:51:580
142547991917,1cyclictest0-21swapper/022:45:560
142547991917,1cyclictest0-21swapper/022:11:560
142547991917,1cyclictest0-21swapper/020:57:000
142547991917,1cyclictest0-21swapper/020:01:030
142547991917,1cyclictest0-21swapper/000:31:080
142547991914,3cyclictest242136-21ntp_offset23:50:200
142547991911,6cyclictest172932-21date20:35:010
14254799190,1cyclictest3056-21PK-Backend23:21:050
14254799190,1cyclictest3056-21PK-Backend23:21:050
14254799190,1cyclictest0-21swapper/023:17:400
14254799190,1cyclictest0-21swapper/022:30:150
14254799190,1cyclictest0-21swapper/020:25:510
14254799190,1cyclictest0-21swapper/020:25:510
14254799190,1cyclictest0-21swapper/019:31:020
14254799190,18cyclictest0-21swapper/022:28:260
14254799190,18cyclictest0-21swapper/019:22:030
142558991816,1cyclictest3056-21PK-Backend20:56:443
142558991815,2cyclictest3056-21PK-Backend22:37:503
14255599182,14cyclictest253203-21grep00:25:192
14255599182,13cyclictest180111-21ls20:55:132
142555991816,1cyclictest0-21swapper/223:36:462
142555991816,1cyclictest0-21swapper/222:21:092
142555991816,1cyclictest0-21swapper/200:00:052
142547991816,1cyclictest0-21swapper/022:58:000
142547991816,1cyclictest0-21swapper/022:43:200
142547991816,1cyclictest0-21swapper/022:01:560
142547991810,6cyclictest883-21dbus-daemon21:03:450
142547991810,6cyclictest194024-21sed21:35:010
142547991810,6cyclictest188916-21awk21:20:110
142547991810,6cyclictest156846-21cron19:50:000
142558991715,1cyclictest3056-21PK-Backend23:08:323
142558991715,1cyclictest3056-21PK-Backend22:31:483
142558991715,1cyclictest3056-21PK-Backend00:26:383
14255899171,3cyclictest184262-21python321:05:223
142555991716,1cyclictest0-21swapper/220:54:162
142555991715,1cyclictest0-21swapper/223:49:372
142555991715,1cyclictest0-21swapper/222:58:002
142555991715,1cyclictest0-21swapper/220:18:542
142555991715,1cyclictest0-21swapper/200:06:062
142555991712,3cyclictest230910-21cpuspeed_turbos23:20:122
142555991712,3cyclictest230910-21cpuspeed_turbos23:20:112
142547991715,1cyclictest0-21swapper/021:49:180
142547991711,5cyclictest13-21ksoftirqd/022:35:210
142558991613,2cyclictest3056-21PK-Backend19:56:553
14254799169,6cyclictest164178-21sed20:10:010
142547991614,1cyclictest0-21swapper/019:25:350
142547991614,1cyclictest0-21swapper/000:25:060
14255899158,5cyclictest203536-21which22:00:243
14255899158,5cyclictest197316-21grep21:45:013
142558991513,1cyclictest3056-21PK-Backend22:56:423
142558991513,1cyclictest3056-21PK-Backend00:07:133
14254799153,10cyclictest186433-21cat21:10:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional