You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-15 - 22:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Mon Jun 15, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
398621721230,5sleep1251rcuc/123:45:131
38878892870,8sleep20-21swapper/219:05:242
388706327447,17sleep10-21swapper/119:05:121
388815727041,23sleep00-21swapper/019:05:310
388824926739,22sleep30-21swapper/319:06:433
40003322480,3sleep20-21swapper/200:25:152
40003322480,3sleep20-21swapper/200:25:152
39590042460,3chrt0-21swapper/122:25:221
39249032420,3sleep00-21swapper/020:50:120
39795082360,11sleep013-21ksoftirqd/023:25:150
39795082360,11sleep013-21ksoftirqd/023:25:140
39424002330,2sleep00-21swapper/021:40:130
39844782320,9sleep234-21ksoftirqd/223:40:132
388868799308,7cyclictest436-20systemd-journal19:40:001
39008092280,5sleep342-21ksoftirqd/319:40:183
39395382270,5sleep20-21swapper/221:30:212
388868799278,5cyclictest3952165-21seq22:05:321
3888687992710,6cyclictest436-20systemd-journal00:25:001
3888691992612,12cyclictest3901512-21unin-run19:45:002
388868799264,5cyclictest3963478-21apt-get22:40:051
3888687992611,3cyclictest3988421-21needreboot23:50:181
388868799259,4cyclictest3907648-21awk20:00:191
388868799258,4cyclictest436-20systemd-journal00:19:561
388868799257,6cyclictest3922538-21cron20:45:001
388869199240,22cyclictest3952937-21tr22:10:122
388868799248,6cyclictest3955332-21ntpq22:15:201
388868799242,9cyclictest3990058-21sort23:55:171
139099240,16rtkit-daemon3056-21PK-Backend22:03:332
388869599236,4cyclictest4002681-21wc00:30:213
3888695992313,6cyclictest0-21swapper/322:00:103
388868799238,6cyclictest883-21dbus-daemon00:05:011
388868799232,6cyclictest3901752-21apt19:45:121
3888687992318,4cyclictest3056-21PK-Backend23:12:011
3888683992318,3cyclictest3921822-21ntp_states20:40:190
3888683992316,6cyclictest883-21dbus-daemon20:32:450
3888683992316,6cyclictest883-21dbus-daemon20:32:440
3888683992316,5cyclictest3985761-21cron23:45:000
39597282220,3sleep3411rcuc/322:30:013
388869599225,15cyclictest3917629-21cpu20:30:123
388869599225,15cyclictest3917629-21cpu20:30:123
3888695992220,1cyclictest0-21swapper/322:21:383
3888695992219,2cyclictest3056-21PK-Backend20:12:473
3888695992217,3cyclictest3932237-21/usr/sbin/munin21:10:203
388869599220,2cyclictest3960774-21cat22:30:223
388869599220,20cyclictest3056-21PK-Backend20:06:523
3888691992220,1cyclictest0-21swapper/220:29:422
3888691992220,1cyclictest0-21swapper/220:29:422
3888691992220,1cyclictest0-21swapper/200:21:402
388869199220,21cyclictest0-21swapper/223:49:022
388869199220,21cyclictest0-21swapper/220:31:172
388869199220,21cyclictest0-21swapper/220:31:172
388868799226,6cyclictest3941872-21cron21:40:001
388868799224,4cyclictest0-21swapper/123:05:141
388868799220,20cyclictest0-21swapper/122:55:111
3888683992220,1cyclictest0-21swapper/023:30:470
3888683992220,1cyclictest0-21swapper/023:16:130
3888683992220,1cyclictest0-21swapper/021:29:260
139099220,16rtkit-daemon3056-21PK-Backend21:05:362
139099220,15rtkit-daemon0-21swapper/222:06:542
139099220,14rtkit-daemon0-21swapper/323:18:313
139099220,14rtkit-daemon0-21swapper/222:27:472
139099220,14rtkit-daemon0-21swapper/221:43:342
388869599215,14cyclictest3973946-21cpu23:10:113
3888695992119,1cyclictest3056-21PK-Backend23:48:103
3888695992119,1cyclictest3056-21PK-Backend00:07:033
3888695992119,1cyclictest0-21swapper/323:21:473
3888695992119,1cyclictest0-21swapper/300:37:313
3888695992114,4cyclictest3891985-21needreboot19:15:173
3888695992113,6cyclictest3917075-21perf20:30:003
3888695992113,6cyclictest3917075-21perf20:30:003
388869599210,2cyclictest39361532sleep321:21:563
388869599210,2cyclictest0-21swapper/321:34:123
388869599210,20cyclictest0-21swapper/300:25:353
388869599210,20cyclictest0-21swapper/300:25:353
388869599210,1cyclictest3056-21PK-Backend19:34:493
388869599210,19cyclictest3056-21PK-Backend00:24:333
388869199213,17cyclictest1095-21runrttasks21:28:472
3888691992119,1cyclictest0-21swapper/221:15:112
3888691992119,1cyclictest0-21swapper/219:22:162
3888691992119,1cyclictest0-21swapper/200:03:552
388869199210,20cyclictest0-21swapper/223:50:512
388869199210,20cyclictest0-21swapper/222:48:532
388869199210,20cyclictest0-21swapper/220:39:482
388869199210,20cyclictest0-21swapper/219:35:272
388869199210,19cyclictest0-21swapper/219:10:462
388868799214,5cyclictest0-21swapper/121:30:001
388868799213,6cyclictest3906079-21python319:55:211
388868799211,7cyclictest3926282-21apt20:55:121
3888687992110,3cyclictest1095-21runrttasks00:07:261
388868799210,20cyclictest0-21swapper/121:57:391
388868799210,19cyclictest3056-21PK-Backend20:37:151
388868799210,15cyclictest3056-21PK-Backend21:44:001
3888683992119,1cyclictest0-21swapper/023:45:330
3888683992119,1cyclictest0-21swapper/020:24:440
3888683992119,1cyclictest0-21swapper/020:16:330
3888683992119,1cyclictest0-21swapper/020:13:290
3888683992119,1cyclictest0-21swapper/019:58:010
3888683992119,1cyclictest0-21swapper/019:13:330
3888683992119,1cyclictest0-21swapper/000:16:150
388868399210,20cyclictest0-21swapper/022:41:460
388868399210,20cyclictest0-21swapper/022:25:500
388868399210,20cyclictest0-21swapper/021:24:130
388868399210,20cyclictest0-21swapper/019:38:060
388868399210,1cyclictest0-21swapper/022:54:300
388868399210,19cyclictest0-21swapper/021:50:020
139099210,15rtkit-daemon3056-21PK-Backend21:00:293
139099210,15rtkit-daemon0-21swapper/222:51:072
139099210,15rtkit-daemon0-21swapper/220:01:022
139099210,14rtkit-daemon3056-21PK-Backend23:03:323
139099210,14rtkit-daemon0-21swapper/321:38:273
139099210,14rtkit-daemon0-21swapper/319:22:253
139099210,14rtkit-daemon0-21swapper/222:40:162
3888695992018,1cyclictest3056-21PK-Backend23:50:463
3888695992018,1cyclictest3056-21PK-Backend19:53:413
3888695992018,1cyclictest3056-21PK-Backend00:16:283
3888695992018,1cyclictest0-21swapper/319:26:313
3888695992014,4cyclictest3995840-21sshd00:10:343
3888695992014,4cyclictest3984476-21tr23:40:133
3888695992010,7cyclictest3906665-21cron20:00:003
388869599200,19cyclictest3056-21PK-Backend22:19:153
388869599200,19cyclictest3056-21PK-Backend00:04:263
388869599200,19cyclictest0-21swapper/323:32:223
388869599200,19cyclictest0-21swapper/321:27:103
3888691992018,1cyclictest0-21swapper/223:32:592
3888691992018,1cyclictest0-21swapper/223:16:382
3888691992018,1cyclictest0-21swapper/223:08:262
3888691992018,1cyclictest0-21swapper/221:50:402
3888691992018,1cyclictest0-21swapper/221:00:102
3888691992018,1cyclictest0-21swapper/220:56:452
3888691992018,1cyclictest0-21swapper/200:38:302
3888691992018,1cyclictest0-21swapper/200:34:382
3888691992018,1cyclictest0-21swapper/200:17:012
3888691992014,4cyclictest978-21snmpd21:47:042
388869199200,19cyclictest0-21swapper/223:59:292
388869199200,19cyclictest0-21swapper/221:22:002
388869199200,19cyclictest0-21swapper/219:31:352
388869199200,19cyclictest0-21swapper/219:18:362
388869199200,19cyclictest0-21swapper/200:11:472
3888687992018,1cyclictest0-21swapper/100:25:151
3888687992018,1cyclictest0-21swapper/100:25:151
3888687992017,2cyclictest0-21swapper/119:52:271
388868799201,6cyclictest3910411-21apt20:10:111
3888687992014,5cyclictest3934495-21taskset21:17:501
388868799200,1cyclictest3056-21PK-Backend00:38:321
388868799200,18cyclictest0-21swapper/122:35:581
3888683992018,1cyclictest3056-21PK-Backend00:37:180
3888683992018,1cyclictest0-21swapper/022:35:240
3888683992018,1cyclictest0-21swapper/021:48:260
3888683992018,1cyclictest0-21swapper/021:38:190
3888683992018,1cyclictest0-21swapper/020:45:060
3888683992018,1cyclictest0-21swapper/020:37:560
3888683992018,1cyclictest0-21swapper/020:07:330
3888683992018,1cyclictest0-21swapper/019:52:330
3888683992018,1cyclictest0-21swapper/019:43:480
3888683992018,1cyclictest0-21swapper/019:34:290
3888683992018,1cyclictest0-21swapper/019:23:200
3888683992018,1cyclictest0-21swapper/000:02:370
3888683992017,2cyclictest3928222-21cstates21:00:130
3888683992017,2cyclictest3056-21PK-Backend22:30:360
3888683992013,5cyclictest3971897-21cron23:05:010
388868399200,1cyclictest0-21swapper/000:25:220
388868399200,1cyclictest0-21swapper/000:25:220
388868399200,19cyclictest0-21swapper/022:22:400
388868399200,19cyclictest0-21swapper/021:15:480
139099200,14rtkit-daemon3056-21PK-Backend22:54:153
139099200,14rtkit-daemon3056-21PK-Backend21:16:463
139099200,14rtkit-daemon3056-21PK-Backend20:10:132
139099200,14rtkit-daemon3056-21PK-Backend19:58:502
139099200,14rtkit-daemon0-21swapper/223:22:502
139099200,14rtkit-daemon0-21swapper/222:55:522
139099200,14rtkit-daemon0-21swapper/222:17:392
139099200,14rtkit-daemon0-21swapper/219:52:222
139099200,14rtkit-daemon0-21swapper/219:47:262
139099200,13rtkit-daemon0-21swapper/220:40:332
139099200,13rtkit-daemon0-21swapper/220:05:242
3888695991917,1cyclictest3056-21PK-Backend22:13:543
3888695991917,1cyclictest3056-21PK-Backend19:35:503
3888695991917,1cyclictest0-21swapper/320:38:163
388869599191,4cyclictest3969573-21python322:55:213
3888695991914,3cyclictest3982971-21hddtemp_smartct23:35:153
3888695991912,5cyclictest3889074-21seq19:10:043
3888695991911,6cyclictest436-20systemd-journal21:55:003
388869599190,18cyclictest3973434-21sshd23:07:363
388869599190,18cyclictest0-21swapper/320:00:103
3888691991917,1cyclictest0-21swapper/219:28:442
3888691991913,4cyclictest3915218-21sshd20:23:352
3888691991912,5cyclictest3957286-21sed22:20:222
3888691991911,6cyclictest3960758-21sed22:30:232
388869199190,1cyclictest0-21swapper/200:06:332
3888687991917,1cyclictest0-21swapper/123:00:521
3888687991912,5cyclictest3967421-21cat22:50:141
3888687991912,1cyclictest0-21swapper/121:05:411
388868799190,1cyclictest0-21swapper/120:45:391
388868799190,18cyclictest3056-21PK-Backend20:31:131
388868799190,18cyclictest3056-21PK-Backend20:31:131
388868799190,18cyclictest0-21swapper/122:45:591
3888683991917,1cyclictest0-21swapper/023:52:220
3888683991917,1cyclictest0-21swapper/022:15:440
3888683991917,1cyclictest0-21swapper/022:14:080
3888683991917,1cyclictest0-21swapper/019:16:030
3888683991917,1cyclictest0-21swapper/000:30:010
3888683991914,3cyclictest3973637-21sh23:10:010
388868399190,1cyclictest0-21swapper/022:57:400
388868399190,1cyclictest0-21swapper/022:48:560
388868399190,1cyclictest0-21swapper/021:10:480
388868399190,1cyclictest0-21swapper/020:00:510
388868399190,18cyclictest0-21swapper/000:06:020
139099190,14rtkit-daemon0-21swapper/321:49:143
139099190,13rtkit-daemon0-21swapper/220:16:582
3888695991816,1cyclictest3056-21PK-Backend20:42:153
3888695991816,1cyclictest0-21swapper/321:42:383
3888695991816,1cyclictest0-21swapper/320:45:263
3888695991815,2cyclictest3056-21PK-Backend19:45:103
3888695991812,4cyclictest1095-21runrttasks23:58:033
3888695991811,5cyclictest436-20systemd-journal21:59:553
3888695991811,5cyclictest42-21ksoftirqd/322:40:193
388869599181,15cyclictest3980568-21perf23:29:593
388869599181,15cyclictest3980568-21perf23:29:593
3888691991816,1cyclictest0-21swapper/221:11:462
3888691991815,2cyclictest0-21swapper/221:37:082
3888691991814,3cyclictest978-21snmpd21:58:312
3888691991813,3cyclictest978-21snmpd23:28:582
3888691991813,3cyclictest978-21snmpd23:28:582
3888691991811,5cyclictest3961794-21expr22:35:122
3888691991811,5cyclictest3926152-21sed20:55:012
3888691991810,6cyclictest3971894-21cron23:05:012
388868799189,7cyclictest3896284-21sh19:30:001
3888687991816,1cyclictest3056-21PK-Backend20:25:191
3888687991816,1cyclictest3056-21PK-Backend20:25:181
3888687991816,1cyclictest3056-21PK-Backend19:13:331
388868799181,4cyclictest26-21ksoftirqd/123:15:211
3888687991811,5cyclictest4002726-21cat00:30:201
3888683991816,1cyclictest0-21swapper/022:00:220
3888683991816,1cyclictest0-21swapper/000:21:430
3888683991812,4cyclictest3902929-21sshd19:46:340
3888683991811,5cyclictest436-20systemd-journal20:57:330
388868399180,1cyclictest0-21swapper/021:32:040
388869599178,7cyclictest3914342-21cat20:20:163
388869599170,15cyclictest3951462-21hddtemp_smartct22:05:163
388869199171,12cyclictest3973906-21apt23:10:092
388868799178,4cyclictest3928857-21awk21:00:201
3888687991715,1cyclictest0-21swapper/123:27:291
3888687991715,1cyclictest0-21swapper/123:27:291
3888687991714,2cyclictest3056-21PK-Backend21:12:441
3888687991710,6cyclictest3956371-21awk22:20:111
3888687991710,5cyclictest436-20systemd-journal22:12:341
3888683991715,1cyclictest3056-21PK-Backend21:56:230
3888683991715,1cyclictest0-21swapper/000:12:510
3888683991713,3cyclictest3991003-21idleruntime-cro00:00:010
3888683991712,4cyclictest3917374-21apt-config20:30:010
3888683991712,4cyclictest3917374-21apt-config20:30:010
3888695991614,1cyclictest3056-21PK-Backend20:15:523
388869199169,6cyclictest3982814-21expr23:35:122
388868799168,6cyclictest3935241-21cat21:20:111
3888687991612,3cyclictest3912339-21cstates20:15:141
3888687991610,5cyclictest3901015-21cat19:40:211
388868399161,12cyclictest3929746-21apt21:05:120
3888683991610,5cyclictest3977970-21cat23:20:180
3888683991610,5cyclictest3951465-21which22:05:160
388869599153,8cyclictest3961517-21apt22:35:093
3888695991511,3cyclictest1095-21runrttasks21:07:413
3888691991510,3cyclictest3923423-21missed_timers20:45:162
388868799159,5cyclictest3981013-21cut23:30:111
388868799157,6cyclictest3908886-21cat20:05:141
3888687991512,2cyclictest3056-21PK-Backend00:11:221
3888687991511,3cyclictest3056-21PK-Backend21:45:071
3888687991511,3cyclictest1095-21runrttasks21:30:031
3888687991510,4cyclictest3925780-21seq20:52:141
3888687991510,3cyclictest3898014-21sh19:35:001
388868399159,4cyclictest3894937-21tr19:25:100
3888683991513,1cyclictest0-21swapper/023:10:390
3888683991511,3cyclictest1095-21runrttasks23:35:150
3888695991411,2cyclictest3927591-21chrt20:58:133
3888695991410,3cyclictest3056-21PK-Backend20:54:013
388868799147,5cyclictest3893831-21open_files19:20:191
3888687991412,1cyclictest3056-21PK-Backend21:52:111
3888687991411,2cyclictest3056-21PK-Backend23:35:541
3888687991410,3cyclictest3056-21PK-Backend22:00:031
3888687991410,3cyclictest3056-21PK-Backend20:23:031
3888687991410,3cyclictest1095-21runrttasks23:41:111
3888687991410,2cyclictest3960556-21open_inodes22:30:211
388868799140,4cyclictest3056-21PK-Backend23:22:351
388869599138,3cyclictest3966218-21timerandwakeup22:45:223
3888687991311,1cyclictest3056-21PK-Backend19:18:471
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional