You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 03:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Jun 30, 2026 00:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
400725621480,4sleep13056-21PK-Backend19:55:001
406658421060,4sleep33056-21PK-Backend22:40:193
139099860,68rtkit-daemon0-21swapper/319:09:203
399053127238,27sleep10-21swapper/119:05:411
399057127039,24sleep00-21swapper/019:06:110
399077326441,16sleep20-21swapper/219:08:472
41052442600,7sleep2341ksoftirqd/200:30:012
40040872380,1sleep34004088-21expr19:45:123
40598802360,4sleep04059851-21apt-get22:25:000
399102699332,5cyclictest0-21swapper/222:20:102
3991026992913,5cyclictest34-21ksoftirqd/219:55:172
399102699289,4cyclictest978-21snmpd21:58:032
3991026992811,3cyclictest4054910-21sh22:10:112
399102699279,3cyclictest4080583-21cpuspeed_turbos23:20:112
399102699278,5cyclictest4063128-21sh22:30:182
399102699277,5cyclictest4102034-21cat00:20:152
399102699277,5cyclictest4026882-21date20:50:002
399102699272,5cyclictest4087651-21apt-get23:40:032
399102699268,4cyclictest4007183-21cat19:55:002
399102699267,5cyclictest4049318-21cat21:55:002
399102699267,5cyclictest4001357-21cat19:35:222
399102699267,4cyclictest0-21swapper/220:10:122
399102699261,7cyclictest978-21snmpd20:50:132
399102699261,7cyclictest4039055-21apt21:25:122
3991026992611,3cyclictest4075265-21/usr/sbin/munin23:05:232
3991026992610,3cyclictest1095-21runrttasks21:01:122
40558832250,5sleep10-21swapper/122:10:221
399102699258,5cyclictest3998195-21pool-udisksd19:28:032
399102699257,5cyclictest4028800-21grep20:55:132
399102699248,3cyclictest0-21swapper/222:15:202
3991026992419,4cyclictest0-21swapper/223:26:332
3991026992419,4cyclictest0-21swapper/220:27:472
399102699241,8cyclictest4084127-21cpuspeed_turbos23:30:112
399102699241,10cyclictest4106152-21python300:30:212
3991026992410,3cyclictest1095-21runrttasks20:22:222
399102199241,21cyclictest4069017-21taskset22:46:331
3991028992317,4cyclictest4037083-21systemd-run21:20:003
3991028992312,8cyclictest4069306-21cron22:50:003
399102699239,4cyclictest978-21snmpd21:33:082
399102699239,3cyclictest4071362-21apt-key22:55:012
399102699232,9cyclictest4097338-21grep00:05:242
399102699231,10cyclictest4009299-21apt20:00:132
3991026992310,3cyclictest4091820-21needreboot23:50:182
3991026992310,2cyclictest4011015-21/usr/sbin/munin20:05:112
3991021992318,3cyclictest4074302-21ntp_states23:00:171
3991021992315,6cyclictest1-21systemd23:59:591
399101799232,19cyclictest4014507-21apt20:15:000
3991017992316,5cyclictest4003548-21cron19:45:000
139099230,16rtkit-daemon3056-21PK-Backend00:37:583
139099230,15rtkit-daemon0-21swapper/319:21:533
39986912220,3sleep30-21swapper/319:30:123
3991028992220,1cyclictest3056-21PK-Backend20:40:053
3991028992220,1cyclictest0-21swapper/320:05:453
399102899221,20cyclictest0-21swapper/323:40:193
399102899220,20cyclictest3056-21PK-Backend22:26:113
399102699229,3cyclictest1095-21runrttasks21:43:472
399102699228,3cyclictest978-21snmpd19:45:172
399102699226,14cyclictest0-21swapper/222:30:012
399102699225,4cyclictest0-21swapper/221:15:182
3991021992215,5cyclictest2156-21gnome-shell00:31:091
399102199220,20cyclictest3056-21PK-Backend00:20:011
3991017992220,1cyclictest0-21swapper/022:55:510
3991017992220,1cyclictest0-21swapper/022:54:430
3991017992220,1cyclictest0-21swapper/022:07:590
3991017992220,1cyclictest0-21swapper/020:29:370
3991017992219,2cyclictest0-21swapper/019:16:170
3991017992214,6cyclictest4094663-21cat00:00:010
3991017992210,8cyclictest883-21dbus-daemon20:41:070
139099220,15rtkit-daemon3056-21PK-Backend20:45:133
40656332210,2sleep00-21swapper/022:40:000
3991028992119,1cyclictest3056-21PK-Backend23:52:573
3991028992119,1cyclictest3056-21PK-Backend20:18:363
3991028992119,1cyclictest3056-21PK-Backend19:17:053
3991028992119,1cyclictest0-21swapper/323:10:533
3991028992119,1cyclictest0-21swapper/322:14:083
3991028992119,1cyclictest0-21swapper/321:46:303
3991028992119,1cyclictest0-21swapper/321:36:163
3991028992119,1cyclictest0-21swapper/321:20:343
3991028992119,1cyclictest0-21swapper/320:22:063
3991028992119,1cyclictest0-21swapper/320:04:233
399102899211,19cyclictest0-21swapper/321:10:133
399102899210,20cyclictest0-21swapper/321:09:393
399102899210,19cyclictest3056-21PK-Backend00:11:093
3991026992119,1cyclictest0-21swapper/223:15:312
3991026992119,1cyclictest0-21swapper/222:43:072
3991026992119,1cyclictest0-21swapper/221:46:562
3991026992119,1cyclictest0-21swapper/220:38:152
3991026992119,1cyclictest0-21swapper/220:31:312
3991026992119,1cyclictest0-21swapper/220:18:212
3991026992119,1cyclictest0-21swapper/219:13:052
3991026992119,1cyclictest0-21swapper/200:17:492
3991026992115,5cyclictest4065564-21cron22:40:002
399102699211,10cyclictest4032120-21apt21:05:122
399102699210,20cyclictest0-21swapper/223:55:332
399102699210,20cyclictest0-21swapper/223:01:252
399102699210,20cyclictest0-21swapper/221:23:512
399102699210,20cyclictest0-21swapper/200:39:472
3991021992114,5cyclictest4096156-21sshd00:02:391
3991021992113,6cyclictest883-21dbus-daemon20:50:001
399102199210,20cyclictest0-21swapper/100:25:211
399102199210,19cyclictest3056-21PK-Backend21:15:211
399102199210,19cyclictest3056-21PK-Backend19:30:091
3991017992119,1cyclictest0-21swapper/023:49:310
3991017992119,1cyclictest0-21swapper/021:39:190
3991017992119,1cyclictest0-21swapper/021:11:480
3991017992119,1cyclictest0-21swapper/021:04:520
3991017992119,1cyclictest0-21swapper/020:30:240
3991017992119,1cyclictest0-21swapper/019:49:420
3991017992119,1cyclictest0-21swapper/019:34:000
3991017992119,1cyclictest0-21swapper/000:10:270
3991017992119,1cyclictest0-21swapper/000:07:560
3991017992119,1cyclictest0-21swapper/000:02:220
3991017992118,2cyclictest3056-21PK-Backend22:27:250
3991017992114,4cyclictest4016243-21grep20:20:000
3991017992113,6cyclictest4021764-21awk20:35:110
3991017992112,7cyclictest4069301-21apt-get22:50:000
399101799210,20cyclictest3056-21PK-Backend20:22:200
399101799210,20cyclictest0-21swapper/021:08:030
399101799210,20cyclictest0-21swapper/000:17:090
399102899203,15cyclictest1095-21runrttasks23:45:203
3991028992018,1cyclictest3056-21PK-Backend22:52:413
3991028992018,1cyclictest3056-21PK-Backend20:28:363
3991028992018,1cyclictest0-21swapper/322:21:103
3991028992018,1cyclictest0-21swapper/321:03:513
3991028992012,6cyclictest4075023-21cron23:05:003
399102899200,19cyclictest3056-21PK-Backend20:39:313
399102899200,19cyclictest3056-21PK-Backend20:11:463
399102899200,19cyclictest3056-21PK-Backend19:44:153
399102899200,19cyclictest0-21swapper/323:25:193
399102899200,19cyclictest0-21swapper/322:30:173
399102899200,19cyclictest0-21swapper/322:05:293
399102899200,19cyclictest0-21swapper/320:59:123
399102899200,19cyclictest0-21swapper/319:25:023
399102899200,19cyclictest0-21swapper/300:21:303
3991026992018,1cyclictest0-21swapper/219:32:452
3991026992018,1cyclictest0-21swapper/200:14:462
3991026992015,4cyclictest978-21snmpd00:04:452
399102699200,19cyclictest0-21swapper/222:46:522
399102699200,19cyclictest0-21swapper/221:39:392
399102699200,19cyclictest0-21swapper/221:14:252
399102699200,19cyclictest0-21swapper/220:44:442
3991021992018,1cyclictest3056-21PK-Backend21:45:291
3991021992017,2cyclictest3056-21PK-Backend23:42:091
3991021992017,2cyclictest0-21swapper/123:50:201
3991021992013,5cyclictest4097334-21ntp_kernel_err00:05:241
3991021992013,5cyclictest4014751-21munin-node20:15:091
3991021992012,6cyclictest4072535-21ntp_kernel_pll_22:55:181
399102199200,19cyclictest3056-21PK-Backend21:07:301
399102199200,19cyclictest0-21swapper/122:50:181
399102199200,19cyclictest0-21swapper/122:05:161
399102199200,19cyclictest0-21swapper/120:38:031
399102199200,18cyclictest3056-21PK-Backend23:36:281
399102199200,18cyclictest3056-21PK-Backend23:16:271
399102199200,18cyclictest3056-21PK-Backend20:10:111
399102199200,18cyclictest3056-21PK-Backend19:57:131
3991017992018,1cyclictest3056-21PK-Backend21:55:350
3991017992018,1cyclictest0-21swapper/023:28:560
3991017992018,1cyclictest0-21swapper/022:41:590
3991017992018,1cyclictest0-21swapper/020:05:100
3991017992018,1cyclictest0-21swapper/019:51:510
3991017992018,1cyclictest0-21swapper/019:22:520
3991017992018,1cyclictest0-21swapper/000:31:080
3991017992018,1cyclictest0-21swapper/000:27:300
3991017992013,5cyclictest4044250-21apt21:40:110
3991017992011,7cyclictest4092610-21apt-get23:55:000
399101799200,1cyclictest0-21swapper/021:50:210
399101799200,19cyclictest0-21swapper/021:29:050
399101799200,19cyclictest0-21swapper/019:14:540
139099200,14rtkit-daemon3056-21PK-Backend00:34:023
3991028991917,1cyclictest3056-21PK-Backend00:29:553
3991028991917,1cyclictest0-21swapper/322:19:083
3991028991917,1cyclictest0-21swapper/321:30:283
3991028991917,1cyclictest0-21swapper/321:28:393
3991028991917,1cyclictest0-21swapper/319:50:313
3991028991916,2cyclictest3056-21PK-Backend22:37:333
3991028991912,5cyclictest4080664-21tr23:20:133
3991028991912,5cyclictest4008274-21ntp_kernel_pll_19:55:183
399102899190,1cyclictest3056-21PK-Backend23:58:183
399102899190,1cyclictest3056-21PK-Backend00:00:483
399102899190,18cyclictest3056-21PK-Backend19:12:383
399102899190,18cyclictest0-21swapper/321:59:283
3991026991917,1cyclictest0-21swapper/219:24:202
399102699190,1cyclictest0-21swapper/223:48:492
3991021991917,1cyclictest3056-21PK-Backend20:04:231
3991021991917,1cyclictest0-21swapper/123:05:391
3991021991917,1cyclictest0-21swapper/122:41:181
3991021991917,1cyclictest0-21swapper/122:38:081
3991021991917,1cyclictest0-21swapper/120:43:021
3991021991917,1cyclictest0-21swapper/120:26:131
3991021991917,1cyclictest0-21swapper/119:16:241
3991021991917,1cyclictest0-21swapper/100:11:021
3991021991911,6cyclictest3991354-21expr19:10:131
399102199190,1cyclictest3056-21PK-Backend23:28:101
399102199190,1cyclictest3056-21PK-Backend23:14:111
399102199190,1cyclictest3056-21PK-Backend21:52:451
399102199190,18cyclictest3056-21PK-Backend20:06:191
399102199190,18cyclictest0-21swapper/121:40:011
399102199190,18cyclictest0-21swapper/120:31:201
399102199190,18cyclictest0-21swapper/120:21:201
399102199190,18cyclictest0-21swapper/119:43:481
399101799191,7cyclictest4085424-21turbostat.cron23:35:010
3991017991917,1cyclictest3056-21PK-Backend20:47:550
3991017991917,1cyclictest0-21swapper/023:40:320
3991017991917,1cyclictest0-21swapper/021:17:150
3991017991917,1cyclictest0-21swapper/020:56:410
3991017991917,1cyclictest0-21swapper/020:03:480
3991017991912,6cyclictest4108509-21basename00:40:000
3991017991911,5cyclictest4073717-21grep23:00:120
399101799190,1cyclictest0-21swapper/022:31:310
399101799190,1cyclictest0-21swapper/000:20:140
399101799190,18cyclictest0-21swapper/019:25:350
399102899188,9cyclictest42-21ksoftirqd/323:15:053
3991028991816,1cyclictest3056-21PK-Backend21:40:283
3991028991816,1cyclictest3056-21PK-Backend00:06:563
3991028991816,1cyclictest0-21swapper/321:52:253
399102899180,6cyclictest4086155-21/usr/sbin/munin23:35:153
3991026991816,1cyclictest0-21swapper/219:42:592
3991026991816,1cyclictest0-21swapper/219:18:182
3991021991816,1cyclictest3056-21PK-Backend23:30:261
3991021991816,1cyclictest0-21swapper/122:27:331
3991021991816,1cyclictest0-21swapper/121:10:071
3991021991815,2cyclictest3056-21PK-Backend21:31:301
3991021991813,3cyclictest4043984-21latency_hist21:40:001
3991021991812,5cyclictest4051074-21dpkg22:00:011
3991021991810,6cyclictest1-21systemd22:25:011
399102199180,1cyclictest3056-21PK-Backend21:21:091
399101799189,7cyclictest4058100-21cron22:19:590
3991017991816,1cyclictest0-21swapper/022:13:190
3991017991816,1cyclictest0-21swapper/019:56:110
3991017991815,2cyclictest3056-21PK-Backend21:31:490
3991017991815,2cyclictest0-21swapper/020:51:060
3991017991813,3cyclictest4047460-21idleruntime-cro21:50:000
399102899175,11cyclictest42-21ksoftirqd/319:37:183
399102899171,3cyclictest4100594-21kernelversion00:15:143
399102899170,3cyclictest4084349-21hddtemp_smartct23:30:133
399102699171,7cyclictest4051419-21/usr/sbin/munin22:00:142
3991026991710,5cyclictest4086324-21cat23:35:172
3991021991715,1cyclictest3056-21PK-Backend22:15:441
3991021991712,3cyclictest4107903-21switchtime00:35:221
3991021991711,5cyclictest3996162-21cat19:20:231
399102199170,1cyclictest3056-21PK-Backend21:28:251
399102199170,16cyclictest3056-21PK-Backend19:28:481
3991017991711,5cyclictest4079483-21awk23:15:190
3991017991710,5cyclictest4000417-21awk19:35:120
139099170,9rtkit-daemon4027948-21sed20:50:243
3991028991613,2cyclictest3056-21PK-Backend20:31:343
399102199169,5cyclictest4051841-21which22:00:161
399102199168,6cyclictest4031334-21wc21:00:221
3991021991614,1cyclictest3056-21PK-Backend20:58:041
3991021991612,3cyclictest4004707-21switchtime19:45:201
3991021991610,4cyclictest4027600-21cut20:50:201
399101799162,12cyclictest4051409-21apt-get22:00:030
3991017991610,4cyclictest4077650-21missed_timers23:10:160
3991026991513,1cyclictest0-21swapper/222:07:042
3991021991511,3cyclictest4100249-21tr00:15:101
399101799158,5cyclictest4085863-21tr23:35:110
3991017991510,3cyclictest1095-21runrttasks23:09:300
139099150,4rtkit-daemon3056-21PK-Backend22:00:573
3991028991412,1cyclictest3056-21PK-Backend23:05:113
399102899140,12cyclictest3056-21PK-Backend22:55:073
399102699147,5cyclictest436-20systemd-journal22:55:042
3991026991410,3cyclictest1095-21runrttasks23:10:322
399102199149,4cyclictest3056-21PK-Backend19:37:521
3991021991411,2cyclictest3056-21PK-Backend22:34:021
399101799147,5cyclictest4081526-21wc23:20:220
3991017991410,3cyclictest4038789-21ls21:25:000
399102199139,2cyclictest4090884-21latency_hist23:50:011
399102199110,9cyclictest3056-21PK-Backend23:25:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional