You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 08:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Apr 24, 2026 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
386287721140,13sleep126-21ksoftirqd/121:20:141
389090921040,3sleep342-21ksoftirqd/322:40:163
381473728253,21sleep00-21swapper/019:05:150
381563427938,14sleep20-21swapper/219:07:472
381570726343,14sleep10-21swapper/119:08:431
381490026142,12sleep30-21swapper/319:05:183
38290232400,1sleep20-21swapper/219:45:122
38184862380,2sleep20-21swapper/219:15:122
38890562350,2sleep23056-21PK-Backend22:35:142
38890562350,2sleep23056-21PK-Backend22:35:142
3815950992621,3cyclictest3833697-21dump-pmu-power20:00:000
139099240,18rtkit-daemon3056-21PK-Backend00:35:161
39070262230,8sleep234-21ksoftirqd/223:25:182
3815958992220,1cyclictest0-21swapper/223:48:542
3815958992220,1cyclictest0-21swapper/223:20:082
3815958992220,1cyclictest0-21swapper/221:34:512
3815958992214,6cyclictest3930404-21sshd00:32:242
3815958992214,6cyclictest3930404-21sshd00:32:242
3815950992220,1cyclictest0-21swapper/023:55:580
3815950992220,1cyclictest0-21swapper/023:28:200
3815950992220,1cyclictest0-21swapper/022:58:530
3815950992219,2cyclictest3056-21PK-Backend00:30:190
3815950992219,2cyclictest3056-21PK-Backend00:30:190
3815950992213,7cyclictest3873889-21ntp_states21:50:200
3815950992211,3cyclictest3825035-21cron19:34:590
381595099220,21cyclictest0-21swapper/000:04:370
39221692210,2sleep20-21swapper/200:10:012
38506122210,11sleep234-21ksoftirqd/220:45:222
3815963992118,2cyclictest0-21swapper/319:15:193
3815963992113,6cyclictest3837163-21cron20:10:013
381596399210,20cyclictest0-21swapper/319:29:193
381596399210,19cyclictest3056-21PK-Backend20:37:273
3815958992119,1cyclictest0-21swapper/223:43:202
3815958992119,1cyclictest0-21swapper/223:39:012
3815958992119,1cyclictest0-21swapper/223:39:012
3815958992119,1cyclictest0-21swapper/223:11:572
3815958992119,1cyclictest0-21swapper/223:07:512
3815958992119,1cyclictest0-21swapper/223:00:412
3815958992119,1cyclictest0-21swapper/221:19:222
3815958992119,1cyclictest0-21swapper/221:11:452
3815958992119,1cyclictest0-21swapper/221:08:212
3815958992119,1cyclictest0-21swapper/220:34:542
3815958992119,1cyclictest0-21swapper/220:22:102
3815958992119,1cyclictest0-21swapper/220:12:032
3815958992119,1cyclictest0-21swapper/220:06:352
3815958992119,1cyclictest0-21swapper/219:20:402
3815958992119,1cyclictest0-21swapper/200:02:192
3815958992118,2cyclictest0-21swapper/222:07:552
3815958992114,5cyclictest3925418-21cron00:20:002
381595899210,20cyclictest0-21swapper/223:34:412
3815955992118,2cyclictest3056-21PK-Backend19:54:291
3815955992113,6cyclictest3905310-21ntp_kernel_err23:20:171
381595599210,20cyclictest0-21swapper/100:11:571
381595599210,1cyclictest3056-21PK-Backend21:56:441
381595599210,19cyclictest3056-21PK-Backend21:50:431
381595599210,19cyclictest3056-21PK-Backend21:25:481
381595599210,19cyclictest3056-21PK-Backend19:21:441
3815950992119,1cyclictest0-21swapper/023:53:420
3815950992119,1cyclictest0-21swapper/023:15:160
3815950992119,1cyclictest0-21swapper/023:13:140
3815950992119,1cyclictest0-21swapper/023:09:070
3815950992119,1cyclictest0-21swapper/022:46:360
3815950992119,1cyclictest0-21swapper/022:42:320
3815950992119,1cyclictest0-21swapper/022:06:480
3815950992119,1cyclictest0-21swapper/021:44:240
3815950992119,1cyclictest0-21swapper/021:35:460
3815950992119,1cyclictest0-21swapper/021:35:460
3815950992119,1cyclictest0-21swapper/021:27:550
3815950992119,1cyclictest0-21swapper/021:16:530
3815950992119,1cyclictest0-21swapper/021:12:400
3815950992119,1cyclictest0-21swapper/020:57:140
3815950992119,1cyclictest0-21swapper/020:44:080
3815950992119,1cyclictest0-21swapper/020:44:080
3815950992119,1cyclictest0-21swapper/020:36:310
3815950992119,1cyclictest0-21swapper/019:46:350
3815950992119,1cyclictest0-21swapper/019:28:580
3815950992119,1cyclictest0-21swapper/019:20:530
3815950992119,1cyclictest0-21swapper/000:36:410
3815950992119,1cyclictest0-21swapper/000:18:360
3815950992113,6cyclictest3925748-21awk00:20:120
381595099210,20cyclictest3856488-21apt-get21:05:030
381595099210,20cyclictest0-21swapper/022:30:410
381595099210,20cyclictest0-21swapper/022:18:460
381595099210,20cyclictest0-21swapper/021:30:320
381595099210,20cyclictest0-21swapper/020:30:300
381595099210,20cyclictest0-21swapper/019:16:210
381595099210,20cyclictest0-21swapper/000:11:130
139099210,14rtkit-daemon3056-21PK-Backend21:20:052
139099210,14rtkit-daemon0-21swapper/200:11:282
3815963992018,1cyclictest3056-21PK-Backend22:17:573
3815963992018,1cyclictest0-21swapper/321:29:243
3815963992017,2cyclictest3056-21PK-Backend22:27:513
3815963992015,4cyclictest3824512-21find19:30:203
3815963992013,5cyclictest3924674-21awk00:15:163
3815963992013,4cyclictest3863350-21python321:20:203
381596399201,18cyclictest3918857-21df_inode00:00:133
381596399201,16cyclictest3841773-21python320:20:213
381596399200,19cyclictest3056-21PK-Backend23:18:213
381596399200,19cyclictest3056-21PK-Backend22:46:573
381596399200,19cyclictest3056-21PK-Backend22:14:193
381596399200,19cyclictest3056-21PK-Backend21:12:343
381596399200,19cyclictest3056-21PK-Backend20:34:023
381596399200,19cyclictest3056-21PK-Backend00:20:463
381596399200,19cyclictest0-21swapper/323:51:123
381596399200,19cyclictest0-21swapper/323:40:173
381596399200,19cyclictest0-21swapper/323:23:143
381596399200,19cyclictest0-21swapper/321:45:063
381596399200,19cyclictest0-21swapper/320:47:403
381596399200,19cyclictest0-21swapper/319:12:233
3815958992018,1cyclictest0-21swapper/223:17:042
3815958992018,1cyclictest0-21swapper/222:19:042
3815958992018,1cyclictest0-21swapper/222:10:462
3815958992018,1cyclictest0-21swapper/221:01:172
3815958992018,1cyclictest0-21swapper/220:56:512
3815958992018,1cyclictest0-21swapper/219:59:192
3815958992018,1cyclictest0-21swapper/219:35:122
3815958992018,1cyclictest0-21swapper/219:11:472
3815958992018,1cyclictest0-21swapper/200:39:102
3815958992013,5cyclictest3868408-21ntpq21:35:192
3815958992013,5cyclictest3868408-21ntpq21:35:192
3815958992011,7cyclictest3823084-21sshd19:27:242
381595899200,1cyclictest0-21swapper/221:54:582
381595899200,1cyclictest0-21swapper/220:37:312
381595899200,19cyclictest0-21swapper/223:51:112
381595899200,19cyclictest0-21swapper/222:32:022
381595899200,19cyclictest0-21swapper/221:56:482
381595899200,19cyclictest0-21swapper/221:47:072
381595899200,19cyclictest0-21swapper/220:17:302
381595899200,19cyclictest0-21swapper/219:52:512
381595599201,9cyclictest3907812-21idleruntime-cro23:30:001
3815955992018,1cyclictest0-21swapper/123:17:421
3815955992018,1cyclictest0-21swapper/123:01:341
3815955992018,1cyclictest0-21swapper/122:30:311
3815955992017,2cyclictest3056-21PK-Backend21:31:501
3815955992017,2cyclictest3056-21PK-Backend20:56:081
3815955992017,2cyclictest3056-21PK-Backend19:58:091
3815955992014,4cyclictest2019-21gmain23:15:001
3815955992013,6cyclictest3848913-21awk20:40:211
3815955992013,6cyclictest3848913-21awk20:40:211
3815955992013,6cyclictest3846485-21diskstats20:35:141
3815955992012,6cyclictest3856488-21apt-get21:05:001
381595599200,1cyclictest3056-21PK-Backend20:09:371
381595599200,1cyclictest3056-21PK-Backend19:12:051
381595599200,19cyclictest0-21swapper/123:58:591
381595599200,19cyclictest0-21swapper/122:55:251
381595599200,19cyclictest0-21swapper/122:20:231
381595599200,19cyclictest0-21swapper/122:16:451
381595599200,19cyclictest0-21swapper/121:18:591
381595599200,19cyclictest0-21swapper/121:13:591
381595599200,18cyclictest3056-21PK-Backend20:50:061
3815950992018,1cyclictest0-21swapper/023:48:070
3815950992018,1cyclictest0-21swapper/023:41:320
3815950992018,1cyclictest0-21swapper/023:38:140
3815950992018,1cyclictest0-21swapper/023:38:140
3815950992018,1cyclictest0-21swapper/023:30:370
3815950992018,1cyclictest0-21swapper/022:25:140
3815950992018,1cyclictest0-21swapper/021:46:130
3815950992018,1cyclictest0-21swapper/020:51:590
3815950992018,1cyclictest0-21swapper/020:03:130
3815950992018,1cyclictest0-21swapper/019:52:170
3815950992018,1cyclictest0-21swapper/000:06:530
3815950992017,2cyclictest1788620-21WebKitWebProces19:37:170
381595099200,1cyclictest0-21swapper/022:14:050
381595099200,19cyclictest0-21swapper/022:00:190
381595099200,19cyclictest0-21swapper/020:46:450
381595099200,19cyclictest0-21swapper/020:20:220
381595099200,19cyclictest0-21swapper/020:05:020
381595099200,19cyclictest0-21swapper/019:11:150
381595099200,18cyclictest3898418-21ntp_states23:00:180
381596399194,13cyclictest3820389-21ls19:20:103
3815963991917,1cyclictest3056-21PK-Backend22:34:063
3815963991917,1cyclictest3056-21PK-Backend21:19:103
3815963991917,1cyclictest3056-21PK-Backend21:05:103
3815963991917,1cyclictest0-21swapper/323:28:213
3815963991916,2cyclictest3056-21PK-Backend21:02:343
3815963991914,3cyclictest3897411-21sh23:00:003
3815963991913,4cyclictest3930626-21turbostat00:35:003
3815963991913,4cyclictest3930626-21turbostat00:35:003
3815963991912,5cyclictest3928146-21ntp_kernel_pll_00:25:193
3815963991912,5cyclictest3910870-21cat23:35:233
3815963991912,5cyclictest3910870-21cat23:35:233
3815963991912,5cyclictest3868235-21ls21:35:173
3815963991912,5cyclictest3868235-21ls21:35:173
3815963991912,5cyclictest3842915-21cat20:25:133
3815963991911,6cyclictest3873548-21which21:50:163
381596399191,16cyclictest3839546-21df_inode20:15:133
3815963991911,6cyclictest3833129-21cat19:55:213
381596399191,16cyclictest3826171-21python319:35:213
381596399190,1cyclictest0-21swapper/322:35:073
381596399190,1cyclictest0-21swapper/322:35:073
381596399190,1cyclictest0-21swapper/321:43:163
381596399190,1cyclictest0-21swapper/300:35:063
381596399190,18cyclictest3056-21PK-Backend22:24:123
381596399190,18cyclictest0-21swapper/321:55:333
381596399190,18cyclictest0-21swapper/320:04:073
3815958991917,1cyclictest0-21swapper/220:41:582
3815958991917,1cyclictest0-21swapper/220:41:572
381595899190,1cyclictest0-21swapper/222:02:142
381595899190,1cyclictest0-21swapper/221:26:112
3815955991917,1cyclictest3056-21PK-Backend21:48:061
3815955991917,1cyclictest3056-21PK-Backend21:42:041
3815955991917,1cyclictest3056-21PK-Backend19:43:211
3815955991917,1cyclictest3056-21PK-Backend19:36:041
3815955991917,1cyclictest3056-21PK-Backend19:19:211
3815955991917,1cyclictest0-21swapper/122:41:191
3815955991917,1cyclictest0-21swapper/122:36:131
3815955991917,1cyclictest0-21swapper/122:36:121
3815955991917,1cyclictest0-21swapper/122:05:511
3815955991917,1cyclictest0-21swapper/121:08:111
3815955991917,1cyclictest0-21swapper/100:15:151
3815955991917,1cyclictest0-21swapper/100:06:361
3815955991917,1cyclictest0-21swapper/100:01:151
3815955991912,5cyclictest3893201-21ntpq22:45:201
3815955991912,5cyclictest3880846-21pluginstate22:10:201
3815955991912,5cyclictest3844650-21expr20:30:121
3815955991911,6cyclictest3894601-21fw_packets22:50:131
381595599190,1cyclictest3056-21PK-Backend21:39:271
381595599190,1cyclictest3056-21PK-Backend21:39:271
381595599190,1cyclictest3056-21PK-Backend20:15:041
381595599190,1cyclictest3056-21PK-Backend19:46:591
381595599190,1cyclictest3056-21PK-Backend19:30:381
381595599190,18cyclictest0-21swapper/122:02:591
3815950991917,1cyclictest0-21swapper/022:50:420
3815950991917,1cyclictest0-21swapper/022:39:260
3815950991917,1cyclictest0-21swapper/022:39:260
3815950991917,1cyclictest0-21swapper/022:21:350
3815950991917,1cyclictest0-21swapper/021:20:190
3815950991917,1cyclictest0-21swapper/019:43:460
381595099190,1cyclictest0-21swapper/023:22:120
381595099190,1cyclictest0-21swapper/020:29:290
139099190,13rtkit-daemon0-21swapper/220:27:242
381596399185,4cyclictest3827827-21which19:40:203
3815963991816,1cyclictest3056-21PK-Backend00:09:183
3815963991816,1cyclictest0-21swapper/320:57:213
3815963991812,4cyclictest3849476-21date20:45:003
3815963991812,4cyclictest3849476-21date20:45:003
381596399181,15cyclictest3876982-21diskstats22:00:133
3815963991811,5cyclictest3838486-21ls20:10:223
381596399180,6cyclictest3851773-21cat20:50:123
381596399180,5cyclictest3866242-21cstates21:30:143
3815958991816,1cyclictest0-21swapper/222:20:052
3815958991816,1cyclictest0-21swapper/221:44:302
381595899181,13cyclictest894-21irqbalance19:40:202
3815958991810,6cyclictest3894927-21awk22:50:172
3815955991816,1cyclictest3056-21PK-Backend23:45:001
3815955991816,1cyclictest3056-21PK-Backend23:39:391
3815955991816,1cyclictest3056-21PK-Backend23:39:391
3815955991816,1cyclictest3056-21PK-Backend20:23:101
3815955991816,1cyclictest3056-21PK-Backend20:10:381
3815955991816,1cyclictest3056-21PK-Backend20:02:341
3815955991816,1cyclictest0-21swapper/123:52:371
3815955991816,1cyclictest0-21swapper/122:25:031
3815955991816,1cyclictest0-21swapper/100:30:021
3815955991816,1cyclictest0-21swapper/100:30:021
3815950991816,1cyclictest0-21swapper/021:55:530
3815950991816,1cyclictest0-21swapper/020:18:330
3815950991816,1cyclictest0-21swapper/020:14:540
381596399179,6cyclictest3913979-21ntpq23:45:163
381596399172,13cyclictest3901791-21needreboot23:10:183
3815963991715,1cyclictest0-21swapper/323:05:433
3815963991714,2cyclictest3056-21PK-Backend23:00:493
381596399171,14cyclictest3829630-21python319:45:193
3815963991710,5cyclictest42-21ksoftirqd/300:00:003
381596399170,15cyclictest3908079-21apt23:30:123
381596399170,15cyclictest3830504-21apt19:50:113
381595899179,6cyclictest3824976-21sshd19:34:242
381595899179,6cyclictest2330-21gdbus20:03:152
381595899172,10cyclictest2156-21gnome-shell22:48:162
3815958991715,1cyclictest0-21swapper/223:55:442
3815958991715,1cyclictest0-21swapper/200:24:092
3815958991713,3cyclictest3895919-21/usr/sbin/munin22:55:172
3815958991711,5cyclictest3885974-21awk22:25:192
3815955991715,1cyclictest0-21swapper/123:05:391
3815955991714,2cyclictest3056-21PK-Backend23:34:181
3815955991713,2cyclictest3844110-21latency_hist20:30:011
3815955991712,3cyclictest3850223-21needreboot20:45:191
3815955991711,4cyclictest3927488-21cat00:25:121
3815950991713,3cyclictest3928008-21missed_timers00:25:170
3815963991614,1cyclictest3056-21PK-Backend22:52:523
3815963991614,1cyclictest3056-21PK-Backend22:05:273
381596399160,15cyclictest39236482sleep300:14:393
3815955991614,1cyclictest3056-21PK-Backend23:49:191
3815950991610,4cyclictest3855211-21cpuspeed_turbos21:00:120
381595899151,8cyclictest3851534-21apt-get20:50:042
3815958991514,1cyclictest0-21swapper/200:29:302
381595599159,4cyclictest3925417-21cron00:20:021
381595899149,3cyclictest3890645-21tr22:40:122
3815955991411,2cyclictest3056-21PK-Backend19:25:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional