You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 08:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Jul 01, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40408121290,3sleep1251rcuc/120:55:221
37732721240,3sleep20-21swapper/219:40:172
41380521220,4sleep1413375-21/usr/sbin/munin21:25:141
139099920,75rtkit-daemon0-21swapper/319:05:243
139099920,75rtkit-daemon0-21swapper/319:05:243
36502627440,28sleep00-21swapper/019:07:030
36502627440,28sleep00-21swapper/019:07:030
36492526344,13sleep20-21swapper/219:05:442
36492526344,13sleep20-21swapper/219:05:442
36468226343,13sleep10-21swapper/119:05:271
36468226343,13sleep10-21swapper/119:05:271
4021372470,2sleep00-21swapper/020:50:200
4192972440,3sleep13056-21PK-Backend21:40:171
4309982330,3sleep10-21swapper/122:15:101
365484992610,14cyclictest442881-21unin-run22:50:001
4447582250,4sleep234-21ksoftirqd/222:55:012
3843322250,4sleep23056-21PK-Backend20:00:182
365493992516,6cyclictest472994-21apt-get00:20:002
36549999237,14cyclictest454513-21timerandwakeup23:20:213
365493992315,6cyclictest394938-21python320:30:212
365484992320,2cyclictest3056-21PK-Backend00:19:491
36548499230,21cyclictest3056-21PK-Backend23:57:451
3950832220,3sleep30-21swapper/320:30:223
365499992215,6cyclictest1-21systemd23:50:003
365499992214,6cyclictest473031-21systemctl00:20:013
365499992214,6cyclictest466369-21ntp_kernel_err23:55:173
365499992214,6cyclictest1-21systemd22:05:013
365484992220,1cyclictest3056-21PK-Backend23:31:351
365484992220,1cyclictest3056-21PK-Backend21:22:051
365484992220,1cyclictest3056-21PK-Backend20:22:361
365484992220,1cyclictest0-21swapper/123:20:411
365484992220,1cyclictest0-21swapper/120:06:271
365484992220,1cyclictest0-21swapper/119:20:441
365484992219,2cyclictest3056-21PK-Backend19:50:591
36548499220,21cyclictest0-21swapper/119:17:331
36548499220,20cyclictest3056-21PK-Backend20:17:501
365477992220,1cyclictest0-21swapper/023:12:520
365477992220,1cyclictest0-21swapper/022:58:540
365477992220,1cyclictest0-21swapper/020:23:400
36547799220,2cyclictest467424-21apt00:00:100
36547799220,21cyclictest0-21swapper/021:26:400
36549999219,10cyclictest420095-21unin-run21:45:003
365499992119,1cyclictest0-21swapper/320:37:363
36549999210,20cyclictest0-21swapper/323:30:403
36549999210,1cyclictest3056-21PK-Backend20:51:353
36549999210,1cyclictest0-21swapper/300:00:113
36549999210,19cyclictest430986-21apt22:15:093
36549999210,19cyclictest0-21swapper/320:40:133
36549999210,19cyclictest0-21swapper/320:40:123
365493992115,4cyclictest1095-21runrttasks23:10:162
365493992114,5cyclictest437499-21sshd22:33:102
36549399210,8cyclictest4014152expr20:50:122
36549399210,20cyclictest0-21swapper/219:18:552
36549399210,19cyclictest0-21swapper/219:31:532
36549399210,19cyclictest0-21swapper/200:06:302
365484992119,1cyclictest3056-21PK-Backend22:12:201
365484992119,1cyclictest3056-21PK-Backend22:00:511
365484992119,1cyclictest3056-21PK-Backend21:00:561
365484992119,1cyclictest3056-21PK-Backend19:43:291
365484992119,1cyclictest0-21swapper/123:27:581
365484992119,1cyclictest0-21swapper/123:17:371
365484992119,1cyclictest0-21swapper/122:37:281
365484992119,1cyclictest0-21swapper/121:14:001
365484992118,2cyclictest3056-21PK-Backend23:07:151
365484992118,2cyclictest3056-21PK-Backend19:13:281
36548499210,20cyclictest3056-21PK-Backend23:45:221
36548499210,20cyclictest3056-21PK-Backend20:27:561
36548499210,20cyclictest0-21swapper/121:30:431
365477992119,1cyclictest0-21swapper/023:18:270
365477992119,1cyclictest0-21swapper/023:05:020
365477992119,1cyclictest0-21swapper/022:11:490
365477992119,1cyclictest0-21swapper/022:09:460
365477992119,1cyclictest0-21swapper/021:02:530
365477992119,1cyclictest0-21swapper/020:57:400
365477992119,1cyclictest0-21swapper/020:34:350
365477992119,1cyclictest0-21swapper/019:52:510
365477992119,1cyclictest0-21swapper/019:43:250
365477992119,1cyclictest0-21swapper/019:29:530
365477992119,1cyclictest0-21swapper/019:20:400
365477992119,1cyclictest0-21swapper/000:36:210
365477992119,1cyclictest0-21swapper/000:36:210
365477992119,1cyclictest0-21swapper/000:06:470
365477992114,5cyclictest883-21dbus-daemon21:48:030
36547799210,20cyclictest0-21swapper/022:21:420
36547799210,20cyclictest0-21swapper/022:02:430
36547799210,20cyclictest0-21swapper/021:07:480
36547799210,1cyclictest0-21swapper/022:33:120
36547799210,19cyclictest0-21swapper/000:15:530
139099210,15rtkit-daemon3056-21PK-Backend20:03:523
139099210,14rtkit-daemon3056-21PK-Backend00:11:193
3777592200,2sleep30-21swapper/319:40:223
365499992018,1cyclictest0-21swapper/323:51:023
365499992018,1cyclictest0-21swapper/321:50:293
365499992018,1cyclictest0-21swapper/321:22:383
365499992018,1cyclictest0-21swapper/319:16:583
365499992014,4cyclictest2156-21gnome-shell19:52:093
365499992013,5cyclictest42-21ksoftirqd/323:10:113
365499992012,6cyclictest387620-21awk20:10:173
365499992012,6cyclictest370676-21python319:20:213
365499992011,8cyclictest414897-21unin-run21:30:003
36549999200,1cyclictest0-21swapper/322:32:473
36549999200,19cyclictest3056-21PK-Backend22:45:593
36549999200,19cyclictest3056-21PK-Backend21:16:563
36549999200,19cyclictest3056-21PK-Backend21:10:553
36549999200,19cyclictest3056-21PK-Backend20:07:203
36549999200,19cyclictest0-21swapper/322:26:463
36549999200,19cyclictest0-21swapper/320:17:493
36549999200,19cyclictest0-21swapper/319:11:243
36549999200,19cyclictest0-21swapper/300:37:393
36549999200,19cyclictest0-21swapper/300:37:383
36549999200,19cyclictest0-21swapper/300:20:293
36549999200,19cyclictest0-21swapper/300:09:263
36549999200,18cyclictest400141-21python320:45:213
36549999200,18cyclictest0-21swapper/319:36:453
36549999200,17cyclictest445714-21python322:55:203
365493992018,1cyclictest0-21swapper/223:43:522
365493992018,1cyclictest0-21swapper/222:48:092
365493992018,1cyclictest0-21swapper/222:39:312
365493992018,1cyclictest0-21swapper/221:48:412
365493992018,1cyclictest0-21swapper/221:31:102
365493992018,1cyclictest0-21swapper/220:40:342
365493992018,1cyclictest0-21swapper/220:40:342
365493992018,1cyclictest0-21swapper/220:25:332
365493992015,4cyclictest978-21snmpd19:52:262
365493992013,5cyclictest381920-21sshd19:55:112
365493992012,6cyclictest407922-21cron21:10:002
36549399200,1cyclictest0-21swapper/223:48:522
36549399200,1cyclictest0-21swapper/222:25:252
36549399200,19cyclictest0-21swapper/221:10:282
36549399200,19cyclictest0-21swapper/220:59:472
36549399200,19cyclictest0-21swapper/220:11:142
36549399200,19cyclictest0-21swapper/200:27:322
365484992018,1cyclictest3056-21PK-Backend23:35:141
365484992018,1cyclictest3056-21PK-Backend23:35:141
365484992018,1cyclictest3056-21PK-Backend23:03:031
365484992018,1cyclictest3056-21PK-Backend22:32:351
365484992018,1cyclictest3056-21PK-Backend22:08:491
365484992018,1cyclictest3056-21PK-Backend19:45:591
365484992018,1cyclictest3056-21PK-Backend19:39:511
365484992018,1cyclictest0-21swapper/122:22:141
365484992018,1cyclictest0-21swapper/121:51:251
365484992018,1cyclictest0-21swapper/120:12:091
36548499200,1cyclictest3056-21PK-Backend23:52:301
36548499200,1cyclictest0-21swapper/123:40:011
36548499200,19cyclictest3056-21PK-Backend00:36:591
36548499200,19cyclictest3056-21PK-Backend00:36:591
36548499200,19cyclictest0-21swapper/121:17:181
36548499200,19cyclictest0-21swapper/120:52:161
365477992018,1cyclictest0-21swapper/023:51:400
365477992018,1cyclictest0-21swapper/021:16:130
365477992018,1cyclictest0-21swapper/020:35:080
365477992018,1cyclictest0-21swapper/020:19:480
365477992018,1cyclictest0-21swapper/020:11:440
365477992018,1cyclictest0-21swapper/000:23:370
36547799200,1cyclictest0-21swapper/023:48:560
36547799200,1cyclictest0-21swapper/022:38:050
36547799200,1cyclictest0-21swapper/020:25:090
36547799200,19cyclictest0-21swapper/023:35:310
36547799200,19cyclictest0-21swapper/023:35:300
36547799200,19cyclictest0-21swapper/023:22:060
36547799200,19cyclictest0-21swapper/022:50:080
36547799200,19cyclictest0-21swapper/022:48:590
36547799200,19cyclictest0-21swapper/022:27:100
36547799200,19cyclictest0-21swapper/021:40:530
36547799200,19cyclictest0-21swapper/019:48:590
36547799200,19cyclictest0-21swapper/019:11:550
36547799200,19cyclictest0-21swapper/000:32:280
139099200,3rtkit-daemon0-21swapper/200:14:312
139099200,14rtkit-daemon0-21swapper/223:25:302
139099200,14rtkit-daemon0-21swapper/220:09:502
365499991917,1cyclictest3056-21PK-Backend23:16:063
365499991917,1cyclictest3056-21PK-Backend21:56:243
365499991917,1cyclictest0-21swapper/323:26:283
36549999191,16cyclictest406833-21df21:05:143
36549999191,16cyclictest372407-21python319:25:193
36549999191,16cyclictest2156-21gnome-shell22:14:093
36549999191,16cyclictest0-21swapper/322:35:013
36549999190,1cyclictest0-21swapper/323:44:253
36549999190,1cyclictest0-21swapper/322:50:173
36549999190,1cyclictest0-21swapper/322:44:513
36549999190,18cyclictest0-21swapper/322:09:223
36549999190,18cyclictest0-21swapper/320:29:593
36549999190,18cyclictest0-21swapper/320:22:123
36549999190,18cyclictest0-21swapper/300:28:473
36549999190,17cyclictest433201-21taskset22:20:173
365493991917,1cyclictest0-21swapper/223:36:152
365493991917,1cyclictest0-21swapper/223:36:152
365493991917,1cyclictest0-21swapper/223:24:112
365493991917,1cyclictest0-21swapper/222:24:162
365493991917,1cyclictest0-21swapper/221:22:322
365493991912,5cyclictest457556-21cat23:30:182
365493991912,5cyclictest436-20systemd-journal22:15:002
365493991912,5cyclictest366032-21expr19:10:132
36549399190,1cyclictest3056-21PK-Backend21:50:102
36549399190,1cyclictest0-21swapper/221:02:512
36549399190,1cyclictest0-21swapper/220:47:152
36549399190,1cyclictest0-21swapper/220:22:362
36549399190,1cyclictest0-21swapper/219:48:292
36549399190,18cyclictest0-21swapper/223:16:212
36549399190,18cyclictest0-21swapper/222:01:252
36549399190,18cyclictest0-21swapper/219:25:512
36549399190,18cyclictest0-21swapper/219:24:432
365484991917,1cyclictest3056-21PK-Backend20:36:291
365484991917,1cyclictest0-21swapper/122:56:211
365484991917,1cyclictest0-21swapper/121:46:581
365484991917,1cyclictest0-21swapper/100:25:361
365484991917,1cyclictest0-21swapper/100:14:011
365484991916,2cyclictest3056-21PK-Backend20:43:111
365484991916,2cyclictest3056-21PK-Backend20:43:111
365484991912,5cyclictest374753-21cron19:35:001
36548499190,1cyclictest3056-21PK-Backend00:08:471
36548499190,1cyclictest0-21swapper/121:55:511
36548499190,1cyclictest0-21swapper/121:08:531
365477991917,1cyclictest0-21swapper/021:57:430
365477991917,1cyclictest0-21swapper/021:36:270
365477991917,1cyclictest0-21swapper/020:43:550
365477991917,1cyclictest0-21swapper/020:43:550
365477991917,1cyclictest0-21swapper/019:56:090
365477991917,1cyclictest0-21swapper/000:10:050
365477991913,4cyclictest467168-40ogrotate00:00:010
365477991911,6cyclictest431719-21ntpq22:15:170
36547799190,1cyclictest0-21swapper/021:52:430
36547799190,1cyclictest0-21swapper/020:49:430
36547799190,18cyclictest3056-21PK-Backend19:17:290
365499991816,1cyclictest0-21swapper/323:05:123
365499991816,1cyclictest0-21swapper/323:02:423
365499991816,1cyclictest0-21swapper/319:47:063
365499991814,3cyclictest978-21snmpd00:30:053
365499991813,3cyclictest417450-21missed_timers21:35:163
365499991813,3cyclictest415719-21memory21:30:173
365499991813,3cyclictest382684-21ntp_states19:55:193
365499991811,6cyclictest421126-21grep21:45:203
36549999180,1cyclictest0-21swapper/321:01:293
365493991816,1cyclictest0-21swapper/223:59:202
365493991816,1cyclictest0-21swapper/223:06:482
365493991816,1cyclictest0-21swapper/222:06:242
365493991816,1cyclictest0-21swapper/221:58:282
365493991816,1cyclictest0-21swapper/220:36:282
365493991816,1cyclictest0-21swapper/200:38:072
365493991816,1cyclictest0-21swapper/200:38:062
365493991814,3cyclictest978-21snmpd21:38:442
365493991813,3cyclictest441120-21turbostat.cron22:45:002
365493991812,4cyclictest472994-21apt-get00:20:022
365493991812,4cyclictest2156-21gnome-shell00:01:092
365493991810,6cyclictest464643-21ntpq23:50:192
36549399180,1cyclictest0-21swapper/219:35:172
365484991816,1cyclictest3056-21PK-Backend21:36:381
365484991816,1cyclictest0-21swapper/123:14:331
365484991815,2cyclictest3056-21PK-Backend22:27:421
365477991816,1cyclictest0-21swapper/023:31:180
365477991816,1cyclictest0-21swapper/022:44:070
365477991816,1cyclictest0-21swapper/021:21:540
365477991816,1cyclictest0-21swapper/021:12:550
365477991816,1cyclictest0-21swapper/020:09:210
365477991816,1cyclictest0-21swapper/019:39:130
365477991814,3cyclictest384192-21latency20:00:170
365477991811,5cyclictest461079-21ls23:40:170
4001172170,2sleep10-21swapper/120:45:201
365499991715,1cyclictest3056-21PK-Backend19:30:303
36549999170,15cyclictest458893-21cut23:35:133
36549999170,15cyclictest458893-21cut23:35:123
36549399179,6cyclictest436-20systemd-journal22:20:012
365493991715,1cyclictest3056-21PK-Backend22:58:562
365493991712,3cyclictest389821-21timerandwakeup20:15:212
36549399170,16cyclictest0-21swapper/221:27:182
365484991715,1cyclictest3056-21PK-Backend19:58:161
365484991715,1cyclictest0-21swapper/122:50:061
365484991715,1cyclictest0-21swapper/119:25:381
365484991714,2cyclictest3056-21PK-Backend22:43:291
365484991714,2cyclictest3056-21PK-Backend00:23:401
365477991715,1cyclictest0-21swapper/023:28:140
365477991715,1cyclictest0-21swapper/023:01:240
365477991715,1cyclictest0-21swapper/021:33:290
365477991712,3cyclictest476206-21xargs00:25:220
139099170,3rtkit-daemon0-21swapper/223:00:232
36549999160,14cyclictest404599-21date21:00:013
365493991614,1cyclictest0-21swapper/200:32:332
36548499169,5cyclictest478057-21cat00:30:241
365484991614,1cyclictest3056-21PK-Backend20:34:261
36547799167,7cyclictest373732-21cat19:30:160
36549399159,4cyclictest2156-21gnome-shell21:44:082
36549399158,5cyclictest436-20systemd-journal21:20:002
36548499159,5cyclictest384095-21ls20:00:161
36548499140,6cyclictest3056-21PK-Backend00:00:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional