You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 23:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Mar 29, 2026 12:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
197195521340,2sleep00-21swapper/009:00:000
193266527641,29sleep10-21swapper/107:08:231
193247827540,30sleep00-21swapper/007:05:570
20088972670,2sleep22000093-21kworker/u8:110:45:012
193208226744,15sleep30-21swapper/307:05:233
193257826345,13sleep20-21swapper/207:07:152
139099540,4rtkit-daemon1389-21rtkit-daemon08:16:272
20220512430,4sleep00-21swapper/011:20:190
20396272360,3sleep10-21swapper/112:10:171
19831692350,1sleep11983174-21grep09:30:171
20259292340,1sleep00-21swapper/011:31:340
20259292340,1sleep00-21swapper/011:31:330
139099340,4rtkit-daemon884-21gmain08:04:322
19989572330,4sleep13056-21PK-Backend10:15:111
139099320,4rtkit-daemon1934841-21munin-node09:11:332
139099310,2rtkit-daemon3056-21PK-Backend10:34:012
139099290,4rtkit-daemon1389-21rtkit-daemon11:10:393
139099290,4rtkit-daemon1389-21rtkit-daemon09:22:493
19493922280,5chrt0-21swapper/307:55:123
1932935992813,12cyclictest2012322-21cron10:55:001
139099270,4rtkit-daemon1389-21rtkit-daemon11:24:113
139099260,4rtkit-daemon1389-21rtkit-daemon11:02:063
139099260,4rtkit-daemon1389-21rtkit-daemon08:08:033
19706612250,4chrt0-21swapper/108:55:131
1932935992514,8cyclictest436-20systemd-journal07:35:001
139099250,4rtkit-daemon1389-21rtkit-daemon10:40:543
139099250,21rtkit-daemon1788620-21WebKitWebProces08:30:143
1932943992413,8cyclictest883-21dbus-daemon12:40:003
193294399241,20cyclictest436-20systemd-journal07:24:353
19852352230,5sleep20-21swapper/209:35:242
19557492230,4sleep13056-21PK-Backend08:10:221
193294399238,13cyclictest20472362chrt12:31:123
193294399232,16cyclictest1095-21runrttasks10:15:113
1932943992311,9cyclictest883-21dbus-daemon10:51:053
1932935992320,2cyclictest3056-21PK-Backend09:42:231
1932935992320,2cyclictest3056-21PK-Backend09:42:231
193293599230,3cyclictest2038142-21ntp_states12:05:191
193293599230,21cyclictest3056-21PK-Backend08:38:351
1932933992315,6cyclictest1-21systemd09:05:000
19603332220,3sleep20-21swapper/208:25:142
1932943992215,4cyclictest1940295-21turbostat.cron07:30:013
1932943992214,5cyclictest82332-21update-manager09:26:033
1932935992220,1cyclictest3056-21PK-Backend07:45:421
1932935992220,1cyclictest3056-21PK-Backend07:45:421
1932935992220,1cyclictest0-21swapper/111:36:331
1932935992219,2cyclictest3056-21PK-Backend08:27:261
193293599220,21cyclictest0-21swapper/109:01:331
1932933992220,1cyclictest0-21swapper/010:57:430
1932933992214,6cyclictest1946194-21df07:45:150
1932933992214,6cyclictest1946194-21df07:45:150
139099220,15rtkit-daemon3056-21PK-Backend12:32:131
193294399219,8cyclictest1095-21runrttasks09:15:113
1932943992118,2cyclictest3056-21PK-Backend10:01:593
1932943992118,2cyclictest3056-21PK-Backend09:04:273
1932943992118,2cyclictest3056-21PK-Backend08:51:513
193294399210,20cyclictest0-21swapper/307:53:243
193294399210,19cyclictest3056-21PK-Backend11:58:343
1932938992118,2cyclictest1992354-21smartctl09:55:192
1932938992115,4cyclictest34-21ksoftirqd/207:36:022
1932938992114,5cyclictest2017519-21cron11:10:012
193293899210,20cyclictest0-21swapper/209:07:132
193293899210,1cyclictest0-21swapper/208:11:432
193293599213,13cyclictest2029348-21taskset11:40:221
1932935992119,1cyclictest3056-21PK-Backend12:24:251
1932935992119,1cyclictest3056-21PK-Backend11:22:471
1932935992119,1cyclictest3056-21PK-Backend10:12:241
1932935992119,1cyclictest3056-21PK-Backend09:35:331
1932935992119,1cyclictest3056-21PK-Backend08:21:581
1932935992119,1cyclictest0-21swapper/108:45:031
1932935992119,1cyclictest0-21swapper/107:35:351
1932935992118,2cyclictest3056-21PK-Backend07:51:431
1932935992114,4cyclictest2005367-21sh10:35:001
1932935992112,6cyclictest436-20systemd-journal09:30:001
1932935992111,7cyclictest2031336-21seq11:45:211
193293599210,20cyclictest3056-21PK-Backend08:32:061
193293599210,20cyclictest3056-21PK-Backend07:55:281
193293599210,20cyclictest0-21swapper/110:00:071
193293599210,19cyclictest3056-21PK-Backend10:41:241
193293599210,19cyclictest3056-21PK-Backend10:21:501
193293599210,19cyclictest3056-21PK-Backend07:12:371
193293399210,20cyclictest0-21swapper/011:12:160
193293399210,20cyclictest0-21swapper/009:46:050
193293399210,20cyclictest0-21swapper/009:44:160
193293399210,20cyclictest0-21swapper/009:44:160
193293399210,20cyclictest0-21swapper/009:32:120
193293399210,20cyclictest0-21swapper/009:28:000
193293399210,20cyclictest0-21swapper/008:06:080
193293399210,1cyclictest0-21swapper/012:29:090
139099210,14rtkit-daemon3056-21PK-Backend12:38:251
139099210,14rtkit-daemon0-21swapper/112:15:311
139099210,14rtkit-daemon0-21swapper/012:06:550
139099210,14rtkit-daemon0-21swapper/011:39:590
20135242200,5chrt3056-21PK-Backend10:55:221
1932943992018,1cyclictest0-21swapper/310:21:073
1932943992015,3cyclictest2002299-21tr10:25:113
1932943992012,6cyclictest436-20systemd-journal10:09:553
193294399201,16cyclictest1956233-21ls08:15:003
1932943992010,8cyclictest1977019-21cron09:15:003
193294399200,1cyclictest3056-21PK-Backend10:37:023
193294399200,1cyclictest3056-21PK-Backend10:37:023
193294399200,1cyclictest3056-21PK-Backend08:46:093
193294399200,1cyclictest3056-21PK-Backend07:42:353
193294399200,1cyclictest3056-21PK-Backend07:42:343
193294399200,1cyclictest0-21swapper/308:15:203
193294399200,19cyclictest3056-21PK-Backend09:51:193
193294399200,19cyclictest3056-21PK-Backend09:30:373
193294399200,19cyclictest0-21swapper/312:27:473
193294399200,19cyclictest0-21swapper/312:05:023
193294399200,19cyclictest0-21swapper/310:49:453
193294399200,19cyclictest0-21swapper/307:49:383
193294399200,19cyclictest0-21swapper/307:49:383
1932938992018,1cyclictest0-21swapper/211:17:462
1932938992015,4cyclictest978-21snmpd09:21:432
1932938992013,5cyclictest2049272-21cron12:40:002
193293899200,1cyclictest0-21swapper/211:43:422
193293899200,1cyclictest0-21swapper/208:40:572
193293899200,1cyclictest0-21swapper/208:40:562
193293899200,19cyclictest0-21swapper/212:26:472
193293899200,19cyclictest0-21swapper/212:21:532
193293899200,19cyclictest0-21swapper/212:19:502
193293899200,19cyclictest0-21swapper/210:35:282
193293899200,19cyclictest0-21swapper/210:35:272
193293899200,19cyclictest0-21swapper/210:24:332
193293899200,19cyclictest0-21swapper/210:12:162
193293899200,19cyclictest0-21swapper/209:03:082
193293899200,19cyclictest0-21swapper/208:48:282
193293899200,19cyclictest0-21swapper/207:51:092
193293899200,19cyclictest0-21swapper/207:41:362
193293899200,19cyclictest0-21swapper/207:41:362
193293899200,19cyclictest0-21swapper/207:29:532
1932935992018,1cyclictest3056-21PK-Backend08:00:431
1932935992018,1cyclictest0-21swapper/111:50:321
1932935992018,1cyclictest0-21swapper/109:47:351
1932935992015,4cyclictest2016238-21grep11:05:131
1932935992014,5cyclictest883-21dbus-daemon11:29:591
1932935992013,5cyclictest26-21ksoftirqd/110:06:031
1932935992012,5cyclictest1965808-21grep08:40:171
1932935992012,5cyclictest1965808-21grep08:40:171
193293599200,1cyclictest3056-21PK-Backend11:17:201
193293599200,1cyclictest3056-21PK-Backend10:45:501
193293599200,19cyclictest0-21swapper/111:31:521
193293599200,19cyclictest0-21swapper/111:31:521
193293599200,19cyclictest0-21swapper/110:37:451
193293599200,19cyclictest0-21swapper/110:37:441
193293599200,19cyclictest0-21swapper/110:27:041
193293599200,19cyclictest0-21swapper/107:27:241
1932933992018,1cyclictest0-21swapper/011:51:100
1932933992018,1cyclictest0-21swapper/007:41:410
1932933992018,1cyclictest0-21swapper/007:41:400
1932933992012,6cyclictest2036406-21ntp_states12:00:160
1932933992012,6cyclictest2030216-21date11:45:010
1932933992012,6cyclictest1-21systemd12:15:000
193293399200,1cyclictest0-21swapper/010:17:080
193293399200,1cyclictest0-21swapper/010:09:170
193293399200,1cyclictest0-21swapper/007:15:590
193293399200,19cyclictest0-21swapper/011:57:390
193293399200,19cyclictest0-21swapper/010:54:530
193293399200,19cyclictest0-21swapper/008:49:200
193293399200,19cyclictest0-21swapper/008:40:010
193293399200,19cyclictest0-21swapper/008:40:010
193293399200,19cyclictest0-21swapper/007:51:140
193293399200,19cyclictest0-21swapper/007:35:050
1932943991917,1cyclictest3056-21PK-Backend07:32:553
1932943991917,1cyclictest0-21swapper/310:13:163
1932943991917,1cyclictest0-21swapper/309:38:133
1932943991917,1cyclictest0-21swapper/308:01:203
1932943991916,2cyclictest3056-21PK-Backend09:07:313
1932943991915,3cyclictest1095-21runrttasks08:26:213
193294399191,16cyclictest1933498-21df07:10:113
193294399191,15cyclictest1963404-21/usr/sbin/munin08:35:213
193294399190,1cyclictest3056-21PK-Backend12:14:213
193294399190,1cyclictest3056-21PK-Backend11:48:273
193294399190,1cyclictest3056-21PK-Backend11:42:583
193294399190,1cyclictest3056-21PK-Backend11:34:413
193294399190,1cyclictest3056-21PK-Backend11:34:413
193294399190,1cyclictest3056-21PK-Backend09:46:053
193294399190,18cyclictest3056-21PK-Backend12:15:233
193294399190,18cyclictest3056-21PK-Backend11:29:143
193294399190,18cyclictest3056-21PK-Backend09:55:453
193294399190,18cyclictest0-21swapper/311:19:053
1932938991917,1cyclictest0-21swapper/211:31:452
1932938991917,1cyclictest0-21swapper/211:31:452
1932938991917,1cyclictest0-21swapper/211:28:542
1932938991917,1cyclictest0-21swapper/211:00:222
1932938991917,1cyclictest0-21swapper/210:45:352
1932938991917,1cyclictest0-21swapper/210:02:362
1932938991917,1cyclictest0-21swapper/209:41:002
1932938991917,1cyclictest0-21swapper/209:40:592
1932938991917,1cyclictest0-21swapper/208:08:532
1932938991917,1cyclictest0-21swapper/207:21:202
1932938991913,4cyclictest2156-21gnome-shell09:50:092
1932938991911,6cyclictest1978509-21cron09:17:002
1932938991911,6cyclictest1959086-21python308:20:202
1932938991911,6cyclictest1950768-21apt-get08:00:002
193293899190,1cyclictest3056-21PK-Backend11:35:242
193293899190,1cyclictest0-21swapper/211:55:042
193293899190,1cyclictest0-21swapper/210:59:202
193293899190,1cyclictest0-21swapper/209:25:322
193293899190,18cyclictest0-21swapper/209:48:032
193293899190,18cyclictest0-21swapper/209:33:002
193293899190,18cyclictest0-21swapper/208:54:092
193293899190,18cyclictest0-21swapper/207:47:242
193293899190,18cyclictest0-21swapper/207:47:232
193293899190,18cyclictest0-21swapper/207:15:542
193293899190,18cyclictest0-21swapper/207:11:482
193293599199,8cyclictest2036533-21python312:00:181
1932935991917,1cyclictest3056-21PK-Backend08:07:121
1932935991917,1cyclictest3056-21PK-Backend07:15:411
1932935991917,1cyclictest0-21swapper/111:55:111
1932935991917,1cyclictest0-21swapper/109:05:391
1932935991917,1cyclictest0-21swapper/107:42:111
1932935991917,1cyclictest0-21swapper/107:42:111
1932935991916,2cyclictest3056-21PK-Backend07:21:561
1932935991913,4cyclictest2044505-21cpuspeed_turbos12:25:131
193293599191,13cyclictest1968629-21apt08:50:111
193293599190,1cyclictest3056-21PK-Backend09:14:111
193293599190,1cyclictest0-21swapper/111:03:471
193293599190,1cyclictest0-21swapper/109:56:291
1932933991917,1cyclictest0-21swapper/011:06:480
1932933991917,1cyclictest0-21swapper/010:40:180
1932933991917,1cyclictest0-21swapper/010:30:260
1932933991917,1cyclictest0-21swapper/010:25:590
1932933991917,1cyclictest0-21swapper/009:52:200
1932933991917,1cyclictest0-21swapper/009:22:120
1932933991917,1cyclictest0-21swapper/009:05:150
1932933991917,1cyclictest0-21swapper/008:36:220
1932933991917,1cyclictest0-21swapper/008:28:380
1932933991917,1cyclictest0-21swapper/007:58:580
1932933991917,1cyclictest0-21swapper/007:12:530
1932933991914,3cyclictest1962363-21grep08:30:170
1932933991912,5cyclictest2029932-21cron11:45:010
193293399190,1cyclictest0-21swapper/010:11:540
193293399190,18cyclictest0-21swapper/012:17:330
193293399190,18cyclictest0-21swapper/009:10:220
193293399190,18cyclictest0-21swapper/008:20:200
193293399190,18cyclictest0-21swapper/007:27:550
139099190,13rtkit-daemon0-21swapper/007:30:320
1932943991816,1cyclictest3056-21PK-Backend12:01:243
1932943991816,1cyclictest3056-21PK-Backend11:39:203
1932943991816,1cyclictest0-21swapper/310:55:273
1932943991816,1cyclictest0-21swapper/309:43:283
1932943991816,1cyclictest0-21swapper/309:43:283
1932943991815,2cyclictest3056-21PK-Backend08:42:313
1932943991815,2cyclictest3056-21PK-Backend08:42:313
193294399180,5cyclictest1935106-21munin-node07:15:033
1932938991816,1cyclictest0-21swapper/211:20:362
1932938991816,1cyclictest0-21swapper/210:16:422
1932938991816,1cyclictest0-21swapper/210:09:382
1932938991812,4cyclictest1095-21runrttasks12:30:202
1932938991811,5cyclictest34-21ksoftirqd/210:51:052
1932935991816,1cyclictest0-21swapper/109:52:021
1932933991816,1cyclictest0-21swapper/012:31:120
1932933991816,1cyclictest0-21swapper/012:22:260
1932933991816,1cyclictest0-21swapper/007:21:260
1932933991811,5cyclictest2020036-21cut11:15:150
1932943991715,1cyclictest0-21swapper/308:55:433
193294399170,15cyclictest20171872sleep311:05:593
1932938991715,1cyclictest0-21swapper/212:06:262
1932938991715,1cyclictest0-21swapper/211:50:232
1932938991715,1cyclictest0-21swapper/208:56:122
1932938991715,1cyclictest0-21swapper/208:37:192
1932938991712,3cyclictest1962153-21if_err_enp2s008:30:172
1932935991715,1cyclictest3056-21PK-Backend11:12:541
193293399174,11cyclictest2024452-21cron11:30:000
1932933991715,1cyclictest0-21swapper/010:47:350
1932933991715,1cyclictest0-21swapper/010:21:330
1932933991715,1cyclictest0-21swapper/008:54:140
1932933991715,1cyclictest0-21swapper/008:18:310
1932933991714,2cyclictest0-21swapper/010:03:140
1932933991712,3cyclictest2014967-21tail11:00:180
1932933991710,5cyclictest2048882-21uptime12:35:210
1932933991710,5cyclictest2007097-21cron10:40:000
1932933991710,5cyclictest2007097-21cron10:40:000
1932933991710,5cyclictest1992195-21ntpq09:55:150
139099170,14rtkit-daemon0-21swapper/308:21:223
1932943991614,1cyclictest3056-21PK-Backend11:51:173
193294399161,13cyclictest2044066-20latency_hist12:25:003
1932943991610,4cyclictest1943092-21grep07:35:223
193293899169,5cyclictest2018081-21cat11:10:132
1932938991614,1cyclictest3056-21PK-Backend10:25:342
1932938991614,1cyclictest0-21swapper/211:49:222
1932933991614,1cyclictest0-21swapper/008:00:250
1932943991513,1cyclictest3056-21PK-Backend10:34:253
193293899159,5cyclictest2036656-21sed12:00:202
1932938991513,1cyclictest0-21swapper/212:13:092
193293399152,11cyclictest1984971-21awk09:35:190
1932933991513,1cyclictest0-21swapper/008:11:490
1932933991511,3cyclictest1977417-21tr09:15:120
193293599140,5cyclictest3056-21PK-Backend08:17:241
193293599140,1cyclictest141rcu_preempt09:20:121
193293899136,5cyclictest883-21dbus-daemon07:35:012
193293599106,3cyclictest19786792sleep109:19:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional