You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 18:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Jul 10, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
158309921310,3sleep30-21swapper/308:35:013
158309921310,3sleep30-21swapper/308:35:003
161473321300,4sleep13056-21PK-Backend10:05:011
139099920,79rtkit-daemon0-21swapper/007:05:290
155118827946,23sleep10-21swapper/107:05:121
15886252770,3sleep31588626-21cpuspeed_turbos08:50:123
155243226541,17sleep20-21swapper/207:07:262
155244826243,13sleep30-21swapper/307:07:393
16538122470,2sleep13056-21PK-Backend11:55:161
15869472470,4sleep13056-21PK-Backend08:45:131
16043962450,5sleep13056-21PK-Backend09:35:131
16554202370,4sleep30-21swapper/312:00:113
1552780993216,14cyclictest1555127-21df_inode07:15:033
155278099310,16cyclictest0-21swapper/312:21:073
155278099310,16cyclictest0-21swapper/307:46:513
155278099310,16cyclictest0-21swapper/307:26:063
155276999310,25cyclictest1612949-21chrt09:59:540
155278099300,22cyclictest0-21swapper/310:34:333
155278099300,17cyclictest0-21swapper/308:05:283
155278099300,16cyclictest1580894-21taskset08:25:453
155278099300,14cyclictest1651653-21apt11:50:113
155278099290,17cyclictest0-21swapper/310:48:363
155278099290,17cyclictest0-21swapper/309:02:313
155278099290,17cyclictest0-21swapper/308:41:373
155278099290,16cyclictest0-21swapper/311:59:303
155278099290,15cyclictest1637471-21taskset11:09:463
155278099282,13cyclictest1631730-21runrttasks10:51:093
1552780992813,13cyclictest1628044-21cat10:40:213
155278099280,17cyclictest0-21swapper/309:11:443
155278099280,16cyclictest1619535-21chrt10:15:293
155278099280,16cyclictest1-21systemd07:35:003
155278099280,16cyclictest0-21swapper/312:18:553
155278099280,16cyclictest0-21swapper/311:14:313
155278099280,16cyclictest0-21swapper/309:35:463
139099280,18rtkit-daemon1559529-21python307:25:232
15974782270,2sleep20-21swapper/209:15:132
1552780992710,15cyclictest1648172-21awk11:40:113
155278099270,16cyclictest0-21swapper/307:53:053
155278099270,14cyclictest0-21swapper/311:39:253
139099270,18rtkit-daemon0-21swapper/211:00:172
139099270,18rtkit-daemon0-21swapper/210:55:172
1552780992610,14cyclictest1650093-21grep11:45:113
155278099260,24cyclictest1562993-21python307:35:203
155278099260,23cyclictest1553838-21python307:10:213
155278099260,16cyclictest1635152-21ps11:00:183
155278099260,14cyclictest1626784-21runrttasks10:39:223
1552777992613,11cyclictest1628613-21unin-run10:45:002
139099260,18rtkit-daemon1659118-21latency12:10:162
139099260,18rtkit-daemon0-21swapper/207:55:022
139099260,17rtkit-daemon1655706-21memory12:00:162
139099260,17rtkit-daemon1591040-21smartctl08:55:222
139099260,17rtkit-daemon0-21swapper/208:00:162
139099260,17rtkit-daemon0-21swapper/208:00:012
1552780992510,13cyclictest1646170-21cron11:35:003
155278099250,23cyclictest1616390-21chrt10:09:363
1552777992516,7cyclictest1554679-21munin-node07:15:032
139099250,17rtkit-daemon1589517-21users08:50:222
139099250,17rtkit-daemon1567280-21ls07:50:012
139099250,17rtkit-daemon0-21swapper/210:50:172
139099250,16rtkit-daemon1619173-21ntp_states10:15:182
139099250,16rtkit-daemon0-21swapper/209:50:012
15765682240,6sleep1261ksoftirqd/108:15:171
15765682240,6sleep1261ksoftirqd/108:15:171
155278099247,15cyclictest1571597-21uname08:00:193
1552780992413,10cyclictest1668394-21sensors_temp12:35:203
1552780992412,11cyclictest1659439-21ps12:10:193
155278099240,23cyclictest1587878-21seq08:46:243
155278099240,22cyclictest436-20systemd-journal07:24:233
155277799248,13cyclictest1626880-21unin-run10:40:002
139099240,17rtkit-daemon0-21swapper/212:10:012
139099240,16rtkit-daemon1613004-21munin-run10:00:012
139099240,16rtkit-daemon1609478-21/usr/sbin/munin09:50:232
155278099239,12cyclictest1578556-21latency08:20:173
155278099237,14cyclictest1613143-21latency_hist10:00:013
155278099236,15cyclictest1657070-21cstates12:05:123
155278099236,15cyclictest1620248-21cpuspeed10:20:113
155278099232,19cyclictest1577627-21latency_hist08:20:013
155278099232,19cyclictest1577627-21latency_hist08:20:013
155278099230,21cyclictest1642153-21sed11:20:213
155278099230,21cyclictest1609478-21/usr/sbin/munin09:50:143
155278099230,21cyclictest0-21swapper/312:30:013
139099230,16rtkit-daemon1587578-21smartctl08:45:222
139099230,15rtkit-daemon0-21swapper/211:21:262
139099230,15rtkit-daemon0-21swapper/209:35:172
139099230,14rtkit-daemon0-21swapper/212:20:162
155278099226,14cyclictest1634178-21date11:00:013
1552780992214,6cyclictest1597881-21pluginstate09:15:203
1552780992213,6cyclictest1663799-21apt12:25:123
1552780992211,9cyclictest1639487-21/usr/sbin/munin11:15:203
155278099221,19cyclictest1569864-21ntp_states07:55:203
155278099220,20cyclictest1605750-21/usr/sbin/munin09:40:143
1552777992220,1cyclictest0-21swapper/210:32:152
1552777992220,1cyclictest0-21swapper/208:34:542
1552777992220,1cyclictest0-21swapper/208:34:542
1552777992214,6cyclictest1645574-21python311:30:202
155277799220,21cyclictest0-21swapper/208:06:562
155277799220,20cyclictest0-21swapper/211:06:152
1552775992219,2cyclictest3056-21PK-Backend09:46:431
155277599220,20cyclictest3056-21PK-Backend08:08:351
1552769992220,1cyclictest0-21swapper/012:14:050
1552769992220,1cyclictest0-21swapper/011:40:260
1552769992220,1cyclictest0-21swapper/010:34:490
1552769992220,1cyclictest0-21swapper/008:11:520
155276999220,3cyclictest1665556-21apt-config12:30:010
139099220,15rtkit-daemon0-21swapper/211:45:012
139099220,14rtkit-daemon0-21swapper/212:35:012
139099220,14rtkit-daemon0-21swapper/211:44:162
139099220,14rtkit-daemon0-21swapper/208:35:222
16647432210,3sleep13056-21PK-Backend12:25:231
15945822210,2sleep13056-21PK-Backend09:05:221
15779662210,2sleep00-21swapper/008:20:020
155278099216,13cyclictest1584032-21proc_pri08:35:203
155278099211,19cyclictest1564786-21python307:40:223
155278099211,18cyclictest1616848-21cpuspeed_turbos10:10:123
155278099210,20cyclictest0-21swapper/309:24:403
155278099210,20cyclictest0-21swapper/308:58:103
155278099210,19cyclictest1614272-21/usr/sbin/munin10:00:213
155278099210,19cyclictest1600676-21sort09:25:143
155278099210,19cyclictest1600676-21sort09:25:143
1552777992119,1cyclictest0-21swapper/211:56:242
1552777992119,1cyclictest0-21swapper/211:27:452
1552777992119,1cyclictest0-21swapper/211:16:022
1552777992119,1cyclictest0-21swapper/211:11:092
1552777992119,1cyclictest0-21swapper/210:24:452
1552777992119,1cyclictest0-21swapper/209:07:322
1552777992119,1cyclictest0-21swapper/208:27:512
1552777992118,2cyclictest0-21swapper/208:40:222
1552777992116,3cyclictest1607393-21latency_hist09:45:002
1552777992112,6cyclictest1579506-21perf08:24:592
155277799210,20cyclictest0-21swapper/212:27:272
1552775992119,1cyclictest3056-21PK-Backend11:42:551
1552775992119,1cyclictest0-21swapper/109:26:421
1552775992119,1cyclictest0-21swapper/109:26:421
1552775992119,1cyclictest0-21swapper/109:23:591
1552775992118,2cyclictest3056-21PK-Backend10:33:551
155277599210,20cyclictest3056-21PK-Backend12:02:571
155277599210,19cyclictest3056-21PK-Backend10:56:051
155276999215,6cyclictest1095-21runrttasks07:30:160
1552769992119,1cyclictest0-21swapper/011:30:190
1552769992119,1cyclictest0-21swapper/010:52:400
1552769992119,1cyclictest0-21swapper/010:43:210
1552769992119,1cyclictest0-21swapper/010:23:130
1552769992119,1cyclictest0-21swapper/009:52:380
1552769992119,1cyclictest0-21swapper/009:45:550
1552769992119,1cyclictest0-21swapper/009:37:510
1552769992119,1cyclictest0-21swapper/009:27:230
1552769992119,1cyclictest0-21swapper/009:27:230
1552769992119,1cyclictest0-21swapper/009:06:340
1552769992119,1cyclictest0-21swapper/008:52:280
1552769992119,1cyclictest0-21swapper/008:04:290
1552769992114,6cyclictest1654876-21basename12:00:000
155276999210,1cyclictest0-21swapper/009:21:280
155276999210,19cyclictest3056-21PK-Backend07:20:490
139099210,15rtkit-daemon0-21swapper/210:25:182
155278099206,12cyclictest1607703-21gdbus09:45:013
155278099200,19cyclictest0-21swapper/310:29:363
155278099200,18cyclictest1603027-21awk09:30:183
155278099200,18cyclictest1594430-21/usr/sbin/munin09:05:203
155278099200,18cyclictest1574947-21taskset08:10:193
1552777992018,1cyclictest3056-21PK-Backend09:28:072
1552777992018,1cyclictest3056-21PK-Backend09:28:072
1552777992018,1cyclictest0-21swapper/210:46:012
1552777992018,1cyclictest0-21swapper/207:39:312
1552777992018,1cyclictest0-21swapper/207:31:542
1552777992014,4cyclictest1665938-21tr12:30:112
155277799200,1cyclictest0-21swapper/211:37:522
155277799200,1cyclictest0-21swapper/209:01:172
155277799200,19cyclictest0-21swapper/210:01:262
155277799200,19cyclictest0-21swapper/207:42:492
1552775992018,1cyclictest3056-21PK-Backend11:45:261
1552775992018,1cyclictest3056-21PK-Backend11:36:141
1552775992018,1cyclictest3056-21PK-Backend11:33:441
1552775992017,2cyclictest3056-21PK-Backend10:48:151
1552775992012,6cyclictest1563649-21cron07:40:001
155277599200,2cyclictest3056-21PK-Backend07:53:481
155277599200,19cyclictest3056-21PK-Backend07:16:031
155277599200,18cyclictest3056-21PK-Backend11:52:431
1552769992018,1cyclictest0-21swapper/012:37:170
1552769992018,1cyclictest0-21swapper/011:22:550
1552769992018,1cyclictest0-21swapper/010:55:240
1552769992018,1cyclictest0-21swapper/010:36:110
1552769992018,1cyclictest0-21swapper/010:06:370
1552769992018,1cyclictest0-21swapper/009:16:280
1552769992018,1cyclictest0-21swapper/009:02:080
1552769992018,1cyclictest0-21swapper/008:38:220
1552769992018,1cyclictest0-21swapper/008:27:410
1552769992018,1cyclictest0-21swapper/007:55:500
1552769992018,1cyclictest0-21swapper/007:41:380
155276999200,1cyclictest0-21swapper/009:11:000
155276999200,19cyclictest0-21swapper/012:01:080
155276999200,19cyclictest0-21swapper/011:54:040
155276999200,19cyclictest0-21swapper/011:36:200
155276999200,19cyclictest0-21swapper/010:49:560
155276999200,19cyclictest0-21swapper/010:29:210
155276999200,19cyclictest0-21swapper/008:45:120
155276999200,19cyclictest0-21swapper/008:40:110
155276999200,19cyclictest0-21swapper/007:35:030
155276999200,19cyclictest0-21swapper/007:29:280
155276999200,19cyclictest0-21swapper/007:16:230
139099200,14rtkit-daemon0-21swapper/210:10:182
139099200,13rtkit-daemon0-21swapper/212:15:012
155278099190,17cyclictest1643152-21grep11:25:113
1552777991917,1cyclictest0-21swapper/210:05:112
1552777991917,1cyclictest0-21swapper/209:11:112
1552777991917,1cyclictest0-21swapper/208:13:522
1552777991917,1cyclictest0-21swapper/207:21:002
1552777991917,1cyclictest0-21swapper/207:12:272
1552775991917,1cyclictest0-21swapper/111:26:271
1552775991914,3cyclictest1557099-21tr07:20:121
1552775991912,5cyclictest1667275-21ls12:35:001
1552775991911,6cyclictest1602303-21awk09:30:111
155277599190,1cyclictest3056-21PK-Backend11:17:011
155277599190,1cyclictest3056-21PK-Backend09:50:011
1552769991917,1cyclictest0-21swapper/012:30:480
1552769991917,1cyclictest0-21swapper/012:17:030
1552769991917,1cyclictest0-21swapper/011:45:530
1552769991917,1cyclictest0-21swapper/011:19:240
1552769991917,1cyclictest0-21swapper/011:13:080
1552769991917,1cyclictest0-21swapper/010:17:520
1552769991917,1cyclictest0-21swapper/010:13:190
1552769991917,1cyclictest0-21swapper/007:52:330
1552769991914,4cyclictest1607241-21apt-get09:45:000
155276999190,1cyclictest0-21swapper/012:05:270
155276999190,1cyclictest0-21swapper/011:08:150
155276999190,1cyclictest0-21swapper/011:01:260
155276999190,1cyclictest0-21swapper/010:02:520
155276999190,1cyclictest0-21swapper/009:30:200
155276999190,1cyclictest0-21swapper/008:55:060
155276999190,18cyclictest0-21swapper/011:27:350
155276999190,18cyclictest0-21swapper/007:45:570
155277799180,1cyclictest3056-21PK-Backend09:23:002
1552775991816,1cyclictest3056-21PK-Backend10:22:051
1552775991816,1cyclictest3056-21PK-Backend07:43:411
1552775991816,1cyclictest3056-21PK-Backend07:27:051
1552775991815,2cyclictest3056-21PK-Backend10:51:531
1552775991815,2cyclictest3056-21PK-Backend10:05:221
1552775991815,1cyclictest3056-21PK-Backend09:11:211
155277599180,1cyclictest3056-21PK-Backend11:24:311
1552769991816,1cyclictest0-21swapper/012:23:320
1552769991816,1cyclictest0-21swapper/008:16:320
1552769991816,1cyclictest0-21swapper/008:16:320
1552769991816,1cyclictest0-21swapper/008:06:450
1552769991816,1cyclictest0-21swapper/007:11:090
16218862170,3sleep13056-21PK-Backend10:25:101
1552777991715,1cyclictest0-21swapper/211:50:572
1552777991710,5cyclictest1576984-21cat08:15:232
1552777991710,5cyclictest1576984-21cat08:15:222
1552775991715,1cyclictest3056-21PK-Backend12:06:151
1552775991715,1cyclictest3056-21PK-Backend10:11:581
1552775991715,1cyclictest3056-21PK-Backend09:57:251
1552775991714,2cyclictest3056-21PK-Backend12:36:501
1552775991714,2cyclictest3056-21PK-Backend10:44:361
1552775991714,2cyclictest3056-21PK-Backend08:01:181
1552775991712,3cyclictest1635770-21sh11:05:001
1552775991710,5cyclictest436-20systemd-journal07:15:001
155277599170,15cyclictest3056-21PK-Backend08:55:061
1552775991614,1cyclictest3056-21PK-Backend12:23:051
1552775991614,1cyclictest3056-21PK-Backend11:11:261
1552775991614,1cyclictest3056-21PK-Backend09:02:021
1552775991614,1cyclictest3056-21PK-Backend08:51:211
1552775991614,1cyclictest3056-21PK-Backend08:35:051
1552775991611,3cyclictest1579833-21cpuspeed08:25:141
155277599160,15cyclictest3056-21PK-Backend08:13:351
155277599160,15cyclictest3056-21PK-Backend07:32:321
155277599160,15cyclictest0-21swapper/109:40:091
1552777991511,3cyclictest978-21snmpd09:30:552
1552775991513,1cyclictest3056-21PK-Backend12:10:071
1552775991513,1cyclictest3056-21PK-Backend10:18:001
155277599150,1cyclictest3056-21PK-Backend12:15:551
155277599150,14cyclictest3056-21PK-Backend07:58:281
155276999158,5cyclictest1582964-21cron08:35:010
155276999158,5cyclictest1582964-21cron08:35:000
155277599148,4cyclictest1598692-21grep09:20:001
1552775991410,3cyclictest3056-21PK-Backend10:37:491
155277599140,1cyclictest3056-21PK-Backend08:42:491
155277599140,1cyclictest3056-21PK-Backend08:32:421
155277599140,1cyclictest3056-21PK-Backend08:32:421
1552775991311,1cyclictest3056-21PK-Backend11:05:181
155277599128,3cyclictest3056-21PK-Backend07:45:381
155277599121,9cyclictest1579511-21munin-run08:25:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional