You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-15 - 16:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Sep 15, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
166095421060,3sleep30-21swapper/309:45:103
170229721030,3sleep13056-21PK-Backend11:40:211
139099770,60rtkit-daemon0-21swapper/307:06:473
160408426545,14sleep00-21swapper/007:05:070
160536826242,14sleep10-21swapper/107:07:331
160533426241,14sleep20-21swapper/207:07:062
16772472500,2sleep00-21swapper/010:30:180
16504982470,4sleep234-21ksoftirqd/209:15:142
17197592410,4sleep234-21ksoftirqd/212:30:222
139099310,23rtkit-daemon1621775-21latency_hist07:55:013
139099270,4rtkit-daemon1389-21rtkit-daemon12:39:041
160570899262,20cyclictest0-21swapper/311:15:133
139099260,4rtkit-daemon1389-21rtkit-daemon12:30:110
1605699992522,2cyclictest436-20systemd-journal10:00:010
1605705992413,9cyclictest1641398-21unin-run08:50:012
1605699992420,3cyclictest1695191-21nm-dispatcher11:22:550
1605699992415,6cyclictest1095-21runrttasks12:20:220
139099240,4rtkit-daemon1389-21rtkit-daemon12:07:251
1605708992315,5cyclictest0-21swapper/311:20:223
1605705992314,7cyclictest1617217-21df_inode07:40:142
160570599230,3cyclictest1671733-21diskstats10:15:132
1605699992320,2cyclictest0-21swapper/010:55:270
1605699992314,6cyclictest436-20systemd-journal11:55:010
139099230,20rtkit-daemon1997-21gvfs-afc-volume11:52:451
160570899229,8cyclictest0-21swapper/307:45:093
1605708992220,1cyclictest0-21swapper/310:34:483
1605708992218,3cyclictest1635104-21hddtemp_smartct08:30:163
1605708992216,4cyclictest1634178-21munin-plugin-st08:30:003
160570899221,19cyclictest1652611-21memory09:20:163
1605708992210,9cyclictest883-21dbus-daemon08:45:013
160570599220,21cyclictest0-21swapper/207:12:492
160570399220,20cyclictest3056-21PK-Backend07:30:561
1605699992220,1cyclictest0-21swapper/012:07:110
1605699992220,1cyclictest0-21swapper/011:36:080
1605699992220,1cyclictest0-21swapper/010:51:140
1605699992220,1cyclictest0-21swapper/009:01:560
16959902210,4sleep13056-21PK-Backend11:25:121
160570899214,5cyclictest1651102-21processes09:15:213
160570899211,17cyclictest1644839-21turbostat09:00:013
1605708992111,7cyclictest1607607-21nin-node07:15:013
160570899211,16cyclictest1674578-21idleruntime-cro10:24:593
160570899210,20cyclictest0-21swapper/311:10:163
1605705992119,1cyclictest0-21swapper/207:46:012
1605705992117,3cyclictest1686942-21/usr/sbin/munin11:00:222
1605705992114,5cyclictest34-21ksoftirqd/210:50:142
1605705992114,5cyclictest1693774-21date11:20:012
1605705992113,6cyclictest1703172-21apt11:45:132
1605703992119,1cyclictest3056-21PK-Backend09:44:461
1605703992119,1cyclictest0-21swapper/112:25:471
1605703992118,2cyclictest3056-21PK-Backend08:23:211
160570399210,20cyclictest3056-21PK-Backend11:38:291
160570399210,20cyclictest3056-21PK-Backend10:46:251
160570399210,20cyclictest0-21swapper/109:05:261
160570399210,19cyclictest3056-21PK-Backend10:00:351
160570399210,19cyclictest3056-21PK-Backend08:30:311
160570399210,19cyclictest3056-21PK-Backend08:19:221
160570399210,19cyclictest3056-21PK-Backend07:46:311
1605699992119,1cyclictest3056-21PK-Backend07:40:050
1605699992119,1cyclictest0-21swapper/012:36:450
1605699992119,1cyclictest0-21swapper/012:15:440
1605699992119,1cyclictest0-21swapper/011:40:080
1605699992119,1cyclictest0-21swapper/011:32:580
1605699992119,1cyclictest0-21swapper/011:00:270
1605699992119,1cyclictest0-21swapper/010:41:130
1605699992119,1cyclictest0-21swapper/010:37:490
1605699992119,1cyclictest0-21swapper/009:54:090
1605699992119,1cyclictest0-21swapper/009:48:200
1605699992119,1cyclictest0-21swapper/009:43:210
1605699992119,1cyclictest0-21swapper/009:23:190
1605699992119,1cyclictest0-21swapper/008:44:340
1605699992119,1cyclictest0-21swapper/008:16:150
1605699992118,2cyclictest0-21swapper/008:48:460
160569999210,20cyclictest0-21swapper/011:06:150
160569999210,20cyclictest0-21swapper/011:06:140
160569999210,20cyclictest0-21swapper/007:22:480
139099210,14rtkit-daemon0-21swapper/309:57:213
160570899208,3cyclictest141rcu_preempt12:30:003
160570899202,13cyclictest1699171-21apt-get11:35:043
1605708992018,1cyclictest0-21swapper/311:00:173
1605708992018,1cyclictest0-21swapper/307:10:073
1605708992017,2cyclictest3056-21PK-Backend12:05:533
1605708992017,2cyclictest0-21swapper/310:27:113
1605708992014,4cyclictest1625231-21cron08:05:003
1605708992013,5cyclictest1628735-21systemd-run08:15:003
1605708992013,4cyclictest1703288-21/usr/sbin/munin11:45:143
1605708992012,6cyclictest1713928-21cstates12:15:133
1605708992012,6cyclictest1617711-21python307:40:213
1605708992012,5cyclictest2173-21JS10:27:483
160570899201,17cyclictest1612291-21netstat07:25:183
160570899201,16cyclictest1656632-21python309:30:213
160570899200,8cyclictest1670488-21python310:10:193
160570899200,8cyclictest1640481-21memory08:45:173
160570899200,1cyclictest3056-21PK-Backend11:05:173
160570899200,1cyclictest3056-21PK-Backend11:05:173
160570899200,1cyclictest0-21swapper/310:45:013
160570899200,19cyclictest3056-21PK-Backend12:37:363
160570899200,19cyclictest3056-21PK-Backend12:01:073
160570899200,19cyclictest0-21swapper/311:29:033
160570899200,19cyclictest0-21swapper/310:57:403
160570899200,19cyclictest0-21swapper/310:38:133
160570899200,19cyclictest0-21swapper/310:16:573
160570899200,19cyclictest0-21swapper/309:27:153
160570899200,19cyclictest0-21swapper/308:15:103
160570899200,18cyclictest3056-21PK-Backend12:20:343
160570899200,18cyclictest3056-21PK-Backend07:35:353
1605705992018,1cyclictest0-21swapper/210:22:032
1605705992018,1cyclictest0-21swapper/209:53:572
1605705992018,1cyclictest0-21swapper/209:05:312
1605705992018,1cyclictest0-21swapper/208:26:442
1605705992017,2cyclictest0-21swapper/208:20:222
1605705992013,5cyclictest1657703-21wc09:35:142
1605705992012,6cyclictest1698892-21lsb_release11:31:412
160570599200,1cyclictest0-21swapper/209:27:552
160570599200,19cyclictest0-21swapper/211:40:442
160570599200,19cyclictest0-21swapper/210:57:442
160570599200,19cyclictest0-21swapper/210:45:082
160570599200,19cyclictest0-21swapper/210:08:242
160570599200,19cyclictest0-21swapper/209:57:212
160570599200,19cyclictest0-21swapper/208:51:312
160570599200,19cyclictest0-21swapper/207:30:132
1605703992018,1cyclictest3056-21PK-Backend10:56:251
1605703992018,1cyclictest0-21swapper/112:24:121
1605703992018,1cyclictest0-21swapper/112:10:201
1605703992018,1cyclictest0-21swapper/111:07:131
1605703992018,1cyclictest0-21swapper/111:07:121
1605703992017,2cyclictest3056-21PK-Backend09:29:121
1605703992017,2cyclictest3056-21PK-Backend07:16:161
160570399200,1cyclictest3056-21PK-Backend09:55:351
160570399200,19cyclictest3056-21PK-Backend11:13:011
160570399200,19cyclictest3056-21PK-Backend09:38:591
160570399200,19cyclictest3056-21PK-Backend09:24:121
160570399200,19cyclictest3056-21PK-Backend07:54:081
160570399200,18cyclictest3056-21PK-Backend10:15:351
1605699992018,1cyclictest0-21swapper/012:02:250
1605699992018,1cyclictest0-21swapper/011:45:430
1605699992018,1cyclictest0-21swapper/011:15:280
1605699992018,1cyclictest0-21swapper/010:46:140
1605699992018,1cyclictest0-21swapper/010:16:460
1605699992018,1cyclictest0-21swapper/010:12:350
1605699992018,1cyclictest0-21swapper/009:35:580
1605699992018,1cyclictest0-21swapper/009:30:570
1605699992018,1cyclictest0-21swapper/009:25:570
1605699992018,1cyclictest0-21swapper/008:52:110
1605699992018,1cyclictest0-21swapper/008:39:330
1605699992018,1cyclictest0-21swapper/008:31:220
1605699992018,1cyclictest0-21swapper/008:07:430
1605699992018,1cyclictest0-21swapper/008:03:100
1605699992018,1cyclictest0-21swapper/007:51:070
1605699992014,4cyclictest1696298-21tail11:25:180
1605699992013,5cyclictest1675002-21tr10:25:130
160569999200,1cyclictest0-21swapper/010:09:100
160569999200,19cyclictest0-21swapper/012:13:200
160569999200,19cyclictest0-21swapper/009:10:560
160569999200,19cyclictest0-21swapper/009:10:560
160569999200,19cyclictest0-21swapper/007:58:580
160569999200,19cyclictest0-21swapper/007:49:180
1605708991917,1cyclictest3056-21PK-Backend11:56:203
1605708991917,1cyclictest0-21swapper/310:53:273
1605708991917,1cyclictest0-21swapper/309:10:393
1605708991917,1cyclictest0-21swapper/309:10:393
1605708991913,3cyclictest1697363-21/usr/sbin/munin11:30:163
1605708991912,5cyclictest1666611-21cat10:00:173
1605708991912,5cyclictest1658528-21sed09:35:233
160570899191,14cyclictest1667870-21/usr/sbin/munin10:05:243
160570899190,18cyclictest0-21swapper/311:50:463
160570899190,18cyclictest0-21swapper/309:50:403
160570899190,18cyclictest0-21swapper/309:43:043
160570899190,18cyclictest0-21swapper/307:23:533
1605705991917,1cyclictest0-21swapper/212:02:412
1605705991917,1cyclictest0-21swapper/210:38:182
1605705991917,1cyclictest0-21swapper/209:22:072
1605705991917,1cyclictest0-21swapper/208:43:062
1605705991917,1cyclictest0-21swapper/208:07:042
1605705991917,1cyclictest0-21swapper/207:20:472
1605705991917,1cyclictest0-21swapper/207:18:032
1605705991913,5cyclictest1623017-21sed07:55:242
1605705991912,5cyclictest1680160-21grep10:40:122
160570599190,1cyclictest0-21swapper/212:19:232
160570599190,1cyclictest0-21swapper/211:08:472
160570599190,1cyclictest0-21swapper/211:08:472
160570599190,1cyclictest0-21swapper/210:32:172
160570599190,1cyclictest0-21swapper/210:29:392
160570599190,1cyclictest0-21swapper/210:11:012
160570599190,1cyclictest0-21swapper/209:12:062
160570599190,1cyclictest0-21swapper/209:12:062
160570599190,1cyclictest0-21swapper/208:38:062
160570599190,1cyclictest0-21swapper/207:40:002
160570599190,18cyclictest0-21swapper/211:28:342
160570599190,18cyclictest0-21swapper/211:11:242
160570599190,18cyclictest0-21swapper/210:00:472
160570599190,18cyclictest0-21swapper/209:32:542
160570599190,18cyclictest0-21swapper/208:15:352
1605703991917,1cyclictest3056-21PK-Backend11:02:131
1605703991917,1cyclictest3056-21PK-Backend08:25:451
1605703991917,1cyclictest0-21swapper/112:17:161
1605703991916,2cyclictest3056-21PK-Backend07:41:301
1605703991912,6cyclictest883-21dbus-daemon12:35:011
160570399190,1cyclictest3056-21PK-Backend10:52:121
160570399190,1cyclictest3056-21PK-Backend10:25:361
160570399190,1cyclictest3056-21PK-Backend10:05:341
160570399190,1cyclictest3056-21PK-Backend08:12:261
160570399190,1cyclictest3056-21PK-Backend07:27:451
160570399190,18cyclictest0-21swapper/110:30:361
1605699991917,1cyclictest0-21swapper/010:02:340
1605699991917,1cyclictest0-21swapper/009:16:440
1605699991917,1cyclictest0-21swapper/008:25:470
1605699991917,1cyclictest0-21swapper/008:20:130
160569999191,3cyclictest1609597-21munin-run07:20:000
160569999190,1cyclictest0-21swapper/011:11:140
160569999190,1cyclictest0-21swapper/010:21:470
160569999190,1cyclictest0-21swapper/007:13:280
160570899182,12cyclictest1646832-21apt09:05:123
1605708991816,1cyclictest0-21swapper/307:58:403
1605708991812,4cyclictest1637097-21grep08:35:203
1605708991812,4cyclictest0-21swapper/311:40:113
1605708991811,5cyclictest1680809-21/usr/sbin/munin10:40:203
1605708991811,5cyclictest1641790-21cut08:50:133
160570899181,13cyclictest1630978-21apt08:20:153
160570599188,2cyclictest1095-21runrttasks08:30:172
1605705991816,1cyclictest0-21swapper/211:35:102
1605705991816,1cyclictest0-21swapper/208:56:322
1605705991813,3cyclictest1713290-21sh12:15:002
1605705991811,6cyclictest1718515-21cron12:30:002
1605705991811,5cyclictest1623969-21grep08:00:132
1605705991811,5cyclictest1611581-21basename07:25:112
160570399184,12cyclictest1606059-21cpuspeed_turbos07:10:131
1605703991816,1cyclictest3056-21PK-Backend08:08:411
1605703991816,1cyclictest3056-21PK-Backend08:01:111
1605703991815,2cyclictest3056-21PK-Backend10:39:481
1605703991815,2cyclictest3056-21PK-Backend07:55:571
1605703991812,4cyclictest1706796-21grep11:55:121
1605699991816,1cyclictest0-21swapper/011:55:160
1605699991816,1cyclictest0-21swapper/008:14:390
1605699991811,5cyclictest1718060-21cat12:25:210
1605708991712,3cyclictest1613686-21df07:30:123
160570899171,14cyclictest1646567-21munin-run09:04:593
160570899171,13cyclictest1720519-21basename12:35:003
1605705991715,1cyclictest0-21swapper/207:54:402
1605705991712,3cyclictest1710530-21if_enp2s012:05:162
1605705991712,3cyclictest1693882-21acpi11:20:102
1605705991710,5cyclictest34-21ksoftirqd/211:50:212
1605705991710,5cyclictest1660687-21cron09:45:002
1605703991715,1cyclictest0-21swapper/112:00:461
1605703991713,3cyclictest26-21ksoftirqd/111:20:001
1605699991711,4cyclictest1647782-21grep09:05:210
1605699991711,4cyclictest1615160-21tr07:35:140
1605699991710,5cyclictest436-20systemd-journal07:28:370
16954942160,4sleep13056-21PK-Backend11:25:011
160570899160,14cyclictest1713017-21sshd12:11:373
160570599169,5cyclictest34-21ksoftirqd/212:35:142
160570599169,5cyclictest34-21ksoftirqd/211:55:192
1605705991614,1cyclictest0-21swapper/208:13:592
160570399167,8cyclictest1680557-21latency10:40:171
1605703991614,1cyclictest3056-21PK-Backend09:49:471
1605703991614,1cyclictest3056-21PK-Backend09:00:391
1605703991614,1cyclictest3056-21PK-Backend08:44:171
1605699991614,1cyclictest0-21swapper/007:30:180
160570599159,4cyclictest1660956-21acpi09:45:102
1605705991511,3cyclictest1646039-21threads09:00:222
160570399158,5cyclictest1648719-21grep09:10:141
160570399158,5cyclictest1648719-21grep09:10:131
1605703991513,1cyclictest3056-21PK-Backend11:45:391
1605703991513,1cyclictest3056-21PK-Backend07:22:501
160569999158,5cyclictest1645039-21date09:00:010
16557772140,4sleep13056-21PK-Backend09:30:121
160570599148,5cyclictest1716046-21sh12:20:192
160570399148,4cyclictest1615034-21seq07:35:111
160570399142,9cyclictest1663564-21chrt09:50:211
1605703991412,1cyclictest3056-21PK-Backend08:55:051
160570399139,3cyclictest3056-21PK-Backend09:17:051
160570399139,3cyclictest3056-21PK-Backend08:45:571
160570399139,3cyclictest3056-21PK-Backend08:35:511
1605703991311,1cyclictest3056-21PK-Backend10:24:481
160570399130,5cyclictest3056-21PK-Backend11:35:001
160570399111,8cyclictest1671090-21perf10:15:001
160570399107,2cyclictest16428032chrt08:51:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional