You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-18 - 14:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Jun 18, 2026 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14920932930,3sleep00-21swapper/000:30:160
139099880,74rtkit-daemon0-21swapper/019:08:300
137587327346,17sleep30-21swapper/319:05:113
137630827144,19sleep20-21swapper/219:05:182
137725826140,14sleep10-21swapper/119:09:111
14729562540,4sleep3411rcuc/323:35:213
139099400,3rtkit-daemon0-21swapper/022:15:010
139099390,3rtkit-daemon0-21swapper/023:43:500
139099380,3rtkit-daemon0-21swapper/021:50:090
139099380,3rtkit-daemon0-21swapper/021:31:510
139099370,3rtkit-daemon0-21swapper/020:39:570
139099370,3rtkit-daemon0-21swapper/020:24:360
139099370,3rtkit-daemon0-21swapper/019:14:130
139099360,3rtkit-daemon0-21swapper/022:39:060
139099360,3rtkit-daemon0-21swapper/022:24:260
139099360,3rtkit-daemon0-21swapper/022:18:490
139099360,3rtkit-daemon0-21swapper/022:18:490
139099360,3rtkit-daemon0-21swapper/022:04:350
139099360,3rtkit-daemon0-21swapper/021:04:430
139099360,3rtkit-daemon0-21swapper/019:33:500
139099270,15rtkit-daemon0-21swapper/121:10:411
1377498992615,9cyclictest1489732-21/usr/sbin/munin00:25:192
13824212250,3chrt0-21swapper/119:20:201
137750199253,20cyclictest1895009-21snapd19:46:133
1377501992518,5cyclictest1443612-21/usr/sbin/munin22:15:103
1377501992518,5cyclictest1443612-21/usr/sbin/munin22:15:103
1377498992518,5cyclictest1476535-21grep23:45:172
1377498992517,6cyclictest1427261-21apt-get21:30:012
137749599255,10cyclictest26-21ksoftirqd/120:58:361
1377501992411,11cyclictest42-21ksoftirqd/323:50:013
1377495992420,2cyclictest251rcuc/121:23:511
1377495992411,3cyclictest26-21ksoftirqd/120:25:481
139099230,15rtkit-daemon3056-21PK-Backend20:41:321
1377501992315,5cyclictest1389474-21python319:40:223
1377498992314,7cyclictest1403476-21ntp_states20:20:202
1377495992311,4cyclictest26-21ksoftirqd/122:48:161
1377490992313,7cyclictest436-20systemd-journal22:34:590
139099220,15rtkit-daemon3056-21PK-Backend21:19:111
139099220,15rtkit-daemon3056-21PK-Backend21:19:101
139099220,15rtkit-daemon3056-21PK-Backend20:25:310
139099220,15rtkit-daemon0-21swapper/019:55:180
1377501992218,3cyclictest42-21ksoftirqd/321:30:003
1377501992214,6cyclictest42-21ksoftirqd/323:40:043
1377498992220,1cyclictest0-21swapper/222:41:432
1377498992220,1cyclictest0-21swapper/221:51:532
1377498992220,1cyclictest0-21swapper/219:55:342
1377498992220,1cyclictest0-21swapper/219:13:292
1377498992213,6cyclictest436-20systemd-journal20:39:592
1377495992220,1cyclictest0-21swapper/122:34:531
1377495992220,1cyclictest0-21swapper/120:50:161
1377495992220,1cyclictest0-21swapper/100:08:151
1377495992219,2cyclictest3056-21PK-Backend23:31:311
1377495992218,2cyclictest1462432-21apt-get23:10:021
1377495992217,3cyclictest14806602sleep123:58:211
1377495992217,3cyclictest14448842sleep122:17:201
1377495992217,3cyclictest14448842sleep122:17:201
1377495992217,3cyclictest14392912sleep122:00:391
1377495992216,2cyclictest141rcu_preempt20:12:071
1377495992214,6cyclictest1401692-21ntp_kernel_err20:15:151
1377495992210,4cyclictest26-21ksoftirqd/100:30:291
1377495992210,4cyclictest26-21ksoftirqd/100:23:401
137749599220,21cyclictest0-21swapper/120:00:481
137749599220,20cyclictest3056-21PK-Backend20:47:181
137749599220,20cyclictest3056-21PK-Backend20:33:471
137749599220,20cyclictest3056-21PK-Backend19:44:261
137749599220,20cyclictest3056-21PK-Backend00:25:051
137749099227,9cyclictest1462926-21runrttasks23:10:030
1377490992220,1cyclictest0-21swapper/022:08:480
1377490992220,1cyclictest0-21swapper/020:55:410
1377490992220,1cyclictest0-21swapper/020:54:530
1377490992220,1cyclictest0-21swapper/000:39:560
137749099221,15cyclictest1397532-21runrttasks20:05:110
137749099220,21cyclictest0-21swapper/022:56:070
13931792210,3sleep20-21swapper/219:50:372
139099210,15rtkit-daemon3056-21PK-Backend21:58:101
139099210,14rtkit-daemon3056-21PK-Backend23:52:291
139099210,13rtkit-daemon0-21swapper/121:48:161
139099210,13rtkit-daemon0-21swapper/120:08:271
137750199212,16cyclictest1439044-21python322:00:203
1377501992119,1cyclictest0-21swapper/321:18:193
1377501992119,1cyclictest0-21swapper/321:18:193
1377501992118,2cyclictest3056-21PK-Backend19:28:283
1377501992118,2cyclictest3056-21PK-Backend19:28:283
1377501992113,6cyclictest1431387-21expr21:40:123
1377501992111,9cyclictest1469896-21apt23:30:093
1377498992119,1cyclictest0-21swapper/223:53:412
1377498992119,1cyclictest0-21swapper/223:41:512
1377498992119,1cyclictest0-21swapper/222:49:122
1377498992119,1cyclictest0-21swapper/221:15:382
1377498992119,1cyclictest0-21swapper/221:15:382
1377498992119,1cyclictest0-21swapper/221:13:282
1377498992119,1cyclictest0-21swapper/220:01:432
1377498992119,1cyclictest0-21swapper/219:39:462
1377498992119,1cyclictest0-21swapper/219:31:072
1377498992119,1cyclictest0-21swapper/200:16:472
1377498992119,1cyclictest0-21swapper/200:16:472
1377498992118,2cyclictest3056-21PK-Backend23:15:212
1377498992118,2cyclictest3056-21PK-Backend23:15:212
1377498992114,5cyclictest28650irq/123-ahci[0021:20:202
137749899210,20cyclictest0-21swapper/200:14:102
137749899210,19cyclictest3056-21PK-Backend00:20:112
137749599217,3cyclictest141rcu_preempt19:15:101
1377495992119,1cyclictest3056-21PK-Backend21:40:121
1377495992119,1cyclictest3056-21PK-Backend19:47:221
1377495992119,1cyclictest0-21swapper/123:40:511
1377495992119,1cyclictest0-21swapper/119:13:301
1377495992116,3cyclictest14307722sleep121:37:291
137749599210,9cyclictest3056-21PK-Backend23:35:151
137749599210,20cyclictest3056-21PK-Backend23:03:061
137749599210,20cyclictest0-21swapper/100:12:271
137749599210,19cyclictest3056-21PK-Backend23:28:201
137749599210,19cyclictest3056-21PK-Backend19:50:211
1377490992119,1cyclictest0-21swapper/023:46:020
1377490992119,1cyclictest0-21swapper/023:02:490
1377490992119,1cyclictest0-21swapper/022:28:290
1377490992119,1cyclictest0-21swapper/021:48:130
1377490992119,1cyclictest0-21swapper/021:23:190
1377490992119,1cyclictest0-21swapper/021:16:170
1377490992119,1cyclictest0-21swapper/021:16:160
1377490992119,1cyclictest0-21swapper/021:14:070
1377490992119,1cyclictest0-21swapper/019:21:530
1377490992119,1cyclictest0-21swapper/000:15:430
1377490992119,1cyclictest0-21swapper/000:15:420
1377490992119,1cyclictest0-21swapper/000:08:540
137749099210,20cyclictest0-21swapper/020:15:260
137749099210,20cyclictest0-21swapper/019:15:370
139099200,14rtkit-daemon0-21swapper/022:40:500
139099200,13rtkit-daemon0-21swapper/119:32:051
137750199207,12cyclictest42-21ksoftirqd/320:55:013
1377501992018,1cyclictest0-21swapper/323:50:283
1377501992018,1cyclictest0-21swapper/322:52:493
1377501992018,1cyclictest0-21swapper/322:42:213
1377501992018,1cyclictest0-21swapper/322:05:103
137750199201,7cyclictest1395899-21df20:00:123
1377501992015,3cyclictest2156-21gnome-shell21:09:093
1377501992015,3cyclictest1447474-21df_inode22:25:123
1377501992013,5cyclictest1394342-21cat19:55:173
137750199201,17cyclictest2156-21gnome-shell20:30:093
137750199200,1cyclictest3056-21PK-Backend21:55:093
137750199200,1cyclictest3056-21PK-Backend20:13:233
137750199200,1cyclictest0-21swapper/321:21:493
137750199200,1cyclictest0-21swapper/320:28:033
137750199200,19cyclictest3056-21PK-Backend20:07:423
137750199200,19cyclictest3056-21PK-Backend00:21:303
137750199200,19cyclictest3056-21PK-Backend00:02:173
137750199200,19cyclictest0-21swapper/320:44:323
1377498992018,1cyclictest0-21swapper/222:57:312
1377498992018,1cyclictest0-21swapper/222:37:102
1377498992018,1cyclictest0-21swapper/222:10:332
1377498992018,1cyclictest0-21swapper/222:09:452
1377498992018,1cyclictest0-21swapper/222:01:542
1377498992018,1cyclictest0-21swapper/221:46:132
1377498992018,1cyclictest0-21swapper/221:37:002
1377498992018,1cyclictest0-21swapper/221:01:302
1377498992018,1cyclictest0-21swapper/220:57:462
1377498992018,1cyclictest0-21swapper/220:51:522
1377498992018,1cyclictest0-21swapper/220:18:392
1377498992018,1cyclictest0-21swapper/200:35:252
1377498992018,1cyclictest0-21swapper/200:31:122
1377498992018,1cyclictest0-21swapper/200:00:312
1377498992016,3cyclictest34-21ksoftirqd/223:35:012
1377498992013,6cyclictest1472894-21sed23:35:212
1377498992011,6cyclictest436-20systemd-journal21:45:002
137749899200,1cyclictest0-21swapper/222:17:012
137749899200,1cyclictest0-21swapper/222:17:012
137749899200,19cyclictest0-21swapper/223:07:592
137749899200,19cyclictest0-21swapper/221:30:312
137749899200,19cyclictest0-21swapper/220:45:572
137749899200,19cyclictest0-21swapper/219:49:262
137749899200,19cyclictest0-21swapper/219:25:262
137749899200,19cyclictest0-21swapper/219:25:262
137749899200,19cyclictest0-21swapper/200:07:192
137749599209,3cyclictest26-21ksoftirqd/121:50:051
137749599208,4cyclictest26-21ksoftirqd/119:38:171
137749599205,4cyclictest26-21ksoftirqd/100:35:491
137749599204,2cyclictest141rcu_preempt22:45:011
1377495992018,1cyclictest3056-21PK-Backend22:14:251
1377495992016,2cyclictest14770552sleep123:46:291
1377495992016,2cyclictest14416052sleep122:09:461
1377495992016,2cyclictest14200812sleep121:07:111
1377495992015,3cyclictest14570872sleep122:53:171
1377495992015,3cyclictest14483622sleep122:27:301
1377495992015,3cyclictest14184572sleep121:03:341
1377495992015,3cyclictest13951042sleep119:58:061
137749599200,19cyclictest0-21swapper/122:21:421
137749599200,19cyclictest0-21swapper/120:21:511
1377490992018,1cyclictest0-21swapper/023:58:390
1377490992018,1cyclictest0-21swapper/022:52:220
1377490992018,1cyclictest0-21swapper/021:29:010
1377490992018,1cyclictest0-21swapper/020:43:520
1377490992018,1cyclictest0-21swapper/019:42:280
1377490992018,1cyclictest0-21swapper/019:37:210
1377490992018,1cyclictest0-21swapper/000:01:170
1377490992012,6cyclictest1490458-21awk00:25:180
137749099200,1cyclictest0-21swapper/022:47:490
137749099200,19cyclictest0-21swapper/023:19:190
137749099200,19cyclictest0-21swapper/023:19:190
137749099200,19cyclictest0-21swapper/021:41:100
137749099200,19cyclictest0-21swapper/021:35:300
137749099200,19cyclictest0-21swapper/020:02:490
137749099200,18cyclictest3056-21PK-Backend19:45:250
139099190,4rtkit-daemon3056-21PK-Backend21:27:071
139099190,13rtkit-daemon0-21swapper/123:17:221
139099190,13rtkit-daemon0-21swapper/123:17:221
139099190,12rtkit-daemon0-21swapper/023:23:090
1377501991917,1cyclictest3056-21PK-Backend19:33:203
1377501991917,1cyclictest0-21swapper/323:02:293
1377501991917,1cyclictest0-21swapper/322:10:503
1377501991917,1cyclictest0-21swapper/320:55:333
1377501991917,1cyclictest0-21swapper/300:39:563
1377501991916,2cyclictest3056-21PK-Backend19:14:013
1377501991914,3cyclictest14343412sleep321:48:503
1377501991912,5cyclictest1490465-21ntpq00:25:173
1377501991912,5cyclictest1427771-21cat21:30:123
137750199191,16cyclictest1468532-21df_inode23:25:133
137750199190,1cyclictest3056-21PK-Backend22:46:063
137750199190,1cyclictest3056-21PK-Backend21:51:383
137750199190,1cyclictest0-21swapper/322:20:173
137750199190,1cyclictest0-21swapper/321:36:103
137750199190,18cyclictest0-21swapper/323:59:403
137750199190,18cyclictest0-21swapper/320:45:203
137750199190,18cyclictest0-21swapper/300:34:563
1377498991917,1cyclictest0-21swapper/223:01:162
1377498991917,1cyclictest0-21swapper/221:56:132
1377498991917,1cyclictest0-21swapper/221:08:342
1377498991917,1cyclictest0-21swapper/220:30:492
137749899190,1cyclictest3056-21PK-Backend19:20:332
137749899190,1cyclictest3056-21PK-Backend19:15:392
137749899190,1cyclictest0-21swapper/222:52:572
137749899190,1cyclictest0-21swapper/220:42:592
137749899190,1cyclictest0-21swapper/219:42:432
137749599192,13cyclictest1466432-21apt23:20:111
1377495991917,1cyclictest3056-21PK-Backend22:57:571
1377495991917,1cyclictest0-21swapper/122:36:291
1377495991917,1cyclictest0-21swapper/119:26:001
1377495991917,1cyclictest0-21swapper/119:26:001
1377495991917,1cyclictest0-21swapper/100:15:521
1377495991917,1cyclictest0-21swapper/100:15:511
1377495991917,1cyclictest0-21swapper/100:04:501
1377495991915,2cyclictest251rcuc/121:32:211
1377495991915,2cyclictest14620792chrt23:05:401
137749099194,13cyclictest1392263-21tr19:50:130
1377490991917,1cyclictest0-21swapper/021:57:260
1377490991913,4cyclictest2156-21gnome-shell00:20:080
1377490991911,6cyclictest1-21systemd23:09:590
137749099190,1cyclictest0-21swapper/023:50:140
1377501991816,1cyclictest3056-21PK-Backend00:15:433
1377501991816,1cyclictest3056-21PK-Backend00:15:433
1377501991816,1cyclictest0-21swapper/323:06:133
1377501991816,1cyclictest0-21swapper/321:14:473
1377501991811,6cyclictest42-21ksoftirqd/319:20:003
1377501991811,5cyclictest1458169-21grep22:55:173
137750199180,7cyclictest1409032-21/usr/sbin/munin20:35:203
1377498991816,1cyclictest0-21swapper/223:57:062
1377498991816,1cyclictest0-21swapper/222:32:372
1377498991816,1cyclictest0-21swapper/222:26:422
1377498991816,1cyclictest0-21swapper/222:20:472
1377498991811,5cyclictest1468029-21cut23:25:012
137749099186,3cyclictest14847922expr00:10:120
1377490991814,2cyclictest1473372-21idleruntime-cro23:40:000
1377490991813,3cyclictest1400727-21dump-pmu-power20:15:010
1377490991811,5cyclictest1419845-21cat21:05:210
137750199174,3cyclictest42-21ksoftirqd/323:15:043
137750199174,3cyclictest42-21ksoftirqd/323:15:043
1377501991715,1cyclictest3056-21PK-Backend22:39:233
1377501991715,1cyclictest3056-21PK-Backend20:19:383
1377501991715,1cyclictest0-21swapper/319:52:273
1377501991711,5cyclictest42-21ksoftirqd/321:00:203
1377498991715,1cyclictest0-21swapper/220:25:082
1377498991711,5cyclictest1463808-21cut23:10:212
1377498991710,5cyclictest1-21systemd20:15:012
1377490991715,1cyclictest0-21swapper/020:49:470
1377490991715,1cyclictest0-21swapper/020:31:420
1377490991710,5cyclictest1468334-21tr23:25:110
137750199165,2cyclictest141rcu_preempt23:13:353
1377501991614,1cyclictest3056-21PK-Backend20:23:103
1377501991614,1cyclictest3056-21PK-Backend19:21:383
1377501991610,4cyclictest978-21snmpd19:35:163
137750199160,14cyclictest1449693-21grep22:30:203
137749899168,6cyclictest1468172-21grep23:25:112
1377495991614,1cyclictest3056-21PK-Backend20:38:061
1377490991614,1cyclictest0-21swapper/019:29:300
1377490991614,1cyclictest0-21swapper/019:29:300
1377490991610,4cyclictest1471568-21dpkg23:35:000
13985382150,4sleep23056-21PK-Backend20:05:212
1377501991511,3cyclictest3056-21PK-Backend23:20:023
1377501991511,2cyclictest42-21ksoftirqd/300:06:093
1377501991410,3cyclictest3056-21PK-Backend00:12:553
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional