You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-25 - 22:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Thu Jun 25, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
154280021470,6sleep1261ksoftirqd/112:00:111
147653721280,3sleep13056-21PK-Backend08:50:131
149103921250,3sleep00-21swapper/009:30:180
14497332960,3sleep30-21swapper/307:35:003
139099940,74rtkit-daemon0-21swapper/307:08:033
143901627745,22sleep20-21swapper/207:05:122
144039827540,13sleep00-21swapper/007:08:550
144026327145,16sleep10-21swapper/107:07:121
15383682510,3sleep10-21swapper/111:45:201
14928662500,13sleep342-21ksoftirqd/309:35:183
15242192400,3sleep23056-21PK-Backend11:05:192
1440621993112,5cyclictest13-21ksoftirqd/012:25:000
144062199310,20cyclictest13-21ksoftirqd/011:55:170
144062199299,5cyclictest13-21ksoftirqd/010:15:010
144062199298,11cyclictest13-21ksoftirqd/008:50:010
144062199288,10cyclictest13-21ksoftirqd/007:50:000
144062899278,4cyclictest42-21ksoftirqd/309:57:383
144062199276,11cyclictest1442279-21dump-pmu-power07:15:000
144062199276,10cyclictest13-21ksoftirqd/012:25:200
144062899268,4cyclictest42-21ksoftirqd/312:30:443
144062399260,24cyclictest1455509-21cstates07:50:121
144062199266,10cyclictest13-21ksoftirqd/007:30:010
144062199260,24cyclictest1497680-21cat09:50:110
144062899257,4cyclictest42-21ksoftirqd/310:18:073
144062199254,11cyclictest13-21ksoftirqd/010:55:150
14893232240,4sleep234-21ksoftirqd/209:25:182
14542692240,3chrt3056-21PK-Backend07:45:193
144062899247,15cyclictest1520819-21threads10:55:233
1440628992416,7cyclictest1442671-21df_inode07:15:023
1440628992416,7cyclictest1442671-21df_inode07:15:023
144062399241,4cyclictest1533881-21latency_hist11:35:001
144062199248,4cyclictest13-21ksoftirqd/011:55:000
144062199246,9cyclictest13-21ksoftirqd/010:45:000
144062199245,9cyclictest13-21ksoftirqd/008:55:190
144062199245,10cyclictest13-21ksoftirqd/007:40:130
144062199244,10cyclictest13-21ksoftirqd/012:05:170
144062199240,7cyclictest13-21ksoftirqd/012:35:150
144062899235,4cyclictest42-21ksoftirqd/308:42:083
1440628992317,5cyclictest42-21ksoftirqd/310:35:013
1440628992314,3cyclictest42-21ksoftirqd/308:32:093
1440628992312,4cyclictest42-21ksoftirqd/312:15:003
1440626992321,1cyclictest0-21swapper/211:50:052
1440626992321,1cyclictest0-21swapper/210:10:292
1440626992321,1cyclictest0-21swapper/209:39:472
144062699230,21cyclictest1471827-21ntp_states08:35:182
144062199238,3cyclictest13-21ksoftirqd/008:25:370
144062199236,8cyclictest13-21ksoftirqd/012:00:210
144062199234,6cyclictest13-21ksoftirqd/009:35:120
144062199233,8cyclictest1517907-21munin-run10:50:010
144062199233,8cyclictest13-21ksoftirqd/007:15:020
144062199233,8cyclictest13-21ksoftirqd/007:15:020
144062199233,7cyclictest13-21ksoftirqd/011:00:130
144062199233,7cyclictest13-21ksoftirqd/011:00:130
144062199233,6cyclictest13-21ksoftirqd/008:55:010
144062199232,10cyclictest13-21ksoftirqd/012:20:000
1440621992319,3cyclictest161rcuc/010:02:000
144062199230,7cyclictest13-21ksoftirqd/007:35:010
144062899228,4cyclictest42-21ksoftirqd/311:45:363
144062899226,4cyclictest42-21ksoftirqd/309:00:443
1440628992217,4cyclictest1549589-21/usr/sbin/munin12:20:123
1440626992220,1cyclictest0-21swapper/210:41:312
1440626992220,1cyclictest0-21swapper/210:17:052
1440626992220,1cyclictest0-21swapper/208:15:042
1440626992220,1cyclictest0-21swapper/207:53:412
1440626992213,7cyclictest883-21dbus-daemon11:20:002
144062699220,21cyclictest0-21swapper/209:42:582
144062699220,21cyclictest0-21swapper/208:30:242
144062199224,9cyclictest13-21ksoftirqd/008:35:390
144062199223,8cyclictest13-21ksoftirqd/011:50:010
144062199223,8cyclictest13-21ksoftirqd/007:25:000
144062199223,5cyclictest13-21ksoftirqd/010:05:120
144062199222,8cyclictest13-21ksoftirqd/007:35:160
144062199222,14cyclictest1506004-21apt10:15:100
1440621992220,1cyclictest0-21swapper/011:39:140
1440621992220,1cyclictest0-21swapper/011:11:360
144062199221,9cyclictest13-21ksoftirqd/010:50:120
1440621992218,3cyclictest161rcuc/007:50:160
144062199220,21cyclictest0-21swapper/011:21:450
144062899218,4cyclictest42-21ksoftirqd/308:35:353
144062899214,2cyclictest141rcu_preempt11:40:113
144062899213,3cyclictest141rcu_preempt11:31:503
144062899213,3cyclictest141rcu_preempt09:48:123
144062899213,3cyclictest141rcu_preempt08:45:333
1440628992116,4cyclictest42-21ksoftirqd/311:55:173
1440628992114,5cyclictest1490132-21cron09:30:003
144062899211,2cyclictest141rcu_preempt12:38:583
144062899210,8cyclictest3056-21PK-Backend09:50:043
144062899210,20cyclictest0-21swapper/307:11:483
144062899210,19cyclictest3056-21PK-Backend11:26:253
144062899210,19cyclictest3056-21PK-Backend10:03:173
144062899210,19cyclictest3056-21PK-Backend09:15:313
144062899210,19cyclictest3056-21PK-Backend09:14:573
144062899210,19cyclictest3056-21PK-Backend08:04:073
1440626992119,1cyclictest0-21swapper/212:31:162
1440626992119,1cyclictest0-21swapper/212:18:522
1440626992119,1cyclictest0-21swapper/212:04:252
1440626992119,1cyclictest0-21swapper/210:35:442
1440626992119,1cyclictest0-21swapper/210:32:322
1440626992119,1cyclictest0-21swapper/210:05:422
1440626992119,1cyclictest0-21swapper/209:16:142
1440626992119,1cyclictest0-21swapper/207:36:582
1440626992119,1cyclictest0-21swapper/207:14:202
1440626992113,7cyclictest1544878-21cat12:05:162
144062699210,20cyclictest0-21swapper/211:14:372
144062699210,20cyclictest0-21swapper/211:01:322
144062699210,20cyclictest0-21swapper/211:01:322
144062699210,1cyclictest0-21swapper/209:02:552
144062399217,4cyclictest1442281-21cron07:15:001
1440623992114,5cyclictest1455055-21date07:50:011
1440623992113,6cyclictest1475642-21seq08:45:231
144062399210,1cyclictest0-21swapper/108:20:131
144062399210,19cyclictest3056-21PK-Backend11:58:471
144062399210,19cyclictest3056-21PK-Backend10:30:531
144062399210,19cyclictest3056-21PK-Backend09:33:481
144062399210,19cyclictest3056-21PK-Backend07:19:161
144062399210,19cyclictest3056-21PK-Backend07:19:151
144062199212,8cyclictest13-21ksoftirqd/009:25:170
1440621992119,1cyclictest0-21swapper/012:34:450
1440621992119,1cyclictest0-21swapper/011:30:570
1440621992119,1cyclictest0-21swapper/011:17:250
1440621992119,1cyclictest0-21swapper/011:09:000
1440621992119,1cyclictest0-21swapper/010:33:380
1440621992119,1cyclictest0-21swapper/010:23:580
1440621992119,1cyclictest0-21swapper/009:55:260
1440621992119,1cyclictest0-21swapper/009:42:010
1440621992119,1cyclictest0-21swapper/008:41:040
1440621992118,2cyclictest161rcuc/009:23:220
144062199211,7cyclictest13-21ksoftirqd/008:20:140
144062199210,20cyclictest0-21swapper/011:26:040
144062199210,20cyclictest0-21swapper/009:46:260
144062199210,20cyclictest0-21swapper/009:05:240
144062199210,20cyclictest0-21swapper/009:04:500
144062199210,20cyclictest0-21swapper/008:16:090
144062899207,4cyclictest141rcu_preempt10:54:393
144062899207,2cyclictest15054262chrt10:11:113
144062899206,4cyclictest42-21ksoftirqd/311:05:313
144062899204,4cyclictest42-21ksoftirqd/308:15:253
144062899204,2cyclictest141rcu_preempt10:22:373
144062899203,3cyclictest42-21ksoftirqd/312:11:223
1440628992018,1cyclictest3056-21PK-Backend11:35:033
1440628992017,2cyclictest0-21swapper/307:27:303
1440628992012,6cyclictest1461400-21awk08:05:173
1440628992012,2cyclictest42-21ksoftirqd/309:24:193
1440628992011,3cyclictest42-21ksoftirqd/307:37:443
144062899200,6cyclictest3056-21PK-Backend08:25:123
144062899200,19cyclictest3056-21PK-Backend11:22:393
144062899200,19cyclictest3056-21PK-Backend11:15:093
144062899200,19cyclictest0-21swapper/312:28:573
144062899200,19cyclictest0-21swapper/312:03:083
144062899200,19cyclictest0-21swapper/310:48:043
144062899200,19cyclictest0-21swapper/308:55:583
144062899200,19cyclictest0-21swapper/308:53:213
144062899200,17cyclictest1513809-21python310:35:193
1440626992018,1cyclictest0-21swapper/210:20:292
1440626992018,1cyclictest0-21swapper/209:58:262
1440626992018,1cyclictest0-21swapper/209:11:552
1440626992018,1cyclictest0-21swapper/208:50:392
1440626992018,1cyclictest0-21swapper/208:26:192
1440626992018,1cyclictest0-21swapper/208:10:372
1440626992018,1cyclictest0-21swapper/207:33:542
1440626992013,5cyclictest1519593-21cron10:55:002
144062699200,1cyclictest0-21swapper/211:47:492
144062699200,1cyclictest0-21swapper/210:46:452
144062699200,1cyclictest0-21swapper/209:05:332
144062699200,19cyclictest0-21swapper/211:57:162
144062699200,19cyclictest0-21swapper/211:31:342
144062699200,19cyclictest0-21swapper/209:32:362
144062699200,19cyclictest0-21swapper/208:43:222
144062699200,19cyclictest0-21swapper/207:55:372
144062699200,19cyclictest0-21swapper/207:41:582
1440623992017,2cyclictest3056-21PK-Backend10:40:471
1440623992013,6cyclictest1467989-21df08:25:141
1440623992013,5cyclictest1493641-21pluginstate09:40:011
1440623992013,5cyclictest1482946-21apt-get09:10:011
144062399200,1cyclictest0-21swapper/111:19:471
144062399200,1cyclictest0-21swapper/108:30:551
144062399200,19cyclictest0-21swapper/111:36:091
144062399200,19cyclictest0-21swapper/111:29:001
144062399200,19cyclictest0-21swapper/110:24:451
144062399200,19cyclictest0-21swapper/109:45:241
144062399200,19cyclictest0-21swapper/109:40:361
144062399200,19cyclictest0-21swapper/109:20:231
144062399200,19cyclictest0-21swapper/108:01:001
144062199205,7cyclictest13-21ksoftirqd/008:01:310
1440621992018,1cyclictest0-21swapper/008:14:130
1440621992018,1cyclictest0-21swapper/008:05:350
1440621992018,1cyclictest0-21swapper/007:59:260
1440621992017,2cyclictest161rcuc/009:17:200
1440621992016,3cyclictest161rcuc/011:44:330
1440621992016,3cyclictest161rcuc/008:30:580
144062199200,7cyclictest13-21ksoftirqd/010:35:110
144062199200,1cyclictest0-21swapper/010:26:010
139099200,13rtkit-daemon0-21swapper/209:48:302
144062899195,3cyclictest42-21ksoftirqd/308:10:243
144062899195,3cyclictest1452903-21chrt07:40:583
1440628991917,1cyclictest3056-21PK-Backend10:05:063
1440628991917,1cyclictest3056-21PK-Backend09:40:393
144062899191,16cyclictest1458090-21python307:55:213
144062899190,6cyclictest3056-21PK-Backend11:15:003
144062899190,4cyclictest1545277-21python312:05:203
144062899190,1cyclictest3056-21PK-Backend08:21:373
144062899190,18cyclictest0-21swapper/309:34:103
1440626991917,1cyclictest0-21swapper/212:27:362
1440626991917,1cyclictest0-21swapper/207:29:412
1440626991917,1cyclictest0-21swapper/207:15:282
1440626991917,1cyclictest0-21swapper/207:15:282
144062699190,1cyclictest3056-21PK-Backend12:10:412
144062699190,1cyclictest0-21swapper/210:55:432
144062699190,1cyclictest0-21swapper/209:22:162
144062699190,1cyclictest0-21swapper/208:55:532
144062699190,18cyclictest0-21swapper/211:21:122
1440623991917,1cyclictest3056-21PK-Backend07:27:481
1440623991917,1cyclictest0-21swapper/110:13:021
1440623991917,1cyclictest0-21swapper/108:37:101
1440623991917,1cyclictest0-21swapper/108:11:541
1440623991917,1cyclictest0-21swapper/107:57:081
1440623991916,2cyclictest3056-21PK-Backend10:54:461
1440623991913,5cyclictest1526573-21systemctl11:15:001
1440623991912,5cyclictest1461057-21df08:05:131
1440623991910,7cyclictest1554166-21users12:30:201
144062399190,5cyclictest3056-21PK-Backend07:30:111
144062399190,1cyclictest3056-21PK-Backend09:27:331
144062399190,1cyclictest3056-21PK-Backend07:20:251
144062399190,1cyclictest0-21swapper/112:15:031
144062399190,1cyclictest0-21swapper/110:26:471
144062399190,1cyclictest0-21swapper/107:40:251
144062399190,18cyclictest3056-21PK-Backend10:47:291
144062399190,18cyclictest0-21swapper/111:08:451
144062399190,18cyclictest0-21swapper/110:08:351
144062399190,18cyclictest0-21swapper/109:58:421
144062399190,18cyclictest0-21swapper/109:50:241
1440621991917,1cyclictest0-21swapper/009:10:380
144062199190,18cyclictest0-21swapper/012:10:110
139099190,12rtkit-daemon0-21swapper/210:26:042
139099190,12rtkit-daemon0-21swapper/208:07:462
144062899188,3cyclictest141rcu_preempt11:51:283
1440628991816,1cyclictest3056-21PK-Backend10:29:133
1440628991816,1cyclictest3056-21PK-Backend09:07:073
1440628991816,1cyclictest3056-21PK-Backend07:52:243
1440628991814,2cyclictest1522003-21hddtemp_smartct11:00:153
1440628991814,2cyclictest1522003-21hddtemp_smartct11:00:153
1440628991813,3cyclictest1445644-21python307:20:193
144062899180,16cyclictest1515072-21hddtemp_smartct10:40:153
1440626991816,1cyclictest0-21swapper/210:04:262
1440626991816,1cyclictest0-21swapper/209:51:232
144062699180,1cyclictest0-21swapper/212:35:282
1440623991816,1cyclictest3056-21PK-Backend12:28:551
1440623991816,1cyclictest3056-21PK-Backend07:37:211
1440623991816,1cyclictest0-21swapper/109:10:141
1440623991815,2cyclictest3056-21PK-Backend10:57:231
1440623991812,4cyclictest1479701-21sed09:00:001
1440623991811,5cyclictest436-20systemd-journal12:14:121
1440623991810,6cyclictest1555601-21awk12:35:201
139099180,12rtkit-daemon0-21swapper/208:47:192
144062699179,6cyclictest1551067-21cron12:24:592
1440626991715,1cyclictest0-21swapper/211:27:142
1440626991715,1cyclictest0-21swapper/208:20:172
1440626991712,4cyclictest1446268-21systemd-run07:25:002
1440623991711,5cyclictest1540667-21cron11:55:001
1440623991711,5cyclictest1501411-21cat10:00:171
1440623991710,6cyclictest1506998-21sed10:15:211
1440623991710,5cyclictest1536375-21cat11:40:171
1440623991710,5cyclictest1485594-21cat09:15:141
1440626991614,1cyclictest0-21swapper/207:48:062
144062399169,5cyclictest1474275-21cron08:44:591
144062399169,5cyclictest1464672-21wc08:15:171
144062399161,4cyclictest141rcu_preempt12:25:001
1440623991612,2cyclictest26-21ksoftirqd/109:00:111
144062699158,6cyclictest883-21dbus-daemon08:05:002
1440626991510,4cyclictest1534432-21basename11:35:152
144062399151,3cyclictest141rcu_preempt10:35:111
1440623991513,1cyclictest3056-21PK-Backend12:05:231
144062399151,11cyclictest1528806-21apt11:20:111
139099150,7rtkit-daemon978-21snmpd11:43:322
144062399145,2cyclictest141rcu_preempt11:00:211
144062399145,2cyclictest141rcu_preempt11:00:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional