You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-12 - 22:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Jul 12, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
269164422710,5sleep342-21ksoftirqd/307:05:213
279630221300,4sleep10-21swapper/112:00:221
269268021010,4sleep03056-21PK-Backend07:10:010
139099750,66rtkit-daemon0-21swapper/107:06:581
269224826444,14sleep20-21swapper/207:06:142
27973092440,4sleep33056-21PK-Backend12:05:133
27497582400,8sleep2341ksoftirqd/209:50:102
27497582400,8sleep2341ksoftirqd/209:50:102
27147312370,3sleep10-21swapper/108:10:011
2692686993211,9cyclictest42-21ksoftirqd/312:16:433
27605302290,3chrt3056-21PK-Backend10:20:153
2692686992715,3cyclictest42-21ksoftirqd/311:17:283
2692686992513,4cyclictest42-21ksoftirqd/307:46:443
269268699250,6cyclictest3056-21PK-Backend12:30:543
2692682992522,2cyclictest0-21swapper/208:18:382
269268699242,5cyclictest42-21ksoftirqd/307:25:023
2692686992413,3cyclictest42-21ksoftirqd/311:21:093
269268699240,6cyclictest3056-21PK-Backend12:30:003
269268699240,3cyclictest42-21ksoftirqd/307:15:013
269268699240,16cyclictest3056-21PK-Backend09:46:383
269267399243,2cyclictest141rcu_preempt08:15:000
2692686992318,3cyclictest2733599-21cron09:05:003
2692686992312,3cyclictest42-21ksoftirqd/310:05:063
2692686992312,3cyclictest42-21ksoftirqd/308:07:573
269268699230,5cyclictest3056-21PK-Backend09:55:003
269268699230,5cyclictest3056-21PK-Backend09:54:593
269268699230,21cyclictest3056-21PK-Backend10:15:523
269267699230,21cyclictest2808832-21users12:35:231
27357162220,6sleep22735720-21latency_hist09:10:012
2692686992219,2cyclictest3056-21PK-Backend12:38:083
2692686992219,2cyclictest3056-21PK-Backend12:04:283
2692686992219,2cyclictest3056-21PK-Backend07:54:183
2692686992214,6cyclictest2718000-21cron08:20:003
2692686992212,3cyclictest42-21ksoftirqd/312:21:443
2692686992211,3cyclictest42-21ksoftirqd/311:37:133
2692686992210,4cyclictest42-21ksoftirqd/311:11:233
269268699220,6cyclictest3056-21PK-Backend09:15:423
269268699220,6cyclictest3056-21PK-Backend08:50:003
269268699220,6cyclictest3056-21PK-Backend08:42:303
269268699220,4cyclictest3056-21PK-Backend08:38:203
269268699220,3cyclictest2768577-21seq10:42:013
269268699220,20cyclictest3056-21PK-Backend11:49:343
269268699220,20cyclictest3056-21PK-Backend10:45:193
269268299220,20cyclictest0-21swapper/211:50:222
2692676992219,2cyclictest3056-21PK-Backend09:09:471
2692676992218,2cyclictest27055252chrt07:44:501
2692676992215,5cyclictest1-21systemd10:30:011
269267699220,20cyclictest0-21swapper/107:20:441
269267399227,4cyclictest13-21ksoftirqd/012:16:020
139099220,15rtkit-daemon0-21swapper/308:33:533
139099220,15rtkit-daemon0-21swapper/209:03:282
27746992210,4sleep32774703-21tr11:00:133
269268699213,7cyclictest42-21ksoftirqd/310:55:083
2692686992119,1cyclictest3056-21PK-Backend12:11:383
2692686992119,1cyclictest3056-21PK-Backend09:24:143
2692686992119,1cyclictest3056-21PK-Backend08:57:453
2692686992119,1cyclictest3056-21PK-Backend08:57:453
2692686992119,1cyclictest0-21swapper/310:36:133
2692686992119,1cyclictest0-21swapper/309:10:563
2692686992110,3cyclictest42-21ksoftirqd/310:25:383
269268699210,6cyclictest3056-21PK-Backend10:15:013
269268699210,6cyclictest3056-21PK-Backend09:45:013
269268699210,6cyclictest3056-21PK-Backend08:51:003
269268699210,20cyclictest0-21swapper/308:22:363
269268699210,19cyclictest3056-21PK-Backend09:05:013
2692682992119,1cyclictest0-21swapper/207:38:432
2692682992114,5cyclictest2755659-21awk10:05:192
2692676992119,1cyclictest3056-21PK-Backend10:39:101
2692676992119,1cyclictest0-21swapper/110:17:061
2692676992119,1cyclictest0-21swapper/108:35:201
2692676992112,7cyclictest2785534-21memory11:30:181
2692676992112,7cyclictest2773633-21awk10:55:231
269267699210,2cyclictest2790054-21cut11:44:591
269267699210,20cyclictest0-21swapper/111:55:281
269267699210,20cyclictest0-21swapper/110:24:301
269267699210,20cyclictest0-21swapper/109:02:301
269267699210,20cyclictest0-21swapper/108:00:051
269267699210,20cyclictest0-21swapper/107:55:321
269267699210,19cyclictest3056-21PK-Backend08:21:271
269267399219,2cyclictest141rcu_preempt08:01:390
2692673992119,1cyclictest0-21swapper/011:12:210
2692673992119,1cyclictest0-21swapper/007:12:310
2692673992113,6cyclictest2806381-21df12:30:120
2692673992113,6cyclictest2770833-21apt10:50:120
2692673992111,3cyclictest13-21ksoftirqd/010:38:140
269267399210,20cyclictest2707324-21dpkg07:49:150
269267399210,20cyclictest0-21swapper/011:35:460
269267399210,20cyclictest0-21swapper/007:28:400
269267399210,18cyclictest2724955-21latency_hist08:40:000
139099210,15rtkit-daemon3056-21PK-Backend11:52:073
139099210,14rtkit-daemon0-21swapper/211:34:422
139099210,13rtkit-daemon3056-21PK-Backend11:05:523
139099210,13rtkit-daemon0-21swapper/209:44:562
27835612200,8sleep0161rcuc/011:25:120
269268699203,8cyclictest3056-21PK-Backend10:30:163
2692686992018,1cyclictest3056-21PK-Backend11:31:363
2692686992018,1cyclictest3056-21PK-Backend07:25:043
2692686992018,1cyclictest0-21swapper/310:52:293
2692686992018,1cyclictest0-21swapper/308:03:233
2692686992017,2cyclictest3056-21PK-Backend07:17:003
2692686992016,3cyclictest411rcuc/307:36:453
2692686992011,7cyclictest2721480-21sh08:30:003
269268699200,8cyclictest3056-21PK-Backend09:30:013
269268699200,6cyclictest3056-21PK-Backend11:58:093
269268699200,6cyclictest3056-21PK-Backend07:56:543
269268699200,1cyclictest3056-21PK-Backend09:55:173
269268699200,19cyclictest3056-21PK-Backend11:43:463
269268699200,19cyclictest3056-21PK-Backend08:13:173
269268699200,19cyclictest3056-21PK-Backend07:33:083
2692682992018,1cyclictest0-21swapper/208:37:582
2692682992018,1cyclictest0-21swapper/207:54:522
269267699209,9cyclictest2726710-21unin-run08:45:001
2692676992018,1cyclictest0-21swapper/111:15:471
2692676992018,1cyclictest0-21swapper/111:11:071
2692676992018,1cyclictest0-21swapper/109:27:591
2692676992018,1cyclictest0-21swapper/108:57:231
2692676992018,1cyclictest0-21swapper/108:57:231
2692676992018,1cyclictest0-21swapper/107:26:391
2692676992012,6cyclictest2729344-21grep08:50:191
269267699200,2cyclictest2707531-40unattended-upgr07:49:231
269267699200,1cyclictest0-21swapper/111:28:451
269267699200,19cyclictest3056-21PK-Backend10:41:051
269267699200,19cyclictest0-21swapper/112:18:201
269267699200,19cyclictest0-21swapper/112:05:561
269267699200,19cyclictest0-21swapper/111:52:101
269267699200,19cyclictest0-21swapper/111:39:471
269267699200,19cyclictest0-21swapper/109:49:421
269267699200,19cyclictest0-21swapper/109:31:301
269267699200,19cyclictest0-21swapper/107:19:561
269267399208,6cyclictest141rcu_preempt12:27:540
269267399207,4cyclictest13-21ksoftirqd/011:50:560
2692673992018,1cyclictest0-21swapper/010:47:540
2692673992018,1cyclictest0-21swapper/009:18:100
2692673992018,1cyclictest0-21swapper/009:01:280
2692673992018,1cyclictest0-21swapper/007:50:500
2692673992018,1cyclictest0-21swapper/007:40:090
2692673992015,4cyclictest13-21ksoftirqd/008:10:000
2692673992014,5cyclictest2741748-21grep09:25:170
2692673992014,4cyclictest13-21ksoftirqd/009:10:120
269267399200,1cyclictest0-21swapper/012:35:490
269267399200,19cyclictest3056-21PK-Backend08:29:170
269267399200,19cyclictest0-21swapper/011:59:120
269267399200,19cyclictest0-21swapper/010:42:400
269267399200,19cyclictest0-21swapper/010:23:530
269267399200,19cyclictest0-21swapper/010:18:270
269267399200,19cyclictest0-21swapper/010:11:030
269267399200,19cyclictest0-21swapper/007:16:500
139099200,14rtkit-daemon0-21swapper/212:35:512
139099200,14rtkit-daemon0-21swapper/210:16:192
139099200,14rtkit-daemon0-21swapper/209:37:002
139099200,14rtkit-daemon0-21swapper/207:47:352
139099200,13rtkit-daemon0-21swapper/210:14:352
2692686991917,1cyclictest3056-21PK-Backend11:27:433
2692686991917,1cyclictest3056-21PK-Backend09:34:353
269268699190,7cyclictest3056-21PK-Backend07:42:033
269268699190,6cyclictest3056-21PK-Backend10:00:233
2692682991917,1cyclictest0-21swapper/212:10:232
2692682991917,1cyclictest0-21swapper/208:00:202
2692682991913,4cyclictest34-21ksoftirqd/209:30:002
269268299190,1cyclictest0-21swapper/212:30:092
2692676991917,1cyclictest0-21swapper/110:00:231
2692676991917,1cyclictest0-21swapper/109:35:021
2692676991917,1cyclictest0-21swapper/109:21:091
2692676991917,1cyclictest0-21swapper/107:38:421
2692676991917,1cyclictest0-21swapper/107:30:241
2692676991912,5cyclictest2758185-21pluginstate10:15:011
2692676991912,5cyclictest2752695-21sshd09:56:531
2692676991911,6cyclictest2803191-21grep12:20:181
2692676991910,7cyclictest918-21rs:main1
2692676991910,7cyclictest918-21rs:main1
269267699190,1cyclictest3056-21PK-Backend08:32:561
269267699190,1cyclictest0-21swapper/111:21:011
269267699190,1cyclictest0-21swapper/111:08:231
269267699190,1cyclictest0-21swapper/109:41:161
269267699190,19cyclictest0-21swapper/107:50:591
269267699190,18cyclictest0-21swapper/109:17:381
269267699190,18cyclictest0-21swapper/108:26:481
269267699190,18cyclictest0-21swapper/107:14:491
269267399195,4cyclictest13-21ksoftirqd/007:35:210
2692673991917,1cyclictest0-21swapper/012:11:010
2692673991917,1cyclictest0-21swapper/012:07:440
2692673991917,1cyclictest0-21swapper/011:07:410
2692673991917,1cyclictest0-21swapper/010:31:040
2692673991917,1cyclictest0-21swapper/010:05:020
2692673991917,1cyclictest0-21swapper/009:52:580
2692673991917,1cyclictest0-21swapper/009:52:570
2692673991917,1cyclictest0-21swapper/009:06:350
2692673991917,1cyclictest0-21swapper/008:56:200
2692673991917,1cyclictest0-21swapper/008:56:200
2692673991917,1cyclictest0-21swapper/007:33:130
2692673991917,1cyclictest0-21swapper/007:22:450
2692673991915,3cyclictest0-21swapper/011:20:170
2692673991911,2cyclictest141rcu_preempt09:55:560
269267399190,1cyclictest0-21swapper/008:43:570
269267399190,18cyclictest0-21swapper/011:48:440
269267399190,18cyclictest0-21swapper/011:34:240
269267399190,18cyclictest0-21swapper/009:44:330
269267399190,18cyclictest0-21swapper/008:22:200
269267399190,18cyclictest0-21swapper/007:55:230
139099190,12rtkit-daemon0-21swapper/211:23:482
269268699187,3cyclictest42-21ksoftirqd/309:35:273
2692682991816,1cyclictest0-21swapper/212:19:422
2692682991816,1cyclictest0-21swapper/211:56:512
2692682991816,1cyclictest0-21swapper/211:38:122
2692682991816,1cyclictest0-21swapper/208:50:212
2692682991811,5cyclictest2752047-21cat09:55:162
2692682991810,6cyclictest436-20systemd-journal10:05:002
2692676991816,1cyclictest0-21swapper/109:10:351
2692676991816,1cyclictest0-21swapper/108:47:571
2692676991812,5cyclictest2716376-21sed08:15:011
2692676991811,5cyclictest2804374-21wc12:25:131
2692676991811,5cyclictest2790229-21cat11:45:131
269267399189,6cyclictest436-20systemd-journal09:50:000
269267399183,2cyclictest141rcu_preempt09:35:210
269267399183,2cyclictest141rcu_preempt08:19:190
2692673991816,1cyclictest0-21swapper/011:41:000
2692673991816,1cyclictest0-21swapper/011:17:000
2692673991816,1cyclictest0-21swapper/011:04:230
2692673991816,1cyclictest0-21swapper/008:53:220
2692673991815,2cyclictest1095-21runrttasks10:59:500
2692673991810,6cyclictest2739741-21fw_packets09:20:170
27695182170,3sleep20-21swapper/210:45:162
269268299179,6cyclictest2736940-21users09:10:212
2692682991715,1cyclictest0-21swapper/211:03:452
2692682991715,1cyclictest0-21swapper/210:56:422
2692682991715,1cyclictest0-21swapper/210:51:212
2692682991715,1cyclictest0-21swapper/210:25:052
2692682991715,1cyclictest0-21swapper/209:47:132
2692682991715,1cyclictest0-21swapper/208:09:052
2692682991715,1cyclictest0-21swapper/207:57:362
2692682991715,1cyclictest0-21swapper/207:20:522
2692682991713,3cyclictest2804941-21needreboot12:25:182
2692682991712,4cyclictest2796806-21cron12:04:592
2692682991712,3cyclictest2692884-21apt07:10:122
269268299171,15cyclictest953-21gmain11:28:292
2692682991711,5cyclictest2766185-21cat10:35:172
2692682991711,4cyclictest2768094-21cat10:40:192
2692682991710,6cyclictest2720404-21awk08:25:162
2692682991710,5cyclictest2715433-21cat08:10:182
2692676991712,4cyclictest2764075-21expr10:30:151
269267699171,12cyclictest2769098-21apt10:45:121
2692676991710,5cyclictest2807010-21sed12:30:201
2692676991710,5cyclictest2799743-21cat12:10:211
269267399173,5cyclictest13-21ksoftirqd/008:45:340
2692673991715,1cyclictest0-21swapper/012:21:280
2692673991715,1cyclictest0-21swapper/008:32:280
2692673991713,3cyclictest2795856-21cut12:00:170
269268299169,5cyclictest34-21ksoftirqd/212:20:112
269268299168,6cyclictest913-21gdbus10:35:012
2692682991614,1cyclictest0-21swapper/209:24:422
2692682991614,1cyclictest0-21swapper/209:17:592
2692682991614,1cyclictest0-21swapper/208:40:072
2692682991614,1cyclictest0-21swapper/208:20:472
2692682991614,1cyclictest0-21swapper/207:30:462
2692682991612,3cyclictest978-21snmpd07:26:272
2692682991612,3cyclictest28650irq/123-ahci[0011:40:202
2692682991611,3cyclictest1095-21runrttasks11:07:322
2692682991610,5cyclictest2727118-21tr08:45:122
269268299160,15cyclictest0-21swapper/210:21:272
269268299160,15cyclictest0-21swapper/207:41:272
269267699168,6cyclictest2755663-21ntp_kernel_err10:05:191
2692676991614,1cyclictest0-21swapper/111:03:101
2692673991614,1cyclictest0-21swapper/010:29:080
2692673991611,3cyclictest2754171-21xargs10:00:240
269268299159,5cyclictest2694932-21cat07:15:022
269268299158,5cyclictest34-21ksoftirqd/208:30:122
269268299158,5cyclictest2730706-21expr08:55:142
269268299158,5cyclictest2730706-21expr08:55:142
2692682991511,3cyclictest2778457-21if_enp2s011:10:162
2692682991510,4cyclictest2790342-21grep11:45:132
269268299150,14cyclictest0-21swapper/211:16:082
269267399159,3cyclictest2743519-21ntp_kernel_err09:30:190
269268299149,4cyclictest2797378-21expr12:05:132
269267699149,3cyclictest2771025-21cstates10:50:141
2692676991412,1cyclictest3056-21PK-Backend08:17:421
269268299139,3cyclictest978-21snmpd09:34:342
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional