You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 08:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Jul 21, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
334066523180,19sleep20-21swapper/219:05:242
342663521050,4sleep03056-21PK-Backend23:06:150
334762321050,3sleep20-21swapper/219:25:152
341291221020,4sleep30-21swapper/322:30:013
139099870,73rtkit-daemon0-21swapper/019:08:280
334065627447,16sleep10-21swapper/119:05:251
334129426840,9sleep30-21swapper/319:09:123
34339712560,4sleep30-21swapper/323:25:233
334150099406,19cyclictest0-21swapper/222:33:532
334150099396,19cyclictest0-21swapper/200:21:322
334150099391,21cyclictest0-21swapper/221:45:002
334150099380,27cyclictest3395162-21chrt21:39:312
334150099380,25cyclictest3457121-21chrt00:32:582
334150099376,18cyclictest0-21swapper/223:11:582
334150099376,18cyclictest0-21swapper/220:46:062
334150099372,21cyclictest0-21swapper/222:42:192
334150099372,21cyclictest0-21swapper/222:42:192
334150099371,22cyclictest0-21swapper/219:44:312
334150099371,21cyclictest0-21swapper/221:01:502
334150099363,20cyclictest0-21swapper/220:11:062
334150099362,20cyclictest0-21swapper/223:31:312
334150099362,20cyclictest0-21swapper/222:07:542
334150099362,20cyclictest0-21swapper/220:24:342
334150099361,22cyclictest0-21swapper/223:46:192
334150099361,21cyclictest0-21swapper/223:15:472
334150099361,21cyclictest0-21swapper/221:49:182
334150099361,21cyclictest0-21swapper/221:25:402
334150099361,21cyclictest0-21swapper/220:43:462
334150099361,21cyclictest0-21swapper/219:35:052
334150099361,21cyclictest0-21swapper/219:12:472
334150099361,21cyclictest0-21swapper/200:07:442
334150099360,26cyclictest3350475-21chrt19:33:442
334150099360,21cyclictest0-21swapper/200:27:052
334150099352,20cyclictest0-21swapper/223:41:102
334150099352,20cyclictest0-21swapper/223:41:102
334150099352,20cyclictest0-21swapper/221:19:102
334150099352,20cyclictest0-21swapper/219:54:302
334150099352,19cyclictest0-21swapper/220:02:512
334150099352,19cyclictest0-21swapper/219:59:272
334150099352,19cyclictest0-21swapper/200:01:472
334150099352,18cyclictest0-21swapper/223:21:332
334150099351,20cyclictest0-21swapper/222:15:442
334150099350,20cyclictest3458760-21chrt00:36:472
334150099350,19cyclictest33932732sleep221:32:302
334150099350,19cyclictest0-21swapper/222:01:052
334150099350,19cyclictest0-21swapper/221:07:352
139099350,3rtkit-daemon0-21swapper/000:01:350
334150099342,22cyclictest0-21swapper/222:46:442
334150099342,21cyclictest0-21swapper/221:24:362
334150099342,19cyclictest0-21swapper/223:57:422
334150099342,19cyclictest0-21swapper/219:24:262
334150099342,18cyclictest0-21swapper/220:17:032
334150099340,21cyclictest0-21swapper/223:04:002
334150099340,20cyclictest3450004-21chrt00:14:502
334150099340,19cyclictest0-21swapper/223:25:582
334150099340,19cyclictest0-21swapper/222:27:312
334150099340,19cyclictest0-21swapper/220:07:492
334150099340,19cyclictest0-21swapper/219:18:122
334150099340,18cyclictest0-21swapper/223:05:412
334150099340,18cyclictest0-21swapper/221:59:162
334150099332,19cyclictest0-21swapper/223:52:282
334150099330,22cyclictest3378897-21chrt20:52:402
334150099330,22cyclictest3378897-21chrt20:52:402
334150099330,18cyclictest0-21swapper/219:49:202
334150099330,18cyclictest0-21swapper/219:49:202
334150099330,17cyclictest0-21swapper/222:54:182
334150099322,20cyclictest0-21swapper/222:22:582
334150099320,19cyclictest0-21swapper/220:58:572
334150099310,19cyclictest0-21swapper/222:55:102
334150099300,21cyclictest0-21swapper/220:28:302
334150099300,19cyclictest0-21swapper/221:14:332
334150099300,18cyclictest0-21swapper/222:39:422
334150099300,18cyclictest0-21swapper/222:11:482
334150099300,18cyclictest0-21swapper/220:38:562
334150299281,24cyclictest3455277-21lsb_release00:26:403
139099270,19rtkit-daemon3056-21PK-Backend23:52:221
34056082250,4sleep33405607-21turbostat.cron22:10:003
33533432250,3chrt3056-21PK-Backend19:40:200
334150099251,22cyclictest3450511-21cstates00:15:132
334150099250,24cyclictest0-21swapper/223:39:152
334150099250,23cyclictest3399158-21cstates21:50:142
334149299256,11cyclictest13-21ksoftirqd/020:40:010
334149299253,7cyclictest13-21ksoftirqd/000:18:060
334149299251,22cyclictest3442352-21ntp_states23:50:200
34221792240,13sleep126-21ksoftirqd/122:55:171
3341502992318,3cyclictest3397217-21apt21:45:123
334150099231,20cyclictest3371179-21sshd20:30:152
334149299238,4cyclictest13-21ksoftirqd/020:17:420
334149299235,5cyclictest13-21ksoftirqd/022:21:410
334149299234,7cyclictest13-21ksoftirqd/021:55:140
3341492992314,8cyclictest3385643-21sed21:10:160
139099230,14rtkit-daemon0-21swapper/019:56:280
34221252220,4sleep03056-21PK-Backend22:55:160
3341502992220,1cyclictest0-21swapper/300:10:043
3341502992217,3cyclictest3361419-21grep20:05:143
3341495992220,1cyclictest3056-21PK-Backend20:05:391
3341495992220,1cyclictest3056-21PK-Backend19:12:531
3341495992220,1cyclictest0-21swapper/119:20:231
3341495992219,2cyclictest3056-21PK-Backend23:03:101
3341495992219,2cyclictest3056-21PK-Backend20:23:511
3341495992219,2cyclictest3056-21PK-Backend19:52:001
3341495992214,6cyclictest3441337-21cron23:50:001
334149599220,20cyclictest3056-21PK-Backend23:59:071
334149599220,20cyclictest3056-21PK-Backend21:26:371
334149599220,20cyclictest3056-21PK-Backend21:06:221
334149599220,20cyclictest3056-21PK-Backend20:03:031
334149299227,2cyclictest141rcu_preempt19:16:000
3341492992220,1cyclictest0-21swapper/023:57:500
3341492992220,1cyclictest0-21swapper/022:00:020
3341492992219,2cyclictest161rcuc/021:53:310
3341492992217,4cyclictest13-21ksoftirqd/020:25:010
334149299220,7cyclictest13-21ksoftirqd/021:05:460
334149299220,7cyclictest13-21ksoftirqd/020:00:180
334149299220,6cyclictest13-21ksoftirqd/019:25:210
334149299220,21cyclictest0-21swapper/019:11:230
139099220,15rtkit-daemon3056-21PK-Backend22:03:121
139099220,15rtkit-daemon3056-21PK-Backend19:18:281
139099220,14rtkit-daemon3056-21PK-Backend00:28:470
3341502992120,1cyclictest0-21swapper/322:03:113
3341502992119,1cyclictest0-21swapper/322:48:113
3341502992119,1cyclictest0-21swapper/320:32:393
3341502992119,1cyclictest0-21swapper/319:32:433
3341502992119,1cyclictest0-21swapper/319:24:253
3341502992119,1cyclictest0-21swapper/319:14:113
3341502992119,1cyclictest0-21swapper/300:04:503
334150299211,8cyclictest3386815-21apt21:15:113
3341502992117,3cyclictest42-21ksoftirqd/322:25:003
3341495992119,1cyclictest3056-21PK-Backend21:02:171
3341495992119,1cyclictest3056-21PK-Backend20:13:501
3341495992119,1cyclictest3056-21PK-Backend00:23:271
3341495992119,1cyclictest0-21swapper/122:23:221
3341495992118,2cyclictest3056-21PK-Backend22:53:501
3341495992118,2cyclictest3056-21PK-Backend19:38:501
334149599210,20cyclictest3056-21PK-Backend22:15:311
334149599210,20cyclictest3056-21PK-Backend20:17:221
334149599210,20cyclictest3056-21PK-Backend00:26:511
334149599210,19cyclictest3056-21PK-Backend23:30:201
334149599210,19cyclictest3056-21PK-Backend22:11:121
334149599210,19cyclictest3056-21PK-Backend21:19:131
334149299217,3cyclictest141rcu_preempt00:05:390
334149299217,2cyclictest141rcu_preempt19:20:210
3341492992119,1cyclictest0-21swapper/023:19:510
3341492992118,2cyclictest161rcuc/023:47:060
3341492992117,3cyclictest161rcuc/020:50:160
3341492992117,3cyclictest161rcuc/020:50:160
3341492992111,4cyclictest13-21ksoftirqd/021:45:080
3341492992110,4cyclictest13-21ksoftirqd/021:34:460
334149299210,7cyclictest13-21ksoftirqd/020:25:200
334149299210,6cyclictest1788620-21WebKitWebProces22:37:510
334149299210,6cyclictest13-21ksoftirqd/023:35:310
334149299210,20cyclictest0-21swapper/023:22:280
334149299210,20cyclictest0-21swapper/020:43:090
334149299210,20cyclictest0-21swapper/020:07:140
334149299210,19cyclictest0-21swapper/019:51:320
139099210,15rtkit-daemon3056-21PK-Backend00:30:321
139099210,15rtkit-daemon3056-21PK-Backend00:03:141
3341502992018,1cyclictest0-21swapper/323:50:583
3341502992018,1cyclictest0-21swapper/323:42:193
3341502992018,1cyclictest0-21swapper/323:42:193
3341502992018,1cyclictest0-21swapper/322:18:383
3341502992018,1cyclictest0-21swapper/320:45:093
3341502992018,1cyclictest0-21swapper/300:35:053
334150299200,8cyclictest3369247-21expr20:25:123
334150299200,7cyclictest3374166-21apt20:40:123
334150299200,7cyclictest3353305-21ntpq19:40:193
334150299200,19cyclictest0-21swapper/323:49:093
334150299200,19cyclictest0-21swapper/322:34:463
334150299200,19cyclictest0-21swapper/322:10:133
334150299200,19cyclictest0-21swapper/321:25:313
334150299200,19cyclictest0-21swapper/320:10:433
334150299200,19cyclictest0-21swapper/320:01:493
3341495992018,1cyclictest3056-21PK-Backend22:34:031
3341495992018,1cyclictest3056-21PK-Backend21:50:041
3341495992018,1cyclictest3056-21PK-Backend20:54:191
3341495992018,1cyclictest3056-21PK-Backend20:54:191
3341495992018,1cyclictest0-21swapper/100:07:591
334149599200,1cyclictest3056-21PK-Backend00:16:371
334149599200,19cyclictest3056-21PK-Backend22:06:521
334149599200,19cyclictest3056-21PK-Backend20:33:171
334149599200,19cyclictest3056-21PK-Backend00:14:001
334149599200,19cyclictest0-21swapper/119:56:131
334149599200,18cyclictest3056-21PK-Backend19:27:061
334149299209,4cyclictest13-21ksoftirqd/023:29:380
3341492992018,1cyclictest0-21swapper/022:31:520
3341492992018,1cyclictest0-21swapper/021:42:170
3341492992010,4cyclictest13-21ksoftirqd/023:44:210
3341492992010,4cyclictest13-21ksoftirqd/023:44:210
3341492992010,4cyclictest13-21ksoftirqd/022:10:100
3341492992010,4cyclictest13-21ksoftirqd/022:08:500
3341492992010,4cyclictest13-21ksoftirqd/021:36:300
3341492992010,4cyclictest13-21ksoftirqd/021:01:100
3341492992010,4cyclictest13-21ksoftirqd/020:57:200
3341492992010,4cyclictest13-21ksoftirqd/020:32:500
3341492992010,4cyclictest13-21ksoftirqd/000:22:300
334149299200,8cyclictest13-21ksoftirqd/000:35:180
334149299200,7cyclictest1095-21runrttasks21:15:190
334149299200,6cyclictest13-21ksoftirqd/022:29:030
334149299200,19cyclictest0-21swapper/023:12:010
334149299200,19cyclictest0-21swapper/022:54:570
334149299200,19cyclictest0-21swapper/022:18:340
334149299200,19cyclictest0-21swapper/021:20:550
334149299200,19cyclictest0-21swapper/020:13:420
334149299200,19cyclictest0-21swapper/019:33:270
139099200,13rtkit-daemon3056-21PK-Backend00:37:441
3341502991915,2cyclictest3386538-21sh21:15:003
3341502991914,3cyclictest3401544-21python321:55:223
3341502991914,3cyclictest3366975-21apt20:20:123
3341502991912,5cyclictest3383435-21ls21:05:133
3341502991910,7cyclictest3389626-21users21:20:243
334150299190,7cyclictest3427190-21/usr/sbin/munin23:10:113
334150299190,1cyclictest0-21swapper/323:18:333
334150299190,1cyclictest0-21swapper/319:27:223
334150299190,1cyclictest0-21swapper/300:20:043
334150299190,1cyclictest0-21swapper/300:15:043
334150299190,18cyclictest0-21swapper/323:36:183
334150299190,18cyclictest0-21swapper/321:52:013
3341495991917,1cyclictest3056-21PK-Backend23:38:451
334149599190,1cyclictest3056-21PK-Backend23:45:011
334149599190,1cyclictest0-21swapper/121:10:281
334149599190,18cyclictest3056-21PK-Backend21:42:591
334149599190,18cyclictest3056-21PK-Backend20:36:141
334149299199,4cyclictest13-21ksoftirqd/023:02:590
334149299199,4cyclictest13-21ksoftirqd/020:46:130
334149299195,6cyclictest13-21ksoftirqd/000:12:040
334149299194,4cyclictest13-21ksoftirqd/023:31:340
334149299192,9cyclictest13-21ksoftirqd/022:41:100
334149299192,9cyclictest13-21ksoftirqd/022:41:100
3341492991917,1cyclictest0-21swapper/019:36:520
3341492991910,3cyclictest13-21ksoftirqd/019:48:000
3341492991910,3cyclictest13-21ksoftirqd/019:47:590
334149299190,1cyclictest0-21swapper/021:25:480
334149299190,1cyclictest0-21swapper/000:30:490
139099190,13rtkit-daemon3056-21PK-Backend22:25:491
139099190,13rtkit-daemon3056-21PK-Backend20:44:301
34390862180,4chrt3056-21PK-Backend23:40:201
33592702180,4chrt3359260-21munin-run20:00:003
334150299187,10cyclictest978-21snmpd22:40:383
334150299187,10cyclictest978-21snmpd22:40:383
3341502991816,1cyclictest0-21swapper/323:56:123
3341502991816,1cyclictest0-21swapper/321:41:063
3341502991816,1cyclictest0-21swapper/300:30:053
3341502991815,2cyclictest3365352-21diskstats20:15:153
3341502991813,3cyclictest3425842-21df23:05:143
3341502991813,3cyclictest3420413-21hddtemp_smartct22:50:153
3341502991812,5cyclictest3431171-21seq23:20:063
3341502991811,6cyclictest3355083-21awk19:45:203
3341502991811,6cyclictest3355083-21awk19:45:193
3341502991810,6cyclictest883-21dbus-daemon21:00:003
334150299180,5cyclictest3414814-21apt22:35:113
334150299180,1cyclictest0-21swapper/319:50:143
334150299180,16cyclictest3446995-21grep00:05:113
3341495991816,1cyclictest3056-21PK-Backend21:46:311
3341495991816,1cyclictest3056-21PK-Backend21:36:101
3341495991816,1cyclictest3056-21PK-Backend21:21:571
3341495991816,1cyclictest3056-21PK-Backend19:47:481
3341495991816,1cyclictest3056-21PK-Backend19:47:481
3341495991813,4cyclictest3369132-21cpuspeed_turbos20:25:121
334149599180,1cyclictest3056-21PK-Backend21:56:181
139099180,12rtkit-daemon3056-21PK-Backend19:42:071
334150299179,6cyclictest3351334-21awk19:35:163
3341502991712,4cyclictest3395250-21wc21:40:003
3341502991712,3cyclictest3435069-21hddtemp_smartct23:30:153
3341502991710,5cyclictest3425180-21cut23:05:013
3341502991710,5cyclictest3422116-21df22:55:153
334150299170,3cyclictest3343473-21python319:15:023
334150299170,15cyclictest3392019-21apt21:30:123
334149599178,6cyclictest3418009-21perf22:45:001
334149599178,6cyclictest3418009-21perf22:45:001
3341495991715,1cyclictest3056-21PK-Backend23:09:451
3341495991715,1cyclictest0-21swapper/122:45:321
334149599170,5cyclictest3056-21PK-Backend23:15:211
334149599170,1cyclictest3056-21PK-Backend20:49:401
334149299172,8cyclictest13-21ksoftirqd/022:46:150
334150299161,6cyclictest3377606-21/usr/sbin/munin20:50:193
334150299161,6cyclictest3377606-21/usr/sbin/munin20:50:193
3341502991611,3cyclictest42-21ksoftirqd/321:00:013
334150299160,14cyclictest3373286-21rt-features20:35:223
3341495991614,1cyclictest0-21swapper/121:30:431
3341495991614,1cyclictest0-21swapper/119:30:171
3341495991513,1cyclictest3056-21PK-Backend20:55:411
3341495991513,1cyclictest0-21swapper/123:25:341
3341495991510,3cyclictest3427480-21chrt23:10:131
334149599145,7cyclictest3432667-21cron23:25:001
3341495991311,1cyclictest3056-21PK-Backend22:36:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional