You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 04:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon Sep 14, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
76355723410,6sleep00-21swapper/019:05:130
77999921310,4sleep30-21swapper/319:50:133
80951121080,2sleep20-21swapper/221:15:132
80639321020,3sleep00-21swapper/021:05:120
76375327545,21sleep30-21swapper/319:05:183
139099730,59rtkit-daemon0-21swapper/119:05:161
76473626343,14sleep20-21swapper/219:08:552
8626552480,4sleep13056-21PK-Backend23:45:181
7678722410,2sleep3767871-21kernelversion19:15:163
76496399385,19cyclictest0-21swapper/219:37:542
139099370,3rtkit-daemon0-21swapper/019:44:260
8483062350,3sleep30-21swapper/323:05:143
76496399350,19cyclictest851320-21chrt23:14:312
139099350,3rtkit-daemon3056-21PK-Backend20:30:410
139099350,3rtkit-daemon0-21swapper/020:43:570
139099350,3rtkit-daemon0-21swapper/019:28:410
76496399342,18cyclictest0-21swapper/222:39:422
76496399342,17cyclictest0-21swapper/219:57:212
76496399342,16cyclictest0-21swapper/223:46:422
76496399341,24cyclictest833599-21chrt22:22:502
76496399341,20cyclictest864999-21taskset23:52:042
76496399341,17cyclictest0-21swapper/222:10:302
76496399341,17cyclictest0-21swapper/222:09:182
76496399341,17cyclictest0-21swapper/221:09:472
76496399341,17cyclictest0-21swapper/220:53:152
76496399340,17cyclictest781036-21taskset19:52:322
76496399333,15cyclictest0-21swapper/219:49:392
76496399332,16cyclictest0-21swapper/200:31:592
76496399332,15cyclictest0-21swapper/220:31:432
76496399332,15cyclictest0-21swapper/200:28:492
76496399331,18cyclictest835431-21chrt22:29:072
76496399331,17cyclictest0-21swapper/221:52:152
76496399331,17cyclictest0-21swapper/221:52:152
76496399331,17cyclictest0-21swapper/221:34:382
76496399331,16cyclictest0-21swapper/222:56:462
76496399331,16cyclictest0-21swapper/222:54:542
76496399331,16cyclictest0-21swapper/200:21:192
76496399330,20cyclictest821287-21taskset21:45:522
76496399323,15cyclictest0-21swapper/222:19:582
76496399323,15cyclictest0-21swapper/219:26:062
76496399323,14cyclictest0-21swapper/200:37:392
76496399322,15cyclictest0-21swapper/220:29:012
76496399322,15cyclictest0-21swapper/219:43:232
76496399322,15cyclictest0-21swapper/219:17:312
76496399322,15cyclictest0-21swapper/200:16:062
76496399322,14cyclictest0-21swapper/223:02:392
76496399321,17cyclictest0-21swapper/219:23:292
76496399321,16cyclictest0-21swapper/220:59:142
76496399321,16cyclictest0-21swapper/220:59:132
76496399320,18cyclictest0-21swapper/223:06:452
76496399320,17cyclictest798781-21taskset20:44:302
76496399320,17cyclictest788265-21chrt20:13:012
76496399320,17cyclictest0-21swapper/220:49:222
76496399313,15cyclictest0-21swapper/222:47:212
76496399312,16cyclictest0-21swapper/221:20:352
76496399312,15cyclictest0-21swapper/223:37:412
76496399312,15cyclictest0-21swapper/222:32:522
76496399312,15cyclictest0-21swapper/220:01:152
76496399312,15cyclictest0-21swapper/220:01:152
76496399311,16cyclictest0-21swapper/223:16:392
76496399310,17cyclictest788679-21/usr/sbin/munin20:15:142
76496399303,15cyclictest0-21swapper/200:14:302
76496399302,16cyclictest0-21swapper/200:02:222
76496399302,15cyclictest0-21swapper/223:30:592
76496399302,15cyclictest0-21swapper/219:33:572
76496399301,16cyclictest0-21swapper/221:42:072
76496399301,16cyclictest0-21swapper/221:37:542
76496399300,18cyclictest0-21swapper/220:37:082
76496399300,17cyclictest866944-21taskset23:59:462
764955993015,5cyclictest13-21ksoftirqd/022:10:010
76496399292,15cyclictest0-21swapper/223:40:542
76496399290,16cyclictest856334-21chrt23:27:102
76496399282,14cyclictest0-21swapper/222:41:472
8320612270,4sleep1832062-21sleep22:20:001
764955992717,2cyclictest141rcu_preempt22:18:080
76495599270,8cyclictest13-21ksoftirqd/000:20:180
76496399261,22cyclictest808629-21python321:10:212
76495599265,7cyclictest13-21ksoftirqd/021:30:120
76495599263,6cyclictest13-21ksoftirqd/023:35:190
8295492250,4sleep00-21swapper/022:10:160
764958992511,12cyclictest825131-21unin-run22:00:011
76495599259,8cyclictest13-21ksoftirqd/023:35:000
764955992515,2cyclictest141rcu_preempt22:38:100
76495599251,4cyclictest13-21ksoftirqd/021:40:010
764963992411,12cyclictest978-21snmpd21:01:372
76495599245,6cyclictest13-21ksoftirqd/000:03:380
76496699230,21cyclictest858578-21apt23:35:113
76496399233,16cyclictest765140-21/usr/sbin/munin19:10:272
764963992310,11cyclictest853958-21missed_timers23:20:172
764958992320,2cyclictest3056-21PK-Backend23:34:111
8094762220,2sleep13056-21PK-Backend21:15:121
76496699225,6cyclictest782405-21cat19:55:173
764966992220,1cyclictest0-21swapper/319:20:483
76496699221,18cyclictest880598-21python300:35:233
76496699221,18cyclictest862913-21python323:45:213
76496699220,20cyclictest773710-21python319:30:223
76496399228,13cyclictest0-21swapper/220:24:482
76496399228,12cyclictest8265002chrt22:00:562
76496399220,20cyclictest785359-21grep20:05:132
764958992220,1cyclictest0-21swapper/121:26:081
764958992220,1cyclictest0-21swapper/120:48:521
764958992219,2cyclictest3056-21PK-Backend23:19:121
764958992219,2cyclictest3056-21PK-Backend22:48:491
764958992219,2cyclictest3056-21PK-Backend19:43:281
76495899220,20cyclictest3056-21PK-Backend19:26:581
76495599224,9cyclictest13-21ksoftirqd/022:40:220
764955992220,1cyclictest0-21swapper/023:21:550
764955992218,3cyclictest13-21ksoftirqd/020:00:010
764955992214,6cyclictest863501-21cron23:50:000
764955992213,7cyclictest865878-21df_inode23:55:120
764966992119,1cyclictest3056-21PK-Backend20:27:553
764966992119,1cyclictest0-21swapper/322:16:093
764966992119,1cyclictest0-21swapper/321:48:123
764966992119,1cyclictest0-21swapper/321:14:593
764966992119,1cyclictest0-21swapper/320:59:593
764966992119,1cyclictest0-21swapper/320:59:593
764966992119,1cyclictest0-21swapper/320:18:413
764966992119,1cyclictest0-21swapper/319:45:293
764966992119,1cyclictest0-21swapper/319:27:453
764966992119,1cyclictest0-21swapper/319:14:333
764966992119,1cyclictest0-21swapper/300:10:263
764966992113,6cyclictest845771-21cat22:55:223
764966992112,7cyclictest816168-21apt-get21:35:003
76496699211,17cyclictest857707-21python323:30:223
76496699210,20cyclictest0-21swapper/321:16:353
76496699210,20cyclictest0-21swapper/320:53:573
76496699210,20cyclictest0-21swapper/320:00:513
76496699210,20cyclictest0-21swapper/320:00:503
76496399210,20cyclictest0-21swapper/221:57:382
764958992119,1cyclictest3056-21PK-Backend23:36:481
764958992119,1cyclictest3056-21PK-Backend23:10:121
764958992119,1cyclictest3056-21PK-Backend22:57:001
764958992119,1cyclictest3056-21PK-Backend20:01:121
764958992119,1cyclictest3056-21PK-Backend20:01:121
764958992119,1cyclictest3056-21PK-Backend19:52:211
764958992119,1cyclictest3056-21PK-Backend19:12:191
764958992119,1cyclictest3056-21PK-Backend00:12:581
764958992119,1cyclictest0-21swapper/121:24:331
764958992118,2cyclictest3056-21PK-Backend22:35:031
764958992118,2cyclictest3056-21PK-Backend21:09:061
764958992114,6cyclictest791732-21seq20:23:021
76495899210,20cyclictest3056-21PK-Backend20:53:511
76495899210,20cyclictest3056-21PK-Backend20:06:131
76495899210,20cyclictest3056-21PK-Backend00:32:521
76495899210,19cyclictest786972-21apt20:10:131
76495599219,4cyclictest13-21ksoftirqd/021:03:410
76495599214,10cyclictest13-21ksoftirqd/020:56:280
76495599214,10cyclictest13-21ksoftirqd/020:56:270
76495599213,10cyclictest13-21ksoftirqd/023:18:060
76495599212,6cyclictest13-21ksoftirqd/021:15:010
76495599212,6cyclictest13-21ksoftirqd/019:25:010
764955992119,1cyclictest0-21swapper/019:52:280
764955992119,1cyclictest0-21swapper/000:29:090
764955992119,1cyclictest0-21swapper/000:19:410
764955992117,3cyclictest13-21ksoftirqd/021:52:160
764955992117,3cyclictest13-21ksoftirqd/021:52:160
76495599211,3cyclictest13-21ksoftirqd/022:00:230
76495599211,1cyclictest0-21swapper/019:18:480
76495599210,6cyclictest13-21ksoftirqd/022:25:170
76495599210,20cyclictest0-21swapper/023:12:220
76495599210,20cyclictest0-21swapper/023:06:000
76495599210,20cyclictest0-21swapper/020:27:030
8173692200,2sleep13056-21PK-Backend21:35:181
764966992018,1cyclictest0-21swapper/322:45:303
764966992018,1cyclictest0-21swapper/322:33:063
764966992018,1cyclictest0-21swapper/321:20:343
764966992017,2cyclictest3056-21PK-Backend22:01:503
764966992013,6cyclictest834002-21sed22:25:013
764966992013,5cyclictest850243-21cat23:10:123
764966992013,5cyclictest837801-21expr22:35:153
764966992012,6cyclictest829307-21awk22:10:153
76496699200,19cyclictest3056-21PK-Backend00:27:583
76496699200,19cyclictest0-21swapper/323:51:063
76496699200,19cyclictest0-21swapper/323:26:343
76496699200,19cyclictest0-21swapper/321:52:443
76496699200,19cyclictest0-21swapper/321:52:443
76496699200,19cyclictest0-21swapper/320:23:423
76496699200,19cyclictest0-21swapper/320:11:523
76496699200,19cyclictest0-21swapper/319:41:503
76496699200,19cyclictest0-21swapper/300:15:403
76496399201,17cyclictest869746-21/usr/sbin/munin00:05:212
764958992018,1cyclictest3056-21PK-Backend21:53:201
764958992018,1cyclictest3056-21PK-Backend21:53:191
764958992018,1cyclictest3056-21PK-Backend21:10:411
764958992018,1cyclictest3056-21PK-Backend20:17:151
764958992018,1cyclictest3056-21PK-Backend19:58:351
764958992018,1cyclictest0-21swapper/121:40:281
764958992017,2cyclictest3056-21PK-Backend23:42:501
764958992017,2cyclictest3056-21PK-Backend20:43:511
76495899200,1cyclictest3056-21PK-Backend23:59:051
76495899200,1cyclictest3056-21PK-Backend20:31:271
76495899200,1cyclictest3056-21PK-Backend00:38:531
76495899200,19cyclictest3056-21PK-Backend20:35:401
76495899200,19cyclictest3056-21PK-Backend19:39:011
76495899200,19cyclictest3056-21PK-Backend00:01:421
76495599207,4cyclictest13-21ksoftirqd/022:55:180
76495599205,7cyclictest13-21ksoftirqd/020:54:280
764955992018,1cyclictest0-21swapper/021:21:570
764955992018,1cyclictest0-21swapper/020:22:500
764955992018,1cyclictest0-21swapper/000:05:280
764955992016,3cyclictest13-21ksoftirqd/023:45:000
764955992013,2cyclictest141rcu_preempt20:01:590
764955992013,2cyclictest141rcu_preempt20:01:590
76495599201,10cyclictest13-21ksoftirqd/021:25:190
139099200,16rtkit-daemon764948-21cyclictest22:33:330
139099200,14rtkit-daemon0-21swapper/019:49:100
764966991917,1cyclictest3056-21PK-Backend21:29:053
764966991917,1cyclictest0-21swapper/323:44:033
764966991917,1cyclictest0-21swapper/323:02:203
764966991917,1cyclictest0-21swapper/321:07:493
764966991917,1cyclictest0-21swapper/320:44:443
764966991914,3cyclictest853855-21irqstats23:20:163
764966991914,3cyclictest840380-21python322:40:233
764966991913,5cyclictest1-21systemd00:05:013
76496699191,16cyclictest868997-21apt00:05:123
764966991910,7cyclictest785597-21df20:05:133
76496699190,1cyclictest0-21swapper/320:31:183
76496699190,18cyclictest0-21swapper/300:32:233
76496399197,11cyclictest813765-21processes21:25:212
764958991917,1cyclictest3056-21PK-Backend23:04:241
764958991917,1cyclictest0-21swapper/123:24:591
764958991917,1cyclictest0-21swapper/121:30:081
764958991917,1cyclictest0-21swapper/120:27:141
764958991914,3cyclictest873935-21turbostat.cron00:20:011
76495899190,1cyclictest3056-21PK-Backend21:46:371
76495899190,1cyclictest0-21swapper/123:25:461
76495599196,6cyclictest13-21ksoftirqd/020:06:510
76495599196,4cyclictest13-21ksoftirqd/021:41:350
76495599192,7cyclictest13-21ksoftirqd/000:11:560
76495599192,5cyclictest0-21swapper/020:10:190
764955991917,1cyclictest0-21swapper/021:57:330
764955991917,1cyclictest0-21swapper/020:35:130
764955991915,3cyclictest13-21ksoftirqd/023:55:000
76495599191,4cyclictest13-21ksoftirqd/020:46:200
764955991912,5cyclictest855818-21ntpq23:25:200
76495599190,18cyclictest0-21swapper/022:49:170
76495599190,10cyclictest13-21ksoftirqd/022:50:160
139099190,13rtkit-daemon0-21swapper/019:34:300
76496699189,7cyclictest818149-21cron21:40:003
764966991816,1cyclictest3056-21PK-Backend21:57:173
764966991816,1cyclictest0-21swapper/322:29:073
764966991816,1cyclictest0-21swapper/319:37:253
764966991816,1cyclictest0-21swapper/300:21:553
764966991811,5cyclictest851833-21expr23:15:113
764966991811,5cyclictest796300-21head20:35:163
764966991810,6cyclictest843644-21grep22:50:163
76496699180,16cyclictest818404-21/usr/sbin/munin21:40:153
764958991816,1cyclictest3056-21PK-Backend21:00:401
764958991816,1cyclictest0-21swapper/122:05:361
764958991815,2cyclictest3056-21PK-Backend22:30:171
764958991815,2cyclictest3056-21PK-Backend20:56:281
764958991815,2cyclictest3056-21PK-Backend20:56:281
764958991815,2cyclictest3056-21PK-Backend19:16:591
764958991814,3cyclictest876006-21turbostat.cron00:25:001
76495599187,4cyclictest13-21ksoftirqd/023:03:210
76495599187,3cyclictest141rcu_preempt00:34:410
76495599186,5cyclictest13-21ksoftirqd/020:19:270
76495599182,8cyclictest13-21ksoftirqd/022:20:320
764955991815,2cyclictest161rcuc/021:16:220
76495599180,6cyclictest13-21ksoftirqd/019:14:590
8638402170,3chrt3056-21PK-Backend23:50:111
76496699171,14cyclictest0-21swapper/300:00:013
76495899179,7cyclictest3056-21PK-Backend22:15:001
76495899179,6cyclictest849641-21sh23:10:001
76495899178,7cyclictest825949-21memory22:00:181
764958991711,4cyclictest876630-21sh00:25:121
76495599174,3cyclictest13-21ksoftirqd/019:37:520
764955991713,3cyclictest13-21ksoftirqd/021:50:000
764955991713,3cyclictest13-21ksoftirqd/000:40:000
76496699169,5cyclictest828429-21sshd22:08:283
76495899166,2cyclictest141rcu_preempt22:25:181
764958991614,1cyclictest3056-21PK-Backend00:06:561
764958991611,3cyclictest1095-21runrttasks19:30:171
139099160,13rtkit-daemon3056-21PK-Backend19:45:461
8323972150,4sleep13056-21PK-Backend22:20:131
764966991513,1cyclictest0-21swapper/320:49:443
764966991511,3cyclictest0-21swapper/321:00:113
764958991410,3cyclictest3056-21PK-Backend19:21:241
76495899130,5cyclictest3056-21PK-Backend22:40:011
76495899116,4cyclictest3056-21PK-Backend22:53:451
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional