You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-06 - 13:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Thu Aug 06, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
396766021450,2sleep30-21swapper/321:45:183
398204321250,3sleep10-21swapper/122:25:171
398340021130,1sleep00-21swapper/022:30:140
392434021020,3sleep33056-21PK-Backend19:40:273
391099921020,9sleep00-21swapper/019:05:170
391126027752,16sleep30-21swapper/319:05:213
391056327346,17sleep10-21swapper/119:05:121
391096126442,14sleep20-21swapper/219:05:172
39228772550,6sleep234-21ksoftirqd/219:38:442
40051212450,3sleep33056-21PK-Backend23:30:203
3912188993820,16cyclictest3929915-21perf19:59:591
3912188993212,5cyclictest3949495-21expr20:55:131
39352862290,3sleep10-21swapper/120:15:011
40087002280,3chrt34-21ksoftirqd/223:40:172
40017912280,4sleep33056-21PK-Backend23:20:203
139099270,4rtkit-daemon1389-21rtkit-daemon23:02:033
40155502260,3sleep20-21swapper/200:00:182
391219699262,19cyclictest3960200-21apt-get21:25:033
139099260,4rtkit-daemon1389-21rtkit-daemon20:50:423
391218899250,19cyclictest0-21swapper/100:04:461
3912196992420,3cyclictest4014789-21dump-pmu-power00:00:013
3912196992414,7cyclictest3983899-21python322:30:223
139099240,4rtkit-daemon1389-21rtkit-daemon21:31:493
39851662230,3sleep13056-21PK-Backend22:35:151
391219699232,4cyclictest3944973-21python320:40:213
3912196992314,6cyclictest4017667-21python300:05:253
3912196992314,6cyclictest3970360-21cron21:54:593
3912192992315,6cyclictest883-21dbus-daemon22:15:012
3912188992316,1cyclictest0-21swapper/123:40:261
3912188992314,7cyclictest3970384-21cron21:55:001
391218699232,12cyclictest3933674-21apt.systemd.dai20:10:120
40170212220,3sleep00-21swapper/000:05:160
39230552220,4sleep1251rcuc/119:40:001
3912196992213,7cyclictest4013046-21perf23:55:003
3912192992220,1cyclictest0-21swapper/219:31:212
3912192992220,1cyclictest0-21swapper/219:27:292
3912192992219,2cyclictest0-21swapper/219:22:292
391219299220,21cyclictest0-21swapper/223:23:332
3912186992217,3cyclictest4001668-21ntp_states23:20:190
391218699220,6cyclictest3926456-21dump-pmu-power19:50:000
39888642210,4chrt0-21swapper/222:45:152
3912196992118,2cyclictest3972379-21apt22:00:133
391219699211,18cyclictest3978576-21ntp_states22:15:183
391219699211,18cyclictest3953624-21memory21:05:183
391219699210,19cyclictest3056-21PK-Backend23:15:153
3912192992119,1cyclictest0-21swapper/223:54:082
3912192992119,1cyclictest0-21swapper/223:49:232
3912192992119,1cyclictest0-21swapper/223:14:352
3912192992119,1cyclictest0-21swapper/223:09:482
3912192992119,1cyclictest0-21swapper/222:36:352
3912192992119,1cyclictest0-21swapper/222:33:102
3912192992119,1cyclictest0-21swapper/222:19:122
3912192992119,1cyclictest0-21swapper/221:23:562
3912192992119,1cyclictest0-21swapper/221:03:142
3912192992119,1cyclictest0-21swapper/220:59:092
3912192992119,1cyclictest0-21swapper/220:19:132
3912192992119,1cyclictest0-21swapper/220:05:342
3912192992119,1cyclictest0-21swapper/219:58:592
3912192992119,1cyclictest0-21swapper/200:38:362
3912192992119,1cyclictest0-21swapper/200:30:462
3912192992119,1cyclictest0-21swapper/200:30:462
3912192992119,1cyclictest0-21swapper/200:27:422
3912192992119,1cyclictest0-21swapper/200:20:582
3912192992115,4cyclictest436-20systemd-journal22:29:552
391219299210,20cyclictest3056-21PK-Backend20:52:332
391219299210,20cyclictest0-21swapper/222:50:142
391219299210,20cyclictest0-21swapper/221:50:192
391219299210,20cyclictest0-21swapper/221:27:342
391219299210,20cyclictest0-21swapper/221:07:122
391219299210,20cyclictest0-21swapper/220:12:582
391219299210,20cyclictest0-21swapper/219:11:132
3912188992119,1cyclictest3056-21PK-Backend20:49:451
391218899210,20cyclictest0-21swapper/120:21:261
391218899210,1cyclictest3056-21PK-Backend00:27:311
3912186992119,1cyclictest0-21swapper/023:07:550
3912186992119,1cyclictest0-21swapper/021:53:250
3912186992119,1cyclictest0-21swapper/000:26:570
391218699210,20cyclictest0-21swapper/021:13:380
391218699210,19cyclictest3056-21PK-Backend21:31:160
391218699210,19cyclictest0-21swapper/022:40:510
391218699210,19cyclictest0-21swapper/021:55:290
3912196992018,1cyclictest0-21swapper/322:26:283
391219699201,7cyclictest3946582-21ntp_states20:45:193
3912196992017,2cyclictest3056-21PK-Backend20:06:293
3912192992018,1cyclictest0-21swapper/223:18:462
3912192992018,1cyclictest0-21swapper/222:40:352
3912192992018,1cyclictest0-21swapper/222:20:542
3912192992018,1cyclictest0-21swapper/220:21:232
3912192992018,1cyclictest0-21swapper/219:52:032
3912192992018,1cyclictest0-21swapper/219:19:242
3912192992018,1cyclictest0-21swapper/200:11:122
391219299200,1cyclictest3056-21PK-Backend23:35:372
391219299200,1cyclictest0-21swapper/222:59:542
391219299200,1cyclictest0-21swapper/221:43:302
391219299200,19cyclictest0-21swapper/200:18:282
391219299200,19cyclictest0-21swapper/200:08:432
3912188992018,1cyclictest0-21swapper/119:45:371
3912188992018,1cyclictest0-21swapper/100:06:301
3912188992012,6cyclictest3927120-21fw_packets19:50:171
391218899200,1cyclictest1213-21kerneloops20:19:301
3912186992018,1cyclictest0-21swapper/022:14:140
3912186992018,1cyclictest0-21swapper/022:06:110
3912186992018,1cyclictest0-21swapper/021:07:500
3912186992018,1cyclictest0-21swapper/000:30:010
3912186992018,1cyclictest0-21swapper/000:30:010
391218699200,1cyclictest0-21swapper/023:40:190
391218699200,1cyclictest0-21swapper/000:23:530
391218699200,19cyclictest0-21swapper/023:28:300
391218699200,19cyclictest0-21swapper/022:20:500
391218699200,19cyclictest0-21swapper/021:20:000
391218699200,19cyclictest0-21swapper/020:47:020
391218699200,19cyclictest0-21swapper/020:43:510
391218699200,19cyclictest0-21swapper/020:29:100
391218699200,19cyclictest0-21swapper/019:21:380
391218699200,19cyclictest0-21swapper/000:15:270
391218699200,18cyclictest3924734-21cut19:45:000
391218699200,11cyclictest40107152python323:45:190
391219699199,9cyclictest3986229-21cron22:40:003
391219699192,11cyclictest3987958-21apt-get22:45:033
3912196991917,1cyclictest0-21swapper/322:51:563
391219699191,4cyclictest3996584-21python323:05:213
3912196991914,4cyclictest1095-21runrttasks21:35:123
3912196991914,3cyclictest3943827-21cron20:39:593
3912196991912,5cyclictest4020741-21awk00:15:163
3912196991911,6cyclictest3976619-21latency22:10:173
3912196991910,7cyclictest4025831-21df00:30:143
3912196991910,7cyclictest4025831-21df00:30:143
391219699190,7cyclictest4028701-21munin-run00:40:003
391219299195,5cyclictest34-21ksoftirqd/220:00:172
3912192991917,1cyclictest0-21swapper/223:26:372
3912192991917,1cyclictest0-21swapper/221:48:502
3912192991917,1cyclictest0-21swapper/220:42:592
3912192991912,6cyclictest3971046-21smartctl21:55:182
3912192991912,5cyclictest3956268-21systemctl21:15:002
391219299190,1cyclictest0-21swapper/220:45:362
391219299190,1cyclictest0-21swapper/220:26:172
391219299190,18cyclictest0-21swapper/222:01:402
391219299190,18cyclictest0-21swapper/220:37:452
391219299190,18cyclictest0-21swapper/220:33:402
391218899199,8cyclictest883-21dbus-daemon19:20:011
3912188991917,1cyclictest3056-21PK-Backend22:46:051
3912188991917,1cyclictest0-21swapper/121:45:421
3912188991913,4cyclictest4176944-21snapd23:31:371
3912188991912,5cyclictest4010037-21expr23:45:121
3912188991912,5cyclictest3994368-21smartctl23:00:151
391218899190,1cyclictest3056-21PK-Backend22:04:281
391218899190,1cyclictest0-21swapper/123:24:181
391218899190,1cyclictest0-21swapper/119:11:381
3912186991917,1cyclictest3056-21PK-Backend23:00:320
3912186991917,1cyclictest0-21swapper/023:53:170
3912186991917,1cyclictest0-21swapper/023:34:250
3912186991917,1cyclictest0-21swapper/022:15:220
3912186991917,1cyclictest0-21swapper/021:28:310
3912186991917,1cyclictest0-21swapper/021:03:510
3912186991917,1cyclictest0-21swapper/020:50:470
3912186991917,1cyclictest0-21swapper/019:16:510
3912186991917,1cyclictest0-21swapper/000:38:260
391218699190,1cyclictest3056-21PK-Backend21:48:050
391218699190,1cyclictest0-21swapper/022:54:500
391218699190,1cyclictest0-21swapper/021:24:530
391218699190,1cyclictest0-21swapper/019:30:150
391218699190,18cyclictest0-21swapper/022:57:410
391218699190,18cyclictest0-21swapper/022:38:350
391218699190,18cyclictest0-21swapper/021:42:250
391218699190,18cyclictest0-21swapper/020:22:070
391218699190,18cyclictest0-21swapper/020:06:330
391218699190,18cyclictest0-21swapper/019:11:100
391218699190,18cyclictest0-21swapper/000:11:490
391219699184,12cyclictest42-21ksoftirqd/321:55:203
3912196991816,1cyclictest3056-21PK-Backend19:53:593
3912196991813,3cyclictest3957252-21ntp_states21:15:213
3912196991812,5cyclictest3959103-21sensors_temp21:20:233
3912196991812,5cyclictest3941199-21cat20:30:163
3912196991812,4cyclictest3987615-21sshd22:40:443
391219699181,15cyclictest3938910-21apt20:25:113
3912196991810,6cyclictest4015433-21df00:00:143
3912196991810,6cyclictest3966895-21cron21:45:003
391219699180,6cyclictest3952253-21taskset21:01:553
391219699180,4cyclictest3056-21PK-Backend19:35:273
391219699180,1cyclictest3056-21PK-Backend19:55:063
391219299184,12cyclictest3963434-21cron21:34:592
391218899187,9cyclictest3998103-21needreboot23:10:181
3912188991816,1cyclictest3056-21PK-Backend23:59:531
3912188991816,1cyclictest3056-21PK-Backend22:41:041
3912188991816,1cyclictest3056-21PK-Backend21:57:191
3912188991816,1cyclictest0-21swapper/120:08:021
3912188991811,5cyclictest4006562-21expr23:35:131
3912186991816,1cyclictest0-21swapper/023:14:580
3912186991816,1cyclictest0-21swapper/021:39:070
3912186991816,1cyclictest0-21swapper/019:56:180
3912186991816,1cyclictest0-21swapper/019:37:470
3912186991812,5cyclictest3949868-21cat20:55:180
3912186991811,5cyclictest3999190-21apt23:15:100
139099180,14rtkit-daemon1389-21rtkit-daemon21:12:443
39306172170,3sleep00-21swapper/020:00:160
391219699179,6cyclictest3992223-21apt22:55:133
391219699178,2cyclictest141rcu_preempt19:30:103
391219699175,11cyclictest42-21ksoftirqd/322:20:203
3912196991715,1cyclictest3056-21PK-Backend19:29:463
3912196991715,1cyclictest3056-21PK-Backend19:16:143
3912196991714,2cyclictest3056-21PK-Backend23:29:293
3912196991712,3cyclictest4018892-21df00:10:103
391219699170,5cyclictest3997531-21cat23:10:123
3912192991715,1cyclictest0-21swapper/223:58:212
3912192991715,1cyclictest0-21swapper/222:07:082
3912192991715,1cyclictest0-21swapper/219:41:342
3912192991711,4cyclictest4006129-21sed23:35:002
391218899179,6cyclictest3917193-21python319:20:191
3912188991715,1cyclictest3056-21PK-Backend23:54:061
3912188991715,1cyclictest3056-21PK-Backend21:13:511
3912188991715,1cyclictest3056-21PK-Backend00:13:261
3912188991714,2cyclictest3056-21PK-Backend22:51:401
3912188991714,2cyclictest3056-21PK-Backend21:22:571
3912188991711,4cyclictest4000859-21cat23:20:021
3912188991710,5cyclictest4021808-21wc00:20:021
3912188991710,5cyclictest3964232-21grep21:35:171
391218699179,6cyclictest883-21dbus-daemon23:40:000
391218699179,5cyclictest436-20systemd-journal20:20:010
3912186991715,1cyclictest0-21swapper/023:59:130
3912186991715,1cyclictest0-21swapper/022:02:260
3912186991715,1cyclictest0-21swapper/020:37:420
139099170,13rtkit-daemon3056-21PK-Backend19:10:333
3912196991614,1cyclictest3056-21PK-Backend20:21:583
3912196991613,2cyclictest3056-21PK-Backend23:41:043
391219699161,2cyclictest141rcu_preempt20:05:003
391219699161,13cyclictest3974237-21tr22:05:133
391219699161,11cyclictest4011304-21cron23:50:003
391219699160,2cyclictest39250752sleep319:45:123
391219699160,15cyclictest0-21swapper/323:35:173
391219699160,14cyclictest4024132-21hddtemp_smartct00:25:143
3912192991610,5cyclictest3994170-21cat23:00:122
391218899169,5cyclictest4022006-21seq00:20:121
391218899169,5cyclictest3920537-21ntpq19:30:181
391218899168,6cyclictest3923546-21expr19:40:171
391218899166,9cyclictest3939582-21needreboot20:25:191
3912188991614,1cyclictest3056-21PK-Backend21:08:321
3912188991614,1cyclictest0-21swapper/123:26:001
3912188991611,3cyclictest3965859-21idleruntime21:40:161
391218699169,5cyclictest4015559-21kernelversion00:00:170
391218699169,5cyclictest3941183-21sed20:30:160
391218699169,5cyclictest3926876-21cpuspeed_turbos19:50:150
3912196991512,2cyclictest3916053-21turbostat.cron19:24:563
391219699150,13cyclictest3949887-21memory20:55:183
3912192991513,1cyclictest0-21swapper/221:37:472
3912192991511,3cyclictest3925357-21df19:45:122
391219299150,13cyclictest3956809-21cat21:15:142
391218899159,5cyclictest4027260-21ls00:35:131
391218899159,4cyclictest4026382-21grep00:30:231
391218899159,4cyclictest4026382-21grep00:30:221
391218899158,5cyclictest3996204-21cut23:05:151
391218899158,5cyclictest3919060-21cat19:25:211
3912188991511,3cyclictest3056-21PK-Backend20:42:561
3912188991510,3cyclictest3962627-21needreboot21:30:191
3912188991510,3cyclictest3056-21PK-Backend22:05:521
391218899150,6cyclictest3056-21PK-Backend20:32:401
391218899150,14cyclictest3056-21PK-Backend21:18:511
391218699159,5cyclictest3988941-21cat22:45:170
3912186991513,1cyclictest0-21swapper/022:28:010
39305212140,3chrt26-21ksoftirqd/120:00:141
391219699148,4cyclictest42-21ksoftirqd/320:10:133
3912196991411,2cyclictest3056-21PK-Backend00:22:563
391218899147,5cyclictest3976371-21expr22:10:121
3912188991412,1cyclictest3056-21PK-Backend20:50:191
3912188991410,3cyclictest3056-21PK-Backend22:24:291
3912188991410,3cyclictest3056-21PK-Backend20:36:221
391218899140,1cyclictest3056-21PK-Backend21:29:541
391218899140,1cyclictest3056-21PK-Backend21:00:071
391218899140,13cyclictest3056-21PK-Backend22:19:351
391218899140,13cyclictest0-21swapper/122:56:401
391218699148,4cyclictest3918652-21missed_timers19:25:170
3912196991310,2cyclictest3056-21PK-Backend20:16:403
391218899128,3cyclictest3056-21PK-Backend22:31:061
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional