You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 06:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Aug 12, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
313188121270,2sleep33056-21PK-Backend20:45:143
309597821130,10sleep30-21swapper/319:05:163
312177721120,4sleep20-21swapper/220:15:222
309940221060,3sleep20-21swapper/219:15:022
139099730,63rtkit-daemon0-21swapper/019:07:360
31734772680,5sleep33173481-21cpuspeed22:45:133
309595926546,13sleep20-21swapper/219:05:162
309667026343,14sleep10-21swapper/119:05:361
31337022580,4sleep33056-21PK-Backend20:50:153
31319682550,6sleep10-21swapper/120:45:161
31476362500,2sleep30-21swapper/321:30:183
31586022420,8sleep234-21ksoftirqd/222:00:242
139099330,4rtkit-daemon1389-21rtkit-daemon20:40:291
139099330,4rtkit-daemon1389-21rtkit-daemon20:40:291
31685162290,4sleep13056-21PK-Backend22:30:181
31393732270,4sleep33056-21PK-Backend21:05:243
139099270,4rtkit-daemon1389-21rtkit-daemon20:17:341
31115302260,4chrt3056-21PK-Backend19:45:211
309715599250,24cyclictest3122381-21perf20:20:000
3097170992418,4cyclictest3172434-21ntp_states22:40:193
3097170992412,10cyclictest3122386-21cron20:20:003
309716399241,4cyclictest3189498-21irqstats23:30:161
309715599240,22cyclictest1788620-21WebKitWebProces21:44:360
309715599240,22cyclictest1788620-21WebKitWebProces21:44:360
3097170992320,2cyclictest0-21swapper/323:55:483
3097170992313,7cyclictest3171152-21cron22:39:593
3097170992313,7cyclictest3171152-21cron22:39:593
3097170992312,8cyclictest883-21dbus-daemon19:35:003
309717099230,21cyclictest3154768-21strings21:50:163
309717099230,21cyclictest3056-21PK-Backend00:10:563
3097167992315,5cyclictest978-21snmpd19:36:322
309716399230,9cyclictest3165970-21idleruntime-cro22:25:001
31668572220,3sleep30-21swapper/322:25:163
309717099222,14cyclictest3182088-21apt23:10:123
3097170992220,1cyclictest3056-21PK-Backend19:10:023
309717099220,21cyclictest978-21snmpd21:27:453
309717099220,20cyclictest3056-21PK-Backend00:33:403
3097167992214,6cyclictest3116356-21needreboot20:00:222
309716799220,20cyclictest3141710-21munin-run21:15:012
3097163992215,5cyclictest2175-21JS10:27:481
309716399220,20cyclictest3056-21PK-Backend21:50:391
309716399220,20cyclictest3056-21PK-Backend21:40:061
309716399220,20cyclictest3056-21PK-Backend21:40:051
309716399220,20cyclictest0-21swapper/120:25:031
309716399220,19cyclictest3056-21PK-Backend19:26:171
3097155992220,1cyclictest0-21swapper/020:32:310
3097155992220,1cyclictest0-21swapper/020:23:040
3097155992210,6cyclictest3211805-21cron00:35:000
139099220,4rtkit-daemon1389-21rtkit-daemon23:58:280
139099220,4rtkit-daemon1389-21rtkit-daemon22:14:180
31056352210,4sleep13056-21PK-Backend19:30:181
3097170992119,1cyclictest3056-21PK-Backend23:19:393
3097170992119,1cyclictest3056-21PK-Backend20:57:573
3097170992119,1cyclictest3056-21PK-Backend00:35:023
3097170992119,1cyclictest0-21swapper/322:11:453
3097170992119,1cyclictest0-21swapper/321:45:093
3097170992119,1cyclictest0-21swapper/321:36:373
3097170992119,1cyclictest0-21swapper/321:15:143
3097170992119,1cyclictest0-21swapper/320:34:253
3097170992114,6cyclictest3202360-21grep00:05:213
3097170992113,6cyclictest42-21ksoftirqd/322:15:123
309717099211,18cyclictest3108631-21date19:40:003
309717099211,17cyclictest3144414-21ntp_kernel_err21:20:203
309717099210,19cyclictest3056-21PK-Backend00:18:193
309716799218,11cyclictest3174916-21unin-run22:50:002
3097167992118,2cyclictest0-21swapper/220:08:012
3097167992114,5cyclictest34-21ksoftirqd/222:30:132
3097167992114,5cyclictest28650irq/123-ahci[0023:15:212
309716799210,19cyclictest3179577-21sed23:00:202
3097163992113,6cyclictest3136155-21sshd20:55:431
309716399210,20cyclictest3056-21PK-Backend22:13:111
309716399210,20cyclictest3056-21PK-Backend22:02:561
309716399210,19cyclictest3056-21PK-Backend23:07:521
309716399210,19cyclictest3056-21PK-Backend21:15:251
309716399210,19cyclictest3056-21PK-Backend19:15:061
3097155992119,1cyclictest0-21swapper/023:37:250
3097155992119,1cyclictest0-21swapper/023:15:280
3097155992119,1cyclictest0-21swapper/023:06:150
3097155992119,1cyclictest0-21swapper/022:51:420
3097155992119,1cyclictest0-21swapper/022:46:210
3097155992119,1cyclictest0-21swapper/021:59:160
3097155992119,1cyclictest0-21swapper/021:52:140
3097155992119,1cyclictest0-21swapper/021:02:530
3097155992119,1cyclictest0-21swapper/020:44:470
3097155992119,1cyclictest0-21swapper/020:44:470
3097155992119,1cyclictest0-21swapper/020:35:360
3097155992119,1cyclictest0-21swapper/019:53:170
3097155992119,1cyclictest0-21swapper/000:37:140
3097155992119,1cyclictest0-21swapper/000:23:210
3097155992119,1cyclictest0-21swapper/000:19:510
3097155992117,3cyclictest2019-21gmain19:50:000
309715599210,2cyclictest3139369-21smartctl21:05:220
309715599210,20cyclictest0-21swapper/023:12:590
309715599210,20cyclictest0-21swapper/022:34:320
309715599210,20cyclictest0-21swapper/022:04:100
309715599210,20cyclictest0-21swapper/021:39:290
309715599210,20cyclictest0-21swapper/021:23:000
309715599210,1cyclictest0-21swapper/019:30:320
139099210,4rtkit-daemon1389-21rtkit-daemon21:49:020
139099210,3rtkit-daemon1389-21rtkit-daemon19:23:120
139099210,17rtkit-daemon3097149-21cyclictest23:31:030
3097170992018,1cyclictest3056-21PK-Backend23:34:463
3097170992018,1cyclictest3056-21PK-Backend20:20:263
3097170992017,2cyclictest0-21swapper/323:50:003
3097170992015,3cyclictest3099750-21grep19:15:133
309717099201,4cyclictest3138248-21perf21:05:013
309717099200,19cyclictest0-21swapper/322:06:313
309717099200,19cyclictest0-21swapper/322:03:553
309717099200,19cyclictest0-21swapper/320:38:043
309717099200,19cyclictest0-21swapper/319:46:543
309717099200,17cyclictest3103276-21/usr/sbin/munin19:25:123
3097167992018,1cyclictest0-21swapper/223:24:522
3097167992018,1cyclictest0-21swapper/222:50:532
3097167992018,1cyclictest0-21swapper/222:28:212
3097167992018,1cyclictest0-21swapper/221:02:362
3097167992018,1cyclictest0-21swapper/220:55:402
3097167992018,1cyclictest0-21swapper/220:14:372
3097167992018,1cyclictest0-21swapper/200:38:132
3097167992018,1cyclictest0-21swapper/200:31:112
3097167992018,1cyclictest0-21swapper/200:29:002
3097167992013,5cyclictest3205150-21awk00:15:122
3097167992012,6cyclictest3155835-21apt21:55:122
3097167992012,6cyclictest3126825-21ntpq20:30:192
3097167992012,6cyclictest3104028-21ntpq19:25:192
309716799200,1cyclictest0-21swapper/223:27:232
309716799200,1cyclictest0-21swapper/222:37:402
309716799200,1cyclictest0-21swapper/222:37:402
309716799200,1cyclictest0-21swapper/221:32:512
309716799200,1cyclictest0-21swapper/220:26:332
309716799200,19cyclictest3198926-21ntpq23:55:212
309716799200,19cyclictest0-21swapper/223:53:592
309716799200,19cyclictest0-21swapper/223:36:422
309716799200,19cyclictest0-21swapper/223:08:432
309716799200,19cyclictest0-21swapper/222:12:052
309716799200,19cyclictest0-21swapper/221:36:162
309716799200,19cyclictest0-21swapper/219:53:022
309716799200,19cyclictest0-21swapper/219:49:022
309716799200,19cyclictest0-21swapper/219:10:022
3097163992018,1cyclictest3056-21PK-Backend22:16:211
3097163992018,1cyclictest0-21swapper/121:05:101
3097163992012,6cyclictest3202763-21users00:05:261
309716399200,1cyclictest3056-21PK-Backend20:30:031
309716399200,1cyclictest3056-21PK-Backend19:55:211
309716399200,1cyclictest0-21swapper/100:20:121
309716399200,19cyclictest3056-21PK-Backend21:35:111
309716399200,19cyclictest3056-21PK-Backend20:00:181
309716399200,19cyclictest3056-21PK-Backend19:51:361
309716399200,19cyclictest0-21swapper/100:16:401
309716399200,19cyclictest0-21swapper/100:14:301
309716399200,19cyclictest0-21swapper/100:00:041
3097155992018,1cyclictest0-21swapper/023:22:450
3097155992018,1cyclictest0-21swapper/021:16:240
3097155992018,1cyclictest0-21swapper/020:46:430
3097155992018,1cyclictest0-21swapper/020:08:390
3097155992018,1cyclictest0-21swapper/019:36:280
3097155992018,1cyclictest0-21swapper/019:26:130
3097155992018,1cyclictest0-21swapper/019:10:460
3097155992018,1cyclictest0-21swapper/000:29:360
3097155992018,1cyclictest0-21swapper/000:11:250
3097155992018,1cyclictest0-21swapper/000:05:440
309715599200,1cyclictest0-21swapper/019:43:100
309715599200,1cyclictest0-21swapper/019:43:100
309715599200,19cyclictest3056-21PK-Backend21:30:040
309715599200,19cyclictest0-21swapper/022:41:020
309715599200,19cyclictest0-21swapper/020:25:340
309715599200,19cyclictest0-21swapper/020:04:460
139099200,17rtkit-daemon1997-21gvfs-afc-volume22:08:550
139099200,16rtkit-daemon3097149-21cyclictest23:49:580
139099200,16rtkit-daemon3097149-21cyclictest19:58:060
3097170991917,1cyclictest3056-21PK-Backend20:08:103
3097170991917,1cyclictest0-21swapper/323:51:573
3097170991917,1cyclictest0-21swapper/322:59:453
3097170991916,2cyclictest3056-21PK-Backend23:24:593
309717099191,5cyclictest3102423-21rt-features19:20:203
3097170991913,3cyclictest3180356-21apt23:05:133
3097170991911,6cyclictest3200698-21ntp_states00:00:213
309717099190,6cyclictest3108739-21/usr/sbin/munin19:40:133
309717099190,6cyclictest3108739-21/usr/sbin/munin19:40:123
309717099190,18cyclictest3056-21PK-Backend23:41:423
309717099190,18cyclictest0-21swapper/321:13:523
309717099190,16cyclictest3179338-21/usr/sbin/munin23:00:173
309716799193,4cyclictest34-21ksoftirqd/221:25:132
309716799192,16cyclictest3133052-21munin-run20:50:002
3097167991917,1cyclictest0-21swapper/223:30:402
3097167991917,1cyclictest0-21swapper/222:57:352
3097167991917,1cyclictest0-21swapper/221:08:242
3097167991917,1cyclictest0-21swapper/219:32:122
3097167991913,4cyclictest3152704-21expr21:45:122
3097167991912,5cyclictest34-21ksoftirqd/219:40:012
3097167991912,5cyclictest34-21ksoftirqd/219:40:012
3097167991912,5cyclictest3122879-21expr20:20:142
309716799190,1cyclictest0-21swapper/221:22:582
309716799190,1cyclictest0-21swapper/200:11:092
309716799190,18cyclictest0-21swapper/223:11:132
309716799190,18cyclictest0-21swapper/222:16:242
309716799190,18cyclictest0-21swapper/219:58:222
3097163991917,1cyclictest3056-21PK-Backend23:46:461
3097163991917,1cyclictest3056-21PK-Backend19:38:251
3097163991917,1cyclictest0-21swapper/120:22:321
3097163991917,1cyclictest0-21swapper/120:11:571
3097163991917,1cyclictest0-21swapper/100:25:051
3097163991916,2cyclictest3056-21PK-Backend23:14:211
3097163991912,5cyclictest3136877-21cat21:00:131
309716399190,1cyclictest3056-21PK-Backend23:51:461
309716399190,1cyclictest3056-21PK-Backend21:55:121
309716399190,1cyclictest0-21swapper/123:21:381
309716399190,1cyclictest0-21swapper/120:09:571
309716399190,18cyclictest3056-21PK-Backend21:34:371
309716399190,18cyclictest0-21swapper/123:28:201
309716399190,18cyclictest0-21swapper/123:18:341
3097155991917,1cyclictest0-21swapper/023:54:280
3097155991917,1cyclictest0-21swapper/023:25:490
3097155991917,1cyclictest0-21swapper/021:13:060
3097155991917,1cyclictest0-21swapper/020:57:520
3097155991917,1cyclictest0-21swapper/020:51:440
309715599190,1cyclictest0-21swapper/023:03:450
309715599190,1cyclictest0-21swapper/000:04:560
309717099184,9cyclictest3115715-21apt20:00:163
3097170991816,1cyclictest3056-21PK-Backend23:39:123
3097170991816,1cyclictest0-21swapper/321:59:363
3097170991815,2cyclictest3056-21PK-Backend20:41:083
3097170991815,2cyclictest3056-21PK-Backend20:41:073
3097170991813,3cyclictest3175734-21memory22:50:183
309717099181,15cyclictest3164871-21diskstats22:20:143
309717099180,16cyclictest3150640-21apt21:40:113
309717099180,16cyclictest3150640-21apt21:40:113
3097167991816,1cyclictest0-21swapper/223:45:272
3097167991816,1cyclictest0-21swapper/222:41:192
3097167991811,5cyclictest34-21ksoftirqd/222:20:112
3097167991811,5cyclictest3150891-21cat21:40:132
3097167991811,5cyclictest3150891-21cat21:40:122
309716799180,1cyclictest0-21swapper/220:42:062
309716799180,1cyclictest0-21swapper/220:42:052
3097163991816,1cyclictest3056-21PK-Backend19:21:141
3097163991816,1cyclictest0-21swapper/120:39:351
3097163991813,3cyclictest3192475-21sh23:40:011
3097163991810,6cyclictest3212835-21seq00:35:201
3097163991810,6cyclictest3210877-21wc00:30:161
3097163991810,6cyclictest3110233-21cron19:44:591
3097163991810,6cyclictest3110233-21cron19:44:591
3097155991816,1cyclictest3056-21PK-Backend22:25:260
3097155991816,1cyclictest0-21swapper/021:29:300
3097155991816,1cyclictest0-21swapper/019:16:540
3097155991814,3cyclictest13-21ksoftirqd/020:15:000
3097155991811,5cyclictest3169867-21cut22:35:130
3097155991811,5cyclictest3169867-21cut22:35:130
309715599180,1cyclictest0-21swapper/023:40:430
3097170991712,3cyclictest3113088-21python319:50:183
3097170991710,5cyclictest3114231-21expr19:55:143
3097167991715,1cyclictest0-21swapper/220:50:062
3097167991715,1cyclictest0-21swapper/200:08:592
3097167991713,3cyclictest978-21snmpd21:50:272
3097167991711,4cyclictest3194221-21sh23:45:002
3097167991710,6cyclictest3159862-21sed22:05:182
3097167991710,5cyclictest436-20systemd-journal20:35:442
309716399179,6cyclictest3145901-21cat21:25:171
3097163991715,1cyclictest3056-21PK-Backend23:58:081
3097163991711,4cyclictest3193375-21missed_timers23:40:191
309716399170,16cyclictest3056-21PK-Backend21:47:501
3097155991715,1cyclictest0-21swapper/022:20:200
3097155991714,2cyclictest0-21swapper/022:55:200
3097155991711,5cyclictest3164218-21cron22:20:000
3097170991614,1cyclictest3056-21PK-Backend00:22:263
309717099160,15cyclictest0-21swapper/320:10:063
309716799162,2cyclictest141rcu_preempt00:02:142
3097167991614,1cyclictest0-21swapper/219:24:092
3097167991614,1cyclictest0-21swapper/200:21:572
3097163991614,1cyclictest3056-21PK-Backend21:24:171
3097163991614,1cyclictest0-21swapper/122:25:341
3097163991611,4cyclictest3176681-21sed22:55:001
3097163991610,5cyclictest3140794-21cat21:10:171
3097170991510,4cyclictest42-21ksoftirqd/322:30:153
3097170991510,3cyclictest3056-21PK-Backend20:25:213
309716799159,5cyclictest3142376-21ls21:15:162
309716399157,6cyclictest3097346-21apt19:10:101
309716399155,2cyclictest141rcu_preempt22:40:001
309716399155,2cyclictest141rcu_preempt22:39:591
309716399151,11cyclictest3176887-21apt22:55:121
309716399150,1cyclictest3056-21PK-Backend22:08:311
3097170991412,1cyclictest3056-21PK-Backend23:29:463
309717099140,12cyclictest3056-21PK-Backend00:30:003
309716399130,3cyclictest3056-21PK-Backend22:50:001
139099130,10rtkit-daemon3056-21PK-Backend23:04:131
309716399117,3cyclictest3056-21PK-Backend22:44:571
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional