You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 06:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat May 30, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
326950521000,3sleep23056-21PK-Backend22:30:162
139099850,72rtkit-daemon0-21swapper/219:08:582
319757328138,13sleep10-21swapper/119:08:581
319656427744,26sleep30-21swapper/319:05:143
319620527545,20sleep00-21swapper/019:05:110
32023612460,2sleep30-21swapper/319:20:143
32786322330,5sleep30-21swapper/322:55:193
32433772270,2sleep20-21swapper/221:15:212
3197790992717,8cyclictest3263369-21cron22:15:001
3197790992717,8cyclictest3263369-21cron22:14:591
139099270,4rtkit-daemon1389-21rtkit-daemon22:00:011
319779799260,24cyclictest3288645-21irqstats23:25:163
3197790992616,7cyclictest3231395-21cron20:45:001
139099260,4rtkit-daemon1389-21rtkit-daemon21:08:131
319779799252,19cyclictest3214083-21apt19:55:113
139099240,4rtkit-daemon3251736-21ntp_states21:40:190
32116022230,3sleep0161rcuc/019:45:210
3197797992320,2cyclictest0-21swapper/322:50:013
3197797992314,7cyclictest3223819-21users20:20:213
3197790992315,6cyclictest3212351-21apt19:50:111
3197790992315,5cyclictest3208603-21cron19:40:001
319779099230,3cyclictest3275548-21munin-run22:50:011
3197787992318,4cyclictest2019-21gmain00:35:000
3197792992220,1cyclictest0-21swapper/200:06:352
3197792992220,1cyclictest0-21swapper/200:02:232
3197792992213,7cyclictest918-21rs:main2
319779299220,21cyclictest0-21swapper/219:44:082
3197790992214,6cyclictest3254455-21cron21:50:001
3197790992211,8cyclictest883-21dbus-daemon21:45:001
3197787992220,1cyclictest0-21swapper/023:53:180
3197787992220,1cyclictest0-21swapper/019:30:370
319778799220,21cyclictest0-21swapper/021:47:170
139099220,4rtkit-daemon1389-21rtkit-daemon00:28:180
139099220,16rtkit-daemon3056-21PK-Backend20:54:203
319779799210,9cyclictest3216185-21diskstats20:00:123
3197792992119,1cyclictest0-21swapper/223:32:562
3197792992119,1cyclictest0-21swapper/222:42:192
3197792992119,1cyclictest0-21swapper/221:48:322
3197792992119,1cyclictest0-21swapper/221:22:162
3197792992119,1cyclictest0-21swapper/221:05:532
3197792992119,1cyclictest0-21swapper/220:51:132
3197792992119,1cyclictest0-21swapper/220:48:092
3197792992119,1cyclictest0-21swapper/220:24:232
3197792992119,1cyclictest0-21swapper/220:14:292
3197792992119,1cyclictest0-21swapper/220:14:292
3197792992119,1cyclictest0-21swapper/220:06:242
3197792992119,1cyclictest0-21swapper/219:55:032
3197792992113,6cyclictest3226295-21cut20:30:012
3197792992112,7cyclictest3282971-21apt23:10:112
319779299210,20cyclictest0-21swapper/222:26:242
319779299210,20cyclictest0-21swapper/221:43:322
319779299210,20cyclictest0-21swapper/220:35:112
319779299210,20cyclictest0-21swapper/200:27:092
319779299210,19cyclictest3056-21PK-Backend23:55:332
319779299210,19cyclictest0-21swapper/219:14:472
3197790992119,1cyclictest0-21swapper/119:58:591
3197790992118,2cyclictest3056-21PK-Backend20:27:451
3197790992113,6cyclictest3296567-21cron23:50:011
3197790992112,7cyclictest436-20systemd-journal21:14:591
3197790992110,8cyclictest3205181-21unin-run19:30:001
319779099210,20cyclictest3056-21PK-Backend21:54:181
319779099210,19cyclictest3056-21PK-Backend23:22:531
3197787992119,1cyclictest0-21swapper/023:57:310
3197787992119,1cyclictest0-21swapper/022:58:020
3197787992119,1cyclictest0-21swapper/022:50:120
3197787992119,1cyclictest0-21swapper/022:10:030
3197787992119,1cyclictest0-21swapper/022:10:020
3197787992119,1cyclictest0-21swapper/021:52:190
3197787992119,1cyclictest0-21swapper/021:39:070
3197787992119,1cyclictest0-21swapper/021:22:230
3197787992119,1cyclictest0-21swapper/021:11:360
3197787992119,1cyclictest0-21swapper/021:06:020
3197787992119,1cyclictest0-21swapper/020:52:300
3197787992119,1cyclictest0-21swapper/020:07:140
3197787992119,1cyclictest0-21swapper/000:39:350
3197787992119,1cyclictest0-21swapper/000:01:430
3197787992118,2cyclictest3056-21PK-Backend20:00:110
3197787992115,4cyclictest3287323-21grep23:20:200
3197787992113,6cyclictest3198520-21cat19:10:160
319778799210,20cyclictest0-21swapper/022:38:280
319778799210,20cyclictest0-21swapper/022:15:510
319778799210,20cyclictest0-21swapper/020:12:210
319778799210,20cyclictest0-21swapper/020:12:210
319778799210,20cyclictest0-21swapper/019:52:330
319778799210,20cyclictest0-21swapper/019:44:360
139099210,18rtkit-daemon2637-21gmain00:11:440
32714452200,4sleep23056-21PK-Backend22:35:212
3197797992018,1cyclictest0-21swapper/323:49:433
3197797992018,1cyclictest0-21swapper/322:13:103
3197797992018,1cyclictest0-21swapper/322:13:103
3197797992018,1cyclictest0-21swapper/321:50:253
3197797992018,1cyclictest0-21swapper/321:29:223
3197797992017,2cyclictest3056-21PK-Backend23:59:093
3197797992015,3cyclictest3304273-21irqstats00:10:183
3197797992012,6cyclictest3235873-21irqstats20:55:163
3197797992011,7cyclictest3248390-21python321:30:183
319779799200,1cyclictest3056-21PK-Backend22:35:003
319779799200,19cyclictest3056-21PK-Backend19:45:273
319779799200,19cyclictest3056-21PK-Backend19:31:283
319779799200,19cyclictest0-21swapper/323:54:573
319779799200,19cyclictest0-21swapper/323:16:583
319779799200,19cyclictest0-21swapper/323:11:363
319779799200,19cyclictest0-21swapper/320:49:223
319779799200,19cyclictest0-21swapper/320:32:253
319779799200,19cyclictest0-21swapper/300:04:093
3197792992018,1cyclictest0-21swapper/223:52:082
3197792992018,1cyclictest0-21swapper/223:45:182
3197792992018,1cyclictest0-21swapper/223:03:422
3197792992018,1cyclictest0-21swapper/222:11:302
3197792992018,1cyclictest0-21swapper/222:11:302
3197792992018,1cyclictest0-21swapper/221:53:322
3197792992018,1cyclictest0-21swapper/221:01:472
3197792992018,1cyclictest0-21swapper/220:30:042
3197792992018,1cyclictest0-21swapper/219:29:552
3197792992018,1cyclictest0-21swapper/200:16:362
3197792992018,1cyclictest0-21swapper/200:10:482
3197792992017,2cyclictest3056-21PK-Backend21:35:142
3197792992017,2cyclictest1095-21runrttasks00:22:122
3197792992017,2cyclictest0-21swapper/219:24:072
3197792992015,3cyclictest3210788-21cpuspeed_turbos19:45:122
3197792992013,6cyclictest3230805-21python320:40:212
3197792992013,5cyclictest3236714-21python320:58:162
319779299200,1cyclictest0-21swapper/223:17:542
319779299200,1cyclictest0-21swapper/221:32:572
319779299200,19cyclictest0-21swapper/223:42:222
319779299200,19cyclictest0-21swapper/223:22:272
319779299200,19cyclictest0-21swapper/223:06:392
319779299200,19cyclictest0-21swapper/223:06:392
319779299200,19cyclictest0-21swapper/222:59:572
3197790992018,1cyclictest3056-21PK-Backend22:24:121
3197790992018,1cyclictest0-21swapper/123:58:421
3197790992018,1cyclictest0-21swapper/123:10:151
3197790992018,1cyclictest0-21swapper/122:07:361
3197790992018,1cyclictest0-21swapper/119:33:311
3197790992013,5cyclictest3278214-21cat22:55:141
319779099200,1cyclictest3056-21PK-Backend22:17:501
319779099200,1cyclictest3056-21PK-Backend20:17:591
319779099200,19cyclictest3056-21PK-Backend21:03:211
319779099200,19cyclictest3056-21PK-Backend20:48:271
319779099200,19cyclictest3056-21PK-Backend00:33:221
319779099200,19cyclictest0-21swapper/123:25:491
319779099200,19cyclictest0-21swapper/122:34:251
319779099200,19cyclictest0-21swapper/119:45:341
319779099200,19cyclictest0-21swapper/119:13:031
319779099200,19cyclictest0-21swapper/100:11:061
3197787992018,1cyclictest0-21swapper/023:26:140
3197787992018,1cyclictest0-21swapper/023:11:140
3197787992018,1cyclictest0-21swapper/023:01:480
3197787992018,1cyclictest0-21swapper/000:07:310
3197787992015,3cyclictest3285194-21sed23:15:140
3197787992015,3cyclictest3203818-21tr19:25:110
319778799200,1cyclictest0-21swapper/023:43:320
319778799200,1cyclictest0-21swapper/021:57:530
319778799200,1cyclictest0-21swapper/000:15:420
319778799200,19cyclictest0-21swapper/023:06:060
319778799200,19cyclictest0-21swapper/023:06:060
319778799200,19cyclictest0-21swapper/021:01:560
319778799200,19cyclictest0-21swapper/020:56:560
319778799200,19cyclictest0-21swapper/020:34:250
319778799200,19cyclictest0-21swapper/020:22:410
319778799200,19cyclictest0-21swapper/019:56:400
139099200,14rtkit-daemon0-21swapper/320:07:313
32481092190,4sleep03056-21PK-Backend21:30:150
3197797991917,1cyclictest3056-21PK-Backend21:58:303
3197797991917,1cyclictest3056-21PK-Backend20:44:353
3197797991917,1cyclictest3056-21PK-Backend19:36:083
3197797991917,1cyclictest0-21swapper/322:26:083
3197797991917,1cyclictest0-21swapper/321:35:253
3197797991917,1cyclictest0-21swapper/321:00:443
3197797991917,1cyclictest0-21swapper/300:20:453
3197797991917,1cyclictest0-21swapper/300:19:573
3197797991916,2cyclictest3056-21PK-Backend23:37:403
3197797991914,3cyclictest3313198-21ntp_states00:35:193
3197797991914,3cyclictest3220016-21ntp_states20:10:183
3197797991914,3cyclictest3220016-21ntp_states20:10:183
3197797991913,5cyclictest883-21dbus-daemon22:55:013
319779799190,7cyclictest3243846-21cut21:20:003
319779799190,6cyclictest3228945-21ntp_states20:35:183
319779799190,1cyclictest3056-21PK-Backend23:34:423
319779799190,1cyclictest3056-21PK-Backend22:01:003
319779799190,1cyclictest3056-21PK-Backend21:09:163
319779799190,1cyclictest3056-21PK-Backend19:28:103
319779799190,18cyclictest0-21swapper/323:41:583
319779799190,18cyclictest0-21swapper/323:24:283
319779799190,18cyclictest0-21swapper/323:08:403
319779799190,18cyclictest0-21swapper/323:08:393
319779799190,18cyclictest0-21swapper/320:29:333
319779799190,18cyclictest0-21swapper/320:16:233
3197792991917,1cyclictest0-21swapper/223:27:012
3197792991917,1cyclictest0-21swapper/222:22:312
3197792991917,1cyclictest0-21swapper/220:02:062
3197792991910,7cyclictest3313533-21awk00:35:222
319779299190,1cyclictest3056-21PK-Backend22:00:272
319779299190,1cyclictest0-21swapper/222:53:132
319779299190,1cyclictest0-21swapper/222:46:252
319779299190,1cyclictest0-21swapper/219:34:352
319779299190,18cyclictest0-21swapper/220:18:212
319779099194,11cyclictest3298305-21munin-run23:55:011
3197790991917,1cyclictest3056-21PK-Backend21:20:381
3197790991917,1cyclictest0-21swapper/123:00:481
3197790991917,1cyclictest0-21swapper/122:35:481
3197790991911,6cyclictest26-21ksoftirqd/120:00:171
3197790991910,7cyclictest3301063-21ntp_states00:00:191
3197790991910,7cyclictest3208885-21apt19:40:101
319779099190,1cyclictest3056-21PK-Backend23:40:241
319779099190,1cyclictest3056-21PK-Backend23:07:181
319779099190,1cyclictest3056-21PK-Backend23:07:181
319779099190,1cyclictest3056-21PK-Backend22:54:071
319779099190,1cyclictest3056-21PK-Backend22:44:331
319779099190,1cyclictest3056-21PK-Backend20:11:091
319779099190,1cyclictest3056-21PK-Backend20:11:091
319779099190,1cyclictest3056-21PK-Backend19:21:001
319779099190,1cyclictest0-21swapper/123:30:561
319779099190,1cyclictest0-21swapper/121:37:421
319779099190,1cyclictest0-21swapper/121:31:401
3197787991917,1cyclictest0-21swapper/022:06:110
3197787991917,1cyclictest0-21swapper/019:38:350
3197787991913,4cyclictest3245975-21cpuspeed_turbos21:25:130
319778799190,1cyclictest0-21swapper/023:35:400
319778799190,1cyclictest0-21swapper/023:34:530
319778799190,1cyclictest0-21swapper/022:33:490
319778799190,18cyclictest0-21swapper/022:04:490
139099190,15rtkit-daemon3197781-21cyclictest23:37:131
319779799189,7cyclictest3244646-21memory21:20:173
3197797991816,1cyclictest0-21swapper/322:22:163
3197797991816,1cyclictest0-21swapper/321:47:553
3197797991816,1cyclictest0-21swapper/319:54:473
3197797991813,3cyclictest3302931-21switchtime00:05:233
3197797991813,3cyclictest3200957-21ntp_states19:15:173
3197797991813,3cyclictest3200957-21ntp_states19:15:173
319779799180,1cyclictest0-21swapper/322:18:583
319779299189,7cyclictest883-21dbus-daemon19:20:002
319779299189,7cyclictest883-21dbus-daemon19:20:002
3197792991817,1cyclictest0-21swapper/219:54:152
3197792991814,3cyclictest978-21snmpd21:25:392
3197792991812,4cyclictest1095-21runrttasks22:15:112
3197790991816,1cyclictest3056-21PK-Backend20:30:021
3197790991816,1cyclictest3056-21PK-Backend20:22:181
3197790991816,1cyclictest3056-21PK-Backend19:15:531
3197790991816,1cyclictest3056-21PK-Backend19:15:531
3197790991816,1cyclictest0-21swapper/123:19:551
3197790991816,1cyclictest0-21swapper/121:27:551
3197790991811,5cyclictest3217723-21grep20:05:131
3197790991810,6cyclictest436-20systemd-journal20:53:171
3197787991816,1cyclictest0-21swapper/022:23:350
3197787991816,1cyclictest0-21swapper/020:44:530
3197787991812,5cyclictest3201669-21cron19:20:000
3197787991812,5cyclictest3201669-21cron19:20:000
3197787991810,6cyclictest3242910-21memory21:15:160
319778799180,1cyclictest0-21swapper/020:18:230
3197797991715,1cyclictest3056-21PK-Backend23:00:013
3197797991715,1cyclictest3056-21PK-Backend21:12:473
3197797991712,3cyclictest3309084-21cpuspeed_turbos00:25:153
319779799170,15cyclictest3273780-21munin-run22:45:003
319779299177,8cyclictest883-21dbus-daemon00:35:012
3197792991715,1cyclictest3056-21PK-Backend19:35:222
3197792991715,1cyclictest0-21swapper/221:13:312
319779099173,9cyclictest3235399-21apt20:55:111
3197790991715,1cyclictest3056-21PK-Backend22:28:041
3197790991715,1cyclictest3056-21PK-Backend21:16:321
3197790991715,1cyclictest3056-21PK-Backend00:37:211
3197790991715,1cyclictest0-21swapper/122:03:441
319778799173,13cyclictest3203421-21perf19:25:000
3197787991715,1cyclictest0-21swapper/023:49:060
3197787991715,1cyclictest0-21swapper/022:46:060
3197787991713,3cyclictest3268196-21vmstat22:25:210
3197787991712,4cyclictest3225441-21awk20:25:170
3197787991710,6cyclictest3232394-21ntpq20:45:180
3197787991710,5cyclictest3307421-21grep00:20:120
3197797991614,1cyclictest3056-21PK-Backend22:06:483
3197797991614,1cyclictest3056-21PK-Backend21:41:273
3197797991611,3cyclictest3311118-21hddtemp_smartct00:30:153
3197792991614,1cyclictest0-21swapper/222:05:422
3197787991613,2cyclictest3273767-21cron22:45:000
3197787991611,3cyclictest3228366-21cpuspeed_turbos20:35:120
319779799158,5cyclictest3198754-21awk19:10:183
3197797991513,1cyclictest3056-21PK-Backend19:43:313
3197797991511,3cyclictest3199699-21munin-node22:35:123
319779299158,5cyclictest883-21dbus-daemon23:40:012
139099150,12rtkit-daemon3056-21PK-Backend00:29:281
139099140,11rtkit-daemon3056-21PK-Backend00:23:441
319779099131,2cyclictest141rcu_preempt00:05:131
3197790991310,2cyclictest3056-21PK-Backend20:36:511
139099130,10rtkit-daemon3056-21PK-Backend00:15:411
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional