You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-31 - 01:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sat May 30, 2026 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139099810,70rtkit-daemon0-21swapper/007:05:080
348206027643,26sleep30-21swapper/307:06:293
34848882750,2sleep00-21swapper/007:15:110
348223227050,15sleep20-21swapper/207:08:412
348215726140,14sleep10-21swapper/107:07:421
34856022600,2sleep30-21swapper/307:15:183
35546462590,3sleep00-21swapper/010:30:190
139099380,3rtkit-daemon0-21swapper/212:33:352
139099380,3rtkit-daemon0-21swapper/211:27:462
139099380,3rtkit-daemon0-21swapper/211:17:072
139099380,3rtkit-daemon0-21swapper/209:57:512
139099370,3rtkit-daemon0-21swapper/211:21:272
139099370,3rtkit-daemon0-21swapper/210:54:052
139099370,3rtkit-daemon0-21swapper/210:54:042
139099370,3rtkit-daemon0-21swapper/210:42:182
139099370,3rtkit-daemon0-21swapper/209:48:052
139099360,3rtkit-daemon0-21swapper/212:20:452
139099360,3rtkit-daemon0-21swapper/212:12:512
139099360,3rtkit-daemon0-21swapper/211:36:152
139099350,2rtkit-daemon0-21swapper/209:22:342
348246799259,14cyclictest3571053-21sed11:20:013
3482465992515,7cyclictest3583359-21cron11:55:002
35523422240,3sleep20-21swapper/210:25:172
35470402240,2sleep1251rcuc/110:10:151
348246799246,16cyclictest3576474-21cut11:35:003
348246799240,22cyclictest3541930-21/usr/sbin/munin09:55:183
348246199242,19cyclictest3519583-21taskset08:54:541
348246199242,19cyclictest3519583-21taskset08:54:541
3482461992414,8cyclictest883-21dbus-daemon10:25:001
35860192230,10sleep342-21ksoftirqd/312:00:173
3482467992310,11cyclictest3505510-21unin-run08:15:003
3482465992320,2cyclictest0-21swapper/209:52:162
348246199231,3cyclictest3554173-21memory10:30:151
348246199230,21cyclictest3581519-21chrt11:48:351
3482467992220,1cyclictest0-21swapper/309:49:303
3482467992216,5cyclictest2085813-21snapd11:01:143
348246799220,21cyclictest0-21swapper/309:10:373
3482465992220,1cyclictest0-21swapper/209:01:272
3482465992220,1cyclictest0-21swapper/208:46:462
3482465992217,3cyclictest3532001-20cron09:30:012
3482465992212,7cyclictest436-20systemd-journal09:10:002
348246199223,4cyclictest26-21ksoftirqd/107:15:021
3482461992220,1cyclictest0-21swapper/111:26:391
3482461992220,1cyclictest0-21swapper/110:05:351
3482461992219,2cyclictest3056-21PK-Backend10:45:151
3482461992216,3cyclictest3530012-21sh09:25:001
35856342210,3sleep20-21swapper/212:00:112
35092382210,4sleep00-21swapper/008:25:010
35045422210,3chrt3056-21PK-Backend08:10:160
348246799211,9cyclictest3492741-21/usr/sbin/munin07:35:203
3482467992119,1cyclictest0-21swapper/309:05:373
3482467992113,7cyclictest3581914-21apt11:50:133
3482467992113,6cyclictest3595780-21awk12:30:113
3482467992113,6cyclictest3562654-21hddtemp_smartct10:55:163
3482467992113,6cyclictest3544981-21awk10:05:123
348246799210,2cyclictest3556921-21sed10:40:003
348246799210,20cyclictest0-21swapper/308:06:363
348246599212,13cyclictest3543197-21apt10:00:112
3482465992119,1cyclictest0-21swapper/211:44:242
3482465992119,1cyclictest0-21swapper/211:44:232
3482465992119,1cyclictest0-21swapper/211:13:412
3482465992119,1cyclictest0-21swapper/210:15:482
3482465992119,1cyclictest0-21swapper/209:41:482
3482465992119,1cyclictest0-21swapper/208:39:442
3482465992119,1cyclictest0-21swapper/208:24:222
3482465992119,1cyclictest0-21swapper/208:16:512
3482465992119,1cyclictest0-21swapper/207:34:272
3482465992119,1cyclictest0-21swapper/207:23:182
348246599210,20cyclictest0-21swapper/211:06:452
348246599210,20cyclictest0-21swapper/210:08:252
348246199219,9cyclictest883-21dbus-daemon10:30:001
3482461992119,1cyclictest3056-21PK-Backend11:02:111
3482461992119,1cyclictest3056-21PK-Backend10:41:301
3482461992119,1cyclictest3056-21PK-Backend10:03:111
3482461992119,1cyclictest3056-21PK-Backend08:45:441
3482461992119,1cyclictest3056-21PK-Backend07:22:441
3482461992119,1cyclictest0-21swapper/111:35:511
3482461992119,1cyclictest0-21swapper/111:08:541
3482461992119,1cyclictest0-21swapper/110:18:181
3482461992119,1cyclictest0-21swapper/109:18:221
3482461992119,1cyclictest0-21swapper/107:45:491
3482461992110,8cyclictest883-21dbus-daemon09:35:001
3482461992110,8cyclictest3521369-21unin-run09:00:001
348246199210,20cyclictest3056-21PK-Backend12:16:061
348246199210,20cyclictest3056-21PK-Backend09:47:491
348246199210,20cyclictest3056-21PK-Backend08:42:201
348246199210,20cyclictest3056-21PK-Backend08:11:521
348246199210,20cyclictest3056-21PK-Backend08:06:241
348246199210,20cyclictest0-21swapper/112:38:171
348246199210,20cyclictest0-21swapper/112:38:171
348246199210,1cyclictest0-21swapper/108:37:541
348246199210,19cyclictest3484144-21dump-pmu-power07:15:001
348246199210,19cyclictest3056-21PK-Backend12:27:291
348246199210,19cyclictest3056-21PK-Backend12:01:331
348246199210,19cyclictest3056-21PK-Backend11:42:541
348246199210,19cyclictest3056-21PK-Backend11:42:541
348246199210,19cyclictest3056-21PK-Backend10:50:231
348246199210,19cyclictest3056-21PK-Backend10:50:221
348246199210,19cyclictest3056-21PK-Backend07:52:381
3482458992113,6cyclictest3539838-21df_inode09:50:140
348245899210,20cyclictest0-21swapper/008:09:010
139099210,14rtkit-daemon0-21swapper/212:17:372
348246799203,15cyclictest0-21swapper/307:41:043
3482467992018,1cyclictest0-21swapper/311:56:523
3482467992018,1cyclictest0-21swapper/311:23:403
3482467992015,3cyclictest1095-21runrttasks07:20:123
3482467992012,6cyclictest883-21dbus-daemon09:45:003
348246799201,17cyclictest3591647-21users12:15:213
348246799200,8cyclictest3593085-21/usr/sbin/munin12:20:193
348246799200,1cyclictest3056-21PK-Backend09:28:163
348246799200,19cyclictest3056-21PK-Backend11:07:523
348246799200,19cyclictest3056-21PK-Backend09:54:323
348246799200,19cyclictest3056-21PK-Backend08:53:003
348246799200,19cyclictest3056-21PK-Backend08:53:003
348246799200,19cyclictest3056-21PK-Backend08:27:053
348246799200,19cyclictest0-21swapper/311:42:203
348246799200,19cyclictest0-21swapper/311:42:193
348246799200,19cyclictest0-21swapper/310:52:033
348246799200,19cyclictest0-21swapper/310:52:033
348246799200,19cyclictest0-21swapper/310:02:283
348246799200,19cyclictest0-21swapper/308:49:353
3482465992018,1cyclictest0-21swapper/212:36:362
3482465992018,1cyclictest0-21swapper/212:36:352
3482465992018,1cyclictest0-21swapper/211:45:122
3482465992018,1cyclictest0-21swapper/210:49:212
3482465992018,1cyclictest0-21swapper/210:35:422
3482465992018,1cyclictest0-21swapper/208:25:232
3482465992018,1cyclictest0-21swapper/208:05:302
3482465992018,1cyclictest0-21swapper/207:56:112
3482465992018,1cyclictest0-21swapper/207:56:102
3482465992018,1cyclictest0-21swapper/207:15:472
3482465992016,3cyclictest1095-21runrttasks12:28:522
3482465992013,5cyclictest3488813-21cat07:25:152
3482465992012,6cyclictest3584388-21ntpq11:55:182
348246599200,19cyclictest0-21swapper/211:34:572
348246599200,19cyclictest0-21swapper/210:59:492
348246599200,19cyclictest0-21swapper/209:11:402
348246599200,19cyclictest0-21swapper/208:11:102
348246599200,19cyclictest0-21swapper/208:00:502
348246599200,19cyclictest0-21swapper/207:47:512
348246199203,4cyclictest141rcu_preempt08:20:011
3482461992018,1cyclictest3056-21PK-Backend08:00:561
3482461992018,1cyclictest0-21swapper/112:33:581
3482461992018,1cyclictest0-21swapper/111:23:071
3482461992018,1cyclictest0-21swapper/109:29:101
3482461992017,1cyclictest3056-21PK-Backend07:36:011
3482461992011,7cyclictest2270-21gdbus11:10:031
348246199200,1cyclictest3056-21PK-Backend07:58:051
348246199200,1cyclictest3056-21PK-Backend07:58:051
348246199200,19cyclictest3056-21PK-Backend12:13:571
348246199200,19cyclictest3056-21PK-Backend12:06:401
348246199200,19cyclictest3056-21PK-Backend11:33:421
348246199200,19cyclictest0-21swapper/109:40:131
348246199200,19cyclictest0-21swapper/109:14:571
348246199200,19cyclictest0-21swapper/107:27:161
3482458992018,1cyclictest3056-21PK-Backend09:18:440
3482458992018,1cyclictest0-21swapper/012:30:210
3482458992018,1cyclictest0-21swapper/010:29:340
3482458992018,1cyclictest0-21swapper/010:22:240
348245899200,1cyclictest0-21swapper/008:55:320
348245899200,19cyclictest0-21swapper/010:49:010
348245899200,19cyclictest0-21swapper/009:44:050
348245899200,19cyclictest0-21swapper/008:45:460
348245899200,19cyclictest0-21swapper/008:17:410
348245899200,19cyclictest0-21swapper/007:37:380
139099200,14rtkit-daemon0-21swapper/212:06:372
35381432190,5chrt0-21swapper/009:45:170
3482467991917,1cyclictest3056-21PK-Backend09:00:373
3482467991917,1cyclictest0-21swapper/312:13:013
3482467991917,1cyclictest0-21swapper/311:47:263
3482467991917,1cyclictest0-21swapper/309:24:033
3482467991917,1cyclictest0-21swapper/307:12:013
3482467991914,3cyclictest3559188-21hddtemp_smartct10:45:153
3482467991912,5cyclictest42-21ksoftirqd/307:45:103
3482467991911,6cyclictest3573512-21expr11:25:113
348246799190,1cyclictest3056-21PK-Backend10:19:123
348246799190,1cyclictest3056-21PK-Backend09:32:273
348246799190,1cyclictest3056-21PK-Backend08:57:123
348246799190,1cyclictest3056-21PK-Backend07:27:363
348246799190,1cyclictest0-21swapper/310:30:203
348246799190,1cyclictest0-21swapper/307:56:163
348246799190,1cyclictest0-21swapper/307:56:163
348246799190,18cyclictest3056-21PK-Backend10:14:253
348246799190,18cyclictest3056-21PK-Backend08:32:183
348246799190,18cyclictest0-21swapper/307:53:253
3482465991917,1cyclictest0-21swapper/209:35:462
3482465991911,6cyclictest1-21systemd09:35:012
348246599190,1cyclictest0-21swapper/208:41:322
348246599190,1cyclictest0-21swapper/208:31:392
3482461991917,1cyclictest3056-21PK-Backend11:59:241
3482461991911,6cyclictest3508515-21awk08:20:191
3482461991910,7cyclictest3539472-21apt09:50:121
348246199190,1cyclictest3056-21PK-Backend09:59:121
348246199190,1cyclictest0-21swapper/111:52:551
348246199190,1cyclictest0-21swapper/110:39:551
348246199190,1cyclictest0-21swapper/108:32:391
3482458991917,1cyclictest0-21swapper/012:39:330
3482458991917,1cyclictest0-21swapper/012:39:330
3482458991917,1cyclictest0-21swapper/011:32:270
3482458991917,1cyclictest0-21swapper/009:28:170
3482458991917,1cyclictest0-21swapper/009:13:440
3482458991917,1cyclictest0-21swapper/007:54:220
3482458991916,2cyclictest3056-21PK-Backend07:40:010
3482458991914,3cyclictest3522121-21cat09:00:150
3482458991912,6cyclictest3533349-21awk09:30:220
348245899191,16cyclictest3570165-21tr11:15:180
348245899190,1cyclictest0-21swapper/010:15:150
348245899190,1cyclictest0-21swapper/010:07:170
348245899190,1cyclictest0-21swapper/009:55:270
348245899190,1cyclictest0-21swapper/008:02:590
348245899190,18cyclictest0-21swapper/012:07:560
348245899190,18cyclictest0-21swapper/011:45:180
348245899190,18cyclictest0-21swapper/011:42:360
348245899190,18cyclictest0-21swapper/011:42:350
348245899190,18cyclictest0-21swapper/011:21:120
348245899190,18cyclictest0-21swapper/010:52:320
348245899190,18cyclictest0-21swapper/010:52:320
348245899190,18cyclictest0-21swapper/008:51:200
348245899190,18cyclictest0-21swapper/008:51:190
348245899190,18cyclictest0-21swapper/008:39:090
348245899190,18cyclictest0-21swapper/007:56:590
348245899190,18cyclictest0-21swapper/007:56:580
348245899190,18cyclictest0-21swapper/007:10:070
35149682180,2sleep00-21swapper/008:40:120
3482467991816,1cyclictest3056-21PK-Backend12:36:133
3482467991816,1cyclictest3056-21PK-Backend12:36:133
3482467991816,1cyclictest3056-21PK-Backend08:36:443
3482467991816,1cyclictest3056-21PK-Backend07:30:473
3482467991816,1cyclictest0-21swapper/311:12:253
3482467991816,1cyclictest0-21swapper/310:44:343
3482467991816,1cyclictest0-21swapper/308:21:373
3482467991813,3cyclictest3501294-21ntp_states08:00:193
3482467991813,3cyclictest1095-21runrttasks08:19:083
348246799180,1cyclictest0-21swapper/312:09:173
348246799180,16cyclictest3577513-21python311:35:213
3482465991816,1cyclictest3056-21PK-Backend08:55:252
3482465991816,1cyclictest0-21swapper/210:23:592
3482465991816,1cyclictest0-21swapper/207:53:192
3482465991816,1cyclictest0-21swapper/207:44:282
3482465991811,4cyclictest3527672-21python309:15:222
3482465991810,6cyclictest436-20systemd-journal08:55:002
3482465991810,6cyclictest436-20systemd-journal08:54:592
3482461991816,1cyclictest3056-21PK-Backend10:56:181
3482461991816,1cyclictest0-21swapper/109:36:011
3482461991816,1cyclictest0-21swapper/107:30:271
3482461991814,2cyclictest3056-21PK-Backend12:23:581
3482458991816,1cyclictest0-21swapper/011:58:570
3482458991816,1cyclictest0-21swapper/011:09:370
3482458991816,1cyclictest0-21swapper/010:39:140
3482458991816,1cyclictest0-21swapper/010:12:040
3482458991816,1cyclictest0-21swapper/010:03:320
3482458991814,3cyclictest3491571-21idleruntime-cro07:35:000
3482458991811,6cyclictest3582943-21sed11:50:230
3482458991810,6cyclictest3534306-21expr09:35:140
3482458991810,6cyclictest1-21systemd08:35:000
35623652170,3sleep00-21swapper/010:55:130
348246799171,3cyclictest2156-21gnome-shell12:25:093
348246599179,6cyclictest3484154-21basename07:15:002
3482465991715,1cyclictest0-21swapper/210:15:002
3482461991714,2cyclictest3056-21PK-Backend11:18:001
3482461991712,3cyclictest3521782-21grep09:00:121
3482461991710,5cyclictest3509774-21expr08:25:121
348245899170,1cyclictest3056-21PK-Backend11:12:330
3482467991613,2cyclictest3056-21PK-Backend09:35:043
3482467991611,4cyclictest42-21ksoftirqd/308:43:403
348246599169,5cyclictest3564300-21cat11:00:122
3482465991614,1cyclictest0-21swapper/210:34:542
3482465991611,3cyclictest0-21swapper/207:35:152
348246199168,3cyclictest3523548-21cstates09:05:141
348246199161,13cyclictest3493793-21cpuspeed_turbos07:40:121
348245899169,6cyclictest3523686-21expr09:05:130
348245899168,6cyclictest3592445-21grep12:20:120
348245899168,6cyclictest3574661-21munin-run11:30:000
3482458991614,1cyclictest3056-21PK-Backend11:35:120
3482458991614,1cyclictest0-21swapper/007:49:210
3482458991611,3cyclictest3591182-21missed_timers12:15:160
3482458991611,3cyclictest3489037-21needreboot07:25:180
348245899160,1cyclictest0-21swapper/010:41:250
348246799156,7cyclictest3553322-21cron10:30:003
3482467991513,1cyclictest0-21swapper/309:19:513
348245899159,4cyclictest3594905-21smartctl12:25:210
348245899152,11cyclictest3589033-21tr12:10:120
3482458991513,1cyclictest0-21swapper/007:24:000
3482458991510,3cyclictest3585542-21cpuspeed_turbos12:00:120
348246799140,3cyclictest3550096-21chrt10:20:043
348245899147,6cyclictest3529004-21cat09:20:170
3482458991411,2cyclictest35097762sleep008:25:130
348245899125,5cyclictest3564685-21netstat11:00:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional