You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 17:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Apr 23, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
353896521120,3sleep13056-21PK-Backend07:25:121
360170021090,5sleep13601696-21munin-run10:25:011
355203921020,3sleep10-21swapper/108:00:201
139099920,73rtkit-daemon0-21swapper/007:05:050
353252427340,26sleep30-21swapper/307:08:153
353159427144,19sleep20-21swapper/207:05:162
353169926745,14sleep10-21swapper/107:05:181
35508912390,4sleep03550875-21munin-run08:00:000
35930822350,2sleep30-21swapper/310:00:003
35930822350,2sleep30-21swapper/310:00:003
36117212280,4chrt0-21swapper/110:50:191
35579452270,7sleep0131ksoftirqd/008:20:010
3532809992714,5cyclictest26-21ksoftirqd/110:00:301
139099250,3rtkit-daemon1389-21rtkit-daemon07:36:560
3532817992410,4cyclictest34-21ksoftirqd/209:21:422
353281799238,13cyclictest3545399-21cron07:45:012
353281799237,4cyclictest34-21ksoftirqd/208:20:552
3532817992321,1cyclictest0-21swapper/210:04:082
3532817992321,1cyclictest0-21swapper/208:43:042
3532817992312,3cyclictest141rcu_preempt08:35:032
3532809992320,2cyclictest3056-21PK-Backend09:58:551
3532809992320,2cyclictest3056-21PK-Backend09:58:551
3532809992312,3cyclictest26-21ksoftirqd/110:25:481
3532805992320,2cyclictest0-21swapper/007:27:530
3532805992317,4cyclictest3637383-21munin-plugin-st12:05:000
3532805992310,11cyclictest3622724-21cron11:25:000
36036902220,4sleep00-21swapper/010:30:010
3532822992214,6cyclictest3565631-21df_inode08:40:123
353281799228,4cyclictest34-21ksoftirqd/208:26:542
353281799228,4cyclictest34-21ksoftirqd/208:05:442
353281799226,4cyclictest34-21ksoftirqd/212:39:362
353281799226,4cyclictest34-21ksoftirqd/211:41:022
353281799226,4cyclictest34-21ksoftirqd/210:23:182
353281799226,4cyclictest34-21ksoftirqd/209:54:472
353281799225,5cyclictest34-21ksoftirqd/208:45:112
3532817992220,1cyclictest0-21swapper/211:33:172
3532817992220,1cyclictest0-21swapper/210:57:482
3532817992220,1cyclictest0-21swapper/209:25:082
3532817992220,1cyclictest0-21swapper/209:16:562
3532817992220,1cyclictest0-21swapper/208:58:322
3532817992220,1cyclictest0-21swapper/208:58:322
3532817992220,1cyclictest0-21swapper/208:04:512
3532817992220,1cyclictest0-21swapper/207:23:002
3532817992212,8cyclictest3645292-21/usr/sbin/munin12:25:182
3532817992211,4cyclictest34-21ksoftirqd/211:37:282
353281799220,21cyclictest0-21swapper/211:49:342
353280999227,3cyclictest26-21ksoftirqd/108:41:381
3532805992220,1cyclictest0-21swapper/011:27:500
3532805992220,1cyclictest0-21swapper/009:36:570
3532805992220,1cyclictest0-21swapper/009:31:010
3532805992220,1cyclictest0-21swapper/008:45:470
3532805992220,1cyclictest0-21swapper/007:12:450
353280599220,20cyclictest3056-21PK-Backend07:52:000
139099220,4rtkit-daemon1389-21rtkit-daemon11:46:351
139099220,4rtkit-daemon1389-21rtkit-daemon08:36:241
35839612210,5chrt42-21ksoftirqd/309:30:243
3532822992119,1cyclictest3056-21PK-Backend08:28:283
3532822992119,1cyclictest0-21swapper/309:35:393
3532822992118,2cyclictest0-21swapper/307:55:163
3532822992113,6cyclictest3627567-21memory11:35:173
353282299211,18cyclictest3548044-21diskstats07:50:133
353281799217,3cyclictest34-21ksoftirqd/207:56:512
353281799216,4cyclictest34-21ksoftirqd/211:58:532
353281799216,4cyclictest34-21ksoftirqd/211:58:532
353281799214,4cyclictest34-21ksoftirqd/211:50:442
3532817992119,1cyclictest0-21swapper/212:13:062
3532817992119,1cyclictest0-21swapper/212:05:362
3532817992119,1cyclictest0-21swapper/212:04:342
3532817992119,1cyclictest0-21swapper/211:27:222
3532817992119,1cyclictest0-21swapper/209:05:422
3532817992119,1cyclictest0-21swapper/208:50:332
3532817992119,1cyclictest0-21swapper/208:12:082
3532817992119,1cyclictest0-21swapper/207:38:352
3532817992119,1cyclictest0-21swapper/207:33:422
3532817992119,1cyclictest0-21swapper/207:16:382
3532817992119,1cyclictest0-21swapper/207:12:532
3532817992118,2cyclictest0-21swapper/212:23:122
3532817992117,3cyclictest34-21ksoftirqd/211:15:012
3532817992117,3cyclictest34-21ksoftirqd/209:45:002
353281799210,20cyclictest0-21swapper/211:15:332
353280999216,4cyclictest26-21ksoftirqd/112:12:341
353280999216,4cyclictest26-21ksoftirqd/108:19:291
353280999213,2cyclictest141rcu_preempt10:47:471
3532809992119,1cyclictest3056-21PK-Backend11:09:591
3532809992119,1cyclictest0-21swapper/109:31:311
3532809992118,2cyclictest3056-21PK-Backend11:13:371
3532809992114,5cyclictest3561254-21cron08:29:591
3532809992113,6cyclictest3562996-21apt-get08:35:001
353280999211,18cyclictest3544082-21tr07:40:131
353280999210,19cyclictest3056-21PK-Backend10:07:121
3532805992119,1cyclictest3056-21PK-Backend12:08:110
3532805992119,1cyclictest0-21swapper/012:39:070
3532805992119,1cyclictest0-21swapper/012:23:460
3532805992119,1cyclictest0-21swapper/012:15:410
3532805992119,1cyclictest0-21swapper/012:14:400
3532805992119,1cyclictest0-21swapper/011:46:550
3532805992119,1cyclictest0-21swapper/011:39:260
3532805992119,1cyclictest0-21swapper/010:12:400
3532805992119,1cyclictest0-21swapper/009:26:080
3532805992119,1cyclictest0-21swapper/008:12:060
3532805992119,1cyclictest0-21swapper/008:01:400
3532805992118,2cyclictest3056-21PK-Backend07:40:170
3532805992117,3cyclictest3579777-21diskstats09:20:140
353280599210,20cyclictest0-21swapper/012:33:390
353280599210,20cyclictest0-21swapper/010:56:540
353280599210,20cyclictest0-21swapper/009:14:060
139099210,18rtkit-daemon1317-21systemd07:19:521
35918772200,4sleep03056-21PK-Backend09:55:110
35918772200,4sleep03056-21PK-Backend09:55:110
353282299205,13cyclictest42-21ksoftirqd/309:05:223
3532822992018,1cyclictest0-21swapper/312:27:373
3532822992018,1cyclictest0-21swapper/310:28:053
3532822992018,1cyclictest0-21swapper/310:04:273
3532822992017,2cyclictest3056-21PK-Backend11:45:193
3532822992017,2cyclictest3056-21PK-Backend09:26:553
3532822992017,2cyclictest3056-21PK-Backend08:31:323
3532822992014,4cyclictest886-21NetworkManager08:25:003
3532822992012,6cyclictest436-20systemd-journal08:01:253
353282299200,1cyclictest3056-21PK-Backend10:36:033
353282299200,1cyclictest3056-21PK-Backend10:22:103
353282299200,19cyclictest3056-21PK-Backend12:39:193
353282299200,19cyclictest3056-21PK-Backend09:24:523
353282299200,19cyclictest0-21swapper/312:23:583
353282299200,19cyclictest0-21swapper/311:56:143
353282299200,19cyclictest0-21swapper/311:56:133
353282299200,19cyclictest0-21swapper/311:08:283
353282299200,19cyclictest0-21swapper/308:36:383
353281799207,4cyclictest34-21ksoftirqd/210:18:452
353281799205,3cyclictest34-21ksoftirqd/208:34:402
3532817992018,1cyclictest0-21swapper/211:05:472
3532817992018,1cyclictest0-21swapper/210:38:022
3532817992018,1cyclictest0-21swapper/209:33:062
3532817992018,1cyclictest0-21swapper/209:10:482
3532817992018,1cyclictest0-21swapper/209:02:372
3532817992018,1cyclictest0-21swapper/207:54:572
353281799200,1cyclictest0-21swapper/212:15:552
353281799200,1cyclictest0-21swapper/210:32:542
353281799200,19cyclictest0-21swapper/211:21:272
353281799200,19cyclictest0-21swapper/210:05:092
353281799200,19cyclictest0-21swapper/209:45:092
353281799200,19cyclictest0-21swapper/208:19:382
353281799200,19cyclictest0-21swapper/207:26:242
353280999206,3cyclictest26-21ksoftirqd/111:30:391
353280999206,3cyclictest141rcu_preempt09:15:011
353280999205,7cyclictest141rcu_preempt10:15:091
3532809992018,1cyclictest3056-21PK-Backend12:29:281
3532809992018,1cyclictest3056-21PK-Backend12:16:521
3532809992018,1cyclictest3056-21PK-Backend09:43:551
3532809992018,1cyclictest0-21swapper/112:01:571
3532809992018,1cyclictest0-21swapper/109:18:051
3532809992017,2cyclictest3056-21PK-Backend09:00:281
3532809992017,2cyclictest3056-21PK-Backend07:58:571
3532809992016,2cyclictest36354952sleep111:58:161
3532809992016,2cyclictest36354952sleep111:58:161
353280999200,1cyclictest3056-21PK-Backend10:33:221
353280999200,1cyclictest0-21swapper/111:37:161
353280999200,1cyclictest0-21swapper/110:44:581
353280999200,19cyclictest3056-21PK-Backend07:46:131
353280999200,19cyclictest0-21swapper/111:50:151
353280999200,19cyclictest0-21swapper/109:20:081
353280999200,18cyclictest3056-21PK-Backend09:50:231
3532805992018,1cyclictest3056-21PK-Backend10:46:180
3532805992018,1cyclictest0-21swapper/012:27:250
3532805992018,1cyclictest0-21swapper/010:53:020
3532805992018,1cyclictest0-21swapper/010:40:230
3532805992018,1cyclictest0-21swapper/010:38:220
3532805992018,1cyclictest0-21swapper/009:50:020
3532805992018,1cyclictest0-21swapper/008:57:230
3532805992018,1cyclictest0-21swapper/008:57:230
3532805992018,1cyclictest0-21swapper/008:40:460
3532805992018,1cyclictest0-21swapper/008:27:220
3532805992018,1cyclictest0-21swapper/008:07:400
353280599200,1cyclictest3056-21PK-Backend07:45:230
353280599200,1cyclictest0-21swapper/011:16:130
353280599200,19cyclictest0-21swapper/011:40:260
353280599200,19cyclictest0-21swapper/011:31:420
353280599200,19cyclictest0-21swapper/010:06:450
353280599200,19cyclictest0-21swapper/009:45:560
353280599200,19cyclictest0-21swapper/008:51:280
353280599200,19cyclictest0-21swapper/007:33:210
139099200,3rtkit-daemon1389-21rtkit-daemon11:05:010
139099200,17rtkit-daemon1317-21systemd09:36:301
35475972190,4chrt0-21swapper/307:50:003
3532822991917,1cyclictest3056-21PK-Backend11:50:453
3532822991917,1cyclictest3056-21PK-Backend08:06:453
3532822991917,1cyclictest0-21swapper/312:09:113
3532822991917,1cyclictest0-21swapper/312:02:433
3532822991917,1cyclictest0-21swapper/311:13:213
3532822991917,1cyclictest0-21swapper/310:44:003
3532822991917,1cyclictest0-21swapper/309:46:423
3532822991917,1cyclictest0-21swapper/307:16:213
3532822991914,3cyclictest978-21snmpd07:42:563
3532822991911,6cyclictest3623380-21cat11:25:143
3532822991911,6cyclictest3577896-21cat09:15:133
353282299190,1cyclictest0-21swapper/311:00:443
353282299190,1cyclictest0-21swapper/310:56:523
353282299190,1cyclictest0-21swapper/310:30:093
353282299190,1cyclictest0-21swapper/308:59:443
353282299190,1cyclictest0-21swapper/308:59:443
353282299190,1cyclictest0-21swapper/308:12:593
353282299190,1cyclictest0-21swapper/307:37:043
353282299190,1cyclictest0-21swapper/307:31:233
353282299190,18cyclictest0-21swapper/312:13:503
353282299190,18cyclictest0-21swapper/308:52:483
353282299190,18cyclictest0-21swapper/308:15:503
353281799194,3cyclictest141rcu_preempt09:35:102
3532817991917,1cyclictest0-21swapper/212:31:312
3532817991917,1cyclictest0-21swapper/210:41:062
3532817991914,4cyclictest34-21ksoftirqd/210:45:232
3532817991911,6cyclictest3602927-21ntp_states10:25:182
353281799190,1cyclictest0-21swapper/210:10:032
353280999195,4cyclictest26-21ksoftirqd/107:52:061
353280999195,3cyclictest141rcu_preempt11:21:381
353280999194,4cyclictest26-21ksoftirqd/108:24:021
353280999194,3cyclictest26-21ksoftirqd/108:09:101
353280999192,2cyclictest141rcu_preempt07:35:001
3532809991917,1cyclictest0-21swapper/108:50:211
3532809991916,2cyclictest3056-21PK-Backend09:09:341
3532809991915,2cyclictest36440942sleep112:22:221
353280999190,1cyclictest3056-21PK-Backend10:35:111
353280999190,1cyclictest3056-21PK-Backend10:13:541
353280999190,1cyclictest0-21swapper/112:07:581
353280999190,1cyclictest0-21swapper/111:42:571
353280999190,1cyclictest0-21swapper/108:59:261
353280999190,1cyclictest0-21swapper/108:59:261
353280999190,18cyclictest3056-21PK-Backend12:32:531
353280999190,18cyclictest3056-21PK-Backend11:27:101
353280999190,18cyclictest3056-21PK-Backend10:59:051
353280999190,18cyclictest0-21swapper/112:36:591
353280999190,10cyclictest3056-21PK-Backend11:15:181
3532805991917,1cyclictest0-21swapper/009:18:110
3532805991917,1cyclictest0-21swapper/009:03:170
3532805991916,2cyclictest0-21swapper/011:51:350
3532805991916,2cyclictest0-21swapper/010:19:350
3532805991914,3cyclictest3594741-21munin-plugin-st10:05:000
3532805991914,3cyclictest3563003-21dump-pmu-power08:35:010
3532805991913,4cyclictest2156-21gnome-shell09:08:090
353280599190,1cyclictest0-21swapper/011:00:460
353280599190,18cyclictest0-21swapper/010:32:270
35678842180,4sleep3411rcuc/308:45:183
353282299189,6cyclictest3646329-21apt12:30:113
3532822991816,1cyclictest3056-21PK-Backend09:54:403
3532822991816,1cyclictest0-21swapper/311:32:553
3532822991816,1cyclictest0-21swapper/310:47:043
3532822991816,1cyclictest0-21swapper/310:06:293
3532822991815,2cyclictest3056-21PK-Backend11:41:403
3532822991813,3cyclictest3577381-21idleruntime-cro09:15:003
3532822991812,5cyclictest3611262-21smartctl10:50:133
3532822991811,5cyclictest3642788-21sed12:20:003
353282299181,15cyclictest3622127-21python311:20:193
3532822991811,5cyclictest3619511-21cat11:15:103
353282299181,15cyclictest3598464-21apt10:15:113
3532822991811,5cyclictest3536951-21awk07:20:103
3532822991811,5cyclictest3534597-21cut07:15:013
353282299180,16cyclictest3587408-21users09:40:223
353281799187,6cyclictest141rcu_preempt11:00:152
353281799186,3cyclictest34-21ksoftirqd/209:55:092
353281799186,3cyclictest34-21ksoftirqd/209:55:092
3532817991814,3cyclictest34-21ksoftirqd/210:55:002
3532809991816,1cyclictest3056-21PK-Backend08:45:071
3532809991815,2cyclictest3056-21PK-Backend09:46:441
3532809991814,2cyclictest36157602sleep111:04:421
3532809991811,5cyclictest3581856-21ntpq09:25:171
3532809991811,5cyclictest3533149-21tr07:10:111
3532805991816,1cyclictest0-21swapper/009:44:550
3532805991810,6cyclictest3634179-21apt11:55:110
3532805991810,6cyclictest3634179-21apt11:55:100
139099180,15rtkit-daemon1317-21systemd07:20:251
3532822991715,1cyclictest0-21swapper/307:26:573
3532822991713,3cyclictest978-21snmpd09:02:073
3532822991712,3cyclictest3597120-21fw_packets10:10:143
3532817991712,4cyclictest34-21ksoftirqd/207:45:022
353280999170,5cyclictest3056-21PK-Backend08:12:091
353280999170,2cyclictest3056-21PK-Backend07:36:521
3532805991715,1cyclictest0-21swapper/008:22:280
3532805991715,1cyclictest0-21swapper/007:16:310
3532805991712,3cyclictest3600777-21missed_timers10:20:160
3532805991513,1cyclictest0-21swapper/011:10:320
353280599151,11cyclictest3563496-21apt-get08:35:030
3532805991311,1cyclictest0-21swapper/007:24:280
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional