You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-08 - 18:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Tue Sep 08, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
191053821430,9sleep234-21ksoftirqd/209:40:012
187919421280,3sleep00-21swapper/008:10:130
139099870,74rtkit-daemon0-21swapper/307:07:013
185646727638,10sleep00-21swapper/007:06:130
139099740,62rtkit-daemon0-21swapper/207:09:242
19129432660,4sleep00-21swapper/009:45:170
185641226039,13sleep10-21swapper/107:05:311
18628472540,2sleep33056-21PK-Backend07:25:013
19534872510,4sleep13056-21PK-Backend11:40:181
19143202480,2sleep30-21swapper/309:50:133
19369452420,4sleep33056-21PK-Backend10:55:013
19535972390,1sleep20-21swapper/211:40:182
185690799370,21cyclictest1882082-21seq08:19:110
185690799360,20cyclictest1896060-21seq08:56:290
185690799360,16cyclictest0-21swapper/011:10:010
1856907993410,16cyclictest1095-21runrttasks09:09:230
185690799340,21cyclictest1968425-21chrt12:24:440
185690799340,19cyclictest1925917-21chrt10:23:060
185690799340,19cyclictest1890832-21chrt08:44:340
185690799340,17cyclictest1962376-21memory12:05:180
185690799340,17cyclictest1926320-21chrt10:25:060
185690799330,24cyclictest1969902-21taskset12:26:250
185690799330,19cyclictest1946856-21chrt11:21:290
185690799330,19cyclictest1915592-21chrt09:54:030
185690799330,16cyclictest1938276-21chrt10:57:430
185690799330,16cyclictest1897950-21chrt09:03:300
185690799329,15cyclictest1095-21runrttasks11:26:290
185690799324,18cyclictest0-21swapper/009:34:590
185690799324,13cyclictest1095-21runrttasks07:40:540
185690799321,16cyclictest1887084-21taskset08:31:370
185690799320,22cyclictest1928951-21python310:30:230
185690799320,18cyclictest1920563-21chrt10:06:060
185690799320,17cyclictest1903107-21chrt09:17:580
185690799320,17cyclictest1892643-21taskset08:47:140
139099320,24rtkit-daemon1871496-21cron07:50:002
185690799311,15cyclictest0-21swapper/008:25:330
185690799310,18cyclictest1866069-21chrt07:32:520
185690799310,17cyclictest1922473-21seq10:13:240
185690799310,17cyclictest1882162-21perf08:24:550
185690799310,17cyclictest1875072-21taskset07:58:100
185690799310,17cyclictest1864197-21seq07:26:060
185690799310,16cyclictest0-21swapper/011:04:220
185690799300,18cyclictest1936201-21chrt10:51:370
185690799300,18cyclictest1934626-21chrt10:48:410
185690799300,17cyclictest1888914-21chrt08:37:120
185690799300,16cyclictest1932945-21chrt10:44:190
185690799300,16cyclictest1901402-21chrt09:13:210
185690799300,16cyclictest1871380-21chrt07:48:360
185690799290,18cyclictest1862375-21chrt07:23:170
185690799290,16cyclictest1095-21runrttasks08:01:150
185690799290,15cyclictest1963257-21perf12:14:550
185690799290,15cyclictest0-21swapper/009:20:300
185690799290,15cyclictest0-21swapper/007:50:030
185690799280,15cyclictest0-21swapper/011:35:160
185690799280,14cyclictest1859326-21apt07:15:130
1856918992711,15cyclictest1886854-21ls08:30:203
185690799270,15cyclictest0-21swapper/009:26:160
18849052260,3sleep30-21swapper/308:25:203
185690799260,14cyclictest1095-21runrttasks09:40:150
185690799260,14cyclictest0-21swapper/011:20:000
185690799258,15cyclictest1919346-21grep10:05:010
1856907992520,4cyclictest0-21swapper/008:50:070
1856907992520,4cyclictest0-21swapper/008:50:070
18588572240,3chrt0-21swapper/207:15:012
185691899242,2cyclictest141rcu_preempt08:45:003
1856915992415,7cyclictest1913940-21cron09:50:002
1856907992418,5cyclictest0-21swapper/011:46:360
1856915992313,7cyclictest436-20systemd-journal11:50:002
1856915992313,7cyclictest1882156-21cron08:20:002
185690799239,12cyclictest1910260-21dpkg09:40:010
185690799230,21cyclictest1943685-21turbostat11:15:000
1856918992211,8cyclictest42-21ksoftirqd/308:35:173
185691599222,18cyclictest1952596-21cron11:40:012
185691299224,16cyclictest1941941-21grep11:10:001
1856912992220,1cyclictest0-21swapper/111:39:451
1856912992220,1cyclictest0-21swapper/109:03:241
1856912992219,2cyclictest3056-21PK-Backend08:11:251
185691299220,21cyclictest0-21swapper/111:04:031
185691299220,20cyclictest3056-21PK-Backend10:32:121
185691299220,20cyclictest3056-21PK-Backend10:05:021
185691299220,11cyclictest141rcu_preempt12:17:241
185691299220,10cyclictest141rcu_preempt09:30:461
185691299220,10cyclictest141rcu_preempt08:50:561
185691299220,10cyclictest141rcu_preempt08:50:561
185691299220,10cyclictest141rcu_preempt07:17:051
185690799227,13cyclictest0-21swapper/007:14:350
1856907992220,1cyclictest0-21swapper/008:06:340
185690799221,19cyclictest1922886-21apt10:15:130
1856918992118,2cyclictest3056-21PK-Backend11:20:473
1856918992118,2cyclictest3056-21PK-Backend09:03:523
1856918992114,5cyclictest1920280-21python310:05:213
1856918992111,9cyclictest42-21ksoftirqd/309:40:003
185691899210,19cyclictest3056-21PK-Backend12:10:493
185691899210,19cyclictest3056-21PK-Backend11:03:233
185691599210,6cyclictest1887406-21cron08:35:002
185691599210,20cyclictest0-21swapper/207:37:352
185691299219,3cyclictest141rcu_preempt12:09:491
185691299212,8cyclictest0-21swapper/108:15:201
1856912992119,1cyclictest3056-21PK-Backend11:34:181
1856912992119,1cyclictest3056-21PK-Backend08:09:021
1856912992119,1cyclictest0-21swapper/109:16:421
1856912992119,1cyclictest0-21swapper/108:57:361
1856912992119,1cyclictest0-21swapper/108:32:411
1856912992119,1cyclictest0-21swapper/108:26:191
1856912992118,2cyclictest3056-21PK-Backend07:13:051
185691299210,2cyclictest3056-21PK-Backend10:20:101
185691299210,20cyclictest0-21swapper/109:52:301
185691299210,20cyclictest0-21swapper/108:45:261
185691299210,20cyclictest0-21swapper/108:02:541
185691299210,19cyclictest3056-21PK-Backend10:37:401
185691299210,19cyclictest3056-21PK-Backend09:48:251
185691299210,10cyclictest141rcu_preempt11:14:161
185690799217,13cyclictest1959782-21cron12:00:000
185690799216,13cyclictest0-21swapper/011:50:520
185690799215,15cyclictest13-21ksoftirqd/007:37:040
185690799211,19cyclictest1960769-21awk12:00:170
185690799210,19cyclictest1965569-21cstates12:15:120
139099210,15rtkit-daemon0-21swapper/209:02:322
185691899209,8cyclictest42-21ksoftirqd/307:45:173
1856918992018,1cyclictest3056-21PK-Backend09:17:443
1856918992018,1cyclictest0-21swapper/312:17:043
1856918992018,1cyclictest0-21swapper/308:51:283
1856918992018,1cyclictest0-21swapper/308:51:283
1856918992017,2cyclictest3056-21PK-Backend09:13:253
1856918992013,6cyclictest1968467-21cron12:25:003
185691899201,1cyclictest141rcu_preempt10:55:003
185691899201,17cyclictest1958381-21/usr/sbin/munin11:55:123
185691899201,16cyclictest1913341-21python309:45:223
185691899200,1cyclictest3056-21PK-Backend10:26:053
185691899200,1cyclictest3056-21PK-Backend09:09:393
185691899200,1cyclictest3056-21PK-Backend07:55:183
185691899200,1cyclictest0-21swapper/309:34:413
185691899200,19cyclictest3056-21PK-Backend08:03:083
185691899200,19cyclictest0-21swapper/312:31:093
185691899200,19cyclictest0-21swapper/312:05:553
185691899200,19cyclictest0-21swapper/308:22:013
185691899200,19cyclictest0-21swapper/308:19:383
185691899200,19cyclictest0-21swapper/307:35:303
185691899200,18cyclictest3056-21PK-Backend11:10:263
185691599203,14cyclictest1948867-21cron11:30:002
1856915992018,1cyclictest0-21swapper/210:02:272
1856915992018,1cyclictest0-21swapper/208:20:342
1856915992018,1cyclictest0-21swapper/208:10:132
1856915992017,2cyclictest0-21swapper/209:40:172
1856915992015,3cyclictest1950903-21munin-run11:35:012
1856915992014,2cyclictest34-21ksoftirqd/208:05:212
1856915992012,6cyclictest34-21ksoftirqd/209:25:012
1856915992012,6cyclictest34-21ksoftirqd/208:25:142
185691599200,6cyclictest1874130-21df07:55:142
185691599200,20cyclictest0-21swapper/209:16:582
185691599200,1cyclictest0-21swapper/212:04:212
185691599200,1cyclictest0-21swapper/207:41:132
185691599200,19cyclictest0-21swapper/212:38:282
185691599200,19cyclictest0-21swapper/212:25:302
185691599200,19cyclictest0-21swapper/212:23:212
185691599200,19cyclictest0-21swapper/210:52:222
185691599200,19cyclictest0-21swapper/210:47:292
185691599200,19cyclictest0-21swapper/210:37:422
185691599200,19cyclictest0-21swapper/209:33:202
185691599200,19cyclictest0-21swapper/209:08:402
185691599200,19cyclictest0-21swapper/208:55:152
185691599200,19cyclictest0-21swapper/207:33:082
185691299209,2cyclictest141rcu_preempt10:43:391
1856912992018,1cyclictest3056-21PK-Backend12:04:331
1856912992018,1cyclictest3056-21PK-Backend11:50:051
1856912992018,1cyclictest3056-21PK-Backend09:57:241
1856912992018,1cyclictest0-21swapper/109:12:231
1856912992018,1cyclictest0-21swapper/107:45:431
1856912992018,1cyclictest0-21swapper/107:30:501
1856912992018,1cyclictest0-21swapper/107:27:251
1856912992014,4cyclictest1936474-21sh10:55:001
1856912992012,6cyclictest436-20systemd-journal09:24:551
1856912992012,2cyclictest26-21ksoftirqd/112:36:091
1856912992012,2cyclictest26-21ksoftirqd/112:15:001
1856912992012,2cyclictest26-21ksoftirqd/111:20:001
1856912992012,2cyclictest26-21ksoftirqd/109:28:091
185691299200,1cyclictest0-21swapper/111:23:221
185691299200,1cyclictest0-21swapper/107:51:111
185691299200,19cyclictest3056-21PK-Backend10:17:461
185691299200,19cyclictest0-21swapper/110:28:071
185691299200,19cyclictest0-21swapper/108:36:401
185691299200,18cyclictest3056-21PK-Backend10:00:081
185690799207,11cyclictest1972648-21tr12:35:130
185690799200,19cyclictest1971387-21tr12:30:170
185690799200,19cyclictest0-21swapper/011:42:310
185690799200,19cyclictest0-21swapper/010:37:550
185690799200,19cyclictest0-21swapper/009:59:020
139099200,14rtkit-daemon0-21swapper/210:09:252
185691899192,15cyclictest1957428-21processes11:50:193
1856918991917,1cyclictest3056-21PK-Backend10:12:473
1856918991917,1cyclictest0-21swapper/311:16:413
1856918991917,1cyclictest0-21swapper/310:31:313
1856918991917,1cyclictest0-21swapper/310:02:123
1856918991917,1cyclictest0-21swapper/307:20:303
1856918991917,1cyclictest0-21swapper/307:15:433
1856918991916,2cyclictest0-21swapper/311:35:123
185691899191,2cyclictest1970447-21sort12:30:013
1856918991912,6cyclictest1931744-21expr10:40:133
1856918991912,5cyclictest1924806-21grep10:20:113
1856918991911,6cyclictest42-21ksoftirqd/310:35:143
185691899191,14cyclictest1952879-21chrt11:40:123
1856918991910,7cyclictest1895463-21memory08:55:153
185691899190,1cyclictest3056-21PK-Backend11:08:153
185691899190,1cyclictest3056-21PK-Backend10:16:183
185691899190,1cyclictest3056-21PK-Backend09:22:043
185691899190,1cyclictest0-21swapper/312:39:013
185691899190,1cyclictest0-21swapper/311:30:203
1856915991917,1cyclictest0-21swapper/211:59:282
1856915991917,1cyclictest0-21swapper/207:14:092
1856915991915,3cyclictest34-21ksoftirqd/208:45:002
1856915991915,2cyclictest1947140-21sh11:25:002
1856915991912,5cyclictest1916107-21grep09:55:132
1856915991911,6cyclictest1962559-21awk12:05:202
185691599190,1cyclictest0-21swapper/212:31:592
185691599190,1cyclictest0-21swapper/210:32:492
185691599190,1cyclictest0-21swapper/209:14:002
185691599190,1cyclictest0-21swapper/208:39:402
185691599190,1cyclictest0-21swapper/208:03:032
185691599190,1cyclictest0-21swapper/207:54:582
185691299197,3cyclictest141rcu_preempt12:25:491
1856912991917,1cyclictest3056-21PK-Backend11:58:171
1856912991917,1cyclictest3056-21PK-Backend07:36:511
1856912991917,1cyclictest0-21swapper/110:10:431
1856912991917,1cyclictest0-21swapper/109:06:481
1856912991917,1cyclictest0-21swapper/107:43:541
185691299190,1cyclictest3056-21PK-Backend09:36:361
185691299190,1cyclictest3056-21PK-Backend08:23:081
185691299190,1cyclictest0-21swapper/107:55:511
185690799190,1cyclictest0-21swapper/011:31:010
1856918991816,1cyclictest3056-21PK-Backend07:51:393
1856918991816,1cyclictest0-21swapper/312:01:013
1856918991811,6cyclictest1877899-21grep08:05:183
1856918991811,6cyclictest1857103-21apt07:10:113
185691599182,3cyclictest141rcu_preempt08:45:002
1856915991816,1cyclictest341ksoftirqd/210:15:002
1856915991816,1cyclictest3056-21PK-Backend10:59:252
1856915991816,1cyclictest0-21swapper/212:18:272
1856915991816,1cyclictest0-21swapper/212:14:552
1856915991816,1cyclictest0-21swapper/211:15:072
1856915991816,1cyclictest0-21swapper/211:01:352
1856915991814,3cyclictest978-21snmpd07:29:092
185691299182,8cyclictest0-21swapper/108:40:241
1856912991816,1cyclictest0-21swapper/111:25:321
1856912991816,1cyclictest0-21swapper/110:49:231
1856912991810,2cyclictest26-21ksoftirqd/111:45:121
185691299180,1cyclictest3056-21PK-Backend12:23:041
1856918991715,1cyclictest3056-21PK-Backend07:32:533
1856918991713,3cyclictest1933758-21irqstats10:45:163
1856918991712,3cyclictest1892871-21dump-pmu-power08:50:003
1856918991710,5cyclictest436-20systemd-journal09:27:183
185691899170,3cyclictest1879443-21irqstats08:10:173
185691899170,16cyclictest1911707-21sed09:40:223
185691899170,15cyclictest1916821-21python309:55:223
185691599172,11cyclictest1956600-21apt11:50:112
1856915991715,1cyclictest0-21swapper/209:26:382
1856915991713,2cyclictest34-21ksoftirqd/207:25:002
1856915991712,2cyclictest34-21ksoftirqd/209:50:202
1856915991710,2cyclictest34-21ksoftirqd/211:15:012
1856912991715,1cyclictest0-21swapper/110:59:091
185691299171,4cyclictest26-21ksoftirqd/107:25:011
1856918991613,2cyclictest3056-21PK-Backend11:26:143
1856918991610,5cyclictest1955404-21sed11:45:183
185691899160,4cyclictest1868561-21expr07:40:143
1856915991612,2cyclictest34-21ksoftirqd/210:15:242
1856915991611,3cyclictest1095-21runrttasks10:25:132
1856915991611,2cyclictest34-21ksoftirqd/210:40:112
1856915991610,5cyclictest1925364-21ntpq10:20:172
185691599160,1cyclictest141rcu_preempt11:05:172
1856912991614,1cyclictest3056-21PK-Backend12:30:551
1856912991614,1cyclictest3056-21PK-Backend09:41:571
185691599157,2cyclictest141rcu_preempt08:50:202
185691599157,2cyclictest141rcu_preempt08:50:202
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional