You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-03 - 22:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Sep 03, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29962239163,24sleep20-21swapper/207:09:022
28672233162,58sleep30-21swapper/307:07:223
28242223192,21sleep00-21swapper/007:06:470
28202222144,25sleep10-21swapper/107:06:441
946921670,4sleep0315999cyclictest12:15:170
3172996158,2cyclictest0-21swapper/208:00:342
178862550,3sleep217889-21fschecks_count12:35:162
64022540,4sleep3317699cyclictest12:10:023
3172995451,2cyclictest0-21swapper/209:25:492
110822510,2sleep011-21rcuc/009:50:290
302352490,2sleep20-21swapper/209:20:252
310852470,1sleep30-21swapper/311:50:233
79562440,1sleep20-21swapper/211:00:022
103502440,1sleep00-21swapper/008:35:240
317699391,30cyclictest0-21swapper/311:25:303
3159993910,13cyclictest8225-21munin-run07:20:010
136672390,3sleep313670-21proc_pri11:10:223
315999368,13cyclictest26417-21munin-run09:15:000
21432360,4sleep2317299cyclictest09:30:282
315999359,12cyclictest21052-21idleruntime-cro11:30:000
317699303,22cyclictest1584-40dnf11:56:263
317699301,13cyclictest0-21swapper/311:16:583
317699300,23cyclictest0-21swapper/309:45:423
317699295,3cyclictest0-21swapper/312:36:573
317699291,13cyclictest0-21swapper/311:20:103
317699291,13cyclictest0-21swapper/310:25:213
317699291,12cyclictest28335-21ntp_states10:30:253
317699291,12cyclictest14249-21pmu-power12:25:223
317699291,12cyclictest0-21swapper/307:49:493
3176992910,4cyclictest2644-21/usr/sbin/munin10:45:233
317299290,10cyclictest87350irq/126-enp1s0-08:50:262
316599295,7cyclictest0-21swapper/110:09:521
317699281,13cyclictest0-21swapper/307:58:013
317699281,12cyclictest0-21swapper/308:17:573
317299287,12cyclictest27316-21hddtemp_smartct09:15:162
316599280,13cyclictest0-21swapper/112:38:361
317699279,3cyclictest3421-21diskstats09:35:143
317699279,3cyclictest3421-21diskstats09:35:143
317699271,12cyclictest0-21swapper/309:09:303
317699271,12cyclictest0-21swapper/307:16:143
317699271,12cyclictest0-21swapper/307:11:303
317699271,12cyclictest0-21swapper/307:11:303
317699271,11cyclictest0-21swapper/311:04:173
317699271,11cyclictest0-21swapper/307:33:383
317299274,11cyclictest11493-21chrt07:25:182
317299272,10cyclictest2698-21/usr/sbin/munin12:00:152
317699269,3cyclictest0-21swapper/310:06:293
317699269,3cyclictest0-21swapper/307:35:453
317699268,2cyclictest4051-21/usr/sbin/munin10:50:133
317699267,10cyclictest13226-21munin-run08:45:013
317699266,10cyclictest31370-21df09:25:143
317699261,13cyclictest0-21swapper/312:10:503
317699261,13cyclictest0-21swapper/310:41:333
317699261,13cyclictest0-21swapper/310:41:333
317699261,12cyclictest11644-21munin-run09:55:023
317699261,12cyclictest0-21swapper/311:38:383
317699261,11cyclictest0-21swapper/308:58:093
317699261,10cyclictest0-21swapper/307:27:223
3176992610,3cyclictest11023-21munin-run08:40:013
317299266,13cyclictest7601-21ntp_states12:10:252
317299265,11cyclictest5800-21cat10:55:022
3172992612,5cyclictest18470-21chrt10:10:122
3172992610,3cyclictest0-21swapper/207:17:412
293032260,2sleep00-21swapper/009:20:140
14032260,4sleep1221ktimersoftd/111:55:221
317699259,3cyclictest28574-21diskstats08:05:133
317699259,3cyclictest0-21swapper/312:32:103
317699259,3cyclictest0-21swapper/307:22:023
317699258,3cyclictest0-21swapper/312:01:383
317699258,3cyclictest0-21swapper/308:04:573
317699257,3cyclictest21896-21cstates07:50:133
317699252,9cyclictest7532-21hddtemp_smartct08:30:153
317699251,11cyclictest0-21swapper/309:23:183
317699251,10cyclictest24216-21diskmemload09:56:463
317699251,10cyclictest0-21swapper/310:11:413
317699251,10cyclictest0-21swapper/309:42:503
317699251,10cyclictest0-21swapper/308:21:533
3176992510,3cyclictest0-21swapper/308:49:053
317299258,3cyclictest0-21swapper/211:22:202
317299258,2cyclictest22248-21/usr/sbin/munin09:05:282
317299254,6cyclictest21875-1kworker/2:1H09:40:012
317299254,6cyclictest21875-1kworker/2:1H09:40:002
317299251,12cyclictest0-21swapper/211:53:342
316599252,4cyclictest13041-40dnf09:55:221
316599251,12cyclictest0-21swapper/110:35:001
316599250,17cyclictest0-21swapper/109:45:221
317699249,3cyclictest16009-21runrttasks10:04:413
317699249,3cyclictest0-21swapper/311:47:463
317699249,3cyclictest0-21swapper/308:14:213
317699248,3cyclictest21730-21ntp_states10:15:213
317699241,6cyclictest0-21swapper/309:10:533
317699241,12cyclictest0-21swapper/311:41:263
317299248,5cyclictest32714-21ntp_states10:40:212
317299248,5cyclictest32714-21ntp_states10:40:202
317299244,9cyclictest30562-21pmu-power10:35:222
317299242,7cyclictest23820-21/usr/sbin/munin07:55:222
317299242,13cyclictest7239-21sh09:45:002
3165992414,9cyclictest0-21swapper/107:19:531
316599241,11cyclictest0-21swapper/110:50:161
315999246,15cyclictest7259-21perf09:45:010
315999240,14cyclictest16793-21pmu-power08:50:230
317699239,3cyclictest0-21swapper/311:31:303
317699239,3cyclictest0-21swapper/308:26:173
317699239,3cyclictest0-21swapper/308:26:173
317699238,3cyclictest0-21swapper/309:02:453
317699238,3cyclictest0-21swapper/307:43:253
317699237,3cyclictest0-21swapper/312:22:143
317699237,3cyclictest0-21swapper/309:33:583
317299238,7cyclictest8600-21grep11:00:172
317299233,9cyclictest11531-21sendmail_mailqu11:05:232
317299233,6cyclictest87550irq/128-enp1s0-10:45:152
317299231,7cyclictest0-21swapper/212:09:262
317299230,9cyclictest0-21swapper/208:58:342
316599232,6cyclictest17560-40dnf08:54:281
3165992317,3cyclictest8442-21diskstats11:00:161
316599231,16cyclictest0-21swapper/108:24:401
316599231,11cyclictest0-21swapper/110:12:391
3165992310,11cyclictest9216-21kworker/u8:107:55:121
3159992316,4cyclictest20899-21cstates10:15:130
315999231,4cyclictest18762-21kworker/u8:011:52:520
315999230,11cyclictest19217-21perf07:45:020
311702230,3sleep011-21rcuc/009:25:120
317699227,3cyclictest0-21swapper/310:22:413
317699227,3cyclictest0-21swapper/308:51:213
317699221,5cyclictest0-21swapper/312:19:063
317299228,6cyclictest28225-21/usr/sbin/munin11:45:132
317299224,7cyclictest16556-21sendmail_mailqu12:30:272
317299221,13cyclictest28225-21/usr/sbin/munin08:05:152
317299221,11cyclictest0-21swapper/212:18:572
317299220,9cyclictest12498-21/usr/sbin/munin11:10:242
316599227,3cyclictest17468-21latency_hist12:35:011
316599227,12cyclictest6886-21systemd-journal09:40:311
3165992214,3cyclictest0-21swapper/107:27:281
316599221,3cyclictest29450-21grep11:45:241
316599221,18cyclictest17056-21cat11:20:111
316599221,16cyclictest0-21swapper/110:02:131
316599221,16cyclictest0-21swapper/109:18:311
316599221,15cyclictest0-21swapper/108:56:291
316599221,15cyclictest0-21swapper/108:16:221
316599221,11cyclictest13448-21/usr/sbin/munin08:45:261
315999228,8cyclictest0-21swapper/010:36:440
315999221,13cyclictest0-21swapper/009:00:220
317699217,3cyclictest0-21swapper/311:06:423
317699211,6cyclictest0-21swapper/309:17:053
317299218,3cyclictest0-21swapper/208:21:412
317299215,8cyclictest87350irq/126-enp1s0-12:25:182
317299215,6cyclictest22863-21/usr/sbin/munin10:20:252
317299213,6cyclictest20666-21/usr/sbin/munin10:15:242
317299212,6cyclictest8444-21/usr/sbin/munin07:20:132
317299211,6cyclictest808-21snmpd09:48:052
3172992112,6cyclictest21419-21cat07:50:012
3172992111,6cyclictest13041-40dnf09:55:232
317299211,12cyclictest0-21swapper/210:25:282
317299211,11cyclictest0-21swapper/211:18:312
316599218,3cyclictest0-21swapper/109:50:051
316599211,2cyclictest87550irq/128-enp1s0-12:10:241
316599211,15cyclictest0-21swapper/110:44:571
316599211,15cyclictest0-21swapper/110:44:571
316599211,15cyclictest0-21swapper/109:31:261
316599211,14cyclictest0-21swapper/108:40:051
316599211,14cyclictest0-21swapper/108:36:091
316599211,11cyclictest579-21systemd-journal08:10:011
316599211,11cyclictest30412-21grep09:20:251
316599211,11cyclictest19193-21grep11:25:121
316599211,10cyclictest0-21swapper/112:06:091
316599211,10cyclictest0-21swapper/111:10:081
316599211,10cyclictest0-21swapper/111:10:001
316599210,15cyclictest0-21swapper/108:25:261
316599210,15cyclictest0-21swapper/108:25:261
315999211,7cyclictest0-21swapper/009:39:570
315999211,7cyclictest0-21swapper/009:39:570
315999211,11cyclictest25872-21irqstats10:25:170
3159992110,3cyclictest16100-21ntp_states07:35:220
25812211,2sleep00-21swapper/010:45:230
317699209,3cyclictest5893-21/usr/sbin/munin10:55:113
317299207,2cyclictest0-21swapper/207:32:412
317299206,3cyclictest0-21swapper/208:16:212
317299204,3cyclictest0-21swapper/207:41:512
317299202,6cyclictest32276-21acpi11:55:092
3172992012,5cyclictest31636-21/usr/sbin/munin08:10:212
3172992011,5cyclictest21109-21ntp_states09:00:222
317299201,11cyclictest0-21swapper/209:52:532
317299200,8cyclictest0-21swapper/207:55:012
316599209,3cyclictest22339-21basename07:50:171
316599201,8cyclictest0-21swapper/111:19:161
316599201,6cyclictest6910-21netstat10:55:191
316599201,3cyclictest0-21swapper/107:35:201
3165992013,2cyclictest0-21swapper/109:00:371
316599201,2cyclictest0-21swapper/108:01:421
316599201,18cyclictest0-21swapper/112:16:001
316599201,18cyclictest0-21swapper/111:31:371
316599201,14cyclictest0-21swapper/108:31:391
316599201,14cyclictest0-21swapper/107:47:561
316599201,14cyclictest0-21swapper/107:44:001
316599201,11cyclictest30962-21fschecks_time08:10:141
316599201,11cyclictest26389-21cat10:25:241
316599201,11cyclictest24538-21ntpq11:35:201
316599201,11cyclictest13581-21fschecks_count12:25:131
316599201,11cyclictest0-21swapper/112:02:471
316599201,10cyclictest24305-21cat09:10:021
316599201,10cyclictest0-21swapper/110:35:311
316599200,9cyclictest0-21swapper/109:35:111
316599200,9cyclictest0-21swapper/109:35:111
316599200,14cyclictest0-21swapper/110:15:081
315999209,3cyclictest16888-21/usr/sbin/munin11:20:110
315999202,10cyclictest16272-21/usr/sbin/munin10:05:140
315999201,4cyclictest0-21swapper/009:16:120
3159992014,3cyclictest18654-21pmu-power12:35:260
315999201,16cyclictest28223-21diskstats11:45:120
315999201,11cyclictest4328-21/usr/sbin/munin12:05:230
315999201,10cyclictest23138-21sed07:50:260
315999201,10cyclictest21890-21gltestperf11:30:180
315999201,10cyclictest15451-21if_enp2s011:15:150
315999201,10cyclictest13443-21tail07:30:160
315999201,10cyclictest0-21swapper/011:38:220
317699197,2cyclictest0-21swapper/310:35:173
317299197,4cyclictest12680-21sshd12:22:252
317299197,2cyclictest87350irq/126-enp1s0-08:35:262
317299195,2cyclictest20157-21ntp_states11:25:222
317299194,5cyclictest87350irq/126-enp1s0-08:25:162
317299194,5cyclictest87350irq/126-enp1s0-08:25:152
317299193,3cyclictest3378-21cpu07:10:132
317299193,3cyclictest3378-21cpu07:10:132
317299191,7cyclictest0-21swapper/211:34:162
317299191,6cyclictest0-21swapper/210:05:082
3172991911,5cyclictest14415-21diskstats10:00:132
317299191,10cyclictest0-21swapper/208:40:182
3165991916,2cyclictest0-21swapper/111:42:381
316599191,3cyclictest0-21swapper/107:20:451
3165991913,2cyclictest0-21swapper/110:22:181
316599191,17cyclictest0-21swapper/111:54:261
316599191,17cyclictest0-21swapper/110:47:251
316599191,10cyclictest0-21swapper/109:29:131
315999199,2cyclictest7-21ksoftirqd/008:55:170
315999198,1cyclictest0-21swapper/008:22:440
315999196,3cyclictest13172-21switchtime09:55:250
315999195,12cyclictest11-21rcuc/012:20:220
315999191,7cyclictest0-21swapper/009:34:170
315999191,6cyclictest0-21swapper/010:44:060
315999191,6cyclictest0-21swapper/010:44:060
315999191,4cyclictest0-21swapper/012:11:040
317299189,6cyclictest8398-21/usr/sbin/munin08:30:252
317299188,7cyclictest24792-21diskstats09:10:142
317299181,9cyclictest28357-21sshd10:30:252
317299181,14cyclictest15299-21chrt08:48:392
317299181,10cyclictest0-21swapper/211:43:342
317299180,10cyclictest87350irq/126-enp1s0-07:35:152
3165991815,2cyclictest0-21swapper/109:11:471
316599181,10cyclictest0-21swapper/112:22:211
316599180,14cyclictest0-21swapper/107:14:591
316599180,14cyclictest0-21swapper/107:14:581
315999188,3cyclictest13448-21/usr/sbin/munin08:45:130
315999188,2cyclictest5926-21smartctl08:25:240
315999188,2cyclictest5926-21smartctl08:25:240
315999186,3cyclictest29395-21ps08:05:220
315999182,9cyclictest7-21ksoftirqd/007:45:170
315999182,3cyclictest0-21swapper/011:40:270
315999181,9cyclictest23820-21/usr/sbin/munin07:55:120
315999181,7cyclictest13485-21strings11:10:180
315999181,7cyclictest0-21swapper/012:28:160
3159991813,3cyclictest9446-21munin-run09:50:020
3159991813,3cyclictest30842-21grep08:10:140
315999181,15cyclictest0-21swapper/008:01:180
3159991811,4cyclictest15220-21proc_pri10:00:220
315999181,10cyclictest22271-21cpu09:05:130
315999180,7cyclictest9434-21needreboot07:20:190
130272181,2sleep00-21swapper/008:42:390
317299178,6cyclictest24126-21sed11:35:162
3165991713,3cyclictest0-21swapper/107:30:381
315999174,3cyclictest87550irq/128-enp1s0-10:10:240
315999172,12cyclictest3694-21/usr/sbin/munin10:50:130
315999171,8cyclictest0-21swapper/011:08:070
315999171,6cyclictest0-21swapper/010:22:140
315999171,4cyclictest15320-21/usr/sbin/munin12:30:270
3159991712,3cyclictest3456-21cpuspeed_turbos07:10:120
3159991712,3cyclictest3456-21cpuspeed_turbos07:10:120
3159991712,3cyclictest3371-21sendmail_mailtr12:00:240
315999171,15cyclictest0-21swapper/011:55:330
315999171,10cyclictest0-21swapper/008:15:150
3159991710,4cyclictest0-21swapper/007:25:150
315999170,7cyclictest87350irq/126-enp1s0-11:02:230
3159991612,2cyclictest6857-21/usr/sbin/munin08:30:240
3159991610,3cyclictest827-21runrttasks10:30:480
3159991510,3cyclictest5893-21/usr/sbin/munin10:55:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional