You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-06 - 14:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Aug 06, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
310742240162,25sleep00-21swapper/007:06:470
310742240162,25sleep00-21swapper/007:06:470
313212237163,23sleep10-21swapper/107:09:581
313212237163,23sleep10-21swapper/107:09:581
312312230163,55sleep20-21swapper/207:08:512
312312230163,55sleep20-21swapper/207:08:502
293602202165,25sleep30-21swapper/307:05:043
293602202165,25sleep30-21swapper/307:05:043
277172440,1sleep10-21swapper/108:10:251
23352420,1sleep20-21swapper/207:15:112
3141499384,13cyclictest0-21swapper/107:23:211
31414993815,2cyclictest15255-21irqstats10:10:171
3140899389,13cyclictest12027-21munin-run10:05:020
31408993510,12cyclictest21439-21awk11:40:000
3141499341,14cyclictest0-21swapper/110:15:221
3142599331,24cyclictest0-21swapper/307:40:233
3142599330,25cyclictest0-21swapper/308:45:333
3141499332,14cyclictest0-21swapper/108:39:501
3141499331,24cyclictest0-21swapper/110:50:581
3142599321,19cyclictest0-21swapper/308:56:043
3142599321,11cyclictest0-21swapper/309:15:423
31421993229,2cyclictest0-21swapper/208:47:032
3141499321,23cyclictest0-21swapper/107:42:141
3142599311,23cyclictest0-21swapper/308:41:333
3141499311,11cyclictest0-21swapper/107:12:221
3142599301,14cyclictest0-21swapper/307:19:363
3142599300,9cyclictest0-21swapper/309:46:263
3142599300,22cyclictest0-21swapper/309:58:143
3142599300,11cyclictest0-21swapper/310:51:413
3141499301,15cyclictest11960-21df08:50:141
3141499301,13cyclictest0-21swapper/112:03:291
3141499301,12cyclictest0-21swapper/109:21:531
3141499301,12cyclictest0-21swapper/108:18:341
3141499301,12cyclictest0-21swapper/108:18:341
31408993011,16cyclictest4107-21idleruntime-cro07:20:010
3142599291,12cyclictest0-21swapper/308:38:493
3142599290,27cyclictest0-21swapper/307:38:053
3142599290,17cyclictest0-21swapper/311:12:373
3141499291,9cyclictest12598-21diskstats10:05:161
31414992913,14cyclictest31779-21kworker/u8:011:00:271
3141499291,18cyclictest0-21swapper/112:37:531
3141499291,17cyclictest0-21swapper/107:56:581
3141499291,13cyclictest0-21swapper/109:37:051
3141499291,11cyclictest0-21swapper/108:58:291
3141499291,11cyclictest0-21swapper/108:25:021
3140899293,12cyclictest0-21swapper/007:45:020
3140899292,12cyclictest1651-21awk10:55:020
3142599280,10cyclictest0-21swapper/308:32:173
3141499281,11cyclictest0-21swapper/111:57:411
3141499281,11cyclictest0-21swapper/110:57:101
3141499281,11cyclictest0-21swapper/110:35:061
3141499281,11cyclictest0-21swapper/110:35:061
3141499281,11cyclictest0-21swapper/110:27:341
3141499281,11cyclictest0-21swapper/108:07:501
3141499281,11cyclictest0-21swapper/108:00:501
3141499281,11cyclictest0-21swapper/107:29:531
3141499281,10cyclictest0-21swapper/112:17:451
3141499281,10cyclictest0-21swapper/107:19:051
3142599271,9cyclictest0-21swapper/312:16:543
3142599271,14cyclictest0-21swapper/311:07:573
3142599270,17cyclictest0-21swapper/311:27:253
3142599270,10cyclictest0-21swapper/312:38:223
3141499276,4cyclictest0-21swapper/112:11:531
3141499271,16cyclictest0-21swapper/112:08:131
3141499271,12cyclictest0-21swapper/109:41:571
3141499271,12cyclictest0-21swapper/107:30:421
3141499271,11cyclictest0-21swapper/109:56:051
3141499271,10cyclictest0-21swapper/112:23:411
3141499271,10cyclictest0-21swapper/111:32:541
3141499271,10cyclictest0-21swapper/110:24:021
3141499271,10cyclictest0-21swapper/107:53:101
31414992710,3cyclictest0-21swapper/111:45:251
3140899271,13cyclictest28029-21awk11:55:000
3142599268,3cyclictest0-21swapper/307:28:063
3142599261,18cyclictest0-21swapper/307:58:093
3142599261,12cyclictest2321-21ntp_states09:40:223
3142599261,12cyclictest0-21swapper/312:02:003
3142599261,10cyclictest0-21swapper/311:33:023
3142599260,14cyclictest0-21swapper/307:22:323
3142599260,11cyclictest0-21swapper/311:49:383
3142599260,10cyclictest0-21swapper/312:26:223
3141499261,18cyclictest0-21swapper/109:47:331
31414992611,3cyclictest0-21swapper/111:54:011
3141499261,12cyclictest0-21swapper/108:47:061
3141499261,10cyclictest0-21swapper/111:19:181
3141499261,10cyclictest0-21swapper/109:29:011
3141499261,10cyclictest0-21swapper/109:29:011
3141499261,10cyclictest0-21swapper/109:18:331
3142599259,3cyclictest0-21swapper/311:41:563
3142599251,8cyclictest0-21swapper/312:05:063
31425992511,3cyclictest20880-21/usr/sbin/munin11:35:243
3142599251,12cyclictest0-21swapper/308:17:313
3142599251,12cyclictest0-21swapper/308:17:313
3142599251,10cyclictest31739-21cpuspeed_turbos07:10:123
3142599251,10cyclictest0-21swapper/308:21:013
3142599251,10cyclictest0-21swapper/308:08:233
3142599250,11cyclictest0-21swapper/312:14:443
3142599250,11cyclictest0-21swapper/311:04:093
3141499259,3cyclictest4767-21munin-run08:35:021
3141499258,2cyclictest0-21swapper/111:20:221
3141499251,9cyclictest0-21swapper/112:27:131
31414992512,6cyclictest11237-21kworker/u8:111:40:271
3141499251,11cyclictest0-21swapper/111:26:101
3141499251,10cyclictest0-21swapper/112:31:391
3140899259,3cyclictest28843-21kworker/0:209:40:460
3142599249,7cyclictest18434-21memory07:50:193
3142599249,3cyclictest0-21swapper/310:10:593
3142599241,9cyclictest0-21swapper/310:44:293
3142599241,9cyclictest0-21swapper/309:01:123
3142599241,14cyclictest19289-21ntp_states09:05:223
3142599241,11cyclictest0-21swapper/310:21:593
3142599241,11cyclictest0-21swapper/309:54:543
3142599241,11cyclictest0-21swapper/307:45:273
3142599240,9cyclictest0-21swapper/309:25:343
3142599240,9cyclictest0-21swapper/309:25:343
3142599240,8cyclictest0-21swapper/310:16:013
3142599240,16cyclictest0-21swapper/308:52:003
3142599240,11cyclictest0-21swapper/308:26:493
3142599240,10cyclictest0-21swapper/310:29:093
3142599240,10cyclictest0-21swapper/308:03:373
3142199243,8cyclictest8910-40dnf09:55:252
3141499249,3cyclictest0-21swapper/109:52:451
3141499248,3cyclictest0-21swapper/109:11:011
3141499241,11cyclictest0-21swapper/110:04:331
3141499241,10cyclictest0-21swapper/109:32:131
3142599238,8cyclictest0-21swapper/310:33:073
3142599238,3cyclictest0-21swapper/310:05:153
3142599237,7cyclictest0-21swapper/311:57:123
3142599237,4cyclictest0-21swapper/312:20:123
3142599236,3cyclictest0-21swapper/307:30:403
3142599231,6cyclictest0-21swapper/310:58:273
3142599231,14cyclictest0-21swapper/309:30:363
3142599231,11cyclictest0-21swapper/311:50:163
3142599231,10cyclictest0-21swapper/312:32:463
3142599231,10cyclictest0-21swapper/309:22:063
31425992310,3cyclictest0-21swapper/309:37:043
3141499239,3cyclictest0-21swapper/111:11:141
3141499238,3cyclictest0-21swapper/110:34:541
3141499237,3cyclictest0-21swapper/111:05:461
3141499233,3cyclictest0-21swapper/107:49:581
3141499231,10cyclictest0-21swapper/111:38:141
3141499231,10cyclictest0-21swapper/107:39:021
3140899239,12cyclictest0-21swapper/011:55:240
3140899238,14cyclictest0-21swapper/010:50:020
3140899230,11cyclictest28297-21awk08:15:000
3142599229,3cyclictest0-21swapper/310:48:153
3142599228,3cyclictest0-21swapper/308:10:073
3142599225,3cyclictest0-21swapper/310:38:193
3142599225,3cyclictest0-21swapper/310:38:193
3142599221,10cyclictest13926-21ntp_states11:20:213
3142599220,9cyclictest0-21swapper/311:17:373
3142599220,9cyclictest0-21swapper/310:01:143
3142199221,4cyclictest0-21swapper/211:37:012
3141499227,2cyclictest0-21swapper/108:40:141
3141499226,3cyclictest0-21swapper/109:09:371
3141499225,4cyclictest0-21swapper/110:45:141
3141499221,12cyclictest0-21swapper/108:28:141
3140899226,14cyclictest26101-21idleruntime-cro08:10:010
3142599211,11cyclictest0-21swapper/309:12:523
3142199214,10cyclictest14039-21sshd10:07:502
31421992114,4cyclictest15052-21grep10:10:142
3142199211,3cyclictest0-21swapper/209:43:362
3142199211,10cyclictest0-21swapper/212:02:412
31414992114,3cyclictest17090-21ntp_states09:00:211
3140899212,7cyclictest4610-21kworker/0:008:36:230
31408992115,3cyclictest7847-21gltestperf08:40:160
3140899211,12cyclictest29210-21/usr/sbin/munin09:30:240
3140899211,12cyclictest0-21swapper/009:49:380
38602200,5sleep23142199cyclictest08:30:212
3142199201,8cyclictest0-21swapper/210:50:432
3142199201,6cyclictest0-21swapper/211:22:282
31421992015,3cyclictest3507-21/usr/sbin/munin09:45:112
3142199201,3cyclictest0-21swapper/209:12:322
3141499206,3cyclictest0-21swapper/110:43:221
3140899202,15cyclictest29088-21grep10:40:250
3140899201,5cyclictest0-21swapper/009:11:460
31408992015,3cyclictest1758-21grep12:05:260
3140899201,13cyclictest0-21swapper/009:54:330
3140899201,11cyclictest3653-21sshd10:57:500
3140899201,11cyclictest31609-21/usr/sbin/munin07:10:220
3140899201,11cyclictest27013-21/usr/sbin/munin09:25:120
3140899201,11cyclictest27013-21/usr/sbin/munin09:25:120
3140899201,11cyclictest13315-21ntp_states10:05:230
3140899201,11cyclictest12086-21irqstats12:30:170
3142199198,3cyclictest10827-21grep11:15:142
3142199193,7cyclictest17518-21/usr/sbin/munin07:50:172
3142199191,7cyclictest0-21swapper/210:19:432
3142199191,6cyclictest0-21swapper/211:13:412
31421991916,2cyclictest0-21swapper/208:05:472
31421991913,3cyclictest21369-21df10:25:132
3140899193,8cyclictest0-21swapper/008:49:500
3140899192,15cyclictest0-21swapper/008:15:480
3140899192,15cyclictest0-21swapper/008:15:470
3140899191,7cyclictest0-21swapper/010:15:470
3140899191,7cyclictest0-21swapper/009:59:590
3140899191,6cyclictest0-21swapper/011:24:230
3140899191,5cyclictest0-21swapper/011:28:410
3140899191,16cyclictest15807-21awk09:00:010
3140899191,12cyclictest0-21swapper/009:00:110
3140899191,11cyclictest0-21swapper/009:20:460
3140899191,11cyclictest0-21swapper/007:50:250
3140899191,10cyclictest23530-21memory09:15:150
85902181,2sleep00-21swapper/012:21:480
3142199186,3cyclictest87350irq/126-enp1s0-11:45:122
3142199186,3cyclictest87350irq/126-enp1s0-11:40:122
3142199182,5cyclictest0-21swapper/210:43:522
3142199181,8cyclictest0-21swapper/210:55:162
3142199181,7cyclictest7503-21gltestperf12:20:142
3142199181,7cyclictest0-21swapper/210:02:072
3142199181,6cyclictest2269-21munin-run12:10:022
3142199181,6cyclictest10291-21sed07:30:242
3142199181,5cyclictest29888-40dnf11:56:002
31421991812,3cyclictest6589-21sshd12:19:202
31421991811,4cyclictest6239-21grep09:50:132
3142199181,11cyclictest0-21swapper/208:35:202
3140899184,8cyclictest755-21rngd09:08:470
3140899184,2cyclictest89050irq/133-enp2s0-12:15:180
3140899184,11cyclictest19719-21/usr/sbin/munin07:55:150
3140899181,6cyclictest0-21swapper/011:43:030
3140899181,6cyclictest0-21swapper/011:03:310
3140899181,5cyclictest0-21swapper/012:29:540
3140899181,5cyclictest0-21swapper/011:18:100
3140899181,3cyclictest18062-21irqstats11:30:170
60352171,2sleep20-21swapper/207:21:382
3142199174,3cyclictest31706-21needreboot08:20:192
3142199173,3cyclictest87550irq/128-enp1s0-07:10:162
3142199172,12cyclictest20190-21grep07:55:152
3142199171,7cyclictest15093-21latency_hist07:45:012
3142199171,7cyclictest0-21swapper/210:22:052
3142199171,6cyclictest0-21swapper/212:37:362
3142199171,6cyclictest0-21swapper/208:25:142
3142199171,5cyclictest0-21swapper/212:27:112
3142199171,5cyclictest0-21swapper/211:25:022
3142199171,5cyclictest0-21swapper/210:38:152
3142199171,5cyclictest0-21swapper/210:38:142
3142199171,3cyclictest0-21swapper/211:08:092
31421991712,3cyclictest23575-21df_inode10:30:132
31421991712,3cyclictest21919-21acpi08:00:122
31421991712,3cyclictest15807-21cut07:45:142
3142199171,14cyclictest19746-21grep09:05:262
31421991711,3cyclictest18013-21basename09:05:012
31421991711,3cyclictest1346-21munin-node09:35:112
3142199170,5cyclictest0-21swapper/209:25:252
3142199170,5cyclictest0-21swapper/209:25:252
3140899177,3cyclictest808-21snmpd07:37:410
3140899176,3cyclictest2488-21/usr/sbin/munin12:10:150
3140899174,3cyclictest87550irq/128-enp1s0-12:35:220
3140899174,3cyclictest10454-21perf11:15:010
3140899173,11cyclictest8093-21grep07:25:270
3140899171,6cyclictest25811-21sh11:50:000
3140899171,6cyclictest0-21swapper/011:09:020
3140899171,6cyclictest0-21swapper/008:52:550
3140899171,5cyclictest0-21swapper/010:31:140
3140899171,2cyclictest4416-21cpuspeed_turbos07:20:120
31408991712,3cyclictest30911-21fschecks_time12:00:140
3140899171,13cyclictest3783-21head08:30:180
31408991710,5cyclictest0-21swapper/008:25:170
267432171,2sleep20-21swapper/209:24:382
189802170,3sleep20-21swapper/211:32:012
3142199164,3cyclictest32-21ksoftirqd/209:15:142
3142199161,7cyclictest0-21swapper/212:10:152
3142199161,6cyclictest0-21swapper/212:30:182
3142199161,5cyclictest29786-21awk08:15:242
3142199161,5cyclictest29786-21awk08:15:242
3142199161,5cyclictest12433-21grep07:35:222
3142199161,4cyclictest0-21swapper/211:00:492
31421991611,3cyclictest8187-21netstat08:40:202
31421991611,3cyclictest12538-21missed_timers08:50:192
31421991610,3cyclictest827-21runrttasks10:49:242
31421991610,3cyclictest29210-21/usr/sbin/munin09:30:262
3140899165,5cyclictest0-21swapper/007:30:110
3140899165,3cyclictest87550irq/128-enp1s0-10:10:120
3140899161,6cyclictest0-21swapper/009:35:160
31408991612,3cyclictest0-21swapper/008:00:430
31408991611,3cyclictest25763-21df10:35:130
31408991611,3cyclictest25763-21df10:35:130
3140899161,13cyclictest0-21swapper/008:20:150
31408991610,3cyclictest21297-21ls10:25:140
31408991610,3cyclictest20176-21timerandwakeup10:20:240
3142199159,3cyclictest26282-21expr11:50:122
3142199151,6cyclictest7452-21memory07:25:192
3142199151,5cyclictest13820-21/usr/sbin/munin08:55:232
3142199151,4cyclictest26391-21cpuspeed_turbos08:10:122
3140899151,6cyclictest0-21swapper/007:45:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional